oh freaking awesome, this looks amazing! Thank you so much for this!

For most of you suggesting hosting a repository - yes but,

Host forgejo. Just host the git mirror. It comes with a package repo out of the box. Then you have the source code and the docker images

Generally there are not LLMs that do this, but you start building up a workflow. You speak, one service reads in the audio and translates it to text. Then you feed that into an LLM, it responds in text, and you have another service translate that into audio.

Home Assistant is the easiest way to get them all put together.

https://www.home-assistant.io/integrations/assist_pipeline

Edit agree with others below. Use the apps that are made for it.

• Whisper for STT
• Any hosted LLM can work, text-generation-webui or tabbyapi
• I use xttsv2 for TTS

This post is written as if there’s only one “community”. Why does there need to be a primary? I’m here and I’m happy. If I have questions I search online or ask here, same as any other community

It’s very hard to convince people that fedi is a healthy place when the default servers are incredibly toxic. I wish they would at least advertise it as such, maybe hide the default from the number one spot. There are several servers up there that accept users that are way more chill.

Also for new selfhosters making it easier to say “these are some problem instances that are commonly blocked, if you want to start out with them”. I know that starts a new problem of “but then who decides” and it causes more splintering, but for a lot of posters it’s overwhelming the firehouse of vitriol that comes in at first.

Seconded with Matrix. All I’m wanting for it is for someone to make a Discord/Revolt UI frontend for Matrix 2.0 and it’ll be a drop in replacement

Good! I’d be sad to see you folks leave

Try going the other route, and explicitly subscribing to content you enjoy. From what I read you aren’t alone, that all is a firehouse and it’s not all fun. Subscribed to me is a much chiller experience

Do you use all or subscribe to things? I have a good list of communities and I honestly don’t see much if any politics

Good, how is it going over there at feddit.uk? I forgot the last owner just kind of ghosted

Oh yeah, critical component. And vram, in fact I would only consider LLMs on a 3000+ card right now, they require quite a bit of vram

NVidia is great in a server, drivers are a pain but do-able. I have a 3000 series that I use regularly and pass into my kubernetes cluster. NVidia on a gaming rig linux is fine, but there is more overhead with the drivers.

AMD is great in gaming servers, but doesn’t have CUDA, so it’s not as useful in a server environment in my experience - if you’re thinking of doing CUDA workloads like hosting LLMs.

1060 will be a noticeable step in Jellyfin

DNS is of course the preferred approach

I wouldn’t worry about mounting your nfs shares directly to those host unless you need to. Compose has an operator similar to k8s that lets docker itself manage the shares, which is insanely useful if you lose your host. Then you don’t have to have piles of scripts to mount them.

https://stackoverflow.com/questions/45282608/how-to-directly-mount-nfs-share-volume-in-container-using-docker-compose-v3

version: "3.2"

services:
  rsyslog:
    image: jumanjiman/rsyslog
    ports:
      - "514:514"
      - "514:514/udp"
    volumes:
      - type: volume
        source: example
        target: /nfs
        volume:
          nocopy: true
volumes:
  example:
    driver_opts:
      type: "nfs"
      o: "addr=10.40.0.199,nolock,soft,rw"
      device: ":/docker/example"

I would really love an official way to transfer ownership of a server. I suppose as long as the domain can transfer and the docker volumes are zipped up it could be migrated anywhere

LLMs use a ton of VRAM, the more VRAM you have the better.

If you just need an API, then TabbyAPI is pretty great.

If you need a full UI, then Oogabooga’s TextGenration WebUI is a good place to start

They’re two different protocols built for two very different forms of communication.

Yep, need a fork of revolt that uses Matrix 2.0 as a backend and I can get my friends to switch

Is this really feature flags as a service? FFAAS?

Tried it and it works pretty well! I’ll have to keep playing with it, but so far so good!