The problem is that they become a buzz word for at scale companies that need them because they have huge complex architects, but then non at scale companies blindly follow the hype when they were created out of necessity for giant tech stacks that are a totally different use case.
I tried out ollama. It was trivially easy to set up.
Stable diffusion is a bit more work, but any power user should be able to figure it out.