You’re not me. You will have different tastes too me. No point you using the distros I like. Do what everybody else does, experiment, try a few, read about them. The are loads out there all with little quirks and differences. Try L/K/X/Ubuntu, try cachyos / arch, try void, try Gentoo, try popos… Have a look at distrowatch for loads of inspiration.

Any and all distros can use and run any software any other distro runs so they can all do the same stuff, just they all get there in slightly different ways.

Yeah, I know how to do that very easily, but I don’t want to have my pc running 24/7 just so I can use the llm when I’m out of the house. My heaviest use is at home. It’s a large power requirement for minimal use.

Yeah, I have a fairly useable setup although it’s only accessible on my lan. I have a web server but it doesn’t host the llm as it’s not powerful enough.

Rx7600 8gb vram GPU 5700x CPU 64gb 3600mts ram

Hosting ornith 35b a3b iq4_xs Llama.cpp server Pi harness or llama.cpp webui Searxng web search Coding sandbox Restricted filesystem access

Get around 40 tok / sec and it’s really useable. Not as good as Claude sonnet, but good enough for most things.

I host my own Lemmy instance, but only because I spent the last 3 years learning to host other stuff. It’s taken a lot of effort and time to figure it all out and get it working, and the only reason I bothered was because it interested me. Most people aren’t that interested in how it all works and just want to use a working thing.

AHH cool, I didn’t thought both 3.8 models were on 4 architecture, I’ll edit my previous comment 👍

I don’t really understand your comment? Qwen3.8 27b is qwen4 architecture as a pre-release. apparently i was wrong here, it’s still on the 3.5 architecture.

A 27b dense model is really difficult to compare to a 35b a3b Moe model. I tried to run qwen3.8 27b on my 8gb card and ok it runs technically, but in no way is it viable or does it come close to ornith 1.5 35b a3b, mostly because to run qwen3.8 27b, I have to choke it to almost useless levels by running an aggressive quant like a 1, 2, or 3 bit quant, and even then it doesn’t run as fast.

I will cap mine for shorter chats to 32k, but for full blown complex tasks, I’ll turn it up to 130k. I run the q4 k m imatrix version

I run ornith 1.5 35b a3b on my 8gb Rx 7600, get 30 t/s, and have context at 130k. It slows down to around 15 t/s near the end on the context window.

Nothing I have tested comes close to this model.

I use it in pi with search, coding sandbox and limited filesystem access.

Yeah, there is a lot to it. Firstly, if you have loads of ram, and don’t really use it, then discussing swap is pointless, just don’t have a swap device. If ram on your system is marginal for your workload, then swap is a must have as your system will gain performance by being able to swap, even if it is to an SSD. In that case your system will start swapping to ssd as it gets closer to being full. Zswap improves this by using the margin the system has in ram at which it would swap, to compress the pages in ram instead. So you don’t lose any usable ram with zswap. But you gain performance as compressing the pages in ram is faster than writing them to SSD. But it does eat some CPU load in doing this.

Sorry, I may have misunderstood. No, if you don’t use your SSD as a swap device, you won’t use or wear your SSD with swap. With 64gb ram and just playing AAA games, you won’t have much need for any kind of swap. If you decide to do any heavy workloads that are heavy on ram requirements, zswap is an option to keep swap off your SSD and thus prevent wear on the disk. However if you are very limited on ram for your workload, zswap might not provide enough space, then traditional SSD swap will be the final option.

That’s not how it works but yes, if you don’t need to swap, then that is better. Avoiding swapping though is also the wrong angle as depending on amount of ram, swapping can be beneficial, it just depends.

Gaming isn’t heavy on ram. You need fast ram not lots of ram for gaming. Lots of ram helps with heavy workloads, compiling and software development and stuff. Playing a game only needs the recommended amount for that game, then as fast as you can get so as not to be a bottleneck for your CPU.

Yes there are lots of methods to do this. Format the disk to something like ext4, untar a distro to it, then with wsl chroot into the disk, finish installing and configuring everything, then reboot to the new disk. You will need specific time to do an this, but should be fairly straight forward.

I essentially did exactly this for my current setup.

There are other ways to.

My instance of Lemmy doesn’t support down votes, and I’m sure others are the same.

Give onrith 35b a3b a try. On my test suite it makes Gemma 4 look bad. https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B

I run on an rx7600 with 8gb vram, 64gb ddr4 and a 5700x

deleted by creator

Good idea! Makes sense 👍

After about a year on void Linux, I would agree. It’s a great, solid, and underrated distro 👍

My always on server hosting 20+ services draws no more than 50w at peak during the night backup, but that lasts maybe an hour or so, normally runs nearer 5w-10w when the hdd’s are spun down and unmounted 22 hours a day. I use an Intel n97 CPU in a mini pc. Navidrome, immich, adguard home, Lemmy, find my device, forgejo, file browser quantum, sterling pdf, valutwarden just to name some of the services it runs easily. Paying 22p per kWh in the UK.

Anything slightly better than a potato will run all of that except the llm’s. To have a useful llm you need to spend circa £1k on a GPU and even then it won’t come close to Claude or chatgpt. Skip the llm’s for now. Just get yourself a solid little server to do everything else you need. Add dedicated hardware specifically for llm’s in future if you really want to. An old used gaming pc will be plenty for your server. I personally use Debian , but there are loads of suitable options available