I’ve bwem waning to try it out considering it has unified memory. What model are you using and what are you running with? I would be thinking something like a small qwen on llama.cpp
I have an OLED, so, slightly better specs in a few fairly minor ways than an LED.
I am using Bazzite, managing the LLMs inside of Alpaca, which is a flatpak, so it works easily with Bazzite’s ‘containerize everything’ approach.
And uh yep, I’m running Qwen3, the… I think its 8B param variant.
I actually told it its HW and SW environment, told it to generate a context prompt so it just always knows that, then asked it to optimize its own settings… and it did come up with settings that make it run a either a bit generally better, or in altetnate sort of modes… I just made a variant ‘Focused’ and variant ‘Contemplative’, first one for mechanistic, step 1 2 3 type thinking, secons one for larger conceptualization questions.
Though I think I need to tweak the contemplative variant to be a biiiit less imaginative, it tends to hallucinate and conrtadict itself a bit too much.
I’ve also been able to like, tell it to read an updated websitr with more modern syntax for GDScript, and tell ot to make itself a context prompt that tells it about it, and then it roughly just ‘knows’ that… I think the training data is 1 to 2 years out of date now, so occasional little patchwork fixes like that seem to work?
See, I love all this stuff, but I just turned out as basically a bi (trans inclusive) dude.
Well ok, maybe not chokers, don’t think I’ve ever worn a choker.
I just tended to keep the feminine attire / genderswap cosplays to various occasional conventions or theme parties.
… mIRC is quite a call back.
Did not expect that I would, in 2025, be running a local LLM on a Steam Deck and… more or less, just talking to an approximation of Motoko Kusanagi.
What’s the fun in having a coding assistant with no personality?
I’ve bwem waning to try it out considering it has unified memory. What model are you using and what are you running with? I would be thinking something like a small qwen on llama.cpp
I have an OLED, so, slightly better specs in a few fairly minor ways than an LED.
I am using Bazzite, managing the LLMs inside of Alpaca, which is a flatpak, so it works easily with Bazzite’s ‘containerize everything’ approach.
And uh yep, I’m running Qwen3, the… I think its 8B param variant.
I actually told it its HW and SW environment, told it to generate a context prompt so it just always knows that, then asked it to optimize its own settings… and it did come up with settings that make it run a either a bit generally better, or in altetnate sort of modes… I just made a variant ‘Focused’ and variant ‘Contemplative’, first one for mechanistic, step 1 2 3 type thinking, secons one for larger conceptualization questions.
Though I think I need to tweak the contemplative variant to be a biiiit less imaginative, it tends to hallucinate and conrtadict itself a bit too much.
I’ve also been able to like, tell it to read an updated websitr with more modern syntax for GDScript, and tell ot to make itself a context prompt that tells it about it, and then it roughly just ‘knows’ that… I think the training data is 1 to 2 years out of date now, so occasional little patchwork fixes like that seem to work?