![](/static/253f0d9b/assets/icons/icon-96x96.png)
![](https://fry.gs/pictrs/image/c6832070-8625-4688-b9e5-5d519541e092.png)
4·
1 month agoJust beware that like AMD, Intel GPUs suffer a performance hit when using LLMs because of the CUDA specific optimizations in frameworks like llama.cpp
Just beware that like AMD, Intel GPUs suffer a performance hit when using LLMs because of the CUDA specific optimizations in frameworks like llama.cpp
I’ve heard good things about Instander, but it may not satisfy all the requirements of an alternative frontend.
Do you have any tips (or examples) using quadlets? I tried using them but I couldn’t wrap my head around them.