gemma-4-26B-A4B-itgoogle
Ideal fit26.5BQ4_K_M15.1 GB23.4 GB needed~52.7 tok/s
Qwen3.5-35B-A3BQwen
Ideal fit36BQ4_K_M20.5 GB29.2 GB needed~70.2 tok/s
Qwen3.5-35B-A3B-MTPQwen
Ideal fit36BQ4_K_M20.5 GB29.2 GB needed~70.2 tok/s
gpt-oss-20bRedHatAI
Ideal fit21.5BQ4_K_M12.3 GB16 GB needed~58.1 tok/s
gpt-oss-20bopenai
Ideal fit21BQ4_K_M12 GB15.7 GB needed~58.5 tok/s
typhoon2.5-qwen3-30b-a3btyphoon-ai
Ideal fit30.5BQ4_K_M17.4 GB29.6 GB needed~63.1 tok/s
Qwen3.6-35B-A3BQwen
Ideal fit36BQ4_K_M20.5 GB32.7 GB needed~70.2 tok/s
Qwen3.6-35B-A3B-MTPQwen
Ideal fit36BQ4_K_M20.5 GB32.7 GB needed~70.2 tok/s
LFM2-8B-A1BLiquidAI
Ideal fit8.3BQ4_K_M4.8 GB20.7 GB needed~149.7 tok/s
DeepSeek-Coder-V2-Lite-Instructdeepseek-ai
Ideal fit15.7BQ4_K_M8.9 GB20.6 GB needed~87.8 tok/s
DeepSeek-R1-0528-Qwen3-8Bdeepseek-ai
Ideal fit8.2BQ4_K_M4.7 GB22.6 GB needed~32.2 tok/s
DeepSeek-R1-Distill-Qwen-7Bdeepseek-ai
Ideal fit7.6BQ4_K_M4.3 GB11.8 GB needed~34.6 tok/s
typhoon2.5-qwen3-4btyphoon-ai
Ideal fit4BQ4_K_M2.3 GB20.2 GB needed~65.5 tok/s
Qwen2.5-7B-Instruct-1MQwen
Ideal fit7.6BQ4_K_M4.3 GB11.8 GB needed~34.6 tok/s
Hermes-3-Llama-3.1-8BNousResearch
Ideal fit8BQ4_K_M4.6 GB20.6 GB needed~32.8 tok/s
Meta-Llama-3.1-8B-InstructNousResearch
Ideal fit8BQ4_K_M4.6 GB20.6 GB needed~32.8 tok/s
gemma-4-E4B-itgoogle
Ideal fit8BQ4_K_M4.6 GB20.6 GB needed~32.9 tok/s
Mistral-7B-Instruct-v0.3mistralai
Ideal fit7.2BQ4_K_M4.1 GB8.7 GB needed~36.3 tok/s
dolphin-2.9.3-mistral-7B-32kdphn
Ideal fit7.2BQ4_K_M4.1 GB8.7 GB needed~36.3 tok/s
4.7BQ4_K_M2.7 GB8 GB needed~56.5 tok/s
Qwen3.5-4B-MTPQwen
Ideal fit4.7BQ4_K_M2.7 GB8 GB needed~56.5 tok/s
xLAM-7b-rSalesforce
Ideal fit7.2BQ4_K_M4.1 GB8.7 GB needed~36.4 tok/s
granite-3.3-8b-instructibm-granite
Ideal fit8.2BQ4_K_M4.7 GB24.3 GB needed~32.2 tok/s
Qwen2.5-7B-InstructQwen
Ideal fit7.6BQ4_K_M4.3 GB6.9 GB needed~34.6 tok/s
Qwen2.5-VL-7B-InstructQwen
Ideal fit8.3BQ4_K_M4.7 GB12.2 GB needed~31.8 tok/s
gemma-4-E2B-itgoogle
Ideal fit5.1BQ4_K_M2.9 GB19.9 GB needed~51.4 tok/s
Falcon3-7B-Instructtiiuae
Ideal fit7.5BQ4_K_M4.2 GB9 GB needed~35.3 tok/s
Qwen2.5-Math-7B-InstructQwen
Ideal fit7.6BQ4_K_M4.3 GB5.3 GB needed~34.6 tok/s
Phi-3.5-mini-instructmicrosoft
Ideal fit3.8BQ4_K_M2.2 GB26.7 GB needed~68.9 tok/s
Qwen2-7B-InstructQwen
Ideal fit7.6BQ4_K_M4.3 GB6.9 GB needed~34.6 tok/s
SmolLM3-3BHuggingFaceTB
Ideal fit3BQ4_K_M1.7 GB17.3 GB needed~87.8 tok/s
gemma-3n-E4B-itgoogle
Ideal fit8BQ4_K_M4.6 GB11.6 GB needed~32.9 tok/s
9.7BQ4_K_M5.5 GB10.3 GB needed~27.3 tok/s
Qwen3.5-9B-MTPQwen
Ideal fit9.7BQ4_K_M5.5 GB10.3 GB needed~27.3 tok/s
Ministral-8B-Instruct-2410mistralai
Ideal fit8BQ4_K_M4.6 GB9.1 GB needed~32.8 tok/s
Qwen2.5-VL-3B-InstructQwen
Ideal fit3.8BQ4_K_M2.1 GB7.2 GB needed~70.2 tok/s
Meta-Llama-3-8B-Instructmeta-llama
Ideal fit8BQ4_K_M4.6 GB5.8 GB needed~32.8 tok/s
Qwen2.5-3B-InstructQwen
Ideal fit3.1BQ4_K_M1.8 GB3.6 GB needed~85.4 tok/s
3.1BQ4_K_M1.8 GB3.6 GB needed~85.4 tok/s
Mistral-NeMo-Minitron-8B-Instructnvidia
Ideal fit8.4BQ4_K_M4.8 GB6.4 GB needed~31.3 tok/s
gemma-3n-E2B-itgoogle
Ideal fit4BQ4_K_M2.3 GB6.3 GB needed~65.9 tok/s
Mistral-Nemo-Instruct-2407mistralai
Ideal fit12.2BQ4_K_M7 GB26.6 GB needed~21.5 tok/s
Qwen2.5-Coder-7B-InstructQwen
Ideal fit7.6BQ4_K_M4.3 GB6.9 GB needed~34.6 tok/s
gemma-3-12b-itgoogle
Ideal fit12BQ4_K_M6.8 GB15.9 GB needed~22 tok/s
14.8BQ4_K_M8.4 GB28.3 GB needed~17.8 tok/s
27.8BQ4_K_M15.8 GB24.5 GB needed~9.5 tok/s
Phi-4-mini-reasoningmicrosoft
Ideal fit3.8BQ4_K_M2.2 GB4.3 GB needed~69.3 tok/s
phi-3-mini-4k-instructmicrosoft
Ideal fit3.8BQ4_K_M2.2 GB4.2 GB needed~68.9 tok/s
glm-4-9b-chatTHUDM
Ideal fit9.4BQ4_K_M5.4 GB10.9 GB needed~28 tok/s
Yi-1.5-9B-Chat01-ai
Ideal fit8.8BQ4_K_M5 GB6.3 GB needed~29.8 tok/s
gemma-2-9b-itgoogle
Ideal fit9.2BQ4_K_M5.3 GB7.6 GB needed~28.5 tok/s
27.8BQ4_K_M15.8 GB24.5 GB needed~9.5 tok/s
Qwen3.5-27B-MTPQwen
Ideal fit27.8BQ4_K_M15.8 GB24.5 GB needed~9.5 tok/s
2.3BQ4_K_M1.3 GB6.1 GB needed~115.8 tok/s
Qwen3.5-2B-MTPQwen
Ideal fit2.3BQ4_K_M1.3 GB6.1 GB needed~115.8 tok/s
xLAM-8x7b-rSalesforce
Tight46.7BQ4_K_M26.6 GB31.2 GB needed~15.7 tok/s
Qwen2.5-Coder-3B-InstructQwen
Ideal fit3.1BQ4_K_M1.8 GB3.6 GB needed~85.4 tok/s
Qwen2.5-Coder-3BQwen
Ideal fit3.1BQ4_K_M1.8 GB3.6 GB needed~85.4 tok/s
Mistral-Small-24B-Instruct-2501mistralai
Ideal fit24BQ4_K_M13.7 GB21.3 GB needed~11 tok/s
LFM2.5-1.2B-InstructLiquidAI
Ideal fit1.2BQ4_K_M0.7 GB12.8 GB needed~225.1 tok/s