Can a Raspberry Pi 5 run Dolphin 2.6 Mistral 7B DPO laser?
Yes, but slowly. Best reported generation speed: 1.6 tokens/s on Raspberry Pi 5 (Q5_K_M, llama.cpp (GGUF)), per r/LocalLLaMA commenter. Boards tested: Raspberry Pi 5.
Dolphin 2.6 Mistral 7B DPO laser speed by board
- Raspberry Pi 5: 1.6 tokens/s best (Q5_K_M, llama.cpp (GGUF)), 1 test
FAQ
Which edge board runs Dolphin 2.6 Mistral 7B DPO laser fastest?
Of the boards tested, Raspberry Pi 5 is fastest for Dolphin 2.6 Mistral 7B DPO laser: 1.6 tokens/s generation (Q5_K_M, llama.cpp (GGUF)).
How fast does Dolphin 2.6 Mistral 7B DPO laser run on Raspberry Pi 5?
Best reported generation speed for Dolphin 2.6 Mistral 7B DPO laser on Raspberry Pi 5 is 1.6 tokens/s (Q5_K_M, llama.cpp (GGUF)), from 1 sourced test. Source: r/LocalLLaMA commenter (https://www.reddit.com/r/LocalLLaMA/comments/19444pu/phi2_tiny_llama_on_raspberry_pi_5/).
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| Raspberry Pi 58 GB RAM | Dolphin 2.6 Mistral 7B DPO laser7B params · Q5_K_M | llama.cpp (GGUF) | 1.6 | ₹18,643 Buy0.09 t/s per ₹1k | Community r/LocalLLaMA commenter2024-01-11 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).