Can a Raspberry Pi CM4 run Llama 3 8B Instruct?
Technically, but very slowly. Best reported generation speed: 0.9 tokens/s on Raspberry Pi CM4 (Q2_K, Ollama), per hammad93 (GitHub gist). Boards tested: Raspberry Pi CM4.
Llama 3 8B Instruct speed by board
- Raspberry Pi CM4: 0.9 tokens/s best (Q2_K, Ollama), 1 test
FAQ
Which edge board runs Llama 3 8B Instruct fastest?
Of the boards tested, Raspberry Pi CM4 is fastest for Llama 3 8B Instruct: 0.9 tokens/s generation (Q2_K, Ollama).
How fast does Llama 3 8B Instruct run on Raspberry Pi CM4?
Best reported generation speed for Llama 3 8B Instruct on Raspberry Pi CM4 is 0.9 tokens/s (Q2_K, Ollama), from 1 sourced test. Source: hammad93 (GitHub gist) (https://gist.github.com/hammad93/eab28862024e480a2046d51bc39bd462/7e502ea4019400a75f4fc5c369992b273f2a5398).
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| Raspberry Pi CM4 | Llama 3 8B Instruct8B params · Q2_K | Ollama | 0.9prompt 1.0 t/s | ₹6,185 Buy0.14 t/s per ₹1k | Community hammad93 (GitHub gist)2024-05-04 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).