Can a Raspberry Pi 5 run Llama 3.1 8B IT?
Yes, but slowly. Best reported generation speed: 2.0 tokens/s on Raspberry Pi 5 (Q4_K_M, llama.cpp b9307 (CPU, OpenBLAS)), per davidscarth (edge-ai-study). Boards tested: Raspberry Pi 5.
Llama 3.1 8B IT speed by board
- Raspberry Pi 5: 2.0 tokens/s best (Q4_K_M, llama.cpp b9307 (CPU, OpenBLAS)), 3 tests
FAQ
Which edge board runs Llama 3.1 8B IT fastest?
Of the boards tested, Raspberry Pi 5 is fastest for Llama 3.1 8B IT: 2.0 tokens/s generation (Q4_K_M, llama.cpp b9307 (CPU, OpenBLAS)).
How fast does Llama 3.1 8B IT run on Raspberry Pi 5?
Best reported generation speed for Llama 3.1 8B IT on Raspberry Pi 5 is 2.0 tokens/s (Q4_K_M, llama.cpp b9307 (CPU, OpenBLAS)), from 3 sourced tests. Source: davidscarth (edge-ai-study) (https://github.com/davidscarth/edge-ai-study/blob/main/RASPI5.md).
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| Raspberry Pi 516 GB RAM | Llama 3.1 8B IT8.03B params · Q4_K_M | llama.cpp b9307 (CPU, OpenBLAS)NVMe (official 512GB SSD kit) · Active Cooler | 2.0prompt 9.1 t/s | ₹34,191 Buy0.06 t/s per ₹1k | Community davidscarth (edge-ai-study)2026-05-27 |
| Raspberry Pi 516 GB RAM | Llama 3.1 8B IT8.03B params · UD-Q3_K_XL | llama.cpp b9307 (CPU, OpenBLAS)NVMe (official 512GB SSD kit) · Active Cooler | 1.9prompt 6.3 t/s | ₹34,191 Buy0.06 t/s per ₹1k | Community davidscarth (edge-ai-study)2026-05-27 |
| Raspberry Pi 516 GB RAM | Llama 3.1 8B IT8.03B params · Q8_0 | llama.cpp b9307 (CPU, OpenBLAS)NVMe (official 512GB SSD kit) · Active Cooler | 1.1prompt 4.9 t/s | ₹34,191 Buy0.03 t/s per ₹1k | Community davidscarth (edge-ai-study)2026-05-27 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).