Can a Orange Pi 5 (RK3588S) run Phi-3?
Phi-3 speed by board
- Orange Pi 5 (RK3588S): 6.9 tokens/s best (IQ4_XS, llama.cpp (-t 4, ~b2787)), 1 test
- Raspberry Pi CM4: 1.4 tokens/s best (Ollama), 1 test
FAQ
Which edge board runs Phi-3 fastest?
Of the boards tested, Orange Pi 5 (RK3588S) is fastest for Phi-3: 6.9 tokens/s generation (IQ4_XS, llama.cpp (-t 4, ~b2787)). Next is Raspberry Pi CM4 at 1.4 tokens/s.
How fast does Phi-3 run on Orange Pi 5 (RK3588S)?
Best reported generation speed for Phi-3 on Orange Pi 5 (RK3588S) is 6.9 tokens/s (IQ4_XS, llama.cpp (-t 4, ~b2787)), from 1 sourced test. Source: llama.cpp issue #7176 author (https://github.com/ggml-org/llama.cpp/issues/7176).
How fast does Phi-3 run on Raspberry Pi CM4?
Best reported generation speed for Phi-3 on Raspberry Pi CM4 is 1.4 tokens/s (Ollama), from 1 sourced test. Source: hammad93 (GitHub gist) (https://gist.github.com/hammad93/eab28862024e480a2046d51bc39bd462/7e502ea4019400a75f4fc5c369992b273f2a5398).
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| Orange Pi 5 (RK3588S) | Phi-33.8B params · IQ4_XS | llama.cpp (-t 4, ~b2787) | 6.9prompt 10.8 t/s | ₹29,000 Buy0.24 t/s per ₹1k | Community llama.cpp issue #7176 author2024-05-09 |
| Raspberry Pi CM4 | Phi-33.8B params | Ollama | 1.4prompt 2.0 t/s | ₹6,185 Buy0.22 t/s per ₹1k | Community hammad93 (GitHub gist)2024-05-04 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).