Can a Orange Pi 5 (RK3588S) run Granite 4.0 350M?
Yes, usably. Best reported generation speed: 55.2 tokens/s on Orange Pi 5 (RK3588S) (Q4_0, rk-llama.cpp fork (CPU)), per milhcbt/rk-llama.cpp. Boards tested: Orange Pi 5 (RK3588S).
Granite 4.0 350M speed by board
- Orange Pi 5 (RK3588S): 55.2 tokens/s best (Q4_0, rk-llama.cpp fork (CPU)), 4 tests
FAQ
Which edge board runs Granite 4.0 350M fastest?
Of the boards tested, Orange Pi 5 (RK3588S) is fastest for Granite 4.0 350M: 55.2 tokens/s generation (Q4_0, rk-llama.cpp fork (CPU)).
How fast does Granite 4.0 350M run on Orange Pi 5 (RK3588S)?
Best reported generation speed for Granite 4.0 350M on Orange Pi 5 (RK3588S) is 55.2 tokens/s (Q4_0, rk-llama.cpp fork (CPU)), from 4 sourced tests. Source: milhcbt/rk-llama.cpp (https://github.com/milhcbt/rk-llama.cpp).
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| Orange Pi 5 (RK3588S) | Granite 4.0 350M0.35B params · Q4_0 | rk-llama.cpp fork (CPU) | 55.2prompt 340.4 t/s | ₹29,000 Buy1.90 t/s per ₹1k | Community milhcbt/rk-llama.cpp2026 |
| Orange Pi 5 (RK3588S) | Granite 4.0 350M0.35B params · Q8_0 | rk-llama.cpp fork (CPU) | 40.6prompt 163.4 t/s | ₹29,000 Buy1.40 t/s per ₹1k | Community milhcbt/rk-llama.cpp2026 |
| Orange Pi 5 (RK3588S) | Granite 4.0 350M0.35B params · Q4_0 | rk-llama.cpp fork (NPU) | 26.7prompt 163.6 t/s | ₹29,000 Buy0.92 t/s per ₹1k | Community milhcbt/rk-llama.cpp2026 |
| Orange Pi 5 (RK3588S) | Granite 4.0 350M0.35B params · Q8_0 | rk-llama.cpp fork (NPU) | 25.4prompt 311.8 t/s | ₹29,000 Buy0.88 t/s per ₹1k | Community milhcbt/rk-llama.cpp2026 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).