LLMs on the RK3588 (unspecified board)
8 sourced results. 8 configurations reached 5+ tokens/s generation.
Good for
- Small 1–3B (chat, simple agents): usable, median 9.4 t/s (≈7 words/s), best 13.2 t/s, 8 tests
- Not tested yet: Medium 7–8B, Big 14B+. Help fill the gap
No Indian price found yet. Labels: ≥15 t/s fast · 5–15 usable · 2–5 slow · <2 too slow to chat. 1 token ≈ 0.75 words.
Real-world uses
Our data for this board: small (1–3B) models: median 9.4 t/s across 8 tests (usable). Use cases that need chat-speed replies want ≥5 t/s; background jobs can live with 2 t/s.
- Home server + local chatbot: Low-power always-on box running a small model for family chat or Home Assistant.
- NPU-accelerated small models: RK3588's NPU (via rknn-llm or NPU forks) can speed up small models; see our NPU vs CPU rows.
- Private document Q&A: Small RAG setups on 8–32GB RAM variants.
- Media + AI kiosk: Multiple display outputs plus a small model for signage or info desks.
Your board here
Reach makers and engineers choosing edge-AI hardware. Sponsor this slot →
India prices
No Indian retailer listing found yet. Seen one? Tell us.
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| RK3588 (unspecified board) | MiniCPM5 2B2B params · W4A8 mixed (QQQ-Rot) | open_rknpu2_lm_infer v0.6.1 (NPU) | 13.2prompt 245.5 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README)2026-10-20 |
| RK3588 (unspecified board) | Qwen3.5 4B4B params · W4A8 mixed (QQQ-Rot) | open_rknpu2_lm_infer v0.6.1 (NPU, MTP1 speculative decoding) | 10.6prompt 147.8 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README)2026-10-20 |
| RK3588 (unspecified board) | Qwen3 4B4B params · W4A8 mixed (QQQ-Rot) | open_rknpu2_lm_infer v0.6.1 (NPU) | 10.5prompt 169.7 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README)2026-10-20 |
| RK3588 (unspecified board) | MiniCPM5 2B2B params · W8A8 | rknn-llm 1.3.1 (NPU, vendor SDK) | 9.4prompt 211.1 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README, vendor baseline)2026-10-20 |
| RK3588 (unspecified board) | Granite 4.2 3B3B params · W4A8 mixed (QQQ-Rot) | open_rknpu2_lm_infer v0.6.1 (NPU) | 9.3prompt 182.5 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README)2026-10-20 |
| RK3588 (unspecified board) | Granite 4.2 3B3B params · W8A8 | rknn-llm 1.3.1 (NPU, vendor SDK) | 6.8prompt 120.5 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README, vendor baseline)2026-10-20 |
| RK3588 (unspecified board) | Qwen3 4B4B params · W8A8 | rknn-llm 1.3.1 (NPU, vendor SDK) | 5.8prompt 105.0 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README, vendor baseline)2026-10-20 |
| RK3588 (unspecified board) | Qwen3.5 4B4B params · W8A8 | rknn-llm 1.3.1 (NPU, vendor SDK) | 5.5prompt 58.3 t/s | No India listing yet | Community happyme531 (open_rknpu2_lm_infer README, vendor baseline)2026-10-20 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).