LLMs on the Orange Pi 5 Plus (RK3588)
6 sourced results. 4 configurations reached 5+ tokens/s generation.
Good for
- Small 1–3B (chat, simple agents): usable, median 6.2 t/s (≈5 words/s), best 25.9 t/s, 6 tests
- Not tested yet: Medium 7–8B, Big 14B+. Help fill the gap
Cheapest India price seen: ₹84,864 (Amazon.in (seller Waterberry), in stock, checked 2026-10-17). Labels: ≥15 t/s fast · 5–15 usable · 2–5 slow · <2 too slow to chat. 1 token ≈ 0.75 words.
Real-world uses
Our data for this board: small (1–3B) models: median 6.2 t/s across 6 tests (usable). Use cases that need chat-speed replies want ≥5 t/s; background jobs can live with 2 t/s.
- Home server + local chatbot: Low-power always-on box running a small model for family chat or Home Assistant.
- NPU-accelerated small models: RK3588's NPU (via rknn-llm or NPU forks) can speed up small models; see our NPU vs CPU rows.
- Private document Q&A: Small RAG setups on 8–32GB RAM variants.
- Media + AI kiosk: Multiple display outputs plus a small model for signage or info desks.
Your board here
Reach makers and engineers choosing edge-AI hardware. Sponsor this slot →
India price comparison
Prices as seen on retailer pages (via search snapshots) on the date shown. Not live; confirm on the retailer page. Buy links are plain links for now: affiliate tags will be added later where marked.
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| Orange Pi 5 Plus (RK3588)16 GB RAM | Qwen2.5 1.5B1.5B params · W8A8 | rknn-llm (NPU)NVMe data disk (/mnt/nvme) | 25.9 ⚠ unusual, check sourceprompt 493.0 t/s | ₹84,864 Buy0.30 t/s per ₹1k | Community jishuzhan.net author2026-09-14 |
| Orange Pi 5 Plus (RK3588)16 GB RAM | Qwen2.5 3B3B params · W8A8 | rknn-llm (NPU)NVMe data disk (/mnt/nvme) | 14.4prompt 284.0 t/s | ₹84,864 Buy0.17 t/s per ₹1k | Community jishuzhan.net author2026-09-14 |
| Orange Pi 5 Plus (RK3588)16 GB RAM | Qwen2.5 1.5B1.5B params · Q8_0 | llama.cpp NPU forkNVMe data disk (/mnt/nvme) | 6.3prompt 408.0 t/s | ₹84,864 Buy0.07 t/s per ₹1k | Community jishuzhan.net author2026-09-14 |
| Orange Pi 5 Plus (RK3588)16 GB RAM | Qwen2.5 1.5B1.5B params · Q8_0 | llama.cpp CPUNVMe data disk (/mnt/nvme) | 6.1prompt 92.0 t/s | ₹84,864 Buy0.07 t/s per ₹1k | Community jishuzhan.net author2026-09-14 |
| Orange Pi 5 Plus (RK3588)16 GB RAM | Qwen2.5 3B3B params · Q4_0 | llama.cpp CPUNVMe data disk (/mnt/nvme) | 4.7prompt 25.0 t/s | ₹84,864 Buy0.06 t/s per ₹1k | Community jishuzhan.net author2026-09-14 |
| Orange Pi 5 Plus (RK3588)16 GB RAM | Qwen2.5 3B3B params · Q4_0 | llama.cpp NPU forkNVMe data disk (/mnt/nvme) | 4.2prompt 230.0 t/s | ₹84,864 Buy0.05 t/s per ₹1k | Community jishuzhan.net author2026-09-14 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).