LLMs on the Raspberry Pi CM4

Good for

• Small 1–3B (chat, simple agents): slow, median 2.1 t/s (≈2 words/s), best 5.1 t/s, 3 tests
• Medium 7–8B: too slow to chat, median 0.8 t/s (≈1 words/s), best 0.9 t/s, 2 tests
• Not tested yet: Big 14B+. Help fill the gap

Cheapest India price seen: ₹6,185 (Hubtronics, check stock on retailer page, checked 2026-10-17). Labels: ≥15 t/s fast · 5–15 usable · 2–5 slow · <2 too slow to chat. 1 token ≈ 0.75 words.

Real-world uses

Our data for this board: small (1–3B) models: median 2.1 t/s across 3 tests (slow); medium (7–8B) models: median 0.8 t/s across 2 tests (too slow to chat). Use cases that need chat-speed replies want ≥5 t/s; background jobs can live with 2 t/s.

• Embedded product prototypes: Compute-module form factor for custom carrier boards with a tiny model.
• Industrial command parsing: Sub-3B models for simple text tasks on custom hardware.

India price comparison

Prices as seen on retailer pages (via search snapshots) on the date shown. Not live; confirm on the retailer page. Buy links are plain links for now: affiliate tags will be added later where marked.