LLMs on the Jetson Orin Nano Super

Good for

• Small 1–3B (chat, simple agents): fast, feels like chat, median 38.1 t/s (≈29 words/s), best 52.7 t/s, 13 tests
• Medium 7–8B: fast, feels like chat, median 19.1 t/s (≈14 words/s), best 21.8 t/s, 3 tests
• Not tested yet: Big 14B+. Help fill the gap

Cheapest India price seen: ₹67,980 (Amazon.in (seller Gamingicons), in stock, checked 2026-10-17). Labels: ≥15 t/s fast · 5–15 usable · 2–5 slow · <2 too slow to chat. 1 token ≈ 0.75 words.

Real-world uses

Our data for this board: small (1–3B) models: median 38.1 t/s across 13 tests (fast, feels like chat); medium (7–8B) models: median 19.1 t/s across 3 tests (fast, feels like chat). Use cases that need chat-speed replies want ≥5 t/s; background jobs can live with 2 t/s.

• Camera + vision-language assistant: CSI camera input with a vision model, then an LLM describes or answers questions about what it sees.
• ROS 2 robot brain: Runs ROS 2 and an LLM side by side to turn spoken/typed goals into robot tasks.
• Offline voice assistant: Speech-to-text → local LLM → text-to-speech with no cloud dependency.
• Factory / farm inspection: Detect defects or crop issues on camera, with an LLM writing the summary report.
• Private document Q&A: RAG over manuals or company docs that never leave the device.
• Agent controlling hardware: 40-pin header GPIO lets an agent switch relays, read sensors, drive motors.

India price comparison

Prices as seen on retailer pages (via search snapshots) on the date shown. Not live; confirm on the retailer page. Buy links are plain links for now: affiliate tags will be added later where marked.