LLMs on the Jetson AGX Orin 64GB
1 sourced results. 1 configurations reached 5+ tokens/s generation.
Good for
- Not tested yet: Small 1–3B, Medium 7–8B, Big 14B+. Help fill the gap
No Indian price found yet. Labels: ≥15 t/s fast · 5–15 usable · 2–5 slow · <2 too slow to chat. 1 token ≈ 0.75 words.
Real-world uses
No size-class data yet for this board. Use cases that need chat-speed replies want ≥5 t/s; background jobs can live with 2 t/s.
- Camera + vision-language assistant: CSI camera input with a vision model, then an LLM describes or answers questions about what it sees.
- ROS 2 robot brain: Runs ROS 2 and an LLM side by side to turn spoken/typed goals into robot tasks.
- Offline voice assistant: Speech-to-text → local LLM → text-to-speech with no cloud dependency.
- Factory / farm inspection: Detect defects or crop issues on camera, with an LLM writing the summary report.
- Private document Q&A: RAG over manuals or company docs that never leave the device.
- Agent controlling hardware: 40-pin header GPIO lets an agent switch relays, read sensors, drive motors.
SPONSORED
Your board here
Reach makers and engineers choosing edge-AI hardware. Sponsor this slot →
India prices
No Indian retailer listing found yet. Seen one? Tell us.
| Board ↕ | Model ↕ | Runtime & setup ↕ | Gen t/s ↕ | Price ₹ (India) ↕ | Source ↕ |
|---|---|---|---|---|---|
| Jetson AGX Orin 64GB64 GB RAM | Qwen3.8-Flash-Next Gyro-MTQ2_0 (2-bit trellis experts) | Strata Orin fork bb98d09 (CUDA SM87, MTP speculative decoding, MAXN)NVMe | 25.8prompt 555.0 t/s | No India listing yet | Community kim-haneol (Strata-Orin)2026-10-08 |
Each row shows only what its source reported: RAM, quant, prompt speed and setup appear under the main value when published. Rows with full details are listed first, fastest first. Source badges: Community, Vendor, Reproduced; ⚠ marks a speed far from similar results (how we check).