Hermes HQ Subscribe
Setup

Run Hermes Agent locally on your GPU at 93 tokens per second

What it does

Set up Hermes Agent to run locally on an RTX 5090 using Qwen3.8-27B 6-bit quantization with llama.cpp, MTP draft mode, and q8_0 cache. This configuration delivers about 93 tokens per second sustained throughput over hour-long agent runs, with consistent performance of 90-97 tok/s across multiple sessions.

The setup is suitable for complex creative tasks that require long-running inference without cloud API rate limits, as demonstrated by generating a voxel diorama of Mont-Saint-Michel in a single HTML file over one hour.

Why it's worth it

Local inference keeps your data private, removes API rate limits and costs, and provides predictable performance for long-running agent tasks. This setup is especially valuable for iterative creative and analytical work where you control the entire pipeline and can run multi-hour generations without worrying about service availability or pricing changes.

Illustration: Run Hermes Agent locally on your GPU at 93 tokens per second

Step by step

Paste it into Hermes Agent and swap the highlighted parts for your own details.

Prompt

Before you start Have RTX 5090 with 32GB VRAM, configure llama.cpp with Qwen3.8-27B 6-bit model, MTP draft, q8_0 cache, 128k context, and set Hermes Agent to use this local model.

Get Hermes Agent to:

  1. Build a voxel diorama of [specific location] with [visual style] in a single HTML file.
  2. Build another complex HTML project like [project type] with [specific visual details] to verify consistent 90-97 tokens per second performance.

Expected resultRun Hermes Agent locally with consistent 93 tokens per second throughput, suitable for generating hour-long complex projects like interactive 3D scenes in code.

Related use cases

See all
Prompt