healthy + SWE ≥ 71% (or unscored) · auto-switch every 15 min · primary stays FCM

Always a working model
above SWE 71%

A Hermes plugin that scores NVIDIA NIM health, FCM router health, provider diversity, and a persistent learning DB — then atomically updates fallback and MoA models. The primary model is never touched.

Get started See the architecture

Primary (never switched)

fcm:fast-coding

FCM router · localhost:19280

Fallback (NIM)

deepseek-v4-flash

Example · SWE 71.5% · 2nd-best eligible

MoA Aggregator

nemotron-3-ultra

Example · SWE 72.5% · top eligible

Learning DB

Active

SWE, streaks, uptime, outcomes

For newcomers

What this repo does

Four real inputs. One hard filter. A 15-minute cron that writes only the fallback and MoA slots.

1. FCM router daemon

Local OpenAI-compatible endpoint on port 19280. Cross-provider failover (NIM, Groq, Cerebras, GitHub Models, OpenRouter, and more) with circuit breakers.

2. NVIDIA NIM health

Every cycle, real chat completions against the FCM router's active NVIDIA models (falls back to a fixed ~9-model list if the router is unreachable). Ranked by latency and success rate.

3. Learning DB (SQLite)

Persistent SWE scores, streaks, uptime hours, and selection outcomes at ~/.hermes/data/model_learning.db.

4. SWE ≥ 71% gate

A model is ineligible unless it's healthy. If the learning DB has a real SWE-bench-style score for it, that score must also be at least 71% — but a model the curated table hasn't caught up on yet is judged on health alone, not silently rejected.

Scoring — all four terms are used

Eligible models are ranked by a weighted sum. Learning is a real 10% term (historical SWE from the DB), not decoration.

40%

NIM health

Success rate + latency

40%

FCM health

Router live score

10%

Diversity

Non-NVIDIA providers

10%

Learning

DB SWE score (0–100)

combined = NIM(40%) + FCM(40%) + Diversity(10%) + Learning(10%)
filter   = healthy AND (swe_score >= 71 OR swe_score is unknown)
write    = fallback + MoA aggregator + MoA ref   # never model.default

System architecture

Four layers, one write path

Stacked flow — no overlapping boxes. Read top to bottom on any screen.

Hermes Agent

Reads the Hermes config resolved via hermes config path

model.default = fcm:fast-coding

http://localhost:19280/v1

FCM Router Daemon

localhost:19280 · OpenAI-compatible · per-request failover

NVIDIA NIM

Groq

Cerebras

GitHub Models

OpenRouter

Mistral

Cloudflare

SambaNova

+ more

Hybrid Auto-Switch Cron

Every 15 minutes via Hermes Gateway

NIM

40%

FCM

40%

Diversity

10%

Learning

10%

Combined = NIM(40%) + FCM(40%) + Diversity(10%) + Learning(10%)

Filter: healthy, and SWE ≥ 71% or unscored. Writes fallback + MoA only.

Learning DB

SWE scores (seeded + measured)
Success rate, latency, streaks
Uptime hours per model
Selection decisions + outcomes
~/.hermes/data/model_learning.db

config.yaml

model.default: fcm:fast-coding
fallback_model: <best eligible>
moa.aggregator: <top eligible>
moa.reference_models[0]: <2nd>
Atomic write · last 10 backups kept

Installation

Get running in six steps

Not on PyPI. Clone the repo and install editable. Cron job IDs are generated per install.

Prerequisites

Required

Hermes Agent installed
NVIDIA NIM API key build.nvidia.com
Node.js 18+ (for FCM)
Python 3.10+

Optional (cross-provider)

Groq API key
Cerebras API key
GitHub PAT
OpenRouter key

# Verify prerequisites
hermes --version
node --version
python3 --version

Clone and install this package

This project is not published on PyPI. Install from a local clone:

# Clone
git clone https://github.com/rhishi99/hermes-model-auto-switch
cd hermes-model-auto-switch

# Editable install (pulls pyyaml + requests)
pip install -e .

# Optional: tests and linters
pip install -e ".[dev]"

Install the FCM router daemon

# Global CLI — serves http://localhost:19280
npm install -g free-coding-models
free-coding-models --version

This is the OpenAI-compatible router Hermes will use as fcm:fast-coding.

Configure environment

Copy the example config and add your API keys:

# Copy to Hermes config directory
cp config/.env.example ~/.hermes/.env
$EDITOR ~/.hermes/.env

NVIDIA_API_KEY=your_nvidia_nim_key_here  # REQUIRED
GROQ_API_KEY=your_groq_key              # Optional
CEREBRAS_API_KEY=your_cerebras_key          # Optional
GITHUB_TOKEN=your_github_pat               # Optional
OPENROUTER_API_KEY=your_openrouter_key         # Optional

# Switcher tuning
SWE_THRESHOLD=71
NIM_AUTO_SWITCH_ENABLED=true

Point Hermes at the FCM router

Primary stays on the local router for the life of the install:

# Set Hermes to use FCM router
hermes config set model.default fcm:fast-coding
hermes config set model.provider fcm
hermes config set model.base_url http://localhost:19280/v1

# Verify
hermes config show

Start / stop FCM with Hermes

Hook the daemon so it comes up when Hermes does:

# Startup (when Hermes opens)
powershell -ExecutionPolicy Bypass -File hooks/start_fcm_daemon.ps1 -WaitForReady

# Shutdown (when Hermes closes)
powershell -File hooks/stop_fcm_daemon.ps1

Batch equivalents: hooks/start_fcm_daemon.bat / hooks/stop_fcm_daemon.bat

★

Optional: enable the 15-minute cron

Hermes Gateway runs the jobs. IDs are generated per install — they are not universal hex strings.

# One-time gateway install
hermes gateway install

# Discover YOUR job IDs (they differ on every machine)
hermes cron list

# Resume hybrid auto-switch (every 15 min)
hermes cron resume <hybrid-switch-job-id>

# Resume learning-DB maintenance (daily 04:00)
hermes cron resume <db-maintenance-job-id>

After hermes gateway install, run hermes cron list and copy the IDs Hermes printed. Do not reuse IDs from this page or from someone else’s machine.

Configuration

Environment variables

All settings live in ~/.hermes/.env — no secrets in the repo.

Required

NVIDIA_API_KEY

NVIDIA NIM key from build.nvidia.com. Needed for health checks and fallback/MoA.

Cross-provider (optional)

GROQ_API_KEY — GroqCloud

CEREBRAS_API_KEY — Cerebras

GITHUB_TOKEN — GitHub Models

OPENROUTER_API_KEY — OpenRouter

MISTRAL_API_KEY — Mistral

CLOUDFLARE_API_KEY — Cloudflare Workers AI

SAMBANOVA_API_KEY — SambaNova

FCM auto-discovers models from whichever keys you set.

Tuning

SWE_THRESHOLD=71

Min SWE % for eligibility (default 71)

NIM_AUTO_SWITCH_ENABLED=true

Must be true for the switcher to write config (default false)

NIM_MODELS_TO_MONITOR

Comma-separated NIM models to probe. Overrides the FCM active set; falls back to a fixed ~9-model list if unset and the router is unreachable.

FCM_AUTO_RESYNC=true

Auto-resync the FCM set and restart the daemon when a probe finds a retired model (default true)

NIM_HEALTH_TIMEOUT=10

Seconds per request

NIM_MAX_CONCURRENT=4

Parallel health checks

NIM_MAX_LATENCY_MS=15000

Max average latency to count as healthy

NIM_MAX_FAILURE_RATE=0.3

Max 30% failure rate

NIM_MIN_SUCCESS_COUNT=2

Min successes out of 3 probes

Live monitoring

Watch it work

Health, a live switch, and the learning DB — plus how to find your cron IDs.

FCM router health

# Daemon status
curl -s http://localhost:19280/health | jq

# Active set: real model ids, provider, priority
curl -s http://localhost:19280/sets | jq

# Daemon log
tail -f ~/.fcm-daemon/daemon.log

Not /v1/models — that only lists the router's own aliases (fcm, fcm:fast-coding) with no provider or priority to score against.

{"ok":true,"running":true,"activeModelCount":5}

{"activeSet":"fast-coding","sets":{"fast-coding":{"models":[{"model":"nemotron-3-ultra-550b-a55b","provider":"nvidia","priority":1}]}}}

Probing NVIDIA NIM models...

nemotron-3-ultra healthy (661ms)

NIM health check (manual)

# Probe the monitored NIM models
NVIDIA_API_KEY=your_key python3 scripts/nvidia_nim_health_check.py

# Preview a config write (this helper HAS --dry-run)
python3 scripts/nvidia_nim_config_update.py \
  nvidia/nemotron-3-ultra-550b-a55b \
  --fallback --moa-aggregator --moa-refs --dry-run

Starting health check for 9 models...

nemotron-3-ultra: HEALTHY (3/3, avg: 661ms)

deepseek-v4-flash: HEALTHY (3/3, avg: 740ms)

qwen3.5-397b: UNHEALTHY (6–8s latency)

Healthy: 2/9 | top eligible: nemotron-3-ultra

Hybrid auto-switch (writes config)

Running python scripts/hybrid_auto_switch.py with NIM_AUTO_SWITCH_ENABLED=true writes your Hermes config.yaml (resolved via hermes config path, not assumed to be ~/.hermes/) if a better eligible model is found. Pass --dry-run to run the same health checks and scoring without writing config or recording a switch.

# Live run — will write fallback / MoA if a better model wins
NVIDIA_API_KEY=your_key NIM_AUTO_SWITCH_ENABLED=true \
  python3 scripts/hybrid_auto_switch.py

# Rehearsal — health checks + scoring, no config write
NVIDIA_API_KEY=your_key NIM_AUTO_SWITCH_ENABLED=true \
  python3 scripts/hybrid_auto_switch.py --dry-run

Database initialized

Running NVIDIA NIM health check...

FCM Router: 5 models, set=fast-coding

Scored 11 models, 2 eligible (SWE>71% & healthy)

Top: nemotron-3-ultra score uses NIM+FCM+Div+Learn

Primary stays fcm:fast-coding

Updating fallback/MoA (atomic, backup kept)

Learning database

# Learning report
python3 -c "
from scripts.model_learning_db import export_learning_report
import json
print(json.dumps(export_learning_report(), indent=2))
"

# Best available model above the SWE gate
python3 -c "
from scripts.model_learning_db import init_db, get_best_available_model
init_db()
best = get_best_available_model()
print(best)
"

{

"models": [

{"model": "nemotron-3-ultra-550b-a55b", "swe_score": 72.5, "current_streak": 12},

{"model": "deepseek-v4-flash", "swe_score": 71.5, "current_streak": 8}

"providers": [{"provider": "nvidia", "reliable_models": 2, "best_swe": 72.5}]

}

Retired & throttled models

Every probe is classified, not just pass/fail: retired (410/404 — gone upstream), throttled (429 after backoff retries — your rate limit, excluded from the failure rate), healthy, or unhealthy. A retired model triggers a one-time resync and re-probe.

# What the cron does automatically on a retired model
free-coding-models --sync-set <active-set>
free-coding-models --daemon-stop
free-coding-models --daemon-bg
# then re-probes the refreshed set once

Retired upstream (410/404), needs replacing: old-model-v1

Retired models found -- resyncing FCM set via: free-coding-models --sync-set fast-coding

FCM set resynced, restarting daemon to pick it up

FCM daemon back up with the new set

Cron jobs (Hermes Gateway)

Job IDs are minted by Hermes per install. After hermes gateway install, list them and substitute the placeholders.

# List jobs — copy the IDs from this output
hermes cron list

# Status / run / logs for YOUR hybrid-switch job
hermes cron status <hybrid-switch-job-id>
hermes cron run    <hybrid-switch-job-id>
hermes cron log    <hybrid-switch-job-id>

Scheduled Jobs

<hybrid-switch-job-id> [active]

Name: Hybrid Auto Model Switcher (FCM + NIM + Learning)

Schedule: every 15m

Last run: ok

<db-maintenance-job-id> [active]

Name: Learning DB Maintenance (cleanup + report)

Schedule: 0 4 * * *