Why the failover-routing angle

Free quotas on aggregator gateways and middleman sites share three traits: small grants, strict rate limits, short validity. Leaning on one site guarantees outages. But if you put 3–5 sites' free keys behind a single routing layer ordered primary → secondary → fallback, any rate-limit or exhausted grant gets absorbed automatically, and day-to-day calls rarely notice a break.

This is not a 'register more accounts' volume play. It is availability engineering: treat free quotas as unreliable upstreams and let the routing layer absorb the jitter.

Preconditions

  • A self-hostable unified gateway (open-source gateways such as One API / New API, or LiteLLM Proxy) with multi-upstream, weights, retries and failover.
  • Free keys from two or more aggregator/middleman sites (signup trial credit, typically small; check each site's announcement for exact amounts).
  • A machine that stays online (a NAS at home or a minimal cloud instance both work).

Steps

1. Tag each upstream with capabilities

Create a separate channel per aggregator and note: which models it covers (does it include the chat/embedding models you actually use), the nature of the free grant (one-off trial credit / daily refresh / limited-time campaign), and the rate-limit tier (rough RPM/TPM ranges, estimate with 'about').

This determines routing order: put the bigger, looser quotas on the primary path; smaller, tighter ones on the backup path.

2. Configure three routing tiers

  1. Primary: the aggregator with the largest free grant and broadest model coverage, highest weight.
  2. Secondary: medium grant but stable, takes over on 429/5xx from primary.
  3. Fallback: smallest grant but rarely rate-limited, or your own local Ollama small model, guaranteeing 'always an answer'.

Key settings: enable retry-on-failure plus automatic channel disabling; set 2–3 retries; temporarily disable a failing channel for a few minutes before probing again.

3. Map routes per model

Don't point every model at the same channel. Instead: high-frequency lightweight models (small chat models) go to the site whose free grant lasts longest; large models are called only when the primary is available, and degrade to a small model rather than burning backup quota.

4. Add a quota sentinel

Write a cron script that every 10–30 minutes queries each site's balance/usage endpoint (most aggregator consoles expose one) and writes remaining quota into gateway channel weights: below threshold, automatically down-weight or disable. No manual dashboard-watching.

5. Local fallback

When primary and secondary are both down, route to local Ollama (small-parameter qwen or llama). Quality drops, but the workflow does not stop.

Caveats

  • Compliance: use only publicly offered free grants and your own keys. Do not use shared/resold keys or bypass site rate limits. Most aggregators' terms prohibit automated bulk registration and quota arbitrage; violations get accounts banned.
  • Validity differences: one-off trial credit usually has an expiry date; daily-refresh quotas reset by day. Adjust routing weights dynamically and burn near-expiry channels first.
  • Model naming drift: sites name the same model differently; alias-map in the gateway or failover will fail with 'model not found'.
  • Logging and cost visibility: enable request logs and reconcile actual consumption per channel regularly so no channel is silently blown through.
  • Don't expect long-term stability: free quotas are marketing spend and rules change without notice. Design the routing layer to swap upstreams quickly rather than deep-bind to one vendor.

When to use it

  • Individual developers prototyping, running agent workflows or batch text processing with zero budget but a need for 'mostly uninterrupted' service.
  • Small-team internal tools that must stay available without buying commercial quota.
  • A bridge before paid quota goes live: run the business through the routing layer first, then replace with stable upstreams.

One-line summary

Free-token reliability doesn't come from 'who gives the most' — it comes from your ability to switch away seamlessly when they cut you off. Failover routing is the engineering form of that ability.