Why the routing layer matters more than the quota number

Most aggregator free tiers are not a single balance dropped into your account. They are split across models, regions, and rate-limit groups. The quota figure in your dashboard often maps to several upstream channels at once, so the number can still look healthy while one channel is already exhausted.

The right order is therefore: map the routes first, then decide how to spend.

Step-by-step

Step 1: Copy down the model mapping table

On the gateway's model listing page (usually /v1/models or a Model tab in the console), export every model ID. Pay attention to:

  • Whether the same upstream model appears under several aliases (e.g. gpt-4o / gpt-4o-2024-xx / variants with a -free or :free suffix).
  • Which models are marked free / trial / preview.
  • The context limit of each model and whether streaming is supported.

Save this table as a local JSON file. Every script afterwards should read from it, so you never mistype a model name and burn retries on 404s.

Step 2: Probe each channel's rate-limit group with a minimal request

Send one very short request per candidate model (max_tokens set to 5–16) and record the x-ratelimit-* response headers plus the model field in the body. Many gateways echo the actual upstream they routed to — that tells you which provider "free model A" really hit.

Note that different rate-limit groups usually do not affect each other: when one group is saturated, switching to another group often still works.

Step 3: Choose models by capability tier, not by price

Split your models into three tiers:

  1. Cheap tier: small models or free-suffixed variants, used for classification, extraction, formatting, rewriting — high-frequency, low-difficulty work.
  2. Main tier: mid-capability models for the bulk of generation, only escalated to when the cheap tier fails.
  3. Fallback tier: flagship models, used only for final validation or a handful of hard samples.

Feed free quota to the cheap tier first and put a hard daily cap on the fallback tier. This single step is the most effective way to actually use up the free credit.

Step 4: Configure a failure-degradation chain

Set a fallback order on the client or gateway side: cheap-tier model → another alias in the same tier → main tier. Trigger it on 429, 5xx, and timeouts.

If you run a self-hosted gateway such as LiteLLM or One API, you can declare a fallbacks list in the config. With a hosted gateway, wrap your own retry function and switch models based on the status code.

Step 5: Treat regional endpoints as separate quota sources

Some gateways expose multiple regional entry points (global, Asia, Europe), and free quota plus rate-limit groups may be counted separately per region. During the probe phase, send one test request to each regional endpoint and record the rate-limit headers; then have your client rotate across regions instead of using a single base URL.

Whether this pays off depends on whether the gateway really meters quota per region — the probe results will tell you directly. Do not configure this on a guess.

Step 6: Add a per-channel quota dashboard

Write a script that runs once a day: send one probe request per model alias, record success/failure and rate-limit headers, and output a table. After a week of records you can see which channels saturate at which times, and which aliases are actually duplicate entrances to the same channel.

Caveats

  • Free channels are often capability-degraded: free-suffixed models may have reduced context windows or disabled parameters. Verify parameter compatibility with a small request first.
  • Do not run long context on free channels: one oversized request can consume a full day's free quota, and failed retries are billed again.
  • Follow the platform terms: do not farm quota with multiple accounts or forge headers to bypass rate limits. Such behaviour is typically banned and violates the terms of service.
  • A quota number is not availability: if the dashboard shows balance but requests return 429, the rate-limit group is the constraint, not exhausted credit.
  • Model aliases change: free model lists on gateways update frequently. Read them dynamically from the model listing endpoint; never hardcode.

When this fits

  • Individual developers validating a prototype end-to-end without paying.
  • Small teams building internal tools whose load is mostly classification, extraction, and summarization.
  • Running A/B comparisons across multiple upstreams on a tight budget.
  • As a safety net behind a paid channel: degrade to free channels automatically when the primary fails.

When it does not fit

  • Production services that need a stable SLA.
  • Batch jobs requiring long context (a whole book, a full contract).
  • Cases demanding highly consistent output — the upstream behind a free channel may switch between requests.