Why this toolchain

Most "free token" schemes hit one of two walls: they require a credit card, or they require a local CUDA setup. Hugging Face, Google Colab and Kaggle sidestep both — sign-up is email-only, the runtime lives in the browser, and the output is an HTTP-callable endpoint. The trade-off: quotas are per-session or per-hour, instances sleep, and concurrency is limited. So the right mindset is not "unlimited free" but "wire scattered free compute into one callable pipeline."

Steps

Step 1: Spin up a Gradio inference app on HF Spaces

  1. Register a Hugging Face account, go to Spaces, create a Space with the Gradio SDK.
  2. Pick CPU Basic hardware (the free tier); in app.py, load a small-to-mid open model (roughly 1B–3B instruct class) with transformers, exposed as gr.ChatInterface or a custom gr.Interface.
  3. Key settings: enable sleep in the Space settings to conserve free hours; wrap the handler in queue() and keep concurrency_count at 1–2 so the free instance is not overwhelmed.
  4. After deploy you get a public https://<user>-<space>.hf.space URL — that is your token endpoint.

> Conditions: the free CPU tier suits small models and low-frequency calls. Once you need 7B+, free CPU becomes impractically slow — switch to the Colab route below.

Step 2: Use Colab's free GPU runtime for larger models

  1. Create a Colab notebook and select a GPU runtime (the free tier is typically T4-class; check what you are assigned).
  2. Install deps and load the model: pip install transformers accelerate fastapi uvicorn pyngrok, load a 7B-class model with 4-bit quantization to keep VRAM in check.
  3. Wrap it in a FastAPI app exposing an OpenAI-style /v1/chat/completions, then use pyngrok to map the local port to a public URL.
  4. Put the ngrok URL in your local .env; your scripts can now call it like any API.

> Conditions: free Colab runtimes disconnect after idle periods and have a daily usable-hour cap (typically a few hours; check the official docs). Good for on-demand starts, not 24/7 hosting.

Step 3: Keep Kaggle Notebooks as backup compute

Kaggle Notebooks also offer free GPU (quota counted weekly, usually in the tens of hours per week). Reuse the same FastAPI + ngrok script as a fallback when Colab drops — only the model path changes.

Step 4: Add a local endpoint router

Write a minimal Python client that keeps a list of endpoints (HF Space, current Colab URL, current Kaggle URL), probes /health in order, and sends to the first live one; on failure, fall through to the next. This keeps calls alive when an instance sleeps or disconnects.

```python

import requests

ENDPOINTS = [

"https://<user>-<space>.hf.space",

"https://<colab-ngrok-url>",

"https://<kaggle-ngrok-url>",

]

def pick():

for url in ENDPOINTS:

try:

if requests.get(url + "/health", timeout=3).ok:

return url

except Exception:

continue

raise RuntimeError("no endpoint alive")

```

Step 5: Script the refresh

A sleeping HF Space cold-starts on first hit (often tens of seconds); a dropped Colab notebook must be re-run. Schedule a "wake + health check" job (local cron or GitHub Actions) shortly before you need the endpoint each day to cut the wait.

Caveats

  • Read the quota rules: HF Spaces free tier limits runtime by hardware and has sleep policies; free Colab GPU has per-day and per-session caps; Kaggle counts weekly. These numbers change — check the live pages and never plan capacity against a fixed figure.
  • Not for production: free instances can be reclaimed, throttled or slept at any time. Use for personal experiments, prototypes and low-frequency calls.
  • Acceptable use: follow each platform's policy; don't use free instances for mass scraping, bulk generation or resale.
  • Security: a public ngrok URL is a public endpoint — add a simple token check, and never share notebooks containing secrets.
  • Model licenses: confirm the open model's license permits your use before loading it.

When to use it

  • Solo developers without a credit card or a CUDA setup who need a callable inference endpoint fast.
  • Prototyping prompts, small-scale evals, short-text batch jobs.
  • As a fallback layer when a paid API is rate-limited.

How it fits with other approaches

This line answers "do I have a callable endpoint", not "do I have high throughput". For steady high concurrency, look at orchestrating official free-tier rate limits or a resident local Ollama + vLLM deployment. HF + Colab is the zero-barrier start and the emergency backup.