Hugging Face

The Hugging Face Inference Router is OpenAI-compatible (/v1/chat/completions) and dispatches to Together, Novita, Fireworks, SambaNova, Cerebras, Groq, Replicate, and others through a single billing layer. chat-huggingface is a thin layer over Completions that presets the base URL and reads the HF_TOKEN env var for you.

Install

chat-rs = { version = "0.5.1", features = ["huggingface"] }

HuggingFaceBuilder::new() reads HF_TOKEN at build time and defaults to the base URL https://router.huggingface.co/v1. A token is required even on the free tier.

Minimal usage

use chat_rs::{ChatBuilder, ChatOutcome, huggingface::HuggingFaceBuilder, parts, types::messages};

// HF_TOKEN is read automatically.
let client = HuggingFaceBuilder::new()
    .with_model("openai/gpt-oss-120b:fastest")
    .build();
let mut chat = ChatBuilder::new().with_model(client).build();

let mut messages = messages::from_user(parts!["Say hello in one sentence."]);
if let ChatOutcome::Complete(res) = chat.complete(&mut messages).await.map_err(|e| e.err)? {
    println!("{:#?}", res.content);
}

Model slugs

Slugs are org/model plus an optional policy suffix that picks the upstream provider:

For example, deepseek-ai/DeepSeek-R1:cheapest or meta-llama/Llama-3.3-70B-Instruct:novita.

Configuration

with_model(impl Into<String>) takes a slug as above. A few more helpers:

• with_api_key(impl Into<String>) sets the token explicitly (required if HF_TOKEN is not in the environment).
• with_base_url(impl Into<String>) points at a staging environment or a self-hosted HF-compatible proxy.
• with_header(key, value) adds a custom header to every request.

Capabilities

Riding on the Completions wire, Hugging Face supports completion, streaming, and tools. Streaming needs the stream feature. There are no embeddings on this surface.