Cerebras

Cerebras Inference serves an OpenAI-compatible /v1/chat/completions endpoint with extremely high token throughput on their wafer-scale hardware. chat-cerebras is a thin layer over Completions that presets the base URL and reads the CEREBRAS_API_KEY env var for you.

Install

chat-rs = { version = "0.5.1", features = ["cerebras"] }

CerebrasBuilder::new() reads CEREBRAS_API_KEY at build time and defaults to the base URL https://api.cerebras.ai/v1.

Minimal usage

use chat_rs::{ChatBuilder, ChatOutcome, cerebras::CerebrasBuilder, parts, types::messages};

// CEREBRAS_API_KEY is read automatically.
let client = CerebrasBuilder::new().with_model("llama-3.3-70b").build();
let mut chat = ChatBuilder::new().with_model(client).build();

let mut messages = messages::from_user(parts!["Say hello in one sentence."]);
if let ChatOutcome::Complete(res) = chat.complete(&mut messages).await.map_err(|e| e.err)? {
    println!("{:#?}", res.content);
}

Configuration

with_model(impl Into<String>) selects the model. A few more helpers:

• with_api_key(impl Into<String>) overrides the env var when you want to pass the key explicitly.
• with_base_url(impl Into<String>) points at a different host.
• with_header(key, value) adds a custom header to every request.

Capabilities

Riding on the Completions wire, Cerebras supports completion, streaming, and tools. Streaming needs the stream feature. There are no embeddings on this surface.