Apple Foundation Models

chat-applefm talks to the ~3B-parameter model that ships with Apple Intelligence (macOS 26+) through Apple's FoundationModels framework. There is no HTTP and no weights file: the OS owns the model, and this provider owns the translation. Rust can't call Swift directly, so the crate embeds a small Swift package that exposes a few plain C functions; requests and responses cross that boundary as JSON — the same mental model as an HTTP provider's wire format, minus the network. build.rs compiles the bridge automatically, so you only ever run cargo build.

On non-macOS targets the crate still compiles (with a stub bridge) and reports the model as unavailable.

Install

chat-rs = { version = "0.5.3", features = ["applefm"] }

Binary crates on macOS need one build.rs line so the Swift concurrency runtime resolves at load time:

fn main() {
    println!("cargo:rustc-link-arg=-Wl,-rpath,/usr/lib/swift");
}

Check availability first

The model isn't usable everywhere: it needs eligible hardware, Apple Intelligence enabled, and the assets downloaded. availability() is a cheap probe that performs no generation — call it before constructing a client, and let router strategies consult it to decide whether this provider is eligible at all.

use chat_rs::applefm;

let probe = applefm::availability();
if !probe.available {
    println!("unavailable: {}", probe.reason.as_deref().unwrap_or("?"));
    return Ok(());
}

Minimal usage

There's no model slug and no with_model typestate — the model is the one the OS ships. System prompts flow through Messages like any other provider; the provider folds System-role messages into the session's instructions.

use chat_rs::{ChatBuilder, ChatOutcome, applefm::AppleFMBuilder, parts, types::messages};

let client = AppleFMBuilder::new().build()?; // validates config upfront
let mut chat = ChatBuilder::new().with_model(client).build();

let mut messages = messages::from_user(parts!["Say hello in one sentence."]);
if let ChatOutcome::Complete(res) = chat.complete(&mut messages).await.map_err(|e| e.err)? {
    println!("{:#?}", res.content);
}

Configuration

The builder is a pure connection stub: it wires up which model variant to talk to and the decoding defaults, nothing else.

• with_lora(path) applies a LoRA fine-tune over the on-device base model.
• with_temperature(f64) sets the default decoding temperature. Overridden per call by ChatOptions::temperature.
• with_max_tokens(u32) sets the default response-length cap. Overridden per call by ChatOptions::max_tokens.
• with_sampling(Sampling) picks the decoding strategy — the full set FoundationModels exposes:
  • Sampling::Greedy — deterministic.
  • Sampling::TopK { k, seed } — sample among the k most probable tokens.
  • Sampling::TopP { p, seed } — nucleus sampling within cumulative probability p.

build() validates the configuration upfront — a missing .fmadapter path or nonsensical sampling parameters fail there, not on the first request.

Prewarm

The runtime stages the model down between requests, so a turn that follows an idle pause pays seconds of warm-up. When you can predict a turn is coming — on input focus, or when the user starts typing — call client.prewarm() so the warm-up overlaps the wait. It's fire-and-forget: returns immediately, never fails, and is skipped if a turn is already in flight.

LoRA fine-tunes

with_lora loads a .fmadapter package — a LoRA trained against the on-device base model with Apple's adapter training toolkit:

use chat_rs::applefm::AppleFMBuilder;

let client = AppleFMBuilder::new()
    .with_lora("adapters/transcripts.fmadapter")
    .build()?;

Adapters are version-locked to the base model: after a macOS update rolls the model, retrain and reship. Loading an incompatible adapter fails at request time with a provider error.

Capabilities

The on-device model is text-only through this API. Completion and streaming (behind the stream feature) are supported; tools, structured output, embeddings, and non-text parts (images, audio, files) are rejected fast at request time with a clear pointer to what's unsupported, rather than crossing the bridge and failing opaquely.

Token usage is always zero — Apple's FoundationModels API does not expose token counts. Each turn does report wall-clock duration_ms, and a prefill field in provider_specific metadata records whether the turn reused the held session ("incremental") or rebuilt it ("full"). The client keeps a live session across turns, so append-only conversations prefill only the newest message — clones share that session, so use one client per concurrent conversation.