chatkit

Thin async Rust wrapper around any OpenAI-compatible chat API with streaming, tool calling, and structured output support.

Features

• Non-streaming and streaming responses
• Structured output via tool calling (llm_single_tool_call)
• Automatic <think>…</think> extraction for reasoning models
• Manual or derive-based tool schemas
• Optional metrics feature (histograms for TTFT / TTLT)

Quick start

[dependencies]
chatkit = "0.1.0"
tokio = { version = "1", features = ["macros", "rt-current-thread"] }

Simple chat

use chatkit::{llm_call, messages::{ChatKitMessage, Memory, SystemMessage, UserMessage}, types::{CallConfig, CallOptions}};

let config = CallConfig::builder().api_key(api_key.into()).build();
let options = CallOptions::builder().model("gpt-4o-mini".into()).build();
let memory = Memory(vec![
    ChatKitMessage::User(UserMessage { name: None, content: "Hello!".into() }),
]);

let msg = llm_call(config, options, memory, vec![], None)
    .await?
    .expect_message()?;
println!("{}", msg.text.unwrap().text.unwrap_or_default());

Structured output

use chatkit::llm_single_tool_call;
use schemars::JsonSchema;
use serde::Deserialize;

#[derive(Debug, JsonSchema, Deserialize)]
#[schemars(description = "Sentiment analysis result")]
struct Sentiment {
    /// positive, negative, or neutral
    sentiment: String,
    /// 0.0–1.0
    confidence: f32,
}

let (result, _tokens) = llm_single_tool_call::<Sentiment>(config, options, memory).await?;
println!("{} ({:.0}%)", result.sentiment, result.confidence * 100.0);

Core types

Examples

OPENAI_API_KEY=sk-... cargo run --example simple_chat
OPENAI_API_KEY=sk-... cargo run --example structured_output
OPENAI_API_KEY=sk-... cargo run --example streaming

Features

chatkit = { version = "0.1.0", features = ["metrics"] }

Enables metrics histograms: llm_time_to_first_token_ms and llm_time_to_last_token_ms, tagged with service and model.