2 releases (1 stable)

Uses new Rust 2024

2026.7.0 Jul 7, 2026
0.1.3 Jun 16, 2026

#598 in Artificial intelligence

Download history 14/week @ 2026-06-16 75/week @ 2026-06-30 85/week @ 2026-07-07

174 downloads per month

Apache-2.0 OR MIT

185KB
3K SLoC

A concrete LlmProvider backed by the GCP AI platform REST API, authenticated via Workload Identity Federation / Application Default Credentials (handled by gcp_auth: in-cluster metadata, WIF, or a local gcloud login, transparently).

Uses the SSE-streaming streamGenerateContent endpoint (?alt=sse) and adapts each partial response into the streaming Chunk vocabulary the trait expects (text → Chunk::TextDelta per token batch, function calls → tool-call chunks, usage + finish reason → Chunk::Usage/Chunk::Stop). Chunks are yielded as bytes arrive — the harness pushes them down to the client without buffering the whole response, so user-visible latency starts at first-token time. The trait boundary keeps it a single-file change.

Dependencies

~22–40MB
~598K SLoC