2 releases (1 stable)
Uses new Rust 2024
| 2026.7.0 | Jul 7, 2026 |
|---|---|
| 0.1.3 | Jun 16, 2026 |
#598 in Artificial intelligence
174 downloads per month
185KB
3K
SLoC
A concrete LlmProvider backed by the GCP AI platform REST API,
authenticated via Workload Identity Federation / Application Default
Credentials (handled by gcp_auth: in-cluster metadata, WIF, or a local
gcloud login, transparently).
Uses the SSE-streaming streamGenerateContent endpoint
(?alt=sse) and adapts each partial response into the streaming
Chunk vocabulary the trait expects (text → Chunk::TextDelta per
token batch, function calls → tool-call chunks, usage + finish reason
→ Chunk::Usage/Chunk::Stop). Chunks are yielded as bytes arrive
— the harness pushes them down to the client without buffering the
whole response, so user-visible latency starts at first-token time.
The trait boundary keeps it a
single-file change.
Dependencies
~22–40MB
~598K SLoC