Lib.rs

› Keywords #llm #llm-inference #inference-engine #gguf #openai #llama-cpp #metal

#llama


  1. llama-cpp-4

    llama.cpp bindings for Rust

    v0.4.0 750 #gguf #llama #inference #ggml #llm

  2. gguf-rs

    GGUF file parser

    v0.1.8 650 #llama #parser #llm

  3. TinyHarness

    tinyharness - ai coding harness

    v0.3.2 #tinyharness #artificial-intelligence #agent-skills #tui #llm-provider #system-prompt #ollama-provider #llama #llama-cpp #sockudo #agent-mode

  4. oxibonsai-cli

    Pure Rust 1-bit LLM inference engine for PrismML Bonsai models — CLI

    v0.2.2 #gguf #inference-engine #llm-inference #bonsai #ternary #metal #1-bit #apple-silicon #model-path #llama

  5. kalosm-sample

    A common interface for token sampling and helpers for structered llm sampling

    v0.4.1 1.8K #artificial-intelligence #llama #llm #mistral #nlp

  6. llama-herd

    A native Rust TUI and CLI companion for orchestrating, pairing, and routing local LLM services driven by llama.cpp's llama-server

    v1.0.16 #llama #llama-server #orchestration #cli

  7. llama-gguf

    A high-performance Rust implementation of llama.cpp - LLM inference engine with full GGUF support

    v0.14.0 170 #gguf #llama #llm-inference #llm

  8. llama-crab

    Safe, ergonomic and complete Rust bindings for llama.cpp

    v0.1.300 #gguf #llama #inference #llm-inference #llm

  9. wgpu-llm-cli

    Terminal-based chat interface for the wgpu LLM inference engine

    v0.1.1 #llm-inference #inference-engine #wgpu #compute-shader #wgsl-shader #gpu #llama #chat-interface #from-scratch

  10. shimmytok

    Pure Rust tokenizer for GGUF models with llama.cpp compatibility (SentencePiece + BPE + WPM + UGM + RWKV)

    v0.7.1 1.5K #gguf #llama #llm #sentence-piece

  11. trustformers-models

    Model implementations for TrustformeRS

    v0.2.0 #transformer-models #llama #bert

  12. kalosm-language-model

    A common interface for language models/transformers

    v0.4.1 1.8K #artificial-intelligence #mistral #nlp #llama #llm

  13. kalosm-language

    A set of pretrained language models

    v0.4.2 300 #artificial-intelligence #llm #nlp #mistral #llama

  14. blazen-local-llm

    Shared option + request types for Blazen local LLM backends (candle, mistralrs, llama.cpp) and the controlplane local-model factory seam

    v0.6.75 270 #llama #local-llm #blazen #llama-cpp #back-ends #llm-provider #event-driven #prometheus #candle #control-plane

  15. llm_api_access

    A package to query popular LLMs

    v0.1.41 #llm #openai #text-embedding #llama #gemini #llama-server #anthropic #google-gemini #embedding-generation #llm-provider

  16. ferrum-kernels

    Unified compute kernels (CUDA/Metal/CPU) and model runner for Ferrum inference

    v0.7.7 #cuda #llm-inference #inference-engine #llama #ferrum #metal #apple-silicon #moe #openai #open-ai-compatible

  17. kn-runtime

    Dynamic wrapper around CPU and GPU inference

    v0.7.4 #inference #gpu #graph #neural-network #cpu-and-gpu #onnx #llama #intermediate-representation

  18. llama-mcp-server

    Local LLM inference MCP server powered by llama.cpp

    v0.1.1 #llm-inference #gguf #mcp #llama #mcp-server #json-rpc #local-llm #web-server #llama-cpp #mcp-model

  19. tribev2

    TRIBE v2 — multimodal fMRI brain encoding model inference in Rust

    v0.1.0 #multimodal #llama #brain-encoding #neuroscience #fmri

  20. kn-cuda-sys

    A wrapper around the CUDA APIs

    v0.7.4 #inference #cuda #graph #cu-blas #cudnn #neural-network #onnx #llama #happen #intermediate-representation

  21. concerto-cli

    Concerto CLI: a Rust inference multiplexer for self-hosted LLMs. Orchestrates vLLM, llama.cpp, and SGLang processes across 1-8 GPUs with demand-aware routing.

    v0.1.0 #llama #inference #llama-cpp #self-hosted #gpu #vllm #multiplexer #inference-engine #llm #orchestrates

  22. ferrum-kv

    KV cache management with PagedAttention for Ferrum inference

    v0.7.7 #llama #ferrum #inference-engine #llm-inference #apple-silicon #openai #open-ai-compatible #metal #qwen #moe

  23. kalosm

    interface for pretrained AI models

    v0.4.0 430 #whisper #llama #llm #ocr

  24. ferrum-interfaces

    Core trait contracts for the Ferrum LLM inference engine

    v0.7.7 #inference-engine #llm-inference #llama #ferrum #apple-silicon #openai #open-ai-compatible #metal #qwen #moe

  25. embellama

    High-performance Rust library for generating text embeddings using llama-cpp

    v0.10.1 #llama #nlp #machine-learning #api-bindings

  26. ollama-oxide

    integrating with Ollama's native API, providing low-level inference and high-level conveniences

    v0.2.0 #llama #ollama #ollama-api #api-client #llm

  27. ferrum-attention

    Fused flash attention for Metal and CPU — production-grade, framework-independent

    v0.7.3 #llama #ferrum #inference-engine #metal #apple-silicon #llm-inference #fused-flash-attention #open-ai-compatible #openai #production-grade

  28. bloom_lfs

    A high-performance, latch-free log-structured storage layer built for modern flash storage and multi-core systems

    v0.1.1 #storage #lock-free #flash-storage #log-structured #storage-layer #io-uring #concurrency #lfs #llama #flush-buffer

  29. llama-runner

    A straightforward Rust library for running llama.cpp models locally on device

    v2.3.2 #llama #llm

  30. ambi

    A flexible, multi-backend, customizable AI agent framework, entirely based on Rust

    v0.3.8 #ai-agent #llama #openai #llm

  31. ferrum-types

    Shared type definitions for the Ferrum LLM inference engine

    v0.7.7 110 #llama #ferrum #inference-engine #llm-inference #apple-silicon #openai #open-ai-compatible #metal #moe #structured-output

  32. ferrum-testkit

    Testing utilities for Ferrum LLM inference engine

    v0.7.7 #inference-engine #llm-inference #ferrum #llama #apple-silicon #metal #open-ai-compatible #openai #benchmark #structured-output

  33. llmux

    Hook-driven LLM model multiplexer with pluggable switch policy

    v2.4.0 #llm-model #active-model #multiplexer #switching #middleware #llama #proxy #alive #vllm #gpu

  34. octoroute

    Intelligent multi-model router for self-hosted LLMs

    v1.0.0 #prometheus #artificial-intelligence #http-router #model-tier #load-balancing #self-hosted #model-routing #llama #local-llm #model-selection

  35. mlx-models

    Model architectures and weight loading for mlx-server (LLaMA, Mistral, Qwen2/3)

    v0.1.3 #llama #model #mlx #mistral #qwen2 #moe #local-models #tok #apple-silicon

  36. ferrum-scheduler

    Request scheduling for Ferrum LLM inference engine

    v0.7.7 #inference-engine #llama #ferrum #llm-inference #apple-silicon #openai #open-ai-compatible #metal #qwen #moe

  37. mullama

    Comprehensive Rust bindings for llama.cpp with memory-safe API and advanced features

    v0.3.0 #llama #llm-inference #llm

  38. ferrum-quantization

    Weight-format abstraction (Dense / GPTQ / AWQ / GGUF) for Ferrum models

    v0.7.7 #gguf #llama #ferrum #apple-silicon #inference-engine #llm-inference #open-ai-compatible #metal #openai #dense

  39. kalosm-llama

    interface for Llama models

    v0.4.3 250 #llama #mistral #llm #nlp

  40. concerto-core

    Pure routing logic, eviction policies, and memory accounting for Concerto. Zero IO dependencies.

    v0.1.0 #llama #gpu #inference #routing #testing #inference-engine #llama-cpp #accounting #policies #eviction

  41. llama-cpp-sys-4

    Low Level Bindings to llama.cpp

    v0.4.0 1.8K #gguf #inference #ggml #llama #llm

  42. ferrum-models

    Model architectures (LLaMA, Qwen, BERT) for Ferrum inference

    v0.7.7 #llama #inference-engine #llm-inference #ferrum #cuda #apple-silicon #moe #metal #openai #open-ai-compatible

  43. ferrum-engine

    Model orchestration engine for Ferrum LLM inference

    v0.7.7 #llama #inference-engine #llm-inference #ferrum #apple-silicon #openai #open-ai-compatible #metal #qwen #moe

  44. lazyllama

    A lightweight TUI client for Ollama with markdown support and smart scrolling

    v0.5.2 #artificial-intelligence #ollama #llama

  45. ferrum-sampler

    Sampling strategies for Ferrum LLM inference engine

    v0.7.7 #inference-engine #ferrum #llm-inference #llama #apple-silicon #open-ai-compatible #openai #metal #sampling-strategies #qwen

  46. tauri-plugin-llama-crab

    Tauri plugin for llama-crab local inference

    v0.1.8 #tauri-plugin #llama #inference #local-inference #chat-completion #llama-crab #worker-thread #rerank #multimodal #model-loading

  47. lancor

    End-to-end llama.cpp toolkit: API client, HuggingFace Hub, server orchestration, benchmarks

    v0.2.0 330 #artificial-intelligence #openai #llama #llm

  48. ferrum-cuda-kernels

    Custom CUDA kernels and decode runner for Ferrum inference

    v0.6.0 #llama #cuda #inference-engine #ferrum #llm-inference #openai #open-ai-compatible #apple-silicon #metal #qwen

  49. slm_llama

    llama.cpp backend for slm_inference

    v0.1.4 #llama #llama-cpp #slm-inference #llama-model #back-end #metal #vulkan #top-p #top-k #sampler

  50. ferrum-server

    OpenAI-compatible HTTP API server for Ferrum inference

    v0.7.7 #llama #ferrum #openai #open-ai-compatible #inference-engine #apple-silicon #web-api #llm-inference #metal #web-server

  51. sapient-models

    Pre-built LLM architecture graph builders for SAPIENT — Llama, Mistral, Phi, Gemma, GPT-2, BERT, Qwen

    v0.3.1 #llama #sapient #inference-engine #graph #pre-built #gpu #graph-builder #model-info #gpt-2 #mistral

  52. slm_ikllama

    ik_llama.cpp backend for slm_inference

    v0.1.4 #llama #cpp #slm-inference #ik-llama #back-end #llama-model #gguf #kv-cache

  53. kalosm-streams

    A set of streams for pretrained models in Kalosm

    v0.4.0 450 #whisper #llama #pre-trained-models #kalosm #artificial-intelligence #llm #ai-model #mistral #transcription

  54. candle-pipelines

    intuitive pipelines for local LLM inference in Rust, powered by Candle. Inspired by Python's Transformers library.

    v0.0.7 110 #llm-inference #llama #pipeline #candle #hugging-face #tool-calling #local-llm #classification #qwen3 #text-output

  55. ferrum-cli

    CLI for Ferrum — a Rust-native LLM inference engine

    v0.7.7 #llama #llm-inference #inference-engine #apple-silicon #openai #open-ai-compatible #metal #rust-native #moe #qwen

  56. opentslm

    using Burn, WGPU, and llama.cpp

    v0.1.0 #llama #llama-cpp #burn #plot #wgpu #llm #gb #cargo-run #metrics #trainable

  57. oxillama-cli

    Pure Rust LLM inference engine CLI — the sovereign alternative to llama.cpp

    v0.1.3 #gguf #llama #llm-inference #llm

  58. ferrum-runtime

    Backend implementations (Candle, CPU) for Ferrum inference

    v0.7.3 #llama #ferrum #apple-silicon #inference-engine #open-ai-compatible #llm-inference #candle #metal #moe #production-grade

  59. chat-prompts

    Chat prompt template

    v0.35.3 #prompt-template #system-prompt #llama #llm #openai #system-message #string-format

  60. sapient-tokenizers

    HuggingFace-compatible tokenizers for SAPIENT — BPE, WordPiece, SentencePiece, chat templates

    v0.3.1 #sapient #chat #bpe #hugging-face #inference-engine #chat-template #llama #sentence-piece #gemma #qwen

  61. concerto-api

    HTTP API and orchestrator state machine for Concerto: routes, request lifecycle, cold-start dedup, eviction, and graceful shutdown

    v0.1.0 #web-api #llama #graceful-shutdown #inference #gpu #cold-start #self-hosted #eviction #llama-cpp #dedup

  62. llama-runtime

    Execution runtime for llama.rs — oxidizedMLX integration and backend selection

    v0.1.1 #runtime-execution #llama #run-time #integration #back-end #logits #ci-cd #shared-infrastructure #pull-request #agent-workflow

  63. llama_cpp

    High-level bindings to llama.cpp with a focus on just being really, really easy to use

    v0.3.2 1.5K #llama #llama-model #model-context #bindings #token

  64. candelabra

    Desktop-friendly GGUF LLaMA inference wrapper for Candle and Hugging Face Hub

    v0.2.0 #gguf #llama #candle #llm

  65. llama-sampling

    Sampling strategies for llama.rs — temperature, top-k/p, repetition penalty

    v0.1.1 #sampling-strategies #random #top-k #llama #penalty #top-p #reproducible #greedy #scaling #nucleus

  66. curtana

    Simplified zero-cost wrapper over llama.cpp powered by lama-cpp-2

    v0.1.3 #llama #inference #llm #api-bindings

  67. concerto-gpu

    GPU monitoring abstractions for Concerto: GpuMonitor trait, mock implementation, and optional NVML-backed implementation

    v0.1.0 #llama #gpu-monitor #gpu-monitoring #inference #abstraction #mocking #nvml #llama-cpp #telemetry #self-hosted

  68. llama-tokenizer

    Tokenizer crate for llama.rs — deterministic text-to-token conversion

    v0.1.1 #tokenize #llama #deterministic #utf-8 #white-space #ci-cd #shared-infrastructure #chat-template #agent-workflow #authentication

  69. rlx-voxtral

    Mistral Voxtral speech LM for RLX (Whisper encoder + Llama decoder)

    v0.2.6 #speech #llama #rlx-models #mlx #language-model #lm #voxtral #whisper #audio #gguf

  70. log_structured_store

    A high-throughput append-oriented storage engine built in Rust using batched buffer-ring writes, direct I/O (O_DIRECT), and async io_uring dispatch

    v0.1.0 #storage-engine #io-uring #lock-free #log-structured #store #high-throughput #concurrency #logging #llama #byte-offset

  71. ferrum-tokenizer

    Tokenization wrapper for Ferrum inference engine

    v0.7.7 #inference-engine #llama #ferrum #apple-silicon #llm-inference #open-ai-compatible #openai #metal #qwen #moe

  72. oxillama-arch

    Model architecture implementations — LLaMA, Qwen3, Mistral, Gemma, Phi

    v0.1.3 #gguf #llama #llm-inference #neural-network #llm

  73. rlx-flow

    Block assembly-line API for RLX model builders — fusion-first, config-driven

    v0.2.11 550 #llama #ml #compiler #neural-network

  74. ezllama

    An opinionated, simple Rust interface for local LLMs, powered by llama-cpp-2

    v0.3.1 100 #llama #llm #machine-learning #api-bindings

  75. oxillama-runtime

    Inference engine — KV cache, sampling, tokenizer bridge

    v0.1.3 #inference-engine #llm-inference #llama #llm #sampling

  76. wax-bench

    Benchmark types and helpers for wax

    v0.1.0 #benchmark #wax #gguf #safetensors #helper #llama #config-json #mlx #llm

  77. llama-kv

    KV cache management for llama.rs — prefill, decode, paging, eviction

    v0.1.1 #kv-cache #llama #cache-eviction #paging #cache-management

  78. llama-engine

    Narrow-waist engine trait and core types for llama.rs

    v0.1.1 #llama #traits #active #back-ends #metal #ci-cd #core-types #shared-infrastructure #agent-workflow #aws

  79. toktrie_tiktoken

    tiktoken (OpenAI BPE) library support for toktrie and llguidance

    v1.7.6 #tiktoken #structured-output #bpe #openai #llguidance #toktrie #constrained-decoding #context-free-grammar #json-schema #llama

  80. wgpu-llm-core

    A minimalist Llama inference engine built on wgpu and WGSL compute shaders

    v0.1.1 #safetensors #compute-shader #inference-engine #llama #llm-inference #wgpu #wgsl-shader #gpu #gpu-kernel #paged

  81. oxillama-server

    OpenAI-compatible HTTP API server for OxiLLaMa

    v0.1.3 #llama #openai #llm-inference #server-api #llm #llama-server

  82. oxillama

    Pure Rust LLM inference engine — the sovereign alternative to llama.cpp (meta crate)

    v0.1.3 #gguf #llama #llm-inference #llm #pure-rust

  83. llm-manager

    Terminal UI for managing LLMs

    v1.9.0 #llama #llm #tui #terminal

  84. elizaos-plugin-local-ai

    elizaOS Local AI Plugin - Local LLM inference for text, embeddings, vision, and audio

    v2.0.0 #artificial-intelligence #llama #elizaos #local-ai #embedding

  85. concerto-backend

    Backend process lifecycle for Concerto: BackendManager trait, mock implementation, and a real process-spawning implementation for inference engines

    v0.1.0 #inference-engine #llama #gpu #process-spawning #back-end #mocking #llama-cpp #life-cycle #vllm #child-process

  86. kapsl-backends

    Pluggable inference backends (ONNX, llama.cpp) for the Kapsl engine

    v0.1.1 #onnx #inference-engine #llama #back-ends #llama-cpp #kapsl #model-inference

  87. pagedelta

    Deltas: LLAMA's primary units of data

    v0.2.1 #llama #units #units-of-data #delta #primary #evict

  88. ggufy

    Unified GGUF wrapper for llama.cpp and Ollama

    v0.2.0 #gguf #llama #ollama #llama-cpp #unified #cache-directory #multimodal

  89. rlx-neutts

    NeuTTS voice-cloning TTS — GGUF backbone + NeuCodec decoder for RLX

    v0.2.6 #gguf #safetensors #llama #text-to-speech #rlx-models #mlx #voice-cloning #backbone #wgpu #gpu

  90. concerto-config

    TOML configuration parsing for Concerto

    v0.1.0 #config-parser #llama #inference #gpu #llama-cpp #vllm #self-hosted #canonical #multiplexer #llm

  91. llama-cpp-v3

    Safe and ergonomic Rust wrapper for llama.cpp with dynamic loading

    v0.1.8 #llama #llama-cpp #gguf #loading #back-end #dynamic-loading #sampler #dll #cache-back-end #github-api

  92. nimi-shell-tauri

    Shared Tauri host glue for Nimi apps — runtime bridge, daemon lifecycle, session logging, auth/oauth commands, runtime defaults

    v0.1.0 #tauri-app #oauth #daemon #artificial-intelligence #llama #nimi #desktop #life-cycle #tauri-command #path-resolution

  93. kalosm-model-types

    Shared types for Kalosm models

    v0.4.0 1.8K #llama #mistral #llm #nlp

  94. wax-core

    Core inference engine for wax, a small Candle-based local LLM runner

    v0.1.0 #gguf #safetensors #llama #inference-engine #llm #local-llm #llm-inference #mlx #top-p #candle

  95. llama-models

    Model architectures (Llama/Qwen/Mistral blocks) for llama.rs

    v0.1.1 #llama #safetensors #block #architecture #qwen #mlp #rms-norm #ci-cd #mistral #shared-infrastructure

  96. rlx-llama32

    LLaMA 3.2 for RLX

    v0.2.6 #llama #gguf #packing #metal #compile #rlx #lm #causal #3-2

  97. oxillama-wasm

    WebAssembly bindings for OxiLLaMa GGUF parsing and quantization

    v0.1.3 #gguf #llama #llm #wasm

  98. oxillama-py

    Python bindings for OxiLLaMa LLM inference engine

    v0.1.3 #gguf #llama #llm-inference #llm

  99. ushi

    High-performance LLM inference server with llama.cpp FFI bindings

    v0.1.1 #llama #llm-inference #llm #ml

  100. agentkit-provider-groq

    Groq provider adapter for agentkit

    v0.10.2 #agentkit #groq #llama #provider-adapter #streaming #chat-completion #model-parameters #artificial-intelligence

  101. rllama

    Pure Rust implementation of LLaMA-family of models, executable

    v0.3.0 110 #llama #opencl #machine-learning

  102. xybrid-llama

    Safe Rust wrappers over xybrid-llama-sys: RAII model/context handles, streaming trampoline, typed errors. Zero unsafe on the public surface.

    v0.2.2 #llama #llama-cpp #trampoline #streaming #surface #safe-wrapper #raii #model-context #owns #clippy

  103. rlx-llama-base

    Shared Llama-shaped arch config + GGUF reader (Mistral, Phi, Bonsai, OmniCoder, Granite, Cohere — PLAN.md M4 foundation)

    v0.2.9 150 #gguf #llama #m4 #config #mistral #sliding-window #phi #bonsai #cohere #scaling

  104. paca-cli

    Helpers for interacting with llama.cpp

    v0.2.0 #llama #llama-cpp #helper #hugging-face #model #cache-directory #local-models #local-cache

  105. blazen-llm-llamacpp-server

    llama.cpp HTTP-server (llama-server) proxy provider for Blazen — OpenAI-compat + native completion + runtime LoRA adapter toggle

    v0.6.76 #llama #llama-cpp #llama-server #blazen #web-server #proxy #lora #web-api #chat-completion #per-slot

  106. apimyllama

    interacting with the APIMyLlama API

    v2.0.7 #artificial-intelligence #llama #api-bindings

  107. blazen-llm-vllm

    vLLM OpenAI-compat + LoRA adapter management proxy provider for Blazen

    v0.6.76 #vllm #blazen #lora #llama #adapter #proxy #http-proxy #chat-completion #openai

  108. sonr-daemon

    Background daemon for sonr, providing semantic search capabilities using local LLMs

    v0.1.7 #semantic-search #local-llm #daemon #llama #semantic-search-tool #llama-server #model-context-protocol #gpu #hugging-face #embedding

  109. yule-registry

    Model registry: pull, cache, and manage verified model artifacts

    v0.1.0 #gguf #artificial-intelligence #yule #model-registry #vulkan #llama #model-inference

  110. thulpoff-provider

    LLM provider implementations for thulpoff

    v0.1.0 270 #llm-provider #distillation #thulpoff #skill #nim #anthropic #claude #llama #nvidia #mistral

  111. airframe-observe

    FSE-based inference observation layer for Airframe. Selector-first, single-pass capture of layer outputs, logits, and text during inference.

    v0.1.0 #inference #single-pass #capture #text #logits #observer #fact #llama #gguf #airframe

  112. yule-infer

    Inference engine: attention, KV cache, sampling, quantization, token generation

    v0.1.0 #gguf #inference-engine #llama #artificial-intelligence #yule #vulkan #token-generation #gpu #model-inference #vulkan-back-end

  113. hanzo-kernel-macros

    The #[kernel(targets(...))] attribute for hanzo-kernel: one Rust fn, declared backends

    v0.1.0 #safetensors #hanzo #back-ends #target #devices #llama #flash-attention #multimodal #llm #whisper

  114. kalosm-common

    Helpers for kalosm downloads and candle utilities

    v0.4.0 1.9K #whisper #kalosm #llama #pre-trained-models #ai-model #candle #llm #accelerator #mistral #transcription

  115. llama_cpp_rs

    Rust bindings for LLAMA.CPP inference

    v0.3.0 #llama #inference #api-bindings #machine-learning #llama-cpp-bindings

  116. bitnet-llm-sys

    Raw FFI bindings to bitnet.cpp (Microsoft's 1-bit LLM inference engine)

    v1.0.2 #llama #inference-engine #llama-cpp #bitnet #bit-net #llm-inference #1-bit

  117. rmistral

    interface for Mistral models

    v0.1.0 #whisper #llama #pre-trained-models #inference #kalosm #mistral #artificial-intelligence #llm #ai-model #transcription

  118. lmcpp

    Rust bindings for llama.cpp's server with managed toolchain, typed endpoints, and UDS/HTTP support

    v0.1.1 #llama-cpp #artificial-intelligence #inference #llama #llm

  119. weave-writer

    collaborative generative writing

    v0.0.3 170 #openai #chat #weave #llama #story #collaborative #icons #serialization #generative-ai #llama-cpp

  120. rs-llama-cpp

    Automated Rust bindings generation for LLaMA.cpp

    v0.1.67 490 #llama-cpp #llama #llama-cpp-bindings #llm #bindings

  121. thalamus

    A deep learning mesh node server platform for linux/mac/unix

    v0.0.11 #deep-learning #unix #linux #platform #mesh-node #whisper #llama #llama-cpp

  122. llama_cpp_sys

    Automatically-generated bindings to llama.cpp's C API

    v0.3.2 1.5K #llama #bindings #path

  123. kjarni

    Cross-platform ML inference engine. Embeddings, classification, reranking, and RAG search

    v0.0.1-alpha.1 #inference-engine #search-engine #embedding #classification #reranking #rag #ml #cross-platform #onnx #llama

  124. llm-chain-llama

    implementing llm-chains for LLamA. Chains can be use to apply the model series to complete complex tasks, such as agents.

    v0.13.0 100 #langchain #llama #llm-chain #llm

  125. crabml

    core package

    v0.1.0 #llama #artificial-intelligence #inference-engine #llama-cpp #package #web-gpu #simd-accelerated #desktop #objectives

  126. simple_llama

    run llama.cpp in Rust. based on llama-cpp-2

    v0.1.3-b 110 #llama #llama-cpp #llama-cpp-2 #llama-model #prompt

  127. crabml-llama2

    crabml llama2 support

    v0.1.0 #llama #crabml #llama2 #model #inference-engine #llama-cpp #web-gpu

  128. oxidized-transformers

    Transformers library (not functional yet)

    v0.1.1 #llama #hugging-face #transformer #model #bert #albert #cuda #float16

  129. crabml-cli

    crabml cli

    v0.1.0 #model #cli #inference-engine #llama #llama-cpp #web-gpu #simd-accelerated #artificial-intelligence #objectives

  130. duckai

    DuckDuckGo AI to OpenAI API

    v0.3.2 1.2K #artificial-intelligence #openai #llama #duckduckgo #ip #chat #proxy #authentication

    131. Try searching with DuckDuckGo.


  131. kv-cache

    LLM inference in Rust

    v0.1.0 #llm-inference #llama #llm #rust #inference

  132. chat

    AI platform agnostic cli

    v0.2.1 290 #artificial-intelligence #platform #openai #api-key #cli #productivity #llama

  133. kjarni-examples

    Kjarni Examples - Example code for the Kjarni inference engine

    v0.0.1-alpha.1 #inference-engine #kjarni #kjarni-for-kjarni #onnx #llama #whisper

  134. vibesort

    Blazingly fast vibesort implementation in Rust using Llama 4 Maverick on Cerebras Cloud

    v0.3.0 #cerebras #llama #cloud #maverick

  135. llm-chain-llama-sys

    bindings based on bindgen for LLaMA.cpp

    v0.13.0 #langchain #llm-chain #llama #llm

  136. localllm

    qwen2 model lib by llama.cpp

    v0.2.0 #llama #llama-cpp #model #api-bindings

  137. aionic

    unified, user-friendly Rust library for seamless integration with various public Language Model APIs, such as openAI

    v0.1.7 #chatgpt #language-model #language-integration #user-friendly #public #openai-api #llama #sense #bard

  138. llm_devices

    Device management and build system for LLM inference

    v0.0.3 250 #inference #build-system #logging #devices #llama #ram #llm-interface #devices-management #llm-client #metal

  139. round-multiple

    LLM inference in Rust

    v0.1.0 #llm-inference #llama #llm #rust #inference

  140. self_sovereign_ai

    building self-sovereign AI systems with decentralized networking and cryptography

    v0.1.0-beta.3 230 #artificial-intelligence #self-sovereign #decentralized #ai-model #dht #ai-sdk #llm #api-token #llama #deep-learning

  141. llama_link

    A llama.cpp server interface

    v0.2.1 110 #llama #llama-cpp #link #server #prompt

  142. llama2_rs

    port of llama2.c

    v0.1.1 #port #llama #inference #multi-threading

  143. all-close

    LLM inference in Rust

    v0.1.0 #llm-inference #llama #llm #rust #inference

  144. scaled-dot-product-attention

    LLM inference in Rust

    v0.1.0 #llm-inference #llm #llama #rust

  145. llama

    Friendly LLVM bindings

    v0.14.2 #jit-compiler #llvm #jit

  146. llm-base

    The base for llm; provides common structure for model implementations. Not intended for use by end-users.

    v0.1.1 370 #ggml #llm #inference #llama #llama-cpp #ml-model

  147. oxidized-cuda-kernels

    Additional CUDA kernels for Oxidized Transformers

    v0.1.1 #llama #cuda #transformer #oxidized #bert

  148. llm-llama

    LLaMA (Large Language Model Meta AI) for the llm ecosystem

    v0.1.1 410 #large-language-model #artificial-intelligence #llama #ggml #llm #llm-cli #llm-inference

  149. llama-build

    Friendly LLVM bindings (BUILD)

    v0.14.0 #build #llama #llvm

  150. llm-cli

    A CLI for running inference on supported Large Language Models. Powered by the llm library.

    v0.1.1 170 #large-language-model #inference #llama #ggml #model-inference #llm-inference #cli-for-running #llama-cpp

  151. alith-store

    Alith store package

    v0.4.3 #artificial-intelligence #alith #web3 #blockchain #nodejs #embedding #smart-contracts #llama #nft #dao

  152. alith-tools

    Alith Tool package

    v0.4.3 #artificial-intelligence #alith #web3 #blockchain #nodejs #smart-contracts #llama #nft

  153. llama-wasm

    Friendly LLVM bindings (WASM)

    v0.5.0 #llama #jit #llvm #wasm