llm-d is a Kubernetes-native high-performance distributed LLM inference framework

DeepEP: an efficient expert-parallel communication library

Website for llm-d: This repository builds the website seen at llm-d.ai

Inference scheduler for llm-d

NVIDIA NVSHMEM is a parallel programming interface for NVIDIA GPUs based on OpenSHMEM. NVSHMEM can significantly reduce multi-process communication and coordination overheads by allowing programmer…

A high-throughput and memory-efficient inference and serving engine for LLMs

Incubating P/D sidecar for llm-d

llm-d helm charts and deployment examples

Cluster Toolkit is an open-source software offered by Google Cloud which makes it easy for customers to deploy AI/ML and HPC environments on Google Cloud.

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

LLM Instance gateway implementation.

Container Cluster Manager

📜Fork for tracking CNCF projects

Test infrastructure for the Kubernetes project.

Kubernetes community content

Fast JSON encoder/decoder compatible with encoding/json for Go

kcp is a prototype of a Kubernetes API server that is not a Kubernetes cluster - a place to create, update, and maintain Kube-like APis with controllers above or without clusters.

Release tooling for OpenShift Origin

OpenShift 3 - build, deploy, and manage your applications with Docker and Kubernetes

Install an OpenShift cluster

Helpers for going from apis and clients to useful runtime constructs

The Cluster Ingress Operator manages highly available network ingress for OpenShift

CNCF K8s Conformance Working Group