Inferact

We're excited to co-host the vLLM x Dynamo Meetup with the NVIDIA team during vLLM conference week in San Francisco! 🚀 Join us for an evening of tech talks and conversation on what it takes to serve LLMs efficiently at scale — from inference optimization and distributed serving to the practical challenges of running these systems in production. 📅 Monday, August 24 · 6:00–9:00 PM PT On the agenda: • 6:30 PM — vLLM tech talk • 6:45 PM — NVIDIA Dynamo tech talk • 7:00 PM — Food, drinks, and time to connect with engineers, researchers, and builders across the inference community Space is limited and registration is subject to approval — reserve your spot early: https://luma.com/r8o604o0 #LLM #Inference #vLLM #NVIDIA #MachineLearning #AIInfrastructure

The vLLM Conference is coming up in 3 weeks! 🎉 Come learn about the current state and future of AI inference, Aug 24–26 in San Francisco 🌉, hosted by Inferact at Anyscale Ray Summit. We'll have speakers from Inferact, NVIDIA, AMD, Google TPU, Anyscale, PyTorch, Meta, Red Hat, and key builders around vLLM. The talks on the roadmap deep dive into the latest on accelerators, training and serving pipelines, and production-scale inference 🚀 Link to the full schedule below 👇

With Kimi K3 Day-0 on vLLM: Open Frontier Intelligence for Everyone 🚀 At 2.8 trillion parameters, Moonshot AI's Kimi K3 is one of the most powerful open-weight models ever released. Starting today, you can serve it on vLLM the moment the weights are public. What K3 brings: 🧠 2.8T-parameter Mixture-of-Experts (16 of 896 experts active per token) 📚 1M-token context window 🛠️ Native multimodal understanding, including vision ⚡ Kimi Delta Attention: a hybrid of linear and full attention that makes million-token context affordable Huge thank you to Kimi (Moonshot AI) AI for the model release and partnership, Inferact for leading the vLLM optimizations, and to our partners at NVIDIA, AMD, and the broader vLLM community. Recipes, blog, and DSpark draft model in comments below 👇

After a great run at Anyscale, I started my next opportunity last week by joining Inferact (https://inferact.ai/). Inferact was founded by creators and core maintainers of vLLM (https://vllm.ai/), the most widely-used open-source LLM inference engine. Open models are tightening the gap on their closed counterparts, and optimizing the inference layer is becoming increasingly critical for delivering value in production. We're hiring for many roles across the stack. Please reach out if you or someone you know is interested! https://lnkd.in/gTCykeba

Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. https://lnkd.in/gY4NvS5h Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. Cost and latency are important in real-world use cases. Inkling's continuous thinking effort lets you pick your point on the cost/performance curve — reaching the same score with a fraction of the tokens. Inkling natively understands and reasons across text, audio, and images. It’s strong on audio in particular, ranking among the strongest open-weights models on VoiceBench, MMAU, and AudioMC. We’re grateful to our partners for their day-0 support across the open-source ecosystem: NVIDIA, Together AI, Fireworks AI, Databricks, Unsloth AI, Modal, Baseten, LightSeek Foundation, Inferact on VLLM, RadixArk on SGLang. Inkling is the first in a family. We’ve included some details of Inkling-Small, a lighter-weight model trained on a similar recipe, with full weights to follow. We hope you enjoy Inkling, and as always we’re keen to see what you build.

The first vLLM Conference runs as its own track inside Ray Summit 2026, hosted by Inferact. One ticket gets you into both. Speakers include the people building vLLM and the ecosystem around it: → Woosuk Kwon, Creator of vLLM and Cofounder/CTO, Inferact → Simon Mo, Core Maintainer of vLLM and Cofounder/CEO, Inferact → Siyuan Fu, Senior Engineer, NVIDIA → Douglas Lehr, Principal Engineer, AMD → Qi Zhou, Senior Staff Software Engineer, Google TPU → Richard Zou, Senior Staff Software Engineer, Meta → Debarshi Raha, VP/Fellow Engineer, DigitalOcean → Greg Pereira, Senior Machine Learning Engineer, Red Hat Plus Hugging Face, Prime Intellect, Anyscale, Google DeepMind, and Together AI. San Francisco, Marriott Marquis, August 24–26. Early bird pricing ends this Friday, July 17. https://lnkd.in/gcdu7UVy

Announcing the first-ever vLLM Conference — hosted by Inferact at Ray Summit, Aug 24–26 in San Francisco 🎉🌉 This is where we'll get into the work pushing open, high-performance inference forward, such as: 🗺️ Where the vLLM roadmap is headed ⚡ Getting the most out of accelerators including NVIDIA, AMD, TPU 🔗 Wiring vLLM into training and serving pipelines 🚀 Running inference on production scale The summit features Speakers from Inferact, NVIDIA, AMD, Google TPU, Anyscale, PyTorch, Meta, Red Hat, and more 🎤 Come learn where the future of inference, open source, and AI is heading — and meet the leading builders driving it 👇 https://lnkd.in/gBNFDCZM