Breaking the Context Wall in Agentic AI Storage

Great session with Anat and Scott on storage for agentic AI — Supermicro, VAST Data, and Solidigm together on the context memory tier. Fun one to watch. Premieres Aug 25....

"We are just getting started." That's the message from Kevin Cochrane, CMO of Vultr who took the stage at Ai4 - Artificial Intelligence Conferences. We're only at the start of the the next marathon in #AI – one where we can do real, meaningful work with #agenticAI. This new era will require a shift. For agentic AI to reach mass enterprise adoption, we'll need a new infrastructure architecture. We need diversity of CPUs coupled with high performance GPUs deployed everywhere where you are running applications, all backed by a streamlined data pipeline.

AI innovation isn’t stopping at the model layer. The enormous market for AI compute is creating pressure across the entire engineering stack—from GPU architecture to cooling and power delivery. Unlike earlier eras of government-led R&D, today’s breakthroughs are being accelerated by intense commercial demand. Every token creates an incentive to make the underlying infrastructure faster, cooler, and more efficient. AI isn’t just changing software. It’s forcing a new wave of engineering. Watch the full episode on the DCR Media channel: https://lnkd.in/gJyHiG8s #ArtificialIntelligence #Engineering #DataCenters #AIInfrastructure #Innovation Kirk Offel | OVERWATCH | Heron Power | Drew Baglino

"We're just getting started." Vultr CMO Kevin Cochrane joined our partners Supermicro and VAST Data onstage at AMD #AdvancingAI 2026 to walk through his take on where we are in AI and what's next. What makes this moment important is the Infrastructure requirements for mass enterprise adoption to do real work using agentic AI require a new infrastructure architecture. It requires diversity of CPUs coupled with high performance GPUs, all backed by a streamlined data pipeline. Vultr is building an ecosystem to democratize AI, with the help of Supermicro and VAST Data, all built to drive real enterprise results. And we're just getting started.

Accelerate your AI projects with the NVIDIA DGX Spark 2-Pack with Cable Bundle — a compact, enterprise-grade AI desktop supercomputer. Built on Grace Blackwell architecture, it delivers up to 1 petaFLOP of AI performance, 128GB unified memory per unit, and 4TB storage to support model fine-tuning (up to 200B parameters at FP4), inference, and analytics locally. Ideal for R&D teams, AI labs, and businesses wanting secure, high-performance local compute. In stock now: https://wix.to/j43YMY1 #NVIDIA #AI #HPC #MachineLearning #DataScience #EnterpriseAI

Brute-forcing AI scalability by constantly throwing expensive hardware at the problem is hitting a hard wall, but the real secret to breaking the memory bottleneck lies in smarter software and storage architecture. In this clip from the Tech Field Day Podcast, Alastair Cooke, Ron Pagani Jr., Andy Banta, and Brian Martin examine key takeaways from AI Infrastructure Field Day 5. The panel highlights how vendors like Solidigm, MinIO, and CTERA are proving that AI scale is fundamentally a software challenge—demonstrating how high-density SSDs can act as a shared context memory layer for GPUs. By persisting key-value (KV) caches with tools like MinIO MemKV and pre-generating metadata artifacts, enterprises can stop wasting GPU cycles on redundant context re-computations. Translating classic computing principles—such as predictive pre-fetching and multi-layer caching—up the AI stack drastically cuts time-to-first-token, frees up valuable bandwidth, and optimizes enterprise infrastructure budgets. #TFDPodcast #AIInfrastructure #KVCache #EnterpriseAI #StorageTech #GPUOptimization

Nemotron 3.5 Lightning by NVIDIA AI is now supported for training and inference on the Applied Compute Platform. On our agentic coding benchmark, decode throughput, time to first token, and median user latency remained effectively unchanged as concurrency and total token throughput scaled 16x. Its LatentMoE and Mamba architecture lets us scale sparsity, context length, and batch size with minimal overhead, dramatically increasing iteration throughput across post-training runs.

Running an LLM locally is relatively straightforward. Getting it to serve consistently under real workloads is a different problem. I've been working with vLLM for llm inference, focusing on what actually determines inference performance beyond the model's raw benchmark numbers. A few areas I've been paying particular attention to: • Time-to-first-token (TTFT) —> understanding how request scheduling, prompt processing, and queueing affect interactive workloads. • Inter-token latency (ITL) —> maintaining consistent generation speed rather than optimizing only for peak throughput. • KV cache utilization —> understanding how context length and concurrent requests translate into GPU memory pressure. • Continuous batching —> maximizing GPU utilization while maintaining acceptable latency for individual requests. • Memory footprint —> separating model weights, KV cache, activations, and runtime overhead when determining actual serving capacity. • Concurrency vs. latency —> increasing throughput doesn't necessarily mean improving the user experience. The useful operating point depends heavily on the workload. With multiple systems, this also becomes an infrastructure question , whether to distribute a model across nodes or run independent inference instances and scale horizontally. The interesting part of LLM serving is that there is rarely a single "best" configuration. The right architecture depends on model size, context length, concurrency, latency requirements, and available GPU memory. #LLMInference #vLLM #NVIDIA #AIInfrastructure #LLMOps #GenerativeAI #GPU #MachineLearning #SystemsEngineering

Right-sizing Kubernetes for LLM agent fleets — GPU inference pools, CPU retrieval workers, spot preemption handling, and cluster autoscaler settings that scale without bankrupting you. What I cover: → Should agent inference and retrieval run on separate node pools → How do you prevent cluster autoscaler from thrashing during agent traffic spikes → Cluster Autoscaler or Karpenter for agent workloads Read the full article: https://lnkd.in/dqanWSSU #Agent #Cluster #AI #LLM #MachineLearning

https://lnkd.in/gSKQHxJa ...likely gaining access to high-performance GPUs and data‑center infrastructure to accelerate large‑scale model research, training runs, and experimentation. #technology #chatgpt