Understanding AI Systems

I frequently see conversations where terms like LLMs, RAG, AI Agents, and Agentic AI are used interchangeably, even though they represent fundamentally different layers of capability. This visual guides explain how these four layers relate—not as competing technologies, but as an evolving intelligence architecture. Here’s a deeper look: 1. 𝗟𝗟𝗠 (𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹) This is the foundation. Models like GPT, Claude, and Gemini are trained on vast corpora of text to perform a wide array of tasks: – Text generation – Instruction following – Chain-of-thought reasoning – Few-shot/zero-shot learning – Embedding and token generation However, LLMs are inherently limited to the knowledge encoded during training and struggle with grounding, real-time updates, or long-term memory. 2. 𝗥𝗔𝗚 (𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹-𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻) RAG bridges the gap between static model knowledge and dynamic external information. By integrating techniques such as: – Vector search – Embedding-based similarity scoring – Document chunking – Hybrid retrieval (dense + sparse) – Source attribution – Context injection …RAG enhances the quality and factuality of responses. It enables models to “recall” information they were never trained on, and grounds answers in external sources—critical for enterprise-grade applications. 3. 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 RAG is still a passive architecture—it retrieves and generates. AI Agents go a step further: they act. Agents perform tasks, execute code, call APIs, manage state, and iterate via feedback loops. They introduce key capabilities such as: – Planning and task decomposition – Execution pipelines – Long- and short-term memory integration – File access and API interaction – Use of frameworks like ReAct, LangChain Agents, AutoGen, and CrewAI This is where LLMs become active participants in workflows rather than just passive responders. 4. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 This is the most advanced layer—where we go beyond a single autonomous agent to multi-agent systems with role-specific behavior, memory sharing, and inter-agent communication. Core concepts include: – Multi-agent collaboration and task delegation – Modular role assignment and hierarchy – Goal-directed planning and lifecycle management – Protocols like MCP (Anthropic’s Model Context Protocol) and A2A (Google’s Agent-to-Agent) – Long-term memory synchronization and feedback-based evolution Agentic AI is what enables truly autonomous, adaptive, and collaborative intelligence across distributed systems. Whether you’re building enterprise copilots, AI-powered ETL systems, or autonomous task orchestration tools, knowing what each layer offers—and where it falls short—will determine whether your AI system scales or breaks. If you found this helpful, share it with your team or network. If there’s something important you think I missed, feel free to comment or message me—I’d be happy to include it in the next iteration.

In a MAJOR ruling for European copyright law, the Munich Regional Court has sided with Germany’s music rights society GEMA against OpenAI, finding that the company’s ChatGPT model unlawfully used copyrighted song lyrics in its training and responses. The decision, issued this morning, marks the first major European court judgment holding an AI company liable for using protected works without a licence. I got into AI through being Director of Legal Affairs and Regulatory Compliance in IMRO, the Irish counterpart of GEMA - and I know the people in GEMA - so this is very interesting to me. The case centred on GEMA’s allegation that OpenAI trained ChatGPT on its repertoire of German song lyrics, allowing the chatbot to reproduce works by artists such as Helene Fischer and Herbert Grönemeyer. The court agreed, concluding that the model’s ability to reproduce lyrics word for word demonstrated that the works had been used in training. It ruled that OpenAI is liable for copyright infringement and prohibited ChatGPT from reproducing lyrics from GEMA-represented artists unless a licence is obtained. The court also held that the European Union’s Text and Data Mining exceptions cannot shield generative AI systems that “memorise” and reproduce copyrighted material. This reasoning undermines one of the primary legal defences AI developers have relied upon in Europe. While damages will be determined in a separate proceeding, the court’s finding of liability alone sets a powerful precedent. OpenAI has announced plans to appeal. The 42nd Civil Chamber of the Munich Regional Court had indicated its position in September, when it observed that the model’s outputs could not be explained without training on copyrighted material. The final judgment confirmed that assessment. For the wider AI sector, the ruling suggests that AI companies operating in the European Union may need explicit licences for any copyrighted content used in model training or risk litigation. The decision also has regulatory implications. It aligns with growing momentum within the EU to enforce transparency and rights-holder protections under the AI Act and the Copyright in the Digital Single Market Directive. The GEMA v OpenAI ruling diverges sharply from Bartz v Anthropic in the United States. In Bartz, Judge Alsup found that AI training on copyrighted material could qualify as fair use, meaning no licence is required when the use is deemed transformative and non-substitutive. He viewed training as an analytical process that teaches the model general patterns rather than reproducing expression. The Munich court took the opposite view, holding that using protected works in AI training without permission constitutes reproduction requiring a licence. This illustrates the growing divide between the U.S. model, where fair use can exempt AI developers from licensing duties, and the European approach, which treats copyright as an enforceable economic right demanding prior authorisation.

If you’re an AI engineer trying to understand and build with GenAI, RAG (Retrieval-Augmented Generation) is one of the most essential components to master. It’s the backbone of any LLM system that needs fresh, accurate, and context-aware outputs. Let’s break down how RAG works, step by step, from an engineering lens, not a hype one: 🧠 How RAG Works (Under the Hood) 1. Embed your knowledge base → Start with unstructured sources - docs, PDFs, internal wikis, etc. → Convert them into semantic vector representations using embedding models (e.g., OpenAI, Cohere, or HuggingFace models) → Output: N-dimensional vectors that preserve meaning across contexts 2. Store in a vector database → Use a vector store like Pinecone, Weaviate, or FAISS → Index embeddings to enable fast similarity search (cosine, dot-product, etc.) 3. Query comes in - embed that too → The user prompt is embedded using the same embedding model → Perform a top-k nearest neighbor search to fetch the most relevant document chunks 4. Context injection → Combine retrieved chunks with the user query → Format this into a structured prompt for the generation model (e.g., Mistral, Claude, Llama) 5. Generate the final output → LLM uses both the query and retrieved context to generate a grounded, context-rich response → Minimizes hallucinations and improves factuality at inference time 📚 What changes with RAG? Without RAG: 🧠 “I don’t have data on that.” With RAG: 🤖 “Based on [retrieved source], here’s what’s currently known…” Same model, drastically improved quality. 🔍 Why this matters You need RAG when: → Your data changes daily (support tickets, news, policies) → You can’t afford hallucinations (legal, finance, compliance) → You want your LLMs to access your private knowledge base without retraining It’s the most flexible, production-grade approach to bridge static models with dynamic information. 🛠️ Arvind and I are kicking off a hands-on workshop on RAG This first session is designed for beginner to intermediate practitioners who want to move beyond theory and actually build. Here’s what you’ll learn: → How RAG enhances LLMs with real-time, contextual data → Core concepts: vector DBs, indexing, reranking, fusion → Build a working RAG pipeline using LangChain + Pinecone → Explore no-code/low-code setups and real-world use cases If you're serious about building with LLMs, this is where you start. 📅 Save your seat and join us live: https://lnkd.in/gS_B7_7d

Ever wondered where the future of AI is being built? I just visited the data centre in Finland that's making it happen. Nebius’ data centre is the powerhouse where AI models are trained. Thousands of GPUs working in unison. It’s expanding to host up to 60,000 GPUs dedicated to intensive AI workloads. They’re building a full-stack AI cloud platform. Here’s what I learned: 1. There is a scarcity of GPUs in the US • Clusters are being sold in massive packages • People who need smaller requirements can’t find them 2. Nebius are building a self-serve platform • Cover infrastructure requirements from a single GPU to big GPU clusters • They’re not a GPU reseller—they’re designing the servers and the racks from the ground up 3. Applications • Helped Mistral train their multimodal models • Provide full-stack infrastructure for AI model development Something else that was unique about the visit. Nebius cools the servers in Finland using the outside air. The heat that’s generated from the servers is then shipped back into the grid. This means Nebius not only heats the onsite building, But it also heats homes nearby, benefitting the local community. They’re able to recover 70% of the heat generated. And it’s the first in the world to have this heat reuse application connected to the local municipal grid. They’re now investing over $1B in AI data centres in Europe. I feel the future of AI depends on infrastructure like this that balances performance with sustainability. Follow me Alex Banks for daily AI highlights & insights.

We have seen recently a surge in vector databases in this era of generative AI. The idea behind vector databases is to index the data with vectors that relate to that data. Hierarchical Navigable Small World (HNSW) is one of the most efficient ways to build indexes for vector databases. The idea is to build a similarity graph and traverse that graph to find the nodes that are the closest to a query vector. Navigable Small World (NSW) is a process to build efficient graphs for search. We build a graph by adding vectors one after the others and connecting each new node to the most similar neighbors. When building the graph, we need to decide on a metric for similarity such that the search is optimized for the specific metric used to query items. Initially, when adding nodes, the density is low and the edges will tend to capture nodes that are far apart in similarity. Little by little, the density increases and the edges start to be shorter and shorter. As a consequence the graph is composed of long edges that allow us to traverse long distances in the graph, and short edges that capture closer neighbors. Because of it, we can quickly traverse the graph from one side to the other and look for nodes at a specific location in the vector space. When we want to find the nearest neighbor to a query vector, we initiate the search by starting at one node (i.e. node A in that case). Among its neighbors (D, G, C), we look for the closest node to the query (D). We iterate over that process until there are no closer neighbors to the query. Once we cannot move anymore, we found a close neighbor to the query. The search is approximate and the found node may not be the closest as the algorithm may be stuck in a local minima. The problem with NSW, is we spend a lot of iterations traversing the graph to arrive at the right node. The idea for Hierarchical Navigable Small World is to build multiple graph layers where each layer is less dense compared to the next. Each layer represents the same vector space, but not all vectors are added to the graph. Basically, we include a node in the graph at layer L with a probability P(L). We include all the nodes in the final layer (if we have N layers, we have P(N) = 1) and the probability gets smaller as we get toward the first layers. We have a higher chance of including a node in the following layer and we have P(L) < P(L + 1). The first layer allows us to traverse longer distances at each iteration where in the last layer, each iteration will tend to capture shorter distances. When we search for a node, we start first in layer 1 and go to the next layer if the NSW algorithm finds the closest neighbor in that layer. This allows us to find the approximate nearest neighbor in less iterations in average. ---- Find more similar content in my newsletter: TheAiEdge.io Next ML engineering Masterclass starting July 29th: MasterClass.TheAiEdge.io #machinelearning #datascience #artificialintelligence

AI is already impacting 93% of U.S. jobs, and its effects are outpacing expectations by a factor of 4.5x. The broader question for leadership today is no longer if, but how, we channel this transformative power to benefit all levels of society and the economy. Two perspectives stand out. In our TIME article, "AI Should Belong to Workers," my coauthors and I explored how AI disrupts traditional hierarchies by democratizing intelligence. Unlike previous technology waves, AI doesn't demand specialized technical expertise for adoption. Frontline employees — from HVAC technicians to nurses — are using AI-driven diagnostics to expand their leverage and decision-making. When workers shape AI applications tailored to their tasks, value creation accelerates closer to the work itself, enabling better wage leverage and upward mobility. (read here: https://lnkd.in/eSwqbDpT) Our "New Work, New World" research, spanning nearly 1,000 professions, found AI exposure has accelerated well beyond predictions — average exposure scores jumped 30% in just three years, when models forecasted a decade. The yearly rate of jobs impacted by AI has skyrocketed from 2% to 9% annually, underscoring why leaders must design pathways that empower workers to harness this opportunity at every level. (read here: https://lnkd.in/ekWPUxQE) In the Newsweek article, "When Capital Can Think, Who Pays?", we examined AI's fiscal misalignment: digital agents that augment or replace workers contribute nothing toward payroll taxes sustaining Social Security, Medicare, and unemployment insurance, while human labor bears these costs. Temporarily rebalancing this burden — lowering it on labor, raising it on automation — creates an AI-driven economy that rewards augmentation over displacement, much like prior revolutions that generated the 60% of jobs that didn't exist 80 years ago. (read here: https://lnkd.in/eUvMvZMK) AI's potential to create value for the U.S. economy already exceeds $4.5 trillion. But unless enterprise adoption and wider societal architecture move in tandem, these gains could concentrate in narrow economic bands. Leaders today have a choice: manage AI passively, reinforcing the inequities new technologies could correct — or reimagine how intelligence, tasks, and rewards flow through our organizations. The most important innovation of the coming decade may not come from AI itself. It will come from the deliberate systems we create to amplify every worker's potential and ensure technological progress fuels enduring human value.

🚨 AI Privacy Risks & Mitigations Large Language Models (LLMs), by Isabel Barberá, is the 107-page report about AI & Privacy you were waiting for! [Bookmark & share below]. Topics covered: - Background "This section introduces Large Language Models, how they work, and their common applications. It also discusses performance evaluation measures, helping readers understand the foundational aspects of LLM systems." - Data Flow and Associated Privacy Risks in LLM Systems "Here, we explore how privacy risks emerge across different LLM service models, emphasizing the importance of understanding data flows throughout the AI lifecycle. This section also identifies risks and mitigations and examines roles and responsibilities under the AI Act and the GDPR." - Data Protection and Privacy Risk Assessment: Risk Identification "This section outlines criteria for identifying risks and provides examples of privacy risks specific to LLM systems. Developers and users can use this section as a starting point for identifying risks in their own systems." - Data Protection and Privacy Risk Assessment: Risk Estimation & Evaluation "Guidance on how to analyse, classify and assess privacy risks is provided here, with criteria for evaluating both the probability and severity of risks. This section explains how to derive a final risk evaluation to prioritize mitigation efforts effectively." - Data Protection and Privacy Risk Control "This section details risk treatment strategies, offering practical mitigation measures for common privacy risks in LLM systems. It also discusses residual risk acceptance and the iterative nature of risk management in AI systems." - Residual Risk Evaluation "Evaluating residual risks after mitigation is essential to ensure risks fall within acceptable thresholds and do not require further action. This section outlines how residual risks are evaluated to determine whether additional mitigation is needed or if the model or LLM system is ready for deployment." - Review & Monitor "This section covers the importance of reviewing risk management activities and maintaining a risk register. It also highlights the importance of continuous monitoring to detect emerging risks, assess real-world impact, and refine mitigation strategies." - Examples of LLM Systems’ Risk Assessments "Three detailed use cases are provided to demonstrate the application of the risk management framework in real-world scenarios. These examples illustrate how risks can be identified, assessed, and mitigated across various contexts." - Reference to Tools, Methodologies, Benchmarks, and Guidance "The final section compiles tools, evaluation metrics, benchmarks, methodologies, and standards to support developers and users in managing risks and evaluating the performance of LLM systems." 👉 Download it below. 👉 NEVER MISS my AI governance updates: join my newsletter's 58,500+ subscribers (below). #AI #AIGovernance #Privacy #DataProtection #AIRegulation #EDPB

If you’re building a career around AI and Cloud infrastructure ~ this roadmap will help map the journey. It breaks down the Cloud AI Engineer role into 12 focused stages: – Build a strong foundation in cloud platforms and Linux (it’s everywhere), and understand networking, storage, and core infrastructure concepts – Practice containerization and orchestration with Docker and Kubernetes to run scalable AI workloads – Provision infrastructure using Infrastructure as Code (Terraform, Ansible, cloud-native tools) and CI/CD pipelines – Understand AI/ML fundamentals including model architectures, training vs inference workflows, and distributed training concepts – Get familiar with GPU computing, CUDA, and NVIDIA GPU architectures used for AI workloads – Know how high-performance networking works for AI clusters using RDMA, GPUDirect, and optimized network fabrics – Know how to manage AI storage systems including object storage, NVMe, and parallel file systems for large datasets (and why storage can become a bottleneck) – Understand how to run AI workloads on Kubernetes with GPU scheduling, Kubeflow, and ML job orchestration – Learn how to optimize and deploy AI inference pipelines using TensorRT, Triton, batching, and model optimization techniques – Know how to build distributed training infrastructure for large models using NCCL, NVLink, and multi-node GPU clusters – Implement monitoring and observability for AI systems with GPU metrics, tracing, and performance profiling – Operate production AI systems with multi-cluster architectures, disaster recovery, and enterprise-scale AI infrastructure So if you’re building AI models but don’t understand the infrastructure behind them ~ this roadmap helps connect the dots. Resources in the comments below 👇 Hope this helps clarify the systems and skills behind the role. • • • If you found this insightful, feel free to share it so others can learn from it too.

AI is not failing because of bad ideas; it’s "failing" at enterprise scale because of two big gaps: 👉 Workforce Preparation 👉 Data Security for AI While I speak globally on both topics in depth, today I want to educate us on what it takes to secure data for AI—because 70–82% of AI projects pause or get cancelled at POC/MVP stage (source: #Gartner, #MIT). Why? One of the biggest reasons is a lack of readiness at the data layer. So let’s make it simple - there are 7 phases to securing data for AI—and each phase has direct business risk if ignored. 🔹 Phase 1: Data Sourcing Security - Validating the origin, ownership, and licensing rights of all ingested data. Why It Matters: You can’t build scalable AI with data you don’t own or can’t trace. 🔹 Phase 2: Data Infrastructure Security - Ensuring data warehouses, lakes, and pipelines that support your AI models are hardened and access-controlled. Why It Matters: Unsecured data environments are easy targets for bad actors making you exposed to data breaches, IP theft, and model poisoning. 🔹 Phase 3: Data In-Transit Security - Protecting data as it moves across internal or external systems, especially between cloud, APIs, and vendors. Why It Matters: Intercepted training data = compromised models. Think of it as shipping cash across town in an armored truck—or on a bicycle—your choice. 🔹 Phase 4: API Security for Foundational Models - Safeguarding the APIs you use to connect with LLMs and third-party GenAI platforms (OpenAI, Anthropic, etc.). Why It Matters: Unmonitored API calls can leak sensitive data into public models or expose internal IP. This isn’t just tech debt. It’s reputational and regulatory risk. 🔹 Phase 5: Foundational Model Protection - Defending your proprietary models and fine-tunes from external inference, theft, or malicious querying. Why It Matters: Prompt injection attacks are real. And your enterprise-trained model? It’s a business asset. You lock your office at night—do the same with your models. 🔹 Phase 6: Incident Response for AI Data Breaches - Having predefined protocols for breaches, hallucinations, or AI-generated harm—who’s notified, who investigates, how damage is mitigated. Why It Matters: AI-related incidents are happening. Legal needs response plans. Cyber needs escalation tiers. 🔹 Phase 7: CI/CD for Models (with Security Hooks) - Continuous integration and delivery pipelines for models, embedded with testing, governance, and version-control protocols. Why It Matter: Shipping models like software means risk comes faster—and so must detection. Governance must be baked into every deployment sprint. Want your AI strategy to succeed past MVP? Focus and lock down the data. #AI #DataSecurity #AILeadership #Cybersecurity #FutureOfWork #ResponsibleAI #SolRashidi #Data #Leadership

AECOM just dropped $390M on Norwegian AI startup CONSIGLI. Not Autodesk. AECOM. This changes the playbook for how AEC firms compete. For decades, firms have all been using the same few softwares. Innovation meant waiting for the next software update or buying another plugin. Now, we’re seeing companies like AECOM buying their own AI layer and building a competitive moat. Here's what makes this acquisition fascinating: The monolithic software vendors are losing their monopoly. When their biggest customers begin to supply their own software, there's real pressure to level up. Competition accelerates innovation. ⚡️ The margin pressure in AEC is forcing a new strategy. Owning an effective tech stack isn't luxury anymore - it's survival. AECOM goes even further by procuring AI that no competitor can access. This deal signals something bigger: the convergence of construction services and software. The firms that grow their AI stack won't just deliver projects; they'll deliver speed and depth that competitors literally can't match. The old model: Everyone uses Autodesk, compete on execution. The new model: Own your own technology, compete on capabilities others can't buy. $390M says AECOM believes proprietary AI is worth more than a decade of software licenses. So here's my question: If you're running an AEC firm today, what's your AI strategy - build it, buy it, or partner for it?