Xuedong D. Huang

New Course: ACP: Agent Communication Protocol Learn to build agents that communicate and collaborate across different frameworks using ACP in this short course built with IBM Research’s BeeAI, and taught by Sandi Besen, AI Research Engineer & Ecosystem Lead at IBM, and Nicholas Renotte, Head of AI Developer Advocacy at IBM. Building a multi-agent system with agents built or used by different teams and organizations can become challenging. You may need to write custom integrations each time a team updates their agent design or changes their choice of agentic orchestration framework. The Agent Communication Protocol (ACP) is an open protocol that addresses this challenge by standardizing how agents communicate, using a unified RESTful interface that works across frameworks. In this protocol, you host an agent inside an ACP server, which handles requests from an ACP client and passes them to the appropriate agent. Using a standardized client-server interface allows multiple teams to reuse agents across projects. It also makes it easier to switch between frameworks, replace an agent with a new version, or update a multi-agent system without refactoring the entire system. In this course, you’ll learn to connect agents through ACP. You’ll understand the lifecycle of an ACP Agent and how it compares to other protocols, such as MCP (Model Context Protocol) and A2A (Agent-to-Agent). You’ll build ACP-compliant agents and implement both sequential and hierarchical workflows of multiple agents collaborating using ACP. Through hands-on exercises, you’ll build: - A RAG agent with CrewAI and wrap it inside an ACP server. - An ACP Client to make calls to the ACP server you created. - A sequential workflow that chains an ACP server, created with Smolagents, to the RAG agent. - A hierarchical workflow using a router agent that transforms user queries into tasks, delegated to agents available through ACP servers. - An agent that uses MCP to access tools and ACP to communicate with other agents. You’ll finish up by importing your ACP agents into the BeeAI platform, an open-source registry for discovering and sharing agents. ACP enables collaboration between agents across teams and organizations. By the end of this course, you’ll be able to build ACP agents and workflows that communicate and collaborate regardless of framework. Please sign up here: https://lnkd.in/g4gES9CF

8,440

297 Comments

Learn to build your own voice-activated AI assistant that can execute tasks like gathering recent AI news from the web, scripting out a podcast, and using tools to put all that into a multi-speaker podcast. See our new short course: "Building Live Voice Agents with Google’s ADK (Agent Development Kit),” taught by Google’s Lavi Nigam and Sita Lakshmi Sangameswaran. ADK provides modular components that make it easy to build and debug agents. It also includes a built-in web interface for tracing agentic reasoning. This course illustrates these concepts via building a live voice agent that can chain actions to complete a complex task like creating a podcast. This requires maintaining context, implementing guardrails, reasoning, and handling audio streaming, while keeping latency low. You’ll learn to: - Build voice agents that listen, reason, and respond - Guide your agent to follow a specific workflow to accomplish a task - Coordinate specialized agents to build an agentic podcast workflow that researches topics and produces multi-speaker audio - Understand how to deploy an agent into production Even if you’re not yet building voice systems, you'll find understanding how realtime agents stream data and maintain reliability useful for designing modern agentic applications. Please join here: https://lnkd.in/ga6tD5rt

6,433

199 Comments

🎙️Granite 3.2 is Here! Key Takeaways from IBM’s Mixture of Experts Podcast 🎙️ In Episode 44 of the IBM Mixture of Experts podcast, I had the pleasure of joining Tim Hwang, Kate Soule, and Maya Murad to unpack some of the biggest AI announcements of the past week. From Granite 3.2’s launch to multi-agent AI, Anthropic’s Claude 3.7 Sonnet & Code, and the risks of AI misalignment. 🔷 Granite 3.2 – IBM’s Open-Source AI Just Got Stronger IBM continues its push for enterprise-ready open-source AI, balancing performance, governance, and transparency. ☑️ Instruct models (8B & 2B) with chain-of-thought reasoning ☑️ Granite Vision 3.2 (2B)—a multimodal vision-language model optimized for real-world enterprise tasks ☑️ Apache 2.0 licensing—making trustworthy AI accessible to businesses with strict compliance needs. 🐝 IBM's BeeAI – The Next Step in AI Agents? AI is moving beyond single, monolithic models (like ChatGPT) to multi-agent systems. BeeAI is leading this shift with: ☑️ Decentralized AI coordination—specialized agents dynamically collaborate instead of relying on one "supermodel." ☑️ Memory & learning improvements—agents retrieve insights and self-optimize over time ☑️ Error handling & self-correction solving common issues in agentic AI 🎮 Claude 3.7 Sonnet – Smarter, Cheaper, and Pokémon-Powered? Anthropic’s latest model is faster, more efficient, and cost-effective, challenging GPT-4 Turbo, but what really caught our attention? 🔹 Pokémon battles as AI benchmarks! Instead of static tests (like MMLU), this forces AI to: ☑️ Make real-time strategic decisions ☑️ Adapt to uncertain environments ☑️ Optimize long-term reasoning Anthropic's decision to release Claude Code separately ensures more reliable coding and debugging by preventing interference from general-purpose reasoning tasks. ⚠️ The Hidden Risks of AI Fine-Tuning A new study on "Emergent Misalignment" revealed a critical AI safety challenge: ☑️ Fine-tuning AI for software development inadvertently made it better at writing malicious code ☑️ AI skills aren’t isolated—improving one ability can unintentionally weaken safety mechanisms ☑️ AI alignment isn’t static—models evolve in unpredictable ways, requiring continuous monitoring https://lnkd.in/efW6KW9g Takeaways 💡 Granite 3.2: Open-source AI is maturing, setting new standards for enterprise trust and governance 💡 BeeAI: Multi-agent AI is becoming a real alternative to single-model AI assistants 💡 Claude 3.7: AI benchmarking is evolving—games and real-world simulations may be the future 💡 Fine-Tuning Risks: AI safety isn’t a one-time fix—we must rethink how we monitor and align models continuously 📢 Which of these AI trends excites (or concerns) you the most? Curious to see what you think in the comments! 👇 🔗 Watch the full episode here: https://lnkd.in/esxty-bp #MixtureOfExperts #MoEPodcast #IBM #podcasts #GenAI #ClaudeCode #AIAgents #LLM #AITrust #EnterpriseAI

68

In my series on Multi-agent design best-practices, this blog outlines several key design considerations and discusses how to implement a system where agents balance competitive and cooperative behaviors. This balance is essential for us to design a MAS that functions as a holistic system of agents. To achieve this I show how we can use game-theoretic structures that account for both individual incentives and collective outcomes.

50

2 Comments

How LLMs Really Work - Tools, Memory & Workflow Made Simple Most people use LLMs as black boxes. The real power comes when you understand the tools, memory, and workflows driving them Large Language Models (LLMs) like GPT-4, Claude, Gemini, and LLaMA aren’t magic black boxes. They’re complex ecosystems of tools, memory systems, and workflows and understanding them is the key to building the next generation of AI applications. » Inside the LLM Ecosystem → Popular Tools & Frameworks From prompt tools (PromptPerfect, FlowGPT) to vector databases (Pinecone, Weaviate, Qdrant), and fine-tuning with LoRA, PEFT, Hugging Face - these are the building blocks behind every serious LLM application. → Types of Memory in AI Agents LLMs don’t just rely on context windows. They simulate short-term, long-term, working, episodic, semantic, and procedural memory - making them more “agent-like” and adaptive. → LLM Workflow It’s not just input → output. It’s: 👉 Define use case 👉 Tokenize & embed inputs 👉 Prompt engineering (zero/few/CoT) 👉 Retrieval-augmented generation (RAG) 👉 Add memory (STM & LTM) 👉 Secure the system 👉 Deploy & scale → Agent Design Patterns Frameworks like ReAct, Plan-and-Execute, AutoGPT, and Toolformer are changing how AI agents think, act, and learn. » Where It’s Being Applied • Enterprise Knowledge Management → RAG-powered copilots surfacing policies & documents in seconds. • Healthcare → Clinical decision support with retrieval + memory of patient history. • Finance → Intelligent assistants that summarize filings, detect risks, and support compliance. • Software Engineering → Multi-agent frameworks (Planner + Coder + Reviewer) automating dev workflows. • Customer Experience → AI agents that understand context across past conversations for personalized support. » Why this matters • For developers → it’s your roadmap to mastering the LLM stack. • For enterprises → it’s the foundation for secure, scalable AI solutions. • For AI enthusiasts → it’s the bridge between theory and applied intelligence. → The future of AI isn’t just chatbots. The future of work isn’t humans vs AI. It’s Humans + LLMs + Agents + Memory + Tools, working together as the new operating system of business, i.e an autonomous system that reason, learn, and integrate deeply into business and life. → What’s your take? Which part of the LLM workflow will matter most in 2025 - Vector DBs, Memory Systems, or Agentic Workflows? Drop your thoughts ! credit:- Rajeshwar D. follow Devjyoti Seal for more insights

94

15 Comments

“There's Plenty of Room at the Bottom – By Richard Feynman” Introducing BitNet v2, the first native 4-bit activation for 1-bit LLMs (and for any LLMs). It represents a significant technical advancement in 1-bit LLMs, following the introduction of BitNet v1 in October 2023 and BitNet b1.58 in February 2024. Activation outliers present significant challenges to LLM efficiency for low-bit LLMs, particularly in cloud and batch environments. We tackled this issue by proposing an online Hadamard transformation before activation quantization. This technique smooths sharp distributions into more Gaussian-like forms, making them amenable to low-bit representation. Experimental results demonstrate that BitNet v2 trained from scratch achieves performance comparable to BitNet b1.58 at 8-bit activations. More importantly, training with native 4-bit activations yields minimal performance loss, offering substantial reductions in memory footprint and computational cost for batched inference. BitNet v2 is a local optimum for low-bit LLMs – 1.58-bit model weights and 4-bit activations. Following the introduction of BitNet b1.58 2B4T, we anticipate the development of more and larger native 1-bit LLMs. To truly revolutionize AI efficiency, we reiterate our call for custom systems and hardware specifically optimized for these models, advocating for the co-design and co-evolution of model architecture, systems, and hardware. To conclude, echoing Richard Feynman's famous and profound quote "There's Plenty of Room at the Bottom," which envisioned a future of manipulating matter at incredibly small scales, a similar principle may hold true as well for 1-bit LLMs. https://lnkd.in/gy59Wtrd

178

3 Comments

🚀 Docling MCP: Making Docling Agentic 🚀 The Docling ecosystem just got smarter. Docling MCP (Model Context Protocol) brings document intelligence to agentic workflows — enabling seamless conversion, processing, and generation of structured document data. 🧩 Overview Docling MCP builds on the powerful Docling library, which converts unstructured documents like PDFs into structured, machine-readable formats. It exposes this functionality as a set of tools that can be directly called by agents and client applications — making document reasoning truly plug-and-play. ⚙️ Key Features ✅ Conversion Tools — Transform PDFs into structured DoclingDocument JSON formats. ✅ Generation Tools — Create new documents in DoclingDocument form and export across multiple formats. ✅ Local Caching — Speeds up repetitive document access and processing. ✅ Flexible Sources — Handle local files or remote URLs effortlessly. ✅ Memory Management — Efficiently work with large documents. ✅ Robust Logging — Debug and monitor processing with built-in logging. ✅ RAG-Ready — Built-in Milvus integration for document upload and retrieval in Retrieval-Augmented Generation pipelines. With Docling MCP, you can now build agentic systems that read, understand, and generate documents — bridging the gap between traditional document processing and modern AI reasoning. - docling: https://lnkd.in/d4UT-6_2 - docling-mcp: https://lnkd.in/eFWZvHmD #Docling #OpenSource #DocumentAI #LLM #MCP #AIagents #RAG #IBMResearch

556

27 Comments

Gemini 3 benchmarks are impressive. But here’s what matters: Google built this on TPUs, not Nvidia chips. If they open commercial access at competitive pricing, the entire AI infrastructure market reshapes. Watch the hardware play, not just the model.

423

14 Comments

Our advisory board met on February 22 for the 2025 Q1 meeting, focusing on how we continue to thrive following Towards Data Science’s departure from Medium — a shift that left many of their writers and readers in confusion. The collective consensus was that our selectivity must be on par with or exceed that of Towards Data Science. We have found that Medium imposes a daily views cap for each publication. The more selective a publication is, the more views each accepted article naturally receives. While Medium does not publicly disclose publication statistics beyond follower counts, we are always transparent. Our latest acceptance rate stands at 29%. This month, we accepted 104 articles, 26 of which were Boosted — a record-high Boosting ratio of 25% — and 29 are featured in this newsletter. Prior to Towards Data Science’s departure, I noticed that some of the manuscripts we had rejected were published there, which reflects the high quality of our accepted articles. Enjoy your March reads! https://lnkd.in/edkBT3zv

19

◆ Anatomy of an AI Agent   What makes AI agents smart, reliable, and capable of solving complex tasks autonomously? → Memory Stores prior conversations, context, or learned data. ∽ Helps maintain continuity, improve personalization, and adapt over time. ∽ Types: Short-term (current task), Long-term (knowledge base). → Tools External APIs, code interpreters, browsers, or plugins the agent can use. ∽ Extend the agent's capabilities beyond language e.g., fetching real-time data, running code, or querying a database. → Action The actual step the agent performs after decision-making. ∽ Can include responding, triggering an API, sending emails, or initiating a workflow. ∽ Powered by reasoning, rules, or LLM-based decisions. → Planning Strategizes the optimal sequence of steps to achieve a user’s goal. ∽ Involves decomposition of complex problems, setting sub-goals, and adapting based on feedback. ∽ Often implemented using ReAct, CoT (Chain of Thought), or Tree-of-Thought models. 🌟 Follow the AIKaDoctor (Free AI & Data Science Resources)  channel on WhatsApp: https://lnkd.in/dCTCEKKc Follow Habib Shaikh For more such content.

162

14 Comments