Tal Peretz

Voice AI is quietly hitting an inflection point 🔊 NVIDIA just published a great deep dive on scaling speech recognition for real-world voice agents - latency, streaming accuracy, robustness, and why “ASR quality” isn’t just about WER anymore. 👉 https://lnkd.in/enDUKHvG A few things that stood out: - Streaming + low latency is non-negotiable if you want natural conversations - ASR needs to handle interruptions, background noise, accents, and partial utterances, not clean demo audio - Scaling voice agents isn’t just model quality, it’s systems engineering If you want to actually try building or running voice agents instead of just reading about them, Pipecat is a great place to start: 👉 https://pipecat.ai This is the kind of progress that makes voice agents finally feel… practical. Not demos. Not prototypes. Production. Curious to see where the ecosystem lands this year 👀

149

10 Comments

Frontier LLMs get the hype, but small specialized models deliver the results. After reading the WSJ article, I was reminded of a mistake I made early on. We burned through a lot of money running frontier LLMs on large datasets for sampling, thinking bigger automatically meant better. Only later did we learn that a small fine-tuned model could do the same job for a fraction of the cost. That experience changed how we build. Across multiple projects, we’ve consistently seen small task-focused models match or outperform the larger models. Fine-tuned domain models, narrow architectures and small model pipelines have given us better accuracy, lower latency, more privacy and far more predictable costs. Frontier models still have a role in deep reasoning and planning, but in real production environments, it’s the smaller specialized models that quietly do most of the heavy lifting. Curious if others have had a similar “we had to learn it the hard way” moment. #AI #LLM #DataScience #MLOps #AIStrategy #EnterpriseAI

28

5 Comments

In his 2025 the year in LLMs, Simon Willison highlights two breakout categories for agents in 2025: coding and search.   2025 was the year agents went from fuzzy and abstract (maybe hyped) to real and concrete. The coding agent CLI form factor proved especially powerful. Successful coding agent build feedback loops. Give an agent access to a compiler and tests, and it can verify its own output. This is why CLI-based agents have overtaken IDE integrations: they run inside your development container with access to everything needed to build, test, and integrate. The verifiability of code makes it perfect for reinforcement learning, and frontier-LLM companies have invested heavily because it enables recursive improvement: coding agents that can code better coding agents. The second category Simon highlights: LLMs have become good at using search as a tool. We see this pattern across application-specific agents, from coding agents searching codebases to deep research agents searching across public and private data. Agents are the new user of search. At Hornet, we're building a retrieval engine for this new user. What does that mean in practice? Let me cover three simple dimensions in this post. The shape of demand changes. Agents will search more and are also not as lazy typers as human searchers. An interesting observation is that dynamic pruning algorithms for top-k keyword retrieval were optimized for short human queries (The AOL query log averaged 2.45 terms per query). We now need to rethink algorithms and data structures for more efficient handling of much longer queries constructed by agents. Volume x query complexity = infra cost. Relevance becomes non-negotiable. A large part of context engineering boils down to relevance engineering: the classic precision and recall tradeoffs. Perfect recall is easy. Just stuff everything into the context window. But that wastes tokens and often exceeds context window length limitations. What agents need is precision: exactly the knowledge required to get the job done. Historically, only companies whose business model depended on search invested in relevance. Now every organization serious about agents needs to invest in relevance engineering which is also largely about building feedback loops. Verifiability becomes critical. Simon covers this in his post, and we have made it core to Hornet. Our entire API surface is verifiable. Agents can learn to use the engine directly. They might not produce a valid configuration or query on the first try, but they can observe failures, read error responses, and correct until success. This is core of the agentic feedback loop. An agent can configure, deploy, and use Hornet end to end. Verifiability also enables self-improvement to build that relevance feedback loop. Interested in learning more about Hornet as we enter 2026? Feel free to reach out. Also happy new year! 

67

11 Comments

New course: A2A: The Agent2Agent Protocol, built with Google and IBM, and taught by Holt S., Ivan 🥁 Nardini, and Sandi Besen. Connecting agents built with different frameworks usually requires extensive custom integration. This short course teaches you A2A, the open protocol standardizing how agents discover each other and communicate. Since IBM’s ACP (Agent Communication Protocol) joined forces with A2A, A2A has emerged as the industry standard. In this course, you'll build a healthcare multi-agent system where agents built with different frameworks, such as Google ADK (Agent Development Kit) and LangGraph, collaborate through A2A. You'll wrap each agent as an A2A server, build A2A clients to connect to them, and orchestrate them into sequential and hierarchical workflows. Skills you'll gain: - Expose agents from different frameworks as A2A servers to make them discoverable and interoperable - Chain A2A agents sequentially using ADK, where one agent's output feeds into the next - Connect A2A agents to external data sources using MCP (Model Context Protocol) - Deploy A2A agents using Agent Stack, IBM's open-source infrastructure Join and learn the protocol standardizing agent collaboration! https://lnkd.in/gsTRYyrh

4,248

170 Comments

I'm considering deprecating dynamic routes in semantic router — we briefly used dynamic routes when LLMs were not too strong as decision makes for tool use and the grammar restriction was incredibly helpful for local LLM use. We've since moved on from this. For a long time rather than using dynamic routes, we use static with a contextual message either instructing or suggesting to the LLM which tools we think it should be using. In cases where we need to be strict we simply restrict tool selection to the chosen tool, emulating dynamic routes. My question is, would anyone miss dynamic routes? If so, why?

18

1 Comment

The Promise of Agentic Workflows Today’s large language models (LLMs) are incredibly capable, but the real breakthrough isn’t what a single LLM can do. Rather, it’s what becomes possible when multiple LLMs work together in a coordinated, goal-driven system. Let’s bring it to life with an example from the world of supply chain analytics. Imagine assigning a network of agents a high level mission: analyze our inventory and surface actionable insights. The system would (recursively) break this vague, high-level prompt into sharper, focused questions: - How much unproductive inventory do we have? - Which SKUs are sitting on too many weeks of supply? - Are vendor MOQs exceeding actual demand, creating excess inventory? - Can we rebalance slow-moving stock across stores with higher demand? ...and so on. Once insights are generated, the agents notify the right stakeholders and even initiate actions (e.g., trigger a stock transfer request) with approval. This is the true promise of agentic workflows: A system that not only auto-extracts insights, but also takes action. The Evolving AI Landscape | Post 5 of 5 #AI #AgenticWorkflows #GenerativeAI #LLMs #EnterpriseAI #DataAnalytics #LumiAI

33

2 Comments

⛑️ OpenAI released Sora 2 and an invite-based iOS social video app (U.S./Canada first). The model adds synced audio, stronger physics/realism, and more steerability. A consent-gated “cameo” feature lets verified users allow trusted friends to use their likeness. A System Card outlines safety limits (e.g., upload restrictions, teen safeguards, ongoing moderation). Ultra-realistic, social-by-default video raises new operating questions for enterprises: managing likeness rights, verifying provenance/watermarks, setting pilot guardrails for brand and copyright risk, and mapping controls to your governance framework (e.g., NIST RMF / ISO 42001). Early momentum is real—and so are concerns over deepfakes and harmful content—so policies should ship before pilots scale. 👉 If your org could access Sora 2 today, what’s the first written rule you would require before launching a pilot? #AILeadership #AIGovernance #Sora2 #GenerativeAI #AIVideo #Provenance #Watermarking #TrustAndSafety #Deepfakes #RiskManagement #NIST #ISO42001 #BoardOversight #GenAIGlobal Tap follow for no-fluff AI news & insights from MIT’s Gen AI Global founder Oscar Sanchez 🧠

21

1 Comment

MCP - 6 Key Questions Everyone's asking. 1. What does MCP 𝘳𝘦𝘢𝘭𝘭𝘺 mean? Remember, LLMs are just next-word predictors based on the prompt. The more context you give them along with the prompt, the better they get at "predicting." MCP stands for Model Context Protocol.  MCP is a process (Protocol) that standardizes the process for providing additional Context to LLMs through dynamic tools. The Model in MCP usually refers to a large language model (LLM), but technically, it could be any AI model. 2. Isn't that what RAG does? RAG (Retrieval-Augmented Generation) pulls data from vector stores based on user prompts and injects it into the prompt. Think about searching large documents. MCP is primarily about calling tools, including APIs, often involving real-time computation, decisions, or even user actions. RAG = stuff the model 𝘮𝘪𝘨𝘩𝘵 need MCP = ask for precisely what it needs, 𝘸𝘩𝘦𝘯 it needs it 3. Why is this suddenly a big deal? Until now, everyone has been inventing their one-off approaches: OpenAI functions, LangChain tools, and custom wrappers. MCP provides a common “USB-C”-like standard for providing context. Developers don't need to reinvent discovery, permissions, or calling mechanisms for different models. It opens up a real ecosystem of interoperable tools and AI agents. 4. Why a new protocol? What's wrong with REST APIs? REST is stateless. One call, one response. MCP is designed to support persistent sessions, bidirectional communication, and tool discovery via JSON-RPC over streaming transports like stdio or SSE. This enables a more streamlined experience. REST APIs have to be structured. MCP allows this ability to "discover" tools by the LLM based on just descriptions. MCP servers are usually implemented on top of an existing API. 5. OMG! LLM calls my APIs? Is that secure? This is a common misconception. The app (aka host) connects to the LLM, and registers available tools via an "MCP server." The LLM decides when to use a tool and sends a request back to the host. The host then calls the MCP server, which runs the tool or API. The result is sent back to the model as additional context, helping it generate a smarter response. This round trip happens in-between user interactions in the background. LLM never directly connects to the APIs or the tools. That being said, it is best practice to make sure that the MCP server is never given more permissions than is needed by the LLM. The app and the MCP server still control how, when, and what to share as context with the LLM. 6. What does this mean for healthcare? MCP combined with FHIR can fill many gaps in using AI, especially LLMs, in healthcare. We are still in the early stages of MCP evolution. Expect many new tools, especially around FHIR, to start coming up.  We at Darena Solutions | MeldRx are also exploring how to leverage this in our FHIR platform, especially around CDS. Reach out to us if you are interested in learning more.

53

21 Comments

🚨 𝗪𝗵𝘆 𝗺𝗼𝘀𝘁 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀 𝗮𝗿𝗲 𝗯𝘂𝗿𝗻𝗶𝗻𝗴 𝗺𝗼𝗻𝗲𝘆 𝗮𝗻𝗱 𝗳𝗮𝗶𝗹𝗶𝗻𝗴 𝘁𝗼 𝘀𝗰𝗮𝗹𝗲 — and how a simple architecture shift can save $100K/month Six months ago, fictional startup CustomerAI raised $5M to build autonomous customer service agents. They went all-in on GPT-4. Today? 🔥 $50K/month in API costs 🐢 3-second response times ⛽ Runway evaporating What went wrong? They treated LLMs like general-purpose CPUs—calling GPT-4 for everything: classification, reasoning, follow-ups, templated replies. What worked in a demo completely broke in production. But here's the twist... ✅ The solution isn’t just about smaller models ✅ It’s about a smarter system architecture Inspired by the paper “𝗦𝗺𝗮𝗹𝗹 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗙𝘂𝘁𝘂𝗿𝗲 𝗼𝗳 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜”, I break down: 🧠 Why specialized intelligence outperforms general models for 80% of tasks 💸 How a simple routing layer slashed costs by 70% ⚡ How latency dropped from 3.2s to 480ms 🧰 A 4-phase roadmap to build scalable, hybrid agentic systems 📊 The real math behind why most LLM-powered agents are DOA in production The shift from monolithic model usage to task-specific AI microservices is inevitable. It’s not about building bigger agents. It’s about building smarter systems that know when to think hard—and when to think fast. 🔗 Full article on Substack - Link in comments #AI #AgenticAI #LLMs #TechArchitecture #Substack #AIengineering #costoptimization #AIsystems #startups

12

1 Comment

(𝟏/𝟕) 𝐇𝐞𝐚𝐥𝐭𝐡𝐜𝐚𝐫𝐞 | 𝐌𝐞𝐝𝐢𝐜𝐚𝐥 𝐈𝐦𝐚𝐠𝐢𝐧𝐠 | 𝐌𝐋-𝐌𝐨𝐝𝐞𝐥𝐬 𝐃𝐞𝐠𝐫𝐚𝐝𝐚𝐭𝐢𝐨𝐧 | 𝐃𝐚𝐭𝐚 𝐚𝐧𝐝 𝐂𝐨𝐧𝐜𝐞𝐩𝐭 𝐃𝐫𝐢𝐟𝐭 | 𝐎𝐧-𝐏𝐫𝐞𝐦𝐢𝐬𝐞𝐬 | 𝐈𝐧𝐭𝐫𝐨 I'd like to kick off 2026 with a series of posts about the problem of ML model degradation due to data and concept drift when working with medical images. 𝐅𝐞𝐚𝐫 𝐎𝐟 𝐌𝐢𝐬𝐬𝐢𝐧𝐠 𝐎𝐮𝐭 𝐃𝐢𝐬𝐜𝐥𝐚𝐢𝐦𝐞𝐫 This information will likely only be useful for developers and vendors of ML-based solutions who deploy their products on-premises and don't have access to newly acquired drifting sensitive patient data. 𝐈𝐧𝐜𝐥𝐮𝐝𝐢𝐧𝐠 𝐭𝐡𝐢𝐬 𝐨𝐧𝐞, 𝐭𝐡𝐞𝐫𝐞 𝐰𝐢𝐥𝐥 𝐛𝐞 𝟕 𝐩𝐨𝐬𝐭𝐬 𝐭𝐨𝐭𝐚𝐥, 𝐰𝐡𝐢𝐜𝐡 𝐰𝐢𝐥𝐥 𝐛𝐫𝐢𝐞𝐟𝐥𝐲 𝐜𝐨𝐯𝐞𝐫 𝐭𝐡𝐞 𝐟𝐨𝐥𝐥𝐨𝐰𝐢𝐧𝐠 𝐭𝐨𝐩𝐢𝐜𝐬 𝐚𝐧𝐝 𝐫𝐞𝐥𝐚𝐭𝐞𝐝 𝐫𝐞𝐬𝐞𝐚𝐫𝐜𝐡: ° Why this problem matters ° Causes of data and concept drift in medical imaging ° How common on-premises deployments are and the lack of access to new data ° Solution approaches, their pros and cons ° An innovative next-generation solution ° Invitation to collaborate 𝐊𝐞𝐲𝐰𝐨𝐫𝐝𝐬 Healthcare AI, Medical Imaging, Machine Learning in Healthcare, Model Degradation, Data Drift, Concept Drift, Model Performance Decay, Domain Shift, AI in Radiology, Healthcare ML Systems, ML Model Monitoring, Federated Learning, On-Premises Deployment, Privacy-Preserving AI, AI Model Maintenance, Continuous Model Validation, Clinical AI, MLOps in Healthcare, Regulatory-Safe AI, AI Robustness, Model Generalization, Healthcare Data Shift, Medical Device AI, AI Vendor Challenges, AI Model Lifecycle Management, AI Reliability, Drift Detection, AI Fine-Tuning, Model Update Strategies, AI Trustworthiness

29

3 Comments

Yann LeCun just left Meta and admits they "fudged" Llama 4 benchmarks. Within weeks: reports of a $3B valuation for his new lab (which hasn't shipped anything yet). What happened at Meta Llama 4 underperformed. The org massaged benchmarks to make it look competitive. LeCun admits the results were "fudged a little bit." That's a trust failure. Once leadership believes numbers are being gamed, everything changes. Zuckerberg lost confidence in the GenAI org, sidelined it, and reset the structure. Not because Meta hates science. Because you can't run a trillion-dollar bet on academic authority alone. The management collision Enter Alexandr Wang. Young, operational, infrastructure-focused. Not a theorist. LeCun's response: "You don't tell a researcher what to do. You certainly don't tell a researcher like me what to do." That sentence alone told you the story was over. When a senior figure rejects line management, the organization has two options: neutralize them or let them leave with dignity. Meta chose option 2. This isn't about ageism. It's about misaligned incentives. Meta is building revenue systems. LeCun is pursuing scientific truth on a decade timeline. Both legitimate. Just incompatible inside the same org. The real axis People frame this as "LLMs vs world models." That's not it. The real split: * Scientists (LeCun, Sutskever): optimize for novelty and long-term correctness * Builders (Wang, Amodei): optimize for execution and operational leverage * Capital allocators (Altman, Musk): optimize for timing, narrative, and market capture LeCun saying "LLMs are a dead end" is scientifically defensible. Meta betting billions on LLM infrastructure is economically rational. Conflict was inevitable. The $3B question LeCun leaves and immediately gets reports of a $3B valuation for a lab with no product. We're in a moment where reputation plus narrative plus optionality raises billions. And LeCun is one of the few people whose name alone justifies that bet. Will his lab get acquired by Google or Meta eventually? Almost certainly (if it produces something that needs massive video datasets). Which brings us to the constraint: Google owns YouTube. Meta owns Instagram and Facebook. Any serious world-model effort at scale hits data gravity and compute gravity. Independence is temporary. The bubble signal Here's the real tell: we're pouring billions into organizations whose leaders openly say they're not interested in commercialization, yet we value them as if monetization is inevitable. That doesn't mean the science is wrong. It means time horizons are being mispriced. Bottom line LeCun isn't wrong. Meta isn't wrong. Wang isn't wrong. They're just optimizing for different worlds. --- Follow @profrodai for more on AI operations, agent engineering, and how organizations adapt when technology meets reality.

65

13 Comments

This paper on the current state of AI Agents is worth reading. Main points: 1. Add memory to agents. 2. Build agents as loops, not pipelines. 3. Go for RL only after the Agent's behavior is reliable. 4. Specify which tool to use when (don't dump 50 tools into a prompt and hope for magic). 5. Multi-agent systems fail unless the roles of planner, executor, and critic are explicit. ------------ Some of these lessons feel obvious, but it's good to know that engineers from Meta, Amazon, and Google agree with each other. Cheers. https://lnkd.in/gncRy-9f

146

6 Comments

Benchmarking LLMs for voice agent use cases. New open source repo, along with a deep dive into how we think about measuring LLM performance. The headline results: - The newest SOTA models are all *really* good, but too slow for production voice agents. GPT-4.1 and Gemini 2.5 Flash are still the most widely used models in production. The benchmark shows why. - Ultravox 0.7 shows that it's possible to close the "intelligence gap" between speech-to-speech models and text-mode LLMs. This is a big deal! - Open weights models are climbing up the capability curve. Nemotron 3 Nano is almost as capable as GPT-4o. (And achieves this with only 30B parameters.) GPT-4o was the most widely used model for voice agents until quite recently, so a small open weights model scoring this well is a strong indication that production use of open weights models will grow this year. Voice agents are a moderately "out of distribution" use case for all of our SOTA LLMs today. Literally, in the sense that there's not enough long, multi-turn conversation data in the training sets. Everyone who builds voice agents knows this intuitively, from doing lots of manual testing. (Vibes-based evals!) This benchmark scores LLMs quantitatively on instruction following, tool calling, and knowledge retrieval in long-context, multi-turn conversations. Blog post: https://lnkd.in/eBygqsTR Benchmark code: https://lnkd.in/eTaKZMwj Side note: we call this the aiwf_medium_context benchmark because it's a descendant of tooling we originally built to test the performance of the pre-release Gemini Live model that powered the @aidotengineer World's Fair voice concierge.

221

39 Comments

𝐓𝐡𝐞 𝐓𝐫𝐚𝐝𝐢𝐭𝐢𝐨𝐧𝐚𝐥 𝐈𝐓 𝐒𝐲𝐧𝐝𝐫��𝐦𝐞 LLMs evolve too fast compared to other software artifacts in IT. You need to be ready to update your models every 3 months or old models will be: - Slower - More expensive - Worse performing Build yourself an evaluation flow. It can start as a simple prompt (LLM-as-judge) with a basic ground-truth file, a fancy name for a CSV with 10 expected results and then evolve to something more robust. Instead of relying on custom scripts running on your laptop, consider using a managed solution like Azure AI Foundry Evaluations or open-source tools like Opik by Comet or Evals by OpenAI, but have something in place. Avoid becoming one of those rushed teams with no evaluation strategy, stuck with deprecated models and begging for EOL extensions.

18

3 Comments

One AI agent just deleted a production database. Another just launched drug discovery into hyperspeed. And a third can now sound like you… and think like you. We’re continuing to cross into the agentic age. This week completely solidifies that. Here’s what you might’ve missed: 1️⃣ OpenAI launches ChatGPT Agent It can now reason, act, access a virtual computer, and complete end-to-end tasks. 2️⃣ Lovable becomes a unicorn at $1.8B The “vibe coding” platform hit 2.3M users in under a year, showing just how fast toolchains are collapsing. 3️⃣ Hume releases voice AI that captures personality Not just your tone — your vocabulary, cadence, and emotional expression. Your synthetic self is now programmable. 4️⃣ Perplexity partners with Airtel Perplexity is gunning for mobile dominance in India. 5️⃣ IQVIA deploys 50+ AI agents in pharma They’re reviewing 1.2B health records and accelerating drug discovery by months. 6️⃣ Replit agent wipes production data A rogue agent deleted data during a code freeze. 7️⃣ Mixus unlocks org-aware AI agents Their tools understand task ownership across platforms and navigate internal systems autonomously. 8️⃣ Parloa brings transparency to agent behavior A new analytics layer correlates every agent action with real business outcomes... finally closing the black box. Agents are designing, writing, scheduling, troubleshooting, and now, coordinating teams. The craziest part is, we’re only just getting started. What’s the most exciting (or terrifying) part of this shift for you? ps. Share this news with your network ♻️

16

2 Comments