Dan Becker

Building agentic workflows? There are actually only a few key concepts to be aware of. StateGraph orchestration: A StateGraph is the agent’s blueprint. It consists of nodes that handle tasks with edges (directions) to other tasks, defining the order. Together, they provide the agent with a flow to follow. Conditional routing: At decision points, the agent inspects results and follows “if-else” statements that skip failures, handle edge cases, etc. Memory and threads: Thread memory holds recent conversation turns. Essentially, it stores the short-term memory that you are John Doe and were interested in the weather. Long-term memory stores user facts, letting the agent stay context-aware, reduce repetition, and personalize replies, such as that you have been a customer since 2018. Human-in-the-loop: Strategic hand-offs let people review low-confidence outputs, catch critical paths or errors, and feed corrections back to the model, keeping quality and alignment tight. If this still feels too abstract, I’ve included a more detailed explanation in the article below. https://lnkd.in/eAr-6YAQ

14

1 Comment

Your RAG isn’t underperforming because of the LLM. It’s underperforming because of what happens before the LLM ever sees the data. We’ve seen this pattern across almost every retrieval-augmented generation (RAG) project we’ve worked on. Often, the model’s fine, it’s the retrieval pipeline that needs work. When teams go beyond the demo phase of RAG and try to put it into production, three issues usually show up: 1. Irrelevant retrieval - The system keeps surfacing documents that are technically similar in embedding space but contextually useless. 2. Inconsistent answers - Minor query phrasing changes lead to very different results. 3. Latency tradeoffs - Adding complexity to fix 1 and 2 often makes the system too slow for real-time use. Over time, a few techniques have consistently made the biggest difference for us: - Domain-tuned embeddings. We often fine-tune embedding models on the client’s own corpus rather than rely on “top leaderboard” ones. That alone can lift precision significantly. - Hybrid retrieval. Combining vector search with metadata and keyword filters gives you much more control, especially for enterprise data. - Query rewriting. Automatically expanding and reformulating vague user queries improves hit rates without retraining anything. - Dynamic thresholds. Instead of always returning the top-k results, set similarity cutoffs that adapt based on the query. You don’t even need to implement all of this upfront. If you’re building your first serious RAG system, start by improving query formulation and metadata filters. These give the biggest jump in retrieval quality with the least engineering overhead. Once you’ve stabilized retrieval, then worry about model tuning or orchestration.

8

1 Comment

The "Data Wall" isn't a dead end. It’s just an optimization problem we were too lazy to solve. 🧱💥 I just read the new OPUS paper from the Qwen Team (Alibaba) & SJTU, and it fundamentally breaks the assumption that "Pre-training Data" must be static. For years, we've filtered datasets before training (using heuristics like Perplexity or Quality Classifiers like FineWeb-Edu) and hoped that the distribution would be useful for the entire run. OPUS (Online Principled Utility Selection) proves this is inefficient. Data that is valuable at Step 1,000 becomes useless noise by Step 50,000. The Engineering Breakdown: Dynamic Utility: Instead of static filters, it calculates the "Gradient Utility" of data during the training loop. The Math: It selects the specific subset of data that minimizes the approximation error of the full-gradient update. No Heuristics: It removes the need for manual "Curriculum Engineering." The model decides what is hard enough to learn from right now. The Results: It achieves 8x efficiency (reduction in computation) on GPT-XL compared to random selection and beats strong baselines like FineWeb-Edu and DSIR by +2.2% average accuracy. Why it matters: We are running out of high-quality public text. If we can get the same performance with 1/8th of the tokens by simply being smarter about which tokens we show the model, the "Data Wall" gets pushed back by a decade. Question: Do you think "Data Curation" teams will eventually be replaced by "Data Selection Algorithms," or is human intuition still needed for the seed set? #AIArchitecture #LLMPretraining #DataScience #Qwen #Alibaba #OPUS #MachineLearning #DeepLearning #Research #Efficiency

13

1 Comment

Frontier LLMs get the hype, but small specialized models deliver the results. After reading the WSJ article, I was reminded of a mistake I made early on. We burned through a lot of money running frontier LLMs on large datasets for sampling, thinking bigger automatically meant better. Only later did we learn that a small fine-tuned model could do the same job for a fraction of the cost. That experience changed how we build. Across multiple projects, we’ve consistently seen small task-focused models match or outperform the larger models. Fine-tuned domain models, narrow architectures and small model pipelines have given us better accuracy, lower latency, more privacy and far more predictable costs. Frontier models still have a role in deep reasoning and planning, but in real production environments, it’s the smaller specialized models that quietly do most of the heavy lifting. Curious if others have had a similar “we had to learn it the hard way” moment. #AI #LLM #DataScience #MLOps #AIStrategy #EnterpriseAI

28

5 Comments

Every AI engineer building agents in production faces the challenge of trading off agent autonomy and reliability -- in other words, determining how much "determinism" there should actually be in your agent. => The temptation to return to "workflow land" Often times, the solution for reliability is reverting "agents" back deterministic workflows that predefine the desired behavior. But when you hardcode your agents into a rigid graph, you're betting against the future. As models improve, your workflow-based agents can't evolve with them. You're stuck in what Kian Jones likes to call W.F.L ("workflow land"), while truly autonomous agents leap ahead by fully leveraging the LLM's intelligence. => An alternative approach: tool rules Instead of abandoning autonomy, what if we could add lightweight constraints that ensure reliability without sacrificing adaptability? That's where tool rules come in. Think of it like blocking off certain roads you know lead to dead ends or dangerous areas. The agent still chooses its own path, but you've removed the routes that could cause problems. Tool rules constrain an agent's tool-calling patterns, ensuring critical steps happen in the right order while preserving the agent's ability to navigate intelligently. As models become more capable, you can gradually relax these constraints. Your agent grows more autonomous over time, rather than being forever locked in a static workflow. Want to see me build an agent with tool rules? Tutorial link in comments 👇 #AI #AgenticAI #AIAgents #LettaAI #MemGPT #StatefulAgents

125

13 Comments

When building an agentic system to work with data, your math must *always* be right. Here's how we did it: Even with a calculator tool, the agent needs context to not screw up what math it *should* be doing. Reliably multiplying and dividing numbers is not enough if you don't remember which one is appropriate for your problem space! In our experiment, given a simple operation and a sequence of numbers on which to perform the operation, Claude Opus 4.5 succeeded 87% of the time. Given the same prompt, but instructed to remind itself of the rules each time it performs the operation, it succeeds 100% of the time. These numbers are across multiple trials with randomly-generated sequences of numbers. So really we're solving two problems: Preserving context for the calculation, and reliably doing the calculation without screwing up. The solution: The master agent has a sub-agent tool that considers the context before calling a calculator. That way, we can do our complex analysis, synthesize complex data across several sources ... and still reliably get the analysis right at the end!

19

4 Comments

Systems first. AI next. The order matters, a lot. Over the past few weeks, Evan Allen and I have been digging into agentic workflows - reading papers, testing tools, and mapping patterns across different implementations. One insight keeps standing out. Agentic workflows aren’t inherently “AI.” They’re decomposition systems. The core idea is simple: take a complex task, break it into smaller, well-defined steps, execute those steps independently, then reassemble the output.  Who performs each step is secondary. 👉 Some steps are best handled by machines: speed, recall, pattern matching. 👉 Others still benefit from humans: judgment, accountability. 💡 Once you see it this way, the false binary between “automated” and "manual” disappears. For human-in-the-loop systems - like what we're building with Nexus - it shifts the goal from replacing people to assigning work to whoever (or whatever) does it best, step by step. AI becomes a delegate, not an endpoint. That also forces more discipline. You have to be explicit about what the task actually is, what good looks like, and where judgment truly matters. Sloppy workflows don’t magically improve when you add AI - they just fail faster. I suspect the most durable systems won’t be the most automated ones, but the most intentional about where humans stay in the loop. Where have you seen the balance work well, or break down?

14

2 Comments

RAG is a lot tougher than it seems, we’re not quite at the point where long-context LLMs can accurately digest a large corpus of documents. Watching an enterprise data science team's new morning routine after implementing Pulse's end-to-end RAG tools is genuinely impressive. Instead of spending hours manually preparing financial reports, they now drop diverse documents into our system which automatically identifies and semantically chunks content - building a unified knowledge base that connects tables, footnotes, and narrative sections across multiple formats. The most transformative part is how analysts can now ask natural language questions spanning multiple quarters of data and receive contextualized answers with precise citations. Pulse is even handling complex multi-table relationships that previously required custom code for each new document format. 

34

7 Comments

Even though graphs are everywhere, from social networks to recommendation engines, they remain one of the most misunderstood data paradigms. While AI and connected systems cry out for structure, context, along with semantics, we continue to force relationships into flat tables and rigid joins. Why did graphs and more importantly graph databases become so confusing and how do we make them simple? Let's take a look: https://lnkd.in/gPb6vzbT

20