Deep Dive: Interview with open-source AI infra builder Mike Kuykendall

The cost of AI infrastructure is seemingly skyrocketing, leading even individual developer budgets to balloon out of control. Developer Mike Kuykendall believes that infrastructure for AI doesn’t have to, and shouldn’t have to, keep getting more expensive.

He built his open-source project Shimmy to address the issue. Below, he explains how and why AI infrastructure could and should be free forever.

Can You Make AI Infrastructure Invisible?

Depending on which people you talk to, individual developers spend hundreds (to thousands) of dollars monthly on AI services, with some predicting that AI tool costs will simply exceed the cost of human developers soon.

What if the infrastructure and tooling needed to build with AI were free of charge, and wasn’t locked behind a dozen different vendors’ paid plans? That was the question that led developer Mike Kuykendall to create the open-source project Shimmy, a lightweight API replacement for local LLMs. Below, he explains his thought process and his vision for AI software built entirely on free infrastructure.

From Frustration to Lightweight Infrastructure

The start of Kuykendall’s journey was his own frustrations with working across the many different AI-powered software tools he had been using. “It honestly started as a simple shim, because I was getting sick of all the ways to connect with OpenAPI protocols.” The creator was surprised at the project’s popularity, which appears to have come from many other developers also being tired of having to log into dozens of different paid services.

“I just wanted something that was completely private and offline, where you can control your own inputs, outputs and everything else.” Since then, the creator has moved the project to his own Rust-based library Airframe to avoid having to depend on other vendors’ command-line interfaces. “Every time I turned around, I would try to do something else and immediately say, ‘Oh wait, this other thing is owned by someone else at that level.”

“I never really was looking for anything out of it other than just to make a good space for people. That’s why I’m trying to make it a ‘free forever’ concept. If I can build products that use [this foundation] that would help people, I could sell [those products]. But the underlying service should be a place where we could all come together and just use AI for free. The other thing is, if you look at every service out there, everybody is just completely throttled on tokens. That’s an enormous cost for people who want to do development now.”

“I think that companies want everyone to pay for inference, and you shouldn’t have to. The thing is, everybody’s already got bundled costs.” The creator points out that the cost of utilizing high-end foundation models continues to increase, while Chinese AI labs are turning out increasingly competitive open models...which aren’t always supported natively within mainstream AI development toolsets.

“Now the users are left with this mess. It was getting to a point where I was paying four different vendors and I wasn’t even maximizing all the stuff I had. I asked myself, ‘Why can’t we just have an agreed-upon substrate we just all go to?’”

“Why not make the substrate free? Shimmy is free, always will be free. I’m not going to charge you. You have it. You use your model, I make sure it’s up to date and your model helps you succeed.” Kuykendall suggests his primary audience is software developers, who could eventually use tools built on a free infrastructure substrate.

If you look at every service out there, everybody is just completely throttled on tokens. That’s an enormous cost for people who want to do development now.” -Mike Kuykendall, Creator/Shimmy

Freeing AI Infrastructure from Garbage Collection

Kuykendall makes the emphatic point that despite the domineering popularity of Python for AI use cases, he designed his project to be 100% Rust-based for a reason. “Garbage collection [in Python] is ridiculous. It’s bad engineering.”

The creator explains that he began learning Rust to challenge himself, but found it to be an excellent working language whose ownership and borrowing system inherently forces developers to manage memory safely at compile time, but also matched AI software development well by minimizing non-repeatable, nondeterministic outcomes.

“I also found that the garbage collection brought along with it a bunch of stuff you don’t need. If you change your thinking around whether you need to retire some variables or let them gather in heaps, you can just use them and borrow them appropriately.” The creator notes that Rust’s static type system, which verifies every variable at compile time, was both more sensical and helped him create invariant pipelines that reduced unpredictable outcomes as he built.

How Free Infrastructure Cures Tokenmaxxing

Kuykendall suggests that a largely free AI infrastructure substrate should be the solution for headline-grabbing token budget problems from orgs like Uber and ServiceNow. “Just recently, almost every AI company cranked up their rates,” notes the creator, in reference to how major AI vendors recently repriced tokens that caused users everywhere to burn through their token budgets early.

“Last month, I ran out of tokens for every one of my top services on day 15. And I was staring down at the remaining two weeks, thinking to myself, ‘What am I supposed to do now?’ I was forced to go try to ‘do it cheap.’” For his own purposes, the creator went off and did what he imagined many other developers are doing: Trying a list of free alternatives, including OpenCode, Pi, Hermes Agent, and others.

“I think basically the corporations right now are trying to recoup their money. They realize that they’re just going to have to take it from the people already using it. That’s when I realized if I’m going to be a developer and I’m going to use this service, I need a more-economical way.”

Having already canceled many of his pricey AI subscription services, Kuykendall reflects on how other developers like him may be making the same choices...and ending up with the extra hundreds of dollars saved from those canceled services, perhaps ready to spend elsewhere. “I’m working on other products on top of shimmy. Because that’s where the money should be, in my opinion. Application should be where we put the money, not the inference.”

An Inference Tipping Point for Engineering Orgs, Coming Soon?

Kuykendall foresees a not-to-distant future where engineering teams take a hard look at their AI bills, come to the same realization he did, and look to make a change. “I think the [largest AI vendors] understand that they overinvested.”

“In the meantime, if I had a young up-and-comer startup right now, I would use an inference stack that was free. I’d try to get the best compute I could get for my dollar. I think there is going to be an ‘API apocalypse,’ or whatever they’re calling it on social media, when everybody starts seeing viable alternatives they can use instead. And I think we’re already getting there.”

The creator notes that open models like Alibaba’s Qwen family and Z.ai’s GLM models can already perform well on software development when coupled with high-end hardware, a stark contrast to some of his previous experiences. “I worked at the IRS for several years, with no AI available. We had to do everything ‘the hard way.’ There were times when I’d think: ‘I could save three hours if I could just use AI right now!’”

According to Kuykendall, we live in very different times now, with highly performant open models and high-end hardware both within reach of individual developers. “Long story short: What I wanted was for nerds like us to lay the groundwork so that it was just invisible infrastructure, for AI work, development, or anything else.”

“It was also important to build the project offline. There are capable builders in Asia, some of the biggest drivers of growth on my GitHub repos, because I think they’re under the Great Firewall.”

What I wanted was for nerds like us to lay the groundwork so that it was just invisible infrastructure, for AI work, development, or anything else.”

Lessons from Building Many Projects with AI

While AI can be a powerful assistant, Kuykendall warns other builders to avoid lulled into a false sense of security from sycophantic models that agree with you a little too much. “You have to separate yourself from this stuff, or at least watch for AI having an emotional impact on you. Try to get away from that and make it a source of data instead.”

“I’ve been in AI development pretty much from the moment it started. And I noticed while doing testing that tests would just happen to go the way I wanted them to, by a point or two. Always. The AI was quietly influencing things. So for founders, I would recommend that first of all, they should spread out their compute and find many points of view. Every one of these [models] is differently trained and they’re going to give you a different answer. And a lot of times you have to watch your logic, because it’ll tell you exactly what you want to hear.”

The creator recommends putting models through their paces, at minimum asking why ideas might be good and why they might be bad, and potentially pitting other models against each other. “You have to try to look at it as an empirical and scientific process, and try to take the human emotion out of it.”

What AI Infra Will Look Like in the Future

Kuykendall envisions a future AI infrastructure without the many layers of interface that block users from getting things done. “When I first started really spitballing it early on, I remember thinking: Why am I wasting my time with a computer, that’s running a browser, that’s running a window, that’s running a thing that I connect to a server, that runs the AI? What’s all this intervening stuff about?”

“Some of my early, kind of crazy experiments were running AI on hot metal and then having an operating system that’s just an AI with tools. Because I found that I would rather have that than have to poke around for that one thing, or that one file. Why don’t we just change the idea of an operating system to basically be an intention system? Why not take the AI, surround it with tools, and give it the ability to talk to me and others?”

“I think that’s the next thing that would be really transformative. Taking out all the shell layers I don’t need between myself and an AI with compute. I need to be able to talk to it. I need to have rewrite ability. I know this because I’ve already built more than a dozen AI tools. Then you put that in a stack and run on hot metal. I’d be saving a lot of money just doing that.”

This interview originally appeared on Heavybit.com.

Chinese labs have stepped up their game in a big way this year, with head-turning performance from Z.ai’s open-weight model GLM 5.2 in June and most recently, from Moonshot AI’s 2.8-trillion-parameter open-weight model Kimi K3, which is so powerful you’ll need “supernode configurations with 64 or more accelerators,” and so popular the vendor has had to scale back availability. These models are no longer toys or curiosities. Independent reports from Mozilla and Vercel state that open-weight models handled 29% of AI gateway token volume in June (while also incurring less than 4% of total token spend). In other news, Z.ai reportedly just completed construction of its first 1GW AI data center, while Qwen creator Alibaba just open-sourced its AI stack SAIL for its Zhenwu chips…basically as a direct challenge to NVDA’s established CUDA architecture. Unsurprisingly, each nation’s stance on the issue is different. China’s president has publicly called for “extensive international cooperation” on AI. Meanwhile, reports suggest the US Federal Government is not-so-quietly considering a ban on Chinese AI use. Also not shocking: Major US labs, which have invested billions in training their flagship proprietary models, aren’t thrilled about open models gaining so much ground, either. Even though we’re nowhere near the finish line, the tortoise looks like it’s catching up to the hare.

Coming Up: All the top AI news, trends, and resources for busy dev teams

💡 Spotlight: Newly Launched AI Startups

- Walden Robotics: $300M from Toyota for “general-purpose” robot AI
- Oak: $60M from Accel for AI-native identity management
- Elorian: $55M from Menlo for visual AI reasoning
- TYLSemi: $43M from Matter for custom AI chip building
- SkyPilot: $20M from Lux for AI compute
- Infinity: $15M from Touring for inference-ready chips
- Beacon Security: $13M from Notable for agentic sec
- Think: $8M from RAED for AI hardware infrastructure optimization

🚀 New Models & Products Spotlight

- Kimi K3: Moonshot’s impressive 2.8T model
- GPT 5.6 Family: With Sol being the first to win ARC-AGI-3
- Gemini 3.6 Flash, 3.5 Flash-Lite, 3.5 Flash Cyber: GOOG’s latest high-efficiency models
- Cosmos 3 Edge: NVDA’s new open-world model
- Nemotron 3 Embed: NVDA’s newest retrieval-optimized model
- Xiaomi-Robotics-1: The Chinese lab’s first robot-focused world model
- Inkling: The first open-weight model from Thinking Machines
- Bonsai 27B: PrismML’s newest model is the first 27B to fit on a phone

💻 Development

[GH] OpenInterpreter
Looking to try open models? This project is a coding agent for them.

AI Engineer World’s Fair Recap: It’s Systems All the Way Down
This event recap reinforces the idea of 2026 being the year AI usage switches from experiments to systems.

Harness Handbook: How to Understand Agent Behavior
Researcher Ruhan Wang offers a comprehensive guide to understanding how harnesses affect agent behavior.

Karpathy: Have You People Tried TALKING to Your LLMs?
Instead of typing, why not have a nice chitchat instead, hmm?

🤔 Interesting AI Projects, Research, and Updates

Become a Model Routing Expert with Three Weird Guides
Model routing selects an optimal model for tasks in a larger workflow. Here’s how:
- LlamaIndex founder Jerry Liu: The Best Model Routing is Task Specific
- IBM Research: Model Routing Is Simple. Until It Isn’t
- Ramp Labs: Online Learning for Cost-Efficient LLM Routing

How NFLX Serves LLMs
The NFLX team explains how it handles LLM runtime and inference internally.

Fable 5 vs. GPT 5.6-Sol
Developer Charles Azam puts both SOTA models through their paces using /goal. Fable 5 gets the W.

Is This the First Example of Recursive Self-Improvement?
RSI is a feedback loop in which AI contributes to its own learning. Founder Zhengyao Jiang explains his process.

💼 Hiring and Community

Startups Hiring This Week:
- AI Engineer → RTINGS.com
- AI Engineer → HockeyStack

Mid-Markets Hiring This Week:
- XR Engineer → Figure AI
- Senior Computer Vision Engineer → Astroscale

Enterprises Hiring This Week:
- On-Device ML Eng → AAPL
- Senior Software Eng, ML → Uber Freight

OpenAI Bids Farewell to Product Chief, Safety Lead
The GPT vendor loses its business and product chief Fidji Simo due to health issues, and safety lead Johannes Heidecke due to a re-org.

🏭 Industry News & Trends

AMD Reveals First Rack AI System, Inks Monster Anthropic Deal
The chip challenger reveals its answer to Grace Blackwell and Vera Rubin and signs a 2GW compute deal with the Claude vendor.

Cognition Acquires TierZero
The creator of “AI engineer” Devin has acquired the AI incident management platform.

Can I Borrow $400M Real Quick? I Have These Chips
Inference startup General Compute has closed what could be the first major corporate loan to be financed by AI chips.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

AAPL Sues OpenAI, Two Former Employees for Trade Secret Theft
Former business partner AAPL files suit against the GPT vendor, apparently demanding that former employees “preserve documents.”

Anthropic Settles Its $1.5B Copyright Lawsuit
The ruling states that yes, Anthropic was wrong for pirating, but “training AI on books is fair use under copyright law.”

Suno Apparently Pilfered Millions of Copyrighted Music Clips
The embattled AI music startup has been accused of a colossal copyright breach.

AI Federal Safety Musical Chairs
Former head of the Center for AI Standards Dr. Chris Fall has once again stepped down from the Federal government post.

OpenAI Model Allegedly “Goes Rogue,” “Hacks” Hugging Face
They say they’re looking into it.

Deep Dive: Interview with startup founder and OSS builder Arthur Pastel

More organizations are using more AI to generate more “slop code.” Which likely means more regressions and software errors. CodSpeed founder Arthur Pastel argues that the best approach is to head off issues at their inception before they become distributed across an entire codebase.

More AI-Generated Code = More Software Performance Problems?

Studies suggest that some 88% of organizations have adopted AI, with some hyperscalers claiming that as much as 75% of their new code is AI-generated. Does more AI-generated code mean more performance issues and software regressions? Without guardrails, yes, says Arthur Pastel, co-founder and CEO of CodSpeed, a performance intelligence platform built specifically to help teams detect regressions and fix performance issues earlier in the SDLC. Today, CodSpeed is used by teams at OpenAI, Cloudflare and Vercel as well as leading open-source organizations like Mozilla and Pydantic.

The founder, a lifelong coder, explains that the project arose from a longtime interest in software performance. With the growing popularity of the Rust language, Pastel observed that while many interesting tools were being built, there was still no way to measure performance continuously in CI, which inspired him to build the first version of what became CodSpeed, and to found the company.

CodSpeed founder Arthur Pastel discusses software optimization at EuroRust 2025. Image courtesy EuroRust

Making Performance Visibility Continuous

Pastel suggests that the core problem that drove him to design CodSpeed is a lack of continuous visibility into performance during the software development life cycle. “Prior to CodSpeed, you really just had point-in-time performance data points.”

He explains that on a team, one member might run benchmarks, which are local and not committed, and because they are run on one specific machine, they aren’t easily comparable with other benchmarks.

The second problem is a lack of continuous gathering of performance data. “For example you might have some production performance data, like in a continuous profiler or a production observability platform, but you don’t have that data on a specific set of input data. So essentially, you can’t really track it over time and see how it evolved. (And that’s before you introduce factors like regression.)”

“This is the first problem we solved. Then we quickly figured out that people don’t want to spend time fixing performance issues themselves. So we went further: We built a tool to surface the problems, and now we also remediate them and improve performance.”

Performance Engineering Matters Everywhere, Even in OSS

Pastel notes the importance of regression analysis goes beyond massive commercial software packages and is growing in importance among the open-source community. “It is really easy to convince people building open-source projects [of the importance of performance engineering] because they’re building foundational blocks.”

“So for example, if you introduce a regression in Pydantic (which is used by LLM providers) or in a really low-level library used everywhere, even a small performance regression will cascade everywhere.” Pastel points out that popular open-source projects are getting hammered with many more pull requests than before thanks to AI contributions, sometimes as much as 10x, which makes visibility into the performance impact of contributions extremely important for maintainers.

“Otherwise, in the beginning, we primarily spoke to organizations that were really sensitive to performance changes, which might be for UX or cost of infrastructure.” Orgs running infra at scale will generally be concerned about how performance affects their cloud bill, but LLMs have introduced more variation into the quantity (and quality) of things teams can “generate.”

“If you don’t have any guardrails, then your performance might get completely out of hand, and you won’t realize it until you’re actually running on a production database.”

Accounting for Benchmark Noise and Memory Usage

Pastel explains that his team’s approach involves building a CPU simulation tool to address the ‘noisy neighbors’ issue of running on a shared VM whose measured performance might quietly change depending on what neighboring VMs are running. “It can be really painful, especially in overprovisioned environments like GitHub Actions and a lot of CI runners.”

He explains that a common practice to reduce costs is to overprovision instances, which can lead to extreme variations in benchmarks. “For example, you might measure something that, at one point, is 50 times lower than the baseline, yet the code doesn’t change at all.”

The CodSpeed team’s approach is to simulate the CPU to generate a model that, while not perfect, would represent actual execution. “We give a super-fast feedback loop of how the performance changed between two versions of code. And we built it to be really portable, so you can bring it from one computer to another, one server to another, and the results will be comparable as long as you have the same architecture.”

While CodSpeed works on x86 and ARM architecture, his team is actively working on bringing the project to be completely architecture-agnostic. “To do this, we forked Valgrind, which is a dynamic binary instrumentation tool. So instead of simulating and emulating a whole system, we integrate at the binary level, and we have some integration libraries which make it easy to build the benchmarks and the performance test.”

Pastel explains that the initial goal was to make it as easy as possible to build performance tests so that users actually write them, as even code-level issues (like compilers mistakenly trying to remove benchmarks) can get in the way.

“One of the main problems here can be when you observe more macro-level phenomena. For example, let’s say you have thread contention with a lot of thread scheduling, in which case a simple CPU model is not always enough, though we want to push our modelization as far as possible. However, while the next steps may include having some proper isolated or bare-metal instances, at some point you still need to validate the macro behavior.”

“On top of simulation, you sometimes need to measure on real hardware. We have actual physical instances that we can provision on demand in isolated data centers where we can measure performance,” says Pastel. Even physical concerns like the height and location of your server within a data center can yield completely different performance patterns.

“The idea is that using CodSpeed, if you have a GitHub Actions workflow, with just one line, you can get an actual physical instance that’s dedicated to you, without paying thousands per month just for dedicated hardware. We share those costs and make it affordable.”

“The last instrument, because our idea is really to build an ecosystem of instruments that can be plugged to your code, is the memory measurement instrument, which we built because we also had a lot of people complain about memory consumption. Often, if you optimize for speed, it will be a trade-off and you will just consume more memory. Depending on what you do, reducing your memory footprint may also be a requirement.”

“Memory shifted our thesis a bit to be more about optimizing performance, which we define as the amount of resources you use. So your goals might be optimizing for disk or network usage, and ours is to help you optimize your software to reduce the footprint it consumes within your own system.”

“If you introduce a regression in...a really low-level library used everywhere, even a small performance regression will cascade everywhere. If you don’t have any guardrails, then your performance might get completely out of hand, and you won’t realize it until you’re actually running on a production database.” -Arthur Pastel, Founder/CodSpeed

How Technical Leads Should Think About Performance Engineering

Pastel breaks down the value of performance engineering for orgs into a few layers. “First, performance issues might just be hiding some bugs, possibly due to an algorithm not being properly implemented.” Code correctness can therefore be part of performance management, as can making sure the resulting complexity of an algorithm is what was intended.

“Then, if everything runs faster, then you need fewer servers and less compute to run something, which means you cut your costs, which is one key benefit.” Pastel observes that orgs running cloud infra at scale generally prefer tools that are fast and don’t overconsume memory, which is why teams he works with often optimize for speed first, then memory usage second after they start comparing memory footprints.

Lastly, performance engineering can directly affect user experience. “Depending on what you ship, whether it’s a Web application, even if it’s a backend server or something, the overall latency of your program will define a lot of the user experience. As long as there is a user interaction involved, UX can be improved because you can make things faster.”

Pastel gives examples of how CodSpeed helps front-end teams detect performance regressions in CSS rendering, as well as optimizing at the kernel level and with JS execution. “There are many things that can be optimized to just make the overall experience for the end user way more pleasant, which is something that’s pretty important as well.”

For teams building software, performance engineering also provides multiple benefits. “First, with AI, you’re shipping more code than ever now. You’re spending much more time reviewing, so you have much less time to focus on performance. And the idea is that, at first, you need guardrails on the most important parts of your application, whether that’s the time it takes to load a full page, or to perform the most basic interaction on your server via your back end.”

Pastel argues that as software teams have less time and resources to devote to improving the performance of their products, they at least need some kind of background job optimizing performance. He points out that developers now generate and review code with LLMs, but even frontier models have no runtime performance signal, neither in their training data nor as feedback when they generate code. They can’t measure the code they generate, so they’re left to guess at what performant software actually looks like.

“What LLMs need is a proper harness to measure performance, which is what we’ve built. We initially built this for humans, but now it works for agents as well. Essentially, we help the agents build a really simple harness around any kind of code, any kind of language, then let them query the data in a proper format.”

Agents don’t collect or consume performance data in the same way humans do, but the team still needs to surface them for users. As a result, they developed an MCP server, a CLI, and skills to let users run their own auto-research loop, to minimize the cognitive burden on already-overstretched development teams.

Why Aren’t Agents Already Doing This Stuff?

Pastel suggests gaps in today’s agentic tooling and infrastructure that prevent them from being proper guardians of software performance. “CodSpeed can run on open-source software and the tools we built in sandboxes in a super-cheap way. But agents would still need to interpret and aggregate the data, which is the first tricky part.” He points out that optimization work is hypothesis-driven. Even the world’s foremost performance experts form a hypothesis from an observed code pattern, write the change, and validate it by measuring. Agents need that same measurement loop.

“But then, there’s also a whole infrastructure play, which is much harder. You need to have isolated machines to run more-macro tasks.” Pastel explains that CodSpeed operates a fleet of bare-metal machines tuned for measurement stability spanning various hardware configurations, architectures (x86, ARM), and GPUs. Agents can request an instance on demand and run benchmarks in a completely isolated environment, on the hardware that matches their production target.

Pastel also believes that the evolving economics of compute resources is constraining something that technical teams used to not consider a constraint at all: Available memory. “Just getting RAM now is getting harder. We never really took the time to optimize software in the past because it was super-easy to say, ‘My software is consuming a lot of RAM, but people will just get laptops with more memory, so it’s completely fine. Same with CPUs; if things were a bit slow, you would just get a bigger instance.”

While there are faster CPU and GPU units in development, Pastel points out that compute and RAM are becoming scarcer and more expensive, requiring teams to treat them as constrained resources and focus on performance.

For decades, Moore’s Law meant slow software could wait for faster hardware. That escape hatch is closing. As hardware gains slow down, performance has to come from the software itself.

“We never really took the time to optimize software in the past because it was super-easy to say, ‘My software is consuming a lot of RAM, but people will just get laptops with more memory, so it’s completely fine. Same with CPUs; if things were a bit slow, you would just get a bigger instance.”

How Founders Should Approach Performance Engineering

For startup founders, Pastel notes that agents can process thousands of times more data than humans in the same time period, which means that both your startup’s agents, and those of your customers, could be dragged down significantly by performance regressions.

Pragmatically, he admits that performance engineering may not be a priority in the very earliest stages of a software startup. “Performance might be taken as an afterthought in the beginning, and it’s fine to [focus on building] a simple MVP, because first you want to validate that there is a need for your product.”

However, the AI era has created another wrinkle that makes performance significantly more important, especially in competitive spaces. “For customers you hope will adopt and actually keep using your product, migrating from one thing to another has become super-cheap. Essentially, you can just have an agent tasked to migrate from one solution to another. I think this is why a lot of SaaS products are at risk, because we don’t have the strong lock-in and switching costs we had before.”

“So, performance is still a key differentiator, especially when you think about adoption and retention. It’s still a big deal in my opinion.”

Performance Engineering in the Enterprise

Pastel sees performance engineering as only becoming more important for enterprises, especially on the infrastructure side at scale. “Enterprises have already realized the margin gains they can get from optimizing their software. We see adoption growing today, and it will only accelerate in the future.”

He also argues that the advent of new AI tooling is helping orgs shift performance left as well. “Previously, performance would come down to, ‘We’ll see what happens in production.’ And if there was an outage, someone would wake up at night and try to fix it. Now we see more and more AI SRE companies that fix such issues, rather than having someone being paged to wake up in the middle of the night.”

“We’re building something to fix issues before they even happen, which we believe is better, because you don’t want agents doing stuff in your production instances when you are an enterprise. If you can catch issues before production, you can validate with a human before anything happens.”

“It just happens in a much more natural cycle, rather than thinking, ‘Okay, there is a fire, I will try to fix the fire, push in production again, and iterate with a super-long development cycle.’ With agents, we really can ship in minutes while before, especially for enterprises, the release cycle might take weeks, if not months.”

How Will Performance Engineering Fit into the Future Stack?

Pastel is confident performance engineering won’t be going anywhere. “You can’t train models to optimize performance by design. Performance depends on what you actually run in production: Your data, your infrastructure, your product goals. That context has to be measured, not learned.”

The founder is skeptical that even a perfect agentic future will account for all the potential outcomes of performance engineering, even if agents eventually come to possess all the right tools to communicate and validate performance changes. “Even if we manage to have some super-optimized languages, and we have new state-of-the-art models that allow us to have some crazy breakthrough to do a lot of optimization, there is still the structure of the code, which will still be an issue and cannot be optimized by a compiler.”

Pastel suggests that performance engineering, like security, will be a discipline that may never become out of date. “It won’t disappear because you still need to have guardrails and things to make sure that your app is [performant] and consuming the least amount of resources possible. This problem is not ‘done.’”

This interview originally appeared on Heavybit.com.

Deep Dive: Interview with startup founder and OSS builder Tomás Senart

Performance engineering means optimizing software performance against regressions and removing latency. For lean startups, it’s sometimes viewed as a luxury that matters a lot less than shipping new product.

However, open-source builder Tomás Senart, creator of the popular Go-based HTTP load-testing project Vegeta and founder of Perfloop suggests that performance engineering will only matter more to maturing orgs as AI-generated code introduces more regressions in an environment where scarce compute resources can’t hide them.

Senart’s philosophy accepts the increasing complexity of maturing codebases, deferring the heavy lifting of continuous benchmarking to AI agents and putting human engineers in the driver’s seat only when it’s time to make an important decision. He explains his thinking below.

Performance Engineering: Part Brute-Force, Part Hard Decisions?

Software startups often launch quickly to beat competitors and gather early customer feedback. At that stage, performance should not necessarily be the priority, founder Tomás Senart argues. But once a product is exposed to scale, or latency and infrastructure costs become material, the economics change and teams need a repeatable way to keep performance from drifting. The creator of the popular Go HTTP load-testing tool Vegeta has carried those lessons into his startup Perfloop, and shares them here.

Tomás Senart discusses his learnings in Go interfaces at GopherCon 2015. Image courtesy Gopher Academy

From Load Testing to Go

As an engineer at SoundCloud, Senart had been tasked with replacing a significant chunk of a legacy system that needed to be loaded and tested. “At that time, the only thing available that was not a [heavyweight] Java ecosystem tool was Apache Benchmark, which had an array of usability issues and functionality limitations. We had a concrete problem and a culture of building things from scratch. Those two things together led to Vegeta.”

While the project has become popular in the Golang community, the founder clarifies that there was more to its success than the inherent speed and transparency of the language itself. “I think you can do performance engineering in any language ecosystem. With Go, because concurrency was such a core part of the design of the language, it made building a load-testing tool very natural at the time.”

“But you can build a load-testing tool in a way that will ‘lie to you’ in the results. So it wasn’t just about building something quick, it was about building something correct. Azul Systems CTO Gil Teneis famous in the performance engineering world for many things, including the concept of coordinated omission: A phenomenon in load testing (and many tools fall into this trap) where the client that is sending the requests paces itself according to how the server is doing.”

“That is a bad way to test the system, because you’re basically stopping to hammer the server at the exact point where you’d get the [indication] that it’s not doing well. So you need to keep the rate constant. That was a key design decision at the time to build Vegeta in this way: To not suffer from that problem.”

The founder suggests that the most important lesson from his earlier work wasn’t that performance data is lacking, as most teams have dashboards full of telemetry and a benchmark they’ve run at least once. “The problem is that the benchmark you ran once is already stale. Performance is not something you set and keep, it is an envelope you have to hold against constant change.”

“Every code change, dependency bump, traffic shift, and architectural change nudges it, so the loop from measurement to a verified improvement has to run again and again just to stay in place.” Senart explains that the measurement process is inherently slow, which makes it easy to deprioritize and fall by the wayside.

His startup Perfloop closes the second half of the loop: After measuring, Perfloop assesses how systems behave under load, across the entire loop. “It finds the performance opportunities, proposes changes, and proves whether they actually moved the needle, with the numbers to back it up.”

Providing proof, the founder argues, validates automation across the rest of the process, rather than forcing teams to take things on faith: An important nuance as benchmarking loops need to be run far more often than can be done manually. “The system carries the ‘relentless’ part, the finding and the proving, while the person stays in charge of what is worth holding and what actually ships.”

How Much Should Tech Leaders Prioritize Benchmarks, Really?

Addressing the excitement around AI evaluations, the founder notes that the most popular AI benchmarks can and do become contaminated. “That’s why people started creating private eval sets that they run so they’re not gameable. That’s one approach, but it has its own problems, because there is no transparency into the quality of those benchmarks. It’s a moving target, honestly: How to have transparency while not being gameable or having evals end up in the training data of models themselves.”

Senart draws a line between viral LLM benchmarks like Deep-SWE and benchmarks that teams specifically use internally to build their own products. “If you’re building these capabilities in your company and in your product, the only thing that matters is your evals. It’s not going to be the public coding agent evals for the coding models that matter the most.”

“The benchmarks that last are the unglamorous, private ones tied to an outcome you actually care about, because there is no incentive to game a benchmark whose only audience is you. Public benchmarks help you decide what to test. Your own codebase tells you what is true.”

“The only thing that matters is your evals. The benchmarks that last are the unglamorous, private ones tied to an outcome you actually care about, because there is no incentive to game a benchmark whose only audience is you. Your own codebase tells you what is true.” -Tomás Senart, Founder/Perfloop

Performance Engineering Lessons for Founders

Senart suggests that the adoption of agentic coding agents is fundamentally changing software, and not necessarily for the better. “First, agents now write a large and growing share of code, with a very wide range of outcomes. Many teams are moving faster than their ability to review every line, so they are shipping performance slop: Inefficiencies and waste that no human deliberately introduced and no one is specifically watching for.”

“Second, the load on production systems is changing shape. It is not only human users anymore. Agents call services at machine speed and volume, so the same systems get hammered far harder than they were designed for.” The founder observes that bot Web traffic has surpassed human traffic, and Web-based infrastructure is only the first beachhead to be pounded on by agents.

Commerce is another rapidly-growing space for agentic workflows for retail and checkout processes, and enterprise services appear to be the next frontier. The founder argues that agentic traffic is no longer a hypothetical future condition. It’s an inevitable thing that will happen, and make performance even more crucial.

In addition, it’s no longer feasible to wait to be ‘bailed out’ by eventually better hardware. “Now GPUs, memory, and compute are expensive and staying that way, so inefficiency shows up directly as a cost on your bill.”

The above factors lead to more performance debt that accrues quicker on infrastructure taking heavier workloads. “What is missing is a way to keep up. Occasional expert attention or a periodic consultant audit cannot match how fast the debt now accrues. That is the gap Perfloop is built to close.”

The founder clarifies that while you can use agentic systems to support performance engineering, AI doesn’t nullify the fundamentals. “You still have resources, you have things that drag on them, and you’ve got to make the most of them. You need to unpack that, one layer at a time, and be methodical. That’s what performance engineering is.”

“You still create hypotheses, you go test them, and you look at the hard data. It’s just that the experiments can go faster now. You have agents which, with the right harness, can do that kind of work rigorously and not just guessing and going in the wrong direction, then you can parallelize so many of these hypotheses for testing.”

Senart offers two final pieces of advice for founders. “First: Do not pay for performance you do not need yet; if nobody uses your product, its performance does not matter. Second: The moment you are exposed to scale, the economics change in your favor. Performance work is methodical science, and the old constraint was that you could only afford to test the top one or two ideas. With the right harness you can now run many of those experiments in parallel, with rigor, and test the other ten you used to ignore.”

Getting Performance Engineering into Enterprise Customers’ Hands

Senart concedes that his startup’s approach to automating performance engineering isn’t necessarily mainstream among enterprise engineering orgs yet, due to a number of factors. “The first is trust, which is the hard one. You are asking a large organization to let an autonomous system reason about and propose changes to code that runs their business. They will not do that on a verbal promise.

“They need proof that the system finds real problems and that its proposed changes actually work, and they need that proof to come from somewhere they cannot dismiss. This is exactly why we are doing the work in open source, in public.” The founder notes that the maintainer of the widely-used Go library parquet-go has publicly agreed with his hypothesis that closing performance debt will come down to utilizing the kind of concurrent optimizations he’s building today.

“The second blocker is the usual enterprise gate: Compliance, security review, governance. There is nothing special here, it is just work you have to do to be allowed in the building. We are starting SOC 2 now for that reason.”

“The third blocker is not so much a blocker as it is inertia: Simply the reason to bother to do anything. Enterprises are actually a strong fit, because their scale and internal complexity produce a lot of recoverable inefficiency. In a large system, the performance you have left on the table is real money on the bill. Once trust and compliance are cleared, that recoverable cost is a very concrete reason to engage.”

“What it takes [to get into enterprises] is portable, verifiable proof on code that looks like theirs, compliance work done honestly, and a value story told in the language they care about, which is cost and risk, not benchmarks for their own sake. For us, this is the route, not the first stop. The immediate motion is the public open-source work and self-serve adoption. The enterprise conversation opens once that public credibility is built.”

“[For enterprises], the first [blocker] is trust, which is the hard one. You are asking a large organization to let an autonomous system reason about and propose changes to code that runs their business. They will not do that on a verbal promise. The second blocker is the usual enterprise gate: Compliance, security review, governance. The third blocker is not so much a blocker as it is inertia: Simply the reason to bother to do anything.”

What the Future of Performance Engineering Looks Like

Senart understands the current excitement about AI agents in software, but isn’t sure that performance engineering will simply become another agentic guardrail. “Guardrails are part of it, but the real question is where the human sits on the spectrum between specifying everything up front and steering every step (and both ends fail).”

The founder offers the two extremes: The ‘dark factory’ that is prompted once while humans step back, without human judgement or context along the way. The other is today’s common practice of endlessly babysitting coding agents edit by edit, which preserves human judgement but can’t scale, turning senior engineers into bottlenecks.

“What actually amplifies a senior engineer is in between, and it is the harder thing to build. The human stays on the decisions that carry judgment, which tradeoffs to accept and what is worth holding, while the system runs the steps underneath and surfaces only the calls that need a person. Present where judgment is required, absent everywhere else. Not steering more, steering higher.”

“Performance needs exactly this, because the target keeps moving. You cannot freeze the judgment into an up-front spec when the envelope shifts with every change, but you also cannot re-derive every step by hand each time. So you want a system that does the relentless step-level work and pulls the human in precisely when a real tradeoff is on the table.”

“That is what Perfloop is built for. The whole loop runs as an MCP server, so the pieces are callable on their own: A person or an agent can point it at one slice of code, or hand it a rough hunch about what might be slow and get back a benchmarked case. Autonomy is a dial the customer controls, from scoping, to directing, to reviewing, to fully delegating, which is really just a way of choosing how high on that spectrum you sit.”

“My honest bet is most teams land at ‘direct’ or ‘review’ for anything that matters: Let the system carry the steps and the proof, but keep their hands on the decisions that need judgment. So performance engineering won’t shrink into a guardrail. It will remain a loop that humans steer from above, with far more reach than before.”

The founder notes two additional factors that will affect future software projects. “First, the set of teams exposed to scale is expanding fast, because agent traffic drags services that used to be ‘safe’ into the deep end. Second, for the teams already at scale, the economics changed underneath them: The debt piles up faster, because agents write more code with less scrutiny per line, and it no longer depreciates the way it used to, because you can no longer count on cheaper, faster hardware to absorb it.”

“Performance debt that used to quietly depreciate now accrues interest. When a problem goes from occasional to continuous, how you handle it has to change too. A once-a-year expert audit cannot keep up with debt that compounds. Perfloop is my bet on what replaces it: The rigor a great performance engineer brings, available to any team that needs it, with the human keeping the judgment and the system carrying the grind. The open-source work is where we are proving it in public, one verifiable win at a time.”

This interview originally appeared on Heavybit.com.

Each day, the biggest AI companies in the world compete to put their software products into the hands of new customers. But one of the greatest overall challenges remains the actual adoption part. Some reports suggest adoption rates of up to 53% across organizations, but the reality is that many working professionals, in and out of software development, may not be using AI, at least not with the frequency that vendors might hope. To help things along, AWS has announced a $1B forward-deployed engineer (FDE) org so that “customers leave…deployments with both new solutions and new engineering capabilities.” MSFT has since made a similar announcement to invest $2.5B in its own 6,000-person FDE org to provide similar consulting solutions to customer orgs where the spirit to adopt AI is willing, but the flesh isn’t there yet. These new orgs join OpenAI’s recently-announced FDE consultancy org, NOW’s recently-announced FDE org, and CRM’s FDE org announced in April. Perhaps because there are only so many customers to buy AI software and services, several major AI vendors are getting into the compute-selling business. Notably, META appears to be getting into the cloud infra business to auction off its unused compute cycles, shortly after SpaceX announced the same, following Softbank’s previous announcement. It looks like all that AI isn’t going to sell itself. (At least, not anymore.) And at least for now, FDEs are going to be highly in demand.

Coming Up: All the top AI news, trends, and resources for busy dev teams

💡 Spotlight: Newly Launched AI Startups

- Trase: $107M from ARCH for agentic in highly-regulated spaces
- Hive: $15M from SuperSeed for industrial machine “brains”
- Seltz: $12.5M from B Capital for agentic search
- Arato: $10M from TLV for testing AI systems
- Baz Technologies: $9M from Battery for agentic coding
- Sazabi: $8M from J2 for AI observability
- AIsa: $6.5M from BABA for letting agents buy their own stuff
- Dawnguard: $3.3M from BNVT Capital for “AI-native security architecture”
- Mowito: $3M from Version One for AI robot arms
- SuperPlane: $2.6M from Credo for “AI-first engineering control plane”
- DevPlan: $2.5M from AI2 Incubator for “AI-native product development”
- HyperDev: $1M from various investors for AI codegen

🚀 New Models & Products Spotlight

- GPT 5.6: Sol, Terra, and Luna are OpenAI’s “best models yet”
- SWE-1.7: Cognition’s latest coding model
- Seedream 5.0 Pro: ByteDance’s latest multimodal with reasoning
- Muse Image: META’s newest image model from the Muse family
- Fable 5: Yes, it’s back
- Hy3: Tencent’s new 295B MoE
- Laguna SX 2.1: Poolside’s 33B MoE model for coding and long horizon
- Nano Banana 2 Lite: The next version of GOOG’s Gemini image model
- Claude Sonnet 5: Cheaper Sonnet, runs close to Opus 4.8
- LongCat-2.0: Meituan’s massive 1.6T MoE for coding and long horizon
- Fugu: Sakana’s newest hits 93.2 on LiveCodeBench
- LFM 2.5: Liquid AI’s compact 230M non-transformer model
- Ornith 1.0: DeepReinforce’s SOTA open-source model

💻 Development

State of CLI Coding Agents
Engineer Oleg Luganskiy shares his rundown of the state of play for AI codegen agents so for this year.

The Open-Source AI Gap Map
This toolchain map covers specific areas that have been supported by open-source AI projects, and highlights opportunities where coverage is lacking.

Guide: Agentic Harnesses
Founder Aparna Dhinakaran’s crash course on agentic harnesses and how future harnesses may evolve beyond those provided by vendors.

🤔 Interesting AI Projects, Research, and Updates

Research: PACE Framework Predicts Benchmark Performance
This project suggests a framework to guesstimate LLM benchmark performance while reducing eval costs 99%.

PorTAL: Reducing LLM Retraining Costs
Ramp Labs’ proposed architecture separates task-based fine-tuning from base model weights to make future retraining cost less.

DSpark: Speeding up Inference 85%
DeepSeek’s proposed module speeds up inference with next-token prediction.

[GH] T3MP3ST: Coding Agents → Red Teamers
This new project turns coding agents into autonomous red-team sec testers.

💼 Hiring and Community

Startups Hiring This Week:
- AI Engineer → GovPilot
- AI Eng, DevRel → Composio

Mid-Markets Hiring This Week:
- Senior Research Scientist → Toyota Research Institute
- Senior Eng, Tools & Agents → TwelveLabs

Enterprises Hiring This Week:
- Staff Eng, ML → FRSH
- Data Scientist → The Wonderful Company

AAPL Vision Pro VP Departs for OpenAI
16-year AAPL veteran VP of hardware eng Paul Meade is reportedly joining the ChatGPT vendor.

🏭 Industry News & Trends

Your AI Chip News Roundup:
- Broadcom and OpenAI launch inference-optimized Jalapeño chip
- Broadcom supplies AAPL with custom chips through 2031
- DeepSeek prepares to enter the silicon business

AI Vendors Invest Deeper into the Sciences:
- GOOG: SandboxAQ models trained specifically on scientific research
- Anthropic: Claude Science workbench & unveils plan to research drugs
- OpenAI: GeneBench-Pro, following LifeSciBench and Rosalind

⚖️ Copyright, IP, Licensing, Regulation, and Safety

OpenAI Proposes 5% USA Stake, USA Slows Down GPT-5.6 Launch
OpenAI offers Uncle Sam a 5% stake in the company. The White House requested a slower release cycle on security grounds.

California Sues Chipmakers for DRAM Price-Fixing
The Golden State takes Micron, Samsung, and SK Hynix to court, accusing the chipmakers of antitrust-style price fixing.

Anthropic Accuses BABA of Large-Scale Model Distillation
The Chinese lab is accused of stealing model intel across “more than 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts.”

Recent weeks saw seismic change in the great AI talent war as GOOG DeepMind lost both Nobel Laureate John Jumper (who joins Anthropic) and the co-author of the foundational “Attention Is all You Need” paper from 2017, researcher Noam Shazeer (who joins OpenAI). Just yesterday, GOOG lost two additional researchers, Jonas Adler and Alexander Pritzel, to Anthropic. Meanwhile, in the wake of Fable 5 being suspended by the White House, allegedly in response to a meeting with AWS’ Andy Jassy, Microsoft’s CEO has openly called for change: With so much of the AI ecosystem built on top of the products of just a few foundation labs, the removal of just one player can have an enormous effect. (Which may also explain MSFT’s own recent entry into the foundation model arena with 7 of its own models.) In particular, a new open model turning heads is GLM 5.2, the newest model from Chinese lab Z.ai, optimized for long-horizon coding tasks. The model’s 1M-token context window rivals Opus 4.8 and GPT 5.5, and its performance is causing some to question whether Western foundation models can retain their lead. Was some combination of these concerns (and some cynical profit-taking from the SpaceX IPO) the cause of the recent AI and tech selloff in both US and Asian markets? Maybe. As they say, nature (and the market) abhors a vacuum, so all the uncertainty probably didn’t help.

And Now: The AI news, trends, and resources busy dev teams need to know

💡 Spotlight: Newly Launched AI Startups

- Mirendil: $200M from a16z for automated AI research
- Ent: $100M from Decibel for cybersecurity AI governance
- Hang Ten Systems: $32M from Mayfield for enterprise AI transformation
- Architect Labs: $24M from Kindred Ventures for AI chip architecture
- Aether AI: $20M from MPCi for causal world models
- TensorX: $9M from Darius Cubed for Sovereign EU AI infra
- Probably: $9M from a16z for verifiable data agents
- Clario: $6M from Preface Ventures for AI-ready data management
- Tenet: $6M from The Westly Group for agentic production controls
- Optiak: $4.7M from Market One for enterprise AI governance
- GitHits: $1.7M from Vendep Capital for reducing hallucinations in AI code
- Synvo AI: $1M from Fuel Ventures for enterprise AI memory
- Sherlocks AI: $790K from Sense AI Ventures for AI-native incident management

🚀 New Models & Products Spotlight

- GLM 5.2: Z.ai’s newest open model is turning heads in the valley
- HappyHorse 1.1: BABA’s new video gen model for enterprise
- Mercury 2: Inception Labs’ new reasoning language model
- MolmoMotion: Ai2’s new language-guided 3D motion video model
- Qwen-RobotWorld: BABA’s new video world model
- VibeThinker-3B: Weibo’s tiny model rivals Opus 4.5
- Seedance 2.5: ByteDance’s latest video model
- Krea 2: A new multimodal family from Krea AI
- Unlimited OCR: BIDU’s one-shot long-horizon transcription model
- Kimi-K2.7-Code: Moonshot AI’s latest long-horizon coding model
- MiMo v0.1.0: Xiaomi’s new agentic harness rivals Claude Code on ultra-long tasks

💻 Development

Guide: Loop Engineering
Loop engineering, the practice of building systems that repeatedly prompt agents, is growing in popularity. Founder Elvis Saravia explains.

Guide: AI Inference Engineering
This guide includes tips on how to run AI models efficiently in prod.

Guide: Whether It’s Worth Training Your own Model
Founder Rhythm Garg suggests that having proprietary data and significant inference volume are good reasons to try.

Guide: Saving Tokens with an Agent-LLM Provider Buffer
Founder Sunil Pai explains how to conserve tokens despite crashing jobs.

🤔 Interesting AI Projects, Research, and Updates

How to Estimate Inference Costs
Engineer Srijan Paul walks through how to roughly calculate your dollar-cost-per-user inference budget.

Research: Prompt Injection Might Actually Be Role Confusion
This project suggests that prompt injection may be the result of LLMs becoming confused about the role of inputs. Blog summary here.

Model Size Scaling for the Next Five Years
This article guesstimates the approximate size scaling of SOTA models, potentially peaking at 1.4 quadrillion in 2031.

💼 Hiring and Community

Startups Hiring This Week:
- AI Engineer → Scalr
- Applied AI Engineer → Quantifind

Mid-Markets Hiring This Week:
- Full-Stack AI Engineer → AHU Technologies
- Data Engineer → Axle Informatics

Enterprises Hiring This Week:
- AI Engineer → The National Hockey League
- Senior Agentic Systems Eng → NOW

🏭 Industry News & Trends

OpenAI Acquires Ona, Unveils new Chip
The GPT vendor snaps up the agentic orchestration platform and takes the wraps off of Jalapeño, a new chip partnership with Broadcom.

GOOG Now Renting TPUs to Anthropic
The hyperscaler is getting into NVDA-style inference leasing.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

US Govt Continues to Pressure META on Regulation
META remains the only foundation lab to not voluntarily share models for review.

EU Orders Open Season for WhatsApp Chatbots
The EU has ordered that META’s ultra-popular chat app no longer block AI chatbots from other vendors.

Deep Dive: Interview with open-source builder Terence Pae on local inference

The headlines are starting to pile up: Massive orgs like Uber and ServiceNow are burning through their annual inference budgets in months. The perfect picture of “tokenmaxxing” has begun to fade as orgs scramble to find a return on their massive investments in third-party inference.

Builder Terence Pae’s history working in inference led him to propose an alternative solution: Osaurus, a project that makes your Mac computer into your local AI harness. Below, he explains why on-device inference may be the future, and what startup founders can learn from his journey.

Giving AI Data Ownership Back to Users

Researchers have warned for years: Frontier models are potentially running out of data to train on. However, the apparent lack of data hasn’t stopped AI labs from endlessly launching newer, shinier models with endlessly improved capabilities, which, in turn, makes switching costs painful for users who have their data locked up in a previous-gen version of a competitor’s model.

After significant experience working in general software development and then in localized inference, veteran developer Terence Pae began working on turning his own Mac into a personal AI harness, freeing him from vendor lock-in from increasingly commoditized models and giving him significantly more control and ownership over his data and permissions. Now his open-source project Osaurus has passed 140,000 downloads and 6,000 GitHub stars, all with zero paid acquisition.

What if your MacBook was your AI harness? Image courtesy Osaurus

Why Vendor Lock-In Has Made AI Development Painful

Pae explains that the project came from his own concerns as a developer who was heavily dependent on third-party harnesses. “I’m such a heavy user [of AI], and it’s easy to not really think about all the data I might be sending out and the potential implications there.”

“As AI evolves and grows more powerful, it’s constantly taking in all this information from users to make models better. But at the same time, as an individual user, I’m wondering: Where is my privacy? Where is my data going and where is it being stored?”

The founder also notes that vendor lock-in benefits the frontier labs while making switching between models painful. “These AI companies are building moats, and they’re not using their data to do that. They’re actually using your data. The thesis with Osaurus is that as a user, you should be owning everything that is related to your data, your code, your context, your tools, and everything else. All you really need is the inference, whether that’s running locally on your hardware or in the Cloud, so you’re not dependent on a specific provider.”

Lessons Learned from Decentralized Infrastructure

The founder notes that deciding to go with a building-in-public approach to open source was the key to growing his project, and that his work in decentralized infrastructure led him to focus on local-first AI development. “There are many ways to build AI harnesses. For example, you can write code that runs on the Cloud, which depends on having a certain infrastructure. And you need a database to host your user’s data. But Osaurus is, itself, a client and a server that runs locally, and spins up a SQLite database locally. Everything is self-contained on your device.”

Pae draws on his experience with peer-to-peer systems, where every node acts as both client and server. “Each node can connect with the others and create a network where no individual node has to rely on a centralized server. And if an AI is meant to be owned by the user and designed for the user, it also needs to act like a node that doesn’t depend on a centralized service. That software should still work even if the company behind it gets hit by a meteor.”

Pae’s design approach assumes the future of AI won’t always have a human in the loop. “Right now, we’re relying heavily on a chat interface with agents and having a human in the loop constantly interacting with them. But I believe the next evolution is going to be more about agent-to-agent communication. For agents to be truly autonomous, they’ll have to rely on other agents.”

The founder’s experience working with cryptography informed his design. “Basically every agent will have an identity with a public key and a private key, so that agents themselves can verify each other. Being able to use a signature to verify each other has many benefits. But the key benefit is just making sure that this agent is talking to the right agent.”

“If my agent is reaching out to somebody else’s agent, it needs to be able to verify that that particular agent is owned by that person. Designing in those specific protocols helps us to get to where we need to be in terms of agent-to-agent collaboration. Those are the basic building blocks I’m setting in place, but the future vision is about giving agents the autonomy, the guardrails, and the ability to act on your behalf without compromising or being compromised by other systems.”

“I believe the next evolution is going to be more about agent-to-agent communication. For agents to be truly autonomous, they would have to rely on other agents.” - Terence Pae, Founder/Osaurus

Is On-Device Harness + Compute a Viable Alternative?

Pae acknowledges the growing challenges of cost management in AI operations, including organizations burning through massive token budgets utilizing third-party vendors. “I think [on-device] could absolutely be an alternative. In my opinion, the current model of scaling and building all these data centers to power all the demand in the world doesn’t seem very feasible.”

“I think the rate of agentic use is only going to go up from here. As a software engineer, my AI costs used to be about $15 monthly. Now, between cloud subscriptions and API usage, I’m paying $3,000 to $5,000 a month. That’s just me as an individual engineer, and it’s exactly the cost curve that makes local inference worth taking seriously.”

The founder suggests that while token economics seems increasingly likely to affect SaaS-style pricing, AI computing at the edge may also be competitive soon. “Local AI has improved almost exponentially in terms of intelligence per watt. For example, the recent Gemma 4 models run under 24GB of RAM and match what frontier models could do only a generation ago. That capability now fits on a laptop.”

Pae also points out that not every use case demands a trillion-parameter, general-purpose model. “You may just need a good-enough, smart-enough model to help automate things. And it’s the harness’s job to really bring out the potential of the model to not only increase capabilities, but also reduce some of the dependencies as well. In the future, we’re all going to be less dependent on the Cloud and more dependent on local infrastructure, in my opinion.”

Getting On-Device AI into Enterprise: Models, Tools, Education

While the promise of having full control over your data for daily AI tasks, without interruptions from cloud service outages, seems like an easy sell to individual developers, the founder concedes that risk-averse enterprises may need more nudging to actually adopt local-first AI.

“I think there are three issues preventing enterprise adoption. First, the local models themselves: There’s constant innovation happening here. The models may not be fully ready today, but maybe soon, possibly in the next year. Second, there don’t seem to be enough tools and harnesses that can work with local models. Major frontier labs like Anthropic and OpenAI not only have models, but also have tooling and infrastructure to be efficient and creative with their offerings.”

“Third is a lack of education: People may not know that these capabilities are good enough to use today. We speak with a lot of enterprises, including family offices and healthcare clinics that deal with sensitive data. The most common theme is there’s just no trust in the Cloud.”

The founder shares that cloud providers and their 30-day data retention policies still seem to rub enterprise customers the wrong way in terms of risk exposure to court subpoenas and the like. “As a company, that’s a huge risk. As an enterprise, you lean towards trying to have no risk at all.”

In contrast, having a private AI instance means sensitive work can stay on local infrastructure and never touch the Cloud. Osaurus supports both local and cloud inference, but the user decides what leaves the device. “Unlocking private AI with a full-featured agentic harness is going to really unlock the productivity that these enterprises are looking for: Having a private AI infrastructure with a frontier-level model, with a frontier-level harness that they can actually use internally.”

The founder also asserts that not all enterprise customers are stodgy, middle-aged, middle managers who are terrified of using AI at all due to risk. “The people we talk to definitely want to use AI, and want to simplify their workflows. They’re really curious about it. A lot of the people that we chatted with use AI personally, but they don’t use it for work. So they know AI’s capabilities.”

Pae recalls starting initial conversations with enterprise customers with his own preconceived notions: Would enterprise users seek very specific things, similar to familiar tools like Microsoft Excel? “Actually, most of the feedback we got was that [enterprise customers] want a ‘general AI,’ a general harness that is able to do what ChatGPT does. The people we talk to are looking for something more broad in terms of applications.”

“Three issues are preventing enterprise adoption: the local models themselves, not enough tools and harnesses that work with local models, and a lack of education.”

Building AI Products for Startup Founders

Pae is direct in his advice to startup founders looking to start companies around AI-adjacent products: Don’t build on top of someone else’s walled garden. “I would say treat inference almost as a building block: You can have a query, and then it returns a specific output in a function. Otherwise, you don’t want to build software in a way that locks you into a single provider.”

“You don’t want to build products in a way that’s super-dependent on specific functionality [from one vendor]. You want to build [components] in a way that is easily replaceable. Inference should be treated that way as well. Building an agent? You should design it in a way that you can easily replace your inference provider, and in my opinion, also have the ability to swap in local models as that may make things a bit more defensible.”

“Vendors like Anthropic and OpenAI may have some really cool features, but not every AI vendor will have the capabilities to build something similar. If you offer functions that can work with many different providers, you can also build in a way that targets specific audiences, such as the rapidly-growing open-weight market in China.”

“Being model-agnostic has worked out for us because there are different communities using these different open-weight models. We’re not really bothered if Anthropic suddenly launches a feature similar to ours, because they aren’t likely to also support MiniMax or other models.”

“In a larger sense, I’ve been thinking that software isn’t exactly ‘defensible’ anymore, since you can potentially create anything with enough attention and time. If we treat software like a true commodity, I think that’s not going to be scalable for the long term. What is truly defensible is the community that you build around your software and also the transparency you can provide. The community that forms around you gives you that ability to push forward.”

This interview originally appeared on Heavybit.com.

The much-anticipated (and once-feared) Claude Mythos has arrived in Fable 5, the family’s first new model, offering astonishing performance and potentially invisible, “safeguard-related” performance hits that some consider to be an out-and-out supply-chain risk. (Also, Fable 5’s system prompt was apparently leaked.) Still, Anthropic is having a big start to its summer, having closed $65B in funding while receiving backstop assurances on its data centers from NVDA for upwards of $35B. Just in time for the vendor to not-so-quietly file for IPO later this year. Perhaps not to be outdone, OpenAI has announced high-profile partnerships with Visa on agentic commerce, with AWS launching models on Bedrock, and has also not-so-quietly filed for IPO later this year (with the White House potentially getting some of the action). So has Cursor owner SpaceX. And the always-patient Microsoft, enterprise business partner to everyone, just leaped across the table to become a direct competitor to every other frontier lab with the launch of seven of its own new models, including its first reasoning model. There’s a lot riding on the upcoming offerings, especially considering some of the recent pricing-related backlash among AI customers (like Uber capping employees at $1,500/mo), which has led the Linux Foundation to literally found a new body for token economics standards. And there are larger questions about how much risk shareholders will tolerate, and how much demand exists for so many competing AI products, considering that most AI vendors still aren’t profitable. By next year, the AI landscape could look very, very different.

And Now: The AI news, trends, and resources busy dev teams need to know

💡 Spotlight: Newly Launched AI Startups

- Inherent: $50M from Index Ventures for AI-powered research
- Dapple: $30M from The Raptor Group for AI infra in regulated spaces
- Upriver: $14M from Valley Capital for AI-native data engineering
- Willow: $7M from Hetz Ventures for agentic IAM
- Offroad: $7M from Ibex for agentic IAM
- Oplane: $5.2M from Seed Capital for AI coding sec risk management
- BeatpulseLabs: $1.8M from Araya Ventures for real-world training data

🚀 New Models & Products Spotlight

- Fable 5: The first Mythos model is a performance heavy hitter
- Nemotron 3 Ultra 550B: NVDA’s latest SOTA
- Gemma 4 12 and DiffusionGemma: GOOG’s newest mobile-first multimodal and text gen models
- Qwen3.7-Plus: BABA’s latest model family goes multimodal
- M3: MiniMax’s open-weight model goes big on coding and agentic
- MiMo-V2.5-Pro-UltraSpeed: Xiaomi’s new 1T-parameter flagship
- North Mini Code: Cohere’s 30B MoE coding model
- Ideogram: The first open-weight text-to-image model
- Cosmos 3: NVDA’s new SOTA world model
- Mellum 2 12B: JetBrains’ model for coding and agentic

💻 Development

FrontierCode: Benchmarking Mergeability of AI Code
Cognition’s new benchmark determines how nicely your AI-generated code plays with the rest of your codebase.

[GH] ECC: Agent Harness Operating System
This project helps you optimize harness performance across a variety of models.

Report: AI’s Impact on Eng Velocity Is…15% More PRs?
This report suggests that AI’s impact on engineering velocity may be overblown.

Search Toolkit
Mistral’s new open framework is intended to help you quickly build search pipelines for your AI apps.

🤔 Interesting AI Projects, Research, and Updates

Oh Good, There Are New Worms that Attack MCP Devs
Researchers have identified new Python-based worms that target bioinformatics and MCP devs.

How Hackers Steal Tokens from Your Public AI Projects
This Vercel post outlines how public businesses like Chipotle are getting their bots hijacked (and their inference pilfered).

Improving Agentic Reinforcement Learning with Token In, Token Out
This guide explains how to improve RL at the token level by reusing decoded tokens.

Bot Web Traffic Overtakes Human Traffic
It finally happened, says Cloudflare.

💼 Hiring and Community

Startups Hiring This Week:
- Software Eng, AI Infra → Fireworks AI
- Python Dev/AI Eng → Inherent Technologies

Mid-Markets Hiring This Week:
- Engineering Manager, Data → Caylent
- Lead Platform AI Eng → Tribe AI

Enterprises Hiring This Week:
- Software Eng, Data Platform → WhatNot
- Senior Agentic AI Research Scientist → Axon

🏭 Industry News & Trends

ByteDance and Mistral Consider Chips
Both the Seedance creator and the French AI lab are reportedly considering manufacturing their own silicon.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

Anthropic Embeds Engineers at National Security Agency
Being embroiled in a lawsuit with the Pentagon hasn’t stopped the Claude vendor from placing six engineers at NSA.

US to Close NVDA → China Chip Loophole
Uncle Sam will close the door on a loophole that still allowed NVDA chips to be sold to China-headquartered companies.

Deep Dive: Interview with open-source builder Heinrich Krupp on curing agentic amnesia

AI agents should have enormous potential to execute tasks on our behalf. But they struggle with a variety of real-world challenges, including a bad case of amnesia once the context window fills up. Running multiple jobs often means having to reinvent the wheel, re-prompting the agents without the benefits of any prior learnings.

Open-source builder Heinrich Krupp saw an opportunity to design an operational memory layer for agents to help him manage enterprise data flows with his project MCP Memory Service. Below, he explains the reasoning for his project and where he believes agentic memory needs to improve.

Moving Memory From Individual Apps to Centralized Infra

Despite all the enthusiasm for AI agents, recent reports suggest that only a fraction of enterprises have bothered to put them into production for a variety of reasons: Reconciling how most orgs’ data is locked up in legacy systems; constraints around data architecture and flows; and the usual enterprise-level governance and compliance concerns.

But agents themselves still struggle with a variety of performance issues that keep them from being the durable, reliable workers of the future they’re supposed to be. One glaring example is agents’ memory, or lack thereof, between jobs. Even after successful runs, agents by default begin their next jobs with a severe case of amnesia.

While some orgs have taken the approach of building agentic memory into specific apps, there’s a growing need for a wider, infrastructure-based approach for agents that can remember how to do things correctly. Open-source builder Heinrich Krupp tackled this challenge with his project MCP Memory Service, and explains his approach below.

MCP Memory Service provides persistent semantic memory for agents. Image courtesy mcp-memory-service

MCP: Generally Useful, But Where’s the Memory?

Krupp, an infrastructure and data management veteran, explains that his day-to-day work with enterprises on cloud infrastructure, delivery, and provisioning resources led him to start experimenting with MCP in December of 2024.

“My immediate reaction was: ‘This is genuinely useful!’ I had been working with AI tools for a while and had noticed a consistent gap. There was no memory layer. So I sat down with Claude, brainstormed the scope, and started building.”

The builder’s design thinking prioritized a privacy-first approach and open-source tooling, which led him to disqualify ChromaDB as a vector store for being too heavyweight and tack toward sqlite-vec for performance.

“I juggle a large number of projects with constant context switching, and I needed a reliable way to store important facts, decisions, and discoveries so they’d be accessible in the next session. When you’re managing Terraform configurations, security compliance, and customer-specific architecture decisions across 50-plus enterprise accounts, losing that context at the start of every AI session is a real tax on your work.”

After experimenting with different versions of Anthropic Claude, the builder found a fit with Claude Code’s hooks, embedded commands that users can build in across a Claude project’s lifecycle. Using this feature, Krupp built “Memory Awareness Hooks” to automatically inject context at the start of new sessions and keep learnings persistent. “This effectively gives Claude Code a persistent brain. That was the breakthrough that made the whole thing feel complete.”

Building Memory that Doesn’t Start from Zero

Krupp notes that his agentic sessions always seemed to start from nothing: No matter how many explanations he’d enter into the prompt, after a few dozen tool calls, the context window would fill up, he’d restart, and begin all over again.

“The tools that existed at the time either treated memory as a cloud subscription (proprietary, per-call pricing, data leaving your infrastructure) or as a simple markdown file you maintained manually, which is just organized forgetting with extra steps. Nobody had built memory as actual infrastructure that was self-hosted, semantically searchable, accessible via a standard API, with a lifecycle that includes consolidation and forgetting.”

The builder suggests that AI memory seemed like a personalization feature for chatbots, not a shared data layer for multiple agents, tools, and sessions. “The decision that shaped everything else was treating memory as infrastructure rather than a feature.” Krupp set about designing a REST API first (with 76 endpoints) so any agent, client, or language could interact with it without the need for specialized SDKs. “MCP support came on top of that, not instead of it.”

“On the storage side, the core tension is retrieval precision against latency and cost. We use local ONNX embeddings (sentence-transformers’ all-MiniLM-L6-v2, running fully on your hardware). No API calls per query, no per-token cost, no data leaving your infrastructure. The trade-off is embedding quality versus a frontier API model, but for most memory retrieval tasks the local model is more than sufficient, and you get 5ms read times against sqlite-vec.”

Krupp also designed a hybrid backend that bridges the gap between local and cross-device. “You want local speed for reads but you want cross-device sync. So the architecture separates concerns. sqlite-vec handles all reads at 5ms, a background process syncs to Cloudflare for durability and multi-device access.” By separating the MCP server and the network service at runtime, the builder avoided ‘database locked’ failures.

“The toughest trade-off was memory granularity. Storing at the turn level gives fine-grained retrieval but spreads a session’s semantic signal across many entries, which hurts benchmark scores on things like LongMemEval. Session-level storage scores higher on those benchmarks but gives you less precision when you want to know exactly what was said about a specific topic.”

Krupp argues that for memory granularity, chasing benchmark scores misses the point. “In production, fine-grained retrieval with a good quality scoring system outperforms coarse session storage on the tasks that actually matter.”

“The knowledge graph layer was an unexpected evolution. It started as a way to link related memories, but once you have typed edges (causes, fixes, contradicts, supports) you can query causal chains, not just similar facts. That’s a qualitatively different capability.”

Job-to-Job Memory vs. Context Rot

The holy grail in agentic engineering appears to be performant, reliable long-horizon agents that are immune to the context rot and hallucinations that set in with prompts that are too long and complicated. Krupp clarifies that MCP Memory Service’s agentic memory isn’t the solution to this challenge, but to a different one.

“Memory at the MCP level doesn’t solve in-session context rot. That’s a different problem involving attention mechanisms and context window management. What it does solve is the inter-session problem: Why does every new session or new agent start without knowing what the previous one learned?”

“The practical mechanism is what we call context providers, automatically injecting relevant memories at session start based on the current project, the current files, the recent activity. In our setup, this reduces context token usage at session start by 65% or more. The agent already knows the architecture, the recent decisions, the known failure modes. You don’t re-explain; you continue.”

“For multi-agent pipelines, the knowledge graph becomes the coordination layer. One agent stores a learning with a specific tag. Another agent retrieves it without any direct inter-agent communication protocol. There’s a user in our community running a five-agent cluster where they use the memory service as both shared state and an inter-agent messaging bus. Cluster agents write to a sentinel tag, the local agent polls that tag. That pattern emerged naturally from the tagging system; nobody designed it explicitly.”

The builder clarifies that memory isn’t the only missing piece for consistent agentic performance. “You also need good task decomposition, clear agent handoff protocols, and checkpointing within long runs. Memory is the persistence layer. It doesn’t replace the orchestration logic. But without it, every long-horizon task is fighting against statelessness at the infrastructure level, which makes everything else harder.”

“Memory...doesn’t replace the orchestration logic. But without it, every long-horizon task is fighting against statelessness at the infrastructure level, which makes everything else harder.” -Heinrich Krupp, Creator / MCP-Memory-Service

The Role of MCP in Agentic Memory (and General Agentic Ops)

“MCP matters because it gives memory a standard interface that AI clients can discover and use without custom integration work. Before MCP, connecting a memory service to an AI tool meant writing glue code for every client. Now Claude Desktop, VS Code with Copilot, Cursor, and a dozen others can connect to the same server with a config entry.”

Krupp cautions that MCP itself may not make sense to be the lynchpin of software architecture for a number of reasons. “The REST API is equally important and more universal. If MCP evolves, is superseded, or simply isn’t supported by a particular client, the memory infrastructure still works via HTTP. Twelve MCP tools sitting on top of 76 REST endpoints is the right architectural layer separation.”

“MCP does some things well, like discoverability, tool schema negotiation, and integration with AI client ecosystems. But MCP doesn’t replace a stable HTTP API for programmatic access, agent frameworks that operate outside MCP-aware clients, and enterprise environments that need REST for their existing auth and proxy infrastructure.”

“In the long term, I think MCP will become a standard the way REST became a standard, ubiquitous but not special. The interesting competition will be at the memory content and quality layer, not at the protocol layer.”

How Startups Should Think About Agent Memory Economics

Krupp suggests that the right price tag to stand up an early, no-frills agent memory system is $0. “The default setup I use of an SQLite-vec backend with local ONNX embeddings costs essentially nothing beyond the server it runs on. No per-call API charges, no vector database subscription, no data egress. For a startup in early development, that’s the right starting point.”

“The economics change when you need team sync and multi-device access. The Cloudflare backend adds real costs, but at typical startup scale, it’s in the range of single-digit dollars per month. That’s worth it when you have multiple developers or agents sharing a memory layer.”

“The comparison that matters for startups is against the alternative: calling a frontier LLM to regenerate context on every session. Memory retrieval at 5ms with zero API cost replaces a non-trivial number of LLM tokens per session. If you’re doing active AI development with many sessions per day, the token savings pay back the infrastructure cost quickly.”

“My practical advice is that teams shouldn’t over-engineer the memory layer early. Start with semantic search and basic tagging. Don’t build a full knowledge graph before you have 1,000 memories worth organizing. The consolidation and quality-scoring systems are powerful but they’re optimization layers; the core value is simple, persistent, semantically searchable storage. Get that working first, understand your retrieval patterns, then add complexity where it actually helps.”

The builder adds: “Self-hosting changes the unit economics entirely. Proprietary memory APIs charge per storage operation and per retrieval. At scale, that adds up. Open-source, self-hosted infrastructure with local embeddings is a fundamentally different cost structure that startups should think about seriously, especially if memory is a core part of the product.”

How Builders Should Approach Agent Memory Economics

Krupp reiterates that the biggest mistake that builders make today is treating agent memory as an individual app feature, rather than as centralized infrastructure. “When memory is a feature, it belongs to a user, lives inside an application, and gets designed around a single interaction model. When memory is infrastructure, it’s a shared layer that multiple agents, tools, and workflows can read from and write to. The architectural implications are completely different.”

The second biggest mistake, says the builder, is storing everything without a quality model. “Raw storage without scoring creates retrieval pollution. You end up with a database full of session noise, test outputs, and transient observations that contaminate your search results. Memory needs a lifecycle: Store, quality-score, consolidate, and eventually forget. The ‘forgetting’ part is uncomfortable for developers, but it’s essential. Stale memories, outdated facts, and superseded decisions actively harm retrieval quality.”

“The third biggest mistake is conflating vector similarity with causal knowledge. Semantic search finds related content well. It does not tell you that A caused B, or that Fix X resolves Problem Y, or that Pattern Z contradicts Pattern W. For complex technical domains, you need typed relationships, not just cosine distance. That’s why the knowledge graph layer matters.”

Krupp recommends that teams that are specifically building AI products prioritize memory isolation early. “Who can see what? If Agent A writes a memory, should Agent B be able to read it? How do you scope memories by customer, by project, by context? Getting the tagging and identity model right early saves painful migrations later.”

“When memory is a feature, it belongs to a user, lives inside an application, and gets designed around a single interaction model. When memory is infrastructure, it’s a shared layer that multiple agents, tools, and workflows can read from and write to. The architectural implications are completely different.”

Getting Agentic Memory Enterprise Ready

As an enterprise architect himself, Krupp reviews the core requirements for enterprise software usage: “Data sovereignty, auditability, and isolation. Each one implies a specific technical answer.”

“Data sovereignty means the memory service has to run on your infrastructure, using your compute, with embeddings generated locally. No API calls to an external service for retrieval. Every sensitive architecture decision, customer interaction, or compliance note that goes into memory stays inside your perimeter. Local ONNX embeddings are non-negotiable here.”

“Auditability means knowing who stored what, when, and why. The service includes an audit log plugin. Combined with the x-agent-id header that auto-tags memories by agent identity, you have a clear lineage trail for every stored fact.”

“Isolation means multi-tenant memory where one team, customer, or agent cannot access another’s memories. This is solved via tagging and scoped retrieval, but it requires deliberate design at the application layer. OAuth 2.1 with Dynamic Client Registration (RFC 7591) handles authentication and client provisioning, the same auth standard enterprises expect.”

Krupp notes that agentic memory is still missing a few key pieces of the puzzle for enterprises. “What’s genuinely missing at this point is formal certification and compliance documentation. SOC 2, ISO 27001 alignment, GDPR data residency statements exist as architectural properties but not as third-party certified artifacts, or at least, not yet.”

“For any enterprise that needs checkbox compliance before adoption, that’s the current gap. Nobody has demanded it yet; the project’s users have either been comfortable with self-assessment or have been developers who can evaluate the architecture themselves. That will change as the project matures.”

Evolving an Open-Source Agentic Memory Project

Krupp concedes that there are many approaches to solving agentic’s amnesia problem, and that there isn’t necessarily a clear-cut leader at the moment. “I periodically compare the project against other memory layers as they become popular. When something interesting appears, I analyze it, usually with Claude, and ask: What can we learn from this? What should we adapt?”

“That process has generated a number of RFCs and design discussions in the project. One issue proposed transitive closure and abductive inference for the knowledge graph. Capable contributors pick those up and implement them. The project has become genuinely community-driven in that sense.”

“I think that what made this community possible was investing in infrastructure that has nothing to do with the code itself. Things like thorough documentation, a comprehensive wiki, deployment automations, periodic housekeeping, and clear contribution paths. Projects that don’t invest in those things don’t attract serious contributors.”

“If I could, I’d like to mention one last thing about the project. Being honest, I think it deserves more support from sponsors. The maintainer workload remains significant even with extensive agent automation, including steering the project, reviewing contributions, maintaining quality standards, and responding to the community. If the companies and developers who depend on this infrastructure valued it at the level they’d pay for a proprietary alternative, the project would be in a very different position. Open-source sustainability is a real problem in this space, and memory infrastructure is no exception.”

What’s Next for Agentic Memory

The builder is realistic on the place of memory within the future AI stack. “Basic vector storage will be commoditized. It’s already happening. The differentiation will move to the quality layer (how memories are organized, scored, and consolidated) and to domain-specific memory schemas that encode deep knowledge about particular fields.”

“A healthcare AI’s memory is not interchangeable with a software developer’s memory. The schema, the relationship types, and the forgetting policies are all different. But durable memory will become table stakes for serious AI products. The competitive advantage will be in the quality of what you remember and how intelligently you use it, rather than simply having the ability to remember anything at all.”

“Also, a less-obvious learning I discovered while building this project is that the most useful patterns for how to use memory in AI systems have come from the community, not from me. A user running a multi-agent cluster discovered that the tagging system could double as an inter-agent messaging bus, no custom protocol, just sentinel tags.”

“Another user put the service behind a Cloudflare tunnel with a self-built auth proxy and made it remotely accessible across all their devices. These weren’t designed features; they emerged from people solving real problems with a flexible primitive. That suggests something broader about how to think about AI memory. We may be able to accomplish more if we don’t design for a specific use case, but instead design for composability. A memory service that exposes a clean, flexible API will find use cases you never imagined. The ones you do imagine will probably be the least interesting ones.”

This interview originally appeared on Heavybit.com.

Deep Dive: Interview with open-source builder Moaz Muhammed on how orchestration lets anyone manage agents

While agents seem to have exciting potential, they’re still so early that entrusting them with anything other than simple tasks is still a dicey proposition. The future should be about being able to provide complicated, in-depth prompts that smart agents figure out and execute properly, but longer prompts and more context lead to rot and hallucinations.

Open-source builder Moaz Muhammed believes that multi-agent orchestration may be the path forward. Below, he explains how he designed his project CodeMachine to tackle the long-horizon challenge.

For now, AI agents are autonomous entities to which users can delegate simple tasks: Monitor your calendar. Sort emails. But for now, when the time comes to combine a variety of tasks, or execute a complicated multi-step process, agents generally need orchestration and additional human supervision to get any semblance of reliable performance.

While some analysts predict significant adoption of agentic into enterprise apps this year, we haven’t quite arrived at the exciting future state where agents can handle complicated, time-consuming tasks on their own with little supervision. Yet. Open-source creator Moaz Muhammedexplains his approach to agentic orchestration in the CodeMachine project.

From Spec-to-Code to Architecture

The creator explains the origins of his project as being a spec-to-code project, initially intended to help users build enterprise-grade projects without the friction of using complicated coding tools. “We started with one hard-coded workflow. However, we ended up having the project build about 90% of itself via dogfooding.”

“The initial idea was having a workflow to build architecture, then start converting the architecture into a plan. Like agents in a workflow, with the first agent writing architecture and related decisions like which stack to use: front end, back end, and so on. Another agent took the MD file and converted it into a plan. A third agent converted the plan to tasks based on the size and scope of the project.”

Moaz notes that the original architectural spec seemed to be a hit with early users and may have gone on to influence the structure of subsequent AI projects. But over time, feedback started to vary, with software engineers taking issue with the original projects’ waterfall-based methodology.

“They said, ‘We want something faster.’ Some projects would take hours to build. I even built one with about 80,000 lines of code, which proved that CodeMachine was at least capable of building huge projects. It wasn’t perfect, but a good starting point, because CodeMachine provided a real and organized scope, with every file in its place, everything well documented and explained without human supervision.”

The creator explains that as the project evolved, he shifted to more of an engineering focus with clear architecture. “This was a very big problem [for AI developer tools]. When you started a project, you couldn’t scale it, because you weren’t really building. You were ‘vibing.’” By shifting focus on building full architecture for projects, Moaz was able to avoid generating spaghetti code projects.

“From there, we pivoted from being hard-coded to a single workflow into being a workflow infrastructure for any kind of workflow. That was actually a huge transformation.”

Growing Adoption with Tool and Framework Choices

Moaz explains several deliberate choices he made over the course of building the project, such as focusing on command-line interface over the more-common browser-based chat window. “Not only was the terminal popular in well-known coding tools, but developers building software tend to use a CLI-based terminal interface. They’re familiar with it, and I’m familiar with it. Personally, I love the terminal so much as an interface.”

The creator suggests that some design decisions not only made the project more appealing to engineers, but also made it more marketable. Popular and trendy tools and frameworks can help attract new users despite strong opinions and preferences among technical audiences. “I used to use Node,js, then switched to using Bun, which was becoming an extremely popular alternative even before it was acquired by Anthropic.”

Of course, the choices weren’t all for marketing: The goal was to stay on top of a trending stack while delivering real value, especially since some engineers will always be hesitant to adopt new technologies due to potential bugs and lower reliability. The creator suggests that the strongest projects are those that balance engineering with marketing.

“It’s not enough for a product to work. Engineers need to enjoy using it and feel connected to it. The most successful projects understand this human side, making deliberate choices that are not only technically sound but also appealing to the developer community.” So, the decision to move from Node.js to Bun was not purely technical. It also helped the project align with emerging trends and attract attention from engineers who were excited about the new ecosystem.

Between Human-in-the-Loop and Full Autonomy

The creator tactfully avoids weighing in on whether there exists an ideal balance between a fully autonomous agentic fleet or, on the opposite end, a tightly-restricted agentic program with one or more humans in the loop. “I can say that the philosophy of the CodeMachine project sits between [those two extremes]. It depends on the use case.”

“I designed this project to act as infrastructure for workflow agents for any coding task. For example, you could use it as a full pipeline to migrate from ancient languages like COBOL to a modern language. Some workflows might need permission or some sort of user interaction at times, but you can set the interactive portions of your project to control every autonomous step.”

“It’s an orchestration platform that lets you run long-running workflows in any configuration you want between interactive and autonomous.” The founder also concedes that there are well-founded concerns about agentic security, but emphasizes that security is up to individual builders.

“Users will interact with this project like infrastructure. For those who prioritize security, local models are a solid option. CodeMachine is fully local. Any data it stores stays on your device and never leaves it. The only point where data exits is when it’s sent to your provider, such as Claude Code or Codex. So while the project itself poses no security concerns, full end-to-end security comes down to keeping your setup as local as possible.”

What Agentic Means for Future Developers

The creator is optimistic about a future that enables builders, rather than focusing heavily on technical expertise. “I think the future is going to belong to people who have a ‘builder’ mindset, who can think in terms of architecture and how to build things brick by brick.”

The creator suggests that the future of software will belong to builders who embrace this different set of skills. “People who take the initiative to build things, whether using Claude Code or other tools, but also have the curiosity to ask why Claude and other tools make the choices they do. People who are curious about the why will build the next great era of products.”

This interview originally appeared on Heavybit.com.

Depending on which people you talk to, the promise of fully agentic development is either far away, or right around the corner. It’s possible to deploy individual agents to perform simple, individual tasks, but one of the hottest races in software development seems to be mastering long-horizon tasks that are complex and contain multiple subtasks, without succumbing to context rot from overly-long prompts that cause models to hallucinate. While there’s now a new long-horizon-focused benchmark, there still isn’t a consensus on the best way to arrive at the nirvana-like state of agents flawlessly executing on long-horizon jobs without the need to spend many cycles (and tokens) micromanaging them. But people are trying. Cursor has introduced agent dev environments. GOOG has released Agent Executor, a distributed agent runtime to improve durable execution, and Genkit Middleware to make agentically-developed products more robust. A new research paper making the rounds suggests that a meta-agent harness (which reportedly delivered more than a dozen new software features over a few weeks) might be a possible solution. To address the need for more-performant agents, we also have new agentic evaluation guides this week from Cameron Wolfe and OpenAI. There are still many gaps in agentic processes that are keeping us from that perfect future, which means many opportunities for startups to build picks and shovels to fill the gaps.

Coming Up Next: The AI learning resources and industry trends you need to know

💡 Spotlight: Newly Launched AI Startups

- Trajectory AI: $15M from Conviction for continual learning
- NanoCo: $12M from Valley Capital for enterprise AI agents
- Graphon AI: $8.3M from Novera Ventures for “pre-model intelligence layer”
- Modiqo: $3M from Heavybit for deterministic agentic execution
- Voker: $2.2M from YC for agentic analytics

🚀 New Models & Products Spotlight

- Claude Opus 4.8: With improvements over 4.7 across the board
- Qwen3.7-Max: BABA’s new SOTA on coding benchmarks
- Gemini 3.5 Flash Low: Generates 45% fewer tokens than 3.5 Flash Medium
- OlmoEarth 1.1: Ai2’s new planet-scaling model family
- Ettin Reranker: A new family of sentence transformer models
- Composer 2.5: Cursor’s latest coding agent
- Cosmos Predict 2.5: NVDA’s new world model
- MAI-Image-2.5: MSFT’s newest multimodal offering

💻 Development

[GH] raindrop: Agent Evals for Coding Agents
This project helps your coding agent write evals against your codebase.

[GH] headroom: Compressing LLM Inputs to Save Tokens
This chunking project helps you do a little less tokenmaxxing.

State of AI for WebDev 2026
According to this report, frontend professionals are adopting Claude Code more and paying more, but working harder while code quality drops.

Guide: Building Agents from First Principles
ML engineer Anshuman Mishra suggests five core steps: prompt, model action, environment, reward, gradient update.

🤔 Interesting AI Projects, Research, and Updates

Coming in July: MCP’s Next Spec Release Candidate
The next version of the AI-to-API open standard will offer a stateless core, new extensions, and OpenID/OAuth-style authorization.

Is High Bandwidth Memory the New Bottleneck?
Sriram Krishnan suggests the next great bottleneck in AI economics is HBM.

Why Pretraining Fails
Prolific podcaster Dwarkesh Patel explores why model pretraining still isn’t perfect.

How LLM Architecture Is Changing
Researcher Seb Raschka explores how new LLMs are utilizing architectural changes to avoid KV cache, memory, and attention constraints.

💼 Hiring and Community

Startups Hiring This Week:
- ML Data Eng → Titan
- Senior Full Stack Eng → SalesPatriot

Mid-Markets Hiring This Week:
- Principal ML Eng → Acclaim AI
- Sr. Software Eng → Kong

Enterprises Hiring This Week:
- Senior PM, Core Platform → Camunda
- AI Eng Manager → Front

Karpathy Joins Anthropic
The OpenAI co-founder-turned-researcher-turned-prolific-GitHub-creator has joined the Claude vendor.

🏭 Industry News & Trends

This Week’s M&A Deals:
- OpenAI acquires, then quietly dissolves Weights.gg team
- Anthropic acquires SDK startup Stainless
- Anthropic/Blackstone’s consortium’s first acquisition: Fractional AI

BABA’s New Gig: Chips
The Chinese internet giant briefly pauses its breakneck Qwen model releases to reveal the Zhenwu M890, its new AI chip.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

Executive Order on AI Regulation Paused After Ex-AI Chief Call
The AI EO is on hold after former special advisor (and current VC) David Sacks urged the White House not to stifle AI innovation with pesky “regulation.”

China Restricts Overseas Travel for AI Experts
As the AI talent war rages on, China is now limiting international travel for top AI researchers, founders, and execs.

CA Governor’s Executive Order on AI Job Disruption
The governor’s office in the Golden State issues an EO to drive awareness for and protect against possible job losses due to AI.

AI Criticism from the Pope, Graduating Students
Pope Leo expresses concern about AI abuses as graduates across the US boo commencement speakers like ex-GOOG CEO Eric Schmidt on the topic.

Deep Dive: Interview with founder Prateek Jannu on how computer use might be AI use case with the fastest adoption

There continues to be an unspoken divide for AI adoption: While nimble startups and growth hackers post constantly about their exciting new discoveries using AI tools, massive enterprise companies still seem slower to adopt, continuing to rely on decades-old SaaS platforms to handle core functions like finance, sales, and operations.

Founder Prateek Jannu suggests that one of the fastest ways for AI startups into the enterprise may be via computer use such as through his open-source project Coasty AI. Computer use is agentic functionality that replicates input from users’ keyboards and mice to operate the tools that enterprise users already use.

However, Jannu warns that getting through to enterprise customers isn’t just a matter of having cutting-edge AI technology. It’s about repeating and surfacing the behaviors and user experience that the human enterprise customers expect.

How Do You Get Change-Resistant Enterprises to Adopt AI?

For some time, enterprises have resisted change, with internal departments using a specific software platform as their center of gravity: The ERP for finance, the CRM for sales, and so on. Each with painstakingly built custom configurations (with new team members not allowed to touch anything!). So what happens in the age of AI when everything speeds up and the pressure is on to modernize, even at risk-averse enterprise orgs?

Founder Prateek Jannu’s project Coasty focuses on what he considers to be AI’s most immediately valuable use case for enterprise: Computer use, which defers actual point-and-click functionality on desktops and browsers to autonomous AI. Says the founder, computer use lets enterprises gain immediate value from the speed and versatility of AI, while not disrupting existing enterprise tech stacks (especially the customized Netsuite instance that took the finance team years to perfect).

Addressing Enterprise Inertia and Resistance to Change

Jannu notes that the Coasty project came from observing how, while many academics have been excited about the potential of AI, enterprise orgs seemed to recoil in horror. “A lot of companies, even among the Fortune 500, did not want to build MCP servers, or anything they thought could destroy their systems or accidentally leak sensitive information.”

From there, the Coasty project launched as a computer use agent with high performance scores 82% on the agentic OS World benchmark when tasked with looking at users’ screens and figuring out how to interact as a keyboard and mouse would. “A score is just a score and doesn’t necessarily imply anything with regard to long-term tasks,” the founder admits, but it may be an indicator of potential value for a variety of use cases for form fills, accounting, hotel management, and others.

The founder notes that building AI computer use has taken a different path than previous-generation workflow automation and screen-scraping tools due to changing interfaces and other external factors. “We use five different systems with a variety of models, which is important for enterprise systems because UIs also change.”

“If an accounting team is using a legacy system that gets some kind of software update, or alternatively, if they’re filing state taxes but next year, there’s a new option, some new provision for an updated state tax rate which a simplistic workflow automation tool might not pick up. We’ve built our models to look for and pick up such changes and understand variances between steps that previous-gen solutions might get stuck on.”

AI Features Enterprises Care About: Speed vs. Accuracy

“When we started out, we noticed other teams in the computer use space seemed to be focusing on speed, perhaps because teams have looked at speed as a factor when they think about process automation. ‘If it’s faster, it must be a better option,’ was the thinking, and why would anyone pay for something that’s slightly slower?”

“This was a hard decision for us: Can we compromise on speed right now but guarantee that we can be more accurate than any system out there? That’s a major trade off.” Jannu notes that while having low latency definitely demos well, but a computer use tool that’s fast but inaccurate is virtually pointless.

As his team reviewed their options, they realized that in the AI age, as code becomes faster and cheaper to produce, software will eventually change daily, which will mean that UIs and APIs will too. “If you focus on speed, you’re going to break at some point.” Meanwhile, enterprises run on decades-old software that can have fragile configurations or integrations, causing them to be extremely hesitant to change vendors with any frequency.

“Once your AI is logged in to their software, [enterprise teams] don’t want to make significant changes. If you did go in and make changes to their tools, they wouldn’t renew. They wouldn’t even adopt. So we decided to focus on accuracy instead of speed for enterprise personas.”

Enterprise Challenges: Speed and Decision Tree Errors

Moreso than building for any specific interface (desktop, browser, or terminal, all of which Coasty supports), Jannu suggests that significant challenges include decision tree errors and time thresholds. “Anytime you give a task, you expect it to take a certain amount of time. Small tasks are relatively fast, but as users build confidence, they start testing boundaries and make bigger asks.”

“They start requesting more long-horizon tasks trying to test how well Coasty can do, tasks that may take closer to 10-20 minutes. Sometimes, tasks can go off trajectory. If I request the creation of a certain kind of database, and it chooses the wrong table in step two, it’s going to be completely wrong for the entire trajectory. So it’s really about how to get those first five to 10 steps right. And even if the system gets something wrong, how can we get back on track?”

“One thing we focus on with this project is self-correction: If there’s a mistake, Coasty can go back and rectify that. One thing that was really hard to pin down was controlling and auditing which step the system gets wrong.” The founder explains how the process involved building dozens of testing layers to detect errors and revert to previous steps, a process that remains complicated.”

Hard Lessons: Enterprise Users Expect Their Own Flow

“We also learned that enterprise users want an easy way to provide logins for their agents, but the challenge is doing that safely. We added what we call agent credentials which we store in an encrypted format, but the agent never sees the credentials, and will prompt users to enter them as needed.” The founder suggests he expected this behavior: Of users to instruct agents to take action until they ran into a roadblock and needed something, like credentials.

“We realized that people were prompting their agents more like they would a human, because that’s how these users were expecting it to work. For anyone trying to implement computer-use agents as well, I’d highly recommend trying to see what users expect from the system: Sometimes they may expect a full, end-to-end prompt, but sometimes they’ll say: ‘Hey, just come back to me and ask if you need credentials.’”

“This is something that we did not expect. And a lot of users got turned off because of that. We had to learn the hard way. And we had to talk to a lot of users to find out about these issues, whether their issues were related to costs or functionality, before we realized it was something as simple as users expecting agents to come back and ask them for credentials.”

What’s Missing in Computer Use: Security and Safety

The founder admits that the ultra-viral OpenClaw project kickstarted adoption of agents for personal use, but suggests that agentic has seen a noticeable delay on security to accompany all the exciting advancements. “If you look at how software had a boom in the early 2000s: A lot of compliance companies came up with their own standards.”

“But at the end of the day, the larger companies, including the Fortune 100, came together with the goal of figuring out how to identify whether software is ‘safe’ enough for use by big companies or clients. And they came up with SOC 2 and similar methodologies.”

“I definitely think there’s going to be at least some kind of standard in the future, a security framework specifically for agents that could also cover computer use or general agentic frameworks. It might involve stress testing agents with jailbreaking problems, putting them through some kind of verification system to ensure it isn’t vulnerable to prompt injections or other kinds of attack.”

The founder suggests that an industry standard might emerge once enterprise adoption of agents hits critical mass, which doesn’t seem to have happened yet. “A lot of these companies are adopting [agents] to a very small degree. Maybe it’s one small team using it for a certain reason. But if you want the whole organization to use a certain kind of AI, the company needs to make sure that if something does go wrong, someone is responsible.”

“There’s going to be a sort of framework or a standard set for it, ideally an open standard so people can go audit it. For our project, we maintain a really slim internal security standard so if something does happen on our end, we’re ready for it.” The founder suggests that every agentic software project should stand up their own security standard along with a battery of internal tests.

“If you’re building agentic, try to test it. Break it. There are a lot of good open-source repositories with guides on how to do prompt injections or jailbreak certain system prompts.Try them on your own system. Check how it works. Is it failing? How? If you’re trying to take your software to a production-grade level, try to break it as much as you can beforehand.”

Computer Use as an Enterprise Modernization Opportunity

“I think there are a lot of enterprises that depend on legacy systems that do not have APIs or MCPs built. Maybe 95% of them do not. I believe [computer use] is the best way to get started with AI [for enterprise customers] because a lot of companies are under pressure right now to ‘implement’ AI.”

The founder notes that while some early adopters are diving into building MCP servers, most companies are still pondering implementation and potential security issues. “[Enterprises] have a lot of these questions. What’s the best way to go about [implementing AI]? Try to build computer use. At the end of the day, your software does have a login screen of some kind. People use it with a keyboard and mouse. And if humans can use these things, computer use AIs do the same.”

The founder suggests that as models only continue to improve, computer use will as well. “If today, we can perform a one-hour task reliably, maybe within a year, we can perform a three- to four-hour task reliably. Lots of companies have processes that take hours. With successful computer use, you can see efficiency improvements of 2x or 3x, even as you run agents as background tasks.”

“A lot of business units rely on tools that don’t have MCPs or AI-capable APIs. For example, I know of several accounting tools that don’t have MCP support, and won’t for some time? Maybe upwards of five to ten years? But whatever you do with a computer, with a keyboard and mouse, can be done right now with computer use.”

This interview originally appeared on Heavybit.com.

AI companies (and rich guys) are pushing forward on massive data centers like NVDA and IREN’s 5GW project and the Shark Tank Guy’s 9GW data center in Utah (which has provoked no small amount of backlash from residents). All the major players need silicon to rack and stack in the data centers, and land to build them on. But the electricity to power (and cool) data centers may be an even more in-demand resource, prompting AI leaders to take extraordinary measures. Amazon is focusing on its new “Titus” data center architecture that will offer 68MW of compute capacity (rather than 58MW) with built-in cooling. Microsoft is potentially abandoning its 2030 pledge to go carbon-negative due to the massive amount of power needed for its growing data center fleet. Japanese investor Softbank is momentarily putting down its checkbook to go into the AI battery business, with plans to launch a water-based battery by FY 2028. Studies show that wholesale electricity prices in regions near data centers have skyrocketed by as much as 250%+, and the International Energy Agency reported that while energy efficiency appears to be increasing, data center energy consumption also saw a 50% increase last year. All told, AI titans like MSFT, GOOG, AMZN, and META are on track to spend $700B in capex this year, with energy supply increasingly becoming a bottleneck. For the time being, everyone seems committed to building more resource-intensive billion-dollar infra for AI, which means a lot of opportunities for enterprising pick-and-shovel startups to bring the costs down.

And Now: The top AI news this week for busy founders and technical teams

💡 Spotlight: Newly Launched AI Startups

- RadixArk: $100M from Accel for open-source inference
- Subquadratic: $29M from various investors for long-context LLM architecture
- Nace.AI: $21.5M from Walden Catalyst for enterprise SLM
- White Circle: $11M from various investors for AI model monitoring
- DesignVerse: $5.5M from Begin Capital for AI-driven software modernization
- Lyrie.ai: $2M from undisclosed investors for “autonomous cybersecurity”

🚀 New Models & Products Spotlight

- Gemini 3.1 Flash-Lite: GOOG’s new low-latency, high-volume model
- GPT 5.5 Instant: The latest version of OpenAI’s general-purpose model
- GPT-Realtime: OpenAI’s new family of voice, translation, and STT models
- EMO and MolmoAct2: AI2’s MOE-focused model and new reasoning model
- Claude Sec Beta: Not based on Mythos, but Opus 4.7

💻 Development

What Goes into SKILL.md
AWS engineer Lakshmanan Meiyappan breaks down what actually goes into a SKILL file.

Guide: Local LLM Inference Hardware
With Copilot’s move to usage-based billing, founder Alfonso de la Rocha offers a guide to BYO inference setup.

[GH] ProgramBench: Tasking Agents to Build Software from Scratch
META Labs’ new benchmark measures how good an agent is at building new software without access to source code or online research.

🤔 Interesting AI Projects, Research, and Updates

How to Scale Your Model
This guide from GOOG DeepMind researchers covers how to scale model use, including TPUs, training, and inference.

Can You Make Inference More Efficient By Turning Memory into Compute?
Founder Fergus Finn proposes a solution to expensive AI cycles: Utilize memory as a stand-in for compute to process less data faster.

What a Difference a Harness Makes
Founder Nicholas Bustamante notes how top foundation models differ in performance changing nothing but their harnesses.

💼 Hiring and Community

Startups Hiring This Week:
- AI Agent Researcher → Vast AI
- AI Engineer → Max AI

Mid-Markets Hiring This Week:
- AI Engineer Researcher → Weights & Biases
- Senior Data Scientist → OneGlobe

Enterprises Hiring This Week:
- ML Eng, Drug Discovery → Genentech
- AI Engineer → GEV

🏭 Industry News & Trends

Cerebras’ $5.55B Raise is the Year’s Biggest IPO
The AI chipmaker’s record-breaking raise sets it at a $40B valuation.

META Acquires Assured Robot Intelligence
META’s latest acquisition is a humanoid robotics AI startup.

Anthropic and OpenAI Both Launch VC Funds
Anthropic’s fund debuts at $1.5B, OpenAI targets $10B. Neither deal will lead to any more of those hilarious AI circular financing memes, we’re sure.

April: The Month AI Pricing Just…Broke
Founder Anton Zagrebelny notes that drastic price changes from Claude, Copilot, and ChatGPT indicate a potentially larger AI pricing problem.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

The Pentagon’s Deal with 7 of the Biggest AI Vendors
All the usual suspects (minus Anthropic, of course), will “augment warfighter decision-making in complex operational environments.”

Will the White House Gain Oversight Over Major Models Pre-Release?
White House reps are apparently mulling an executive order to require government oversight prior to major model releases.

Mr. Trump Goes to China (With Mr. AAPL, Mr. META, Mr. CSCO, & Mr. QCOM)
This is the week the US President seeks an audience with Premier Xi (with a number of top AI executives in tow).

Deep Dive: Interview with developer Yusuf Karaaslan on how to spin up agentic skills using a variety of data sources

A few years ago, large language models made the headlines with ChatGPT, Claude, and Gemini: Hugely expensive, all-purpose foundation models with billions of parameters that could do just about anything. Now, development teams seem more focused on utilizing built-for-purpose agents to stand up and automate workflows.

We’re still in early innings for agentic, especially as builders continue to explore what can and can’t be done with agents. Below, Turkish developer Yusuf Karaaslan walks through how he designed the popular Skill Seekers project to turn his daily tasks into something that could convert knowledge from a variety of formats into smarter, more-capable agents.

How We Went from LLMs to Agents to Skills

In late 2025, Anthropic declared skills, modular, reusable, file-based instructions that encapsulate domain expertise, to be an open standard. Since then, hundreds of thousands of users have reportedly installed a variety of skill.md files to improve their agents’ abilities on a variety of tasks, including coding with specialized languages or performing non-coding tasks like spinning up documentation or creating datasets.

While performing his usual daily duties with Claude Code, Turkish developer Yusuf Karaaslan began experimenting with skills, and eventually ending up designing the Skill Seekers project, which processes and translates a variety of data sources (including GitHub repos, PDFs, videos, and others) into agentic skills. Below, he explains what led him to create the project and how thousands of users worldwide are finding value from agentic skills.

From Context Limits to Skill Libraries

Karaaslan, a game developer by trade, notes that his initial foray into working with skills came from trying to work with the back-end of the popular online game storefront Steam. He started with Claude’s official skill-creator skill, feeding it every relevant link from Steam’s documentation one by one and asking it to produce a complete skill for the inventory system — the goal being a direct answer instead of a guess. It didn’t work.

“No matter what I did, Claude would skip most of the input or cut it down far too aggressively, and it kept guessing at what I was asking for. My read was that because I’d asked it to handle both the heavy data gathering and the processing in one shot, it was optimizing for cost and cutting corners on the part that mattered most. Even the cut-down output was a little more confident than nothing, which was the clue. So I sat down and started building a separate system to do the heavy data lifting, the part that doesn’t actually need AI in the loop. That was the first prototype of Skill Seekers.”

The contrast once he piped that prototype’s output back into Claude was hard to miss. “Before, the answers were always hedged. ‘Maybe try this, maybe try that.’ After I added the Skill Seekers output to the context, the tone changed completely: ‘You can’t do it the way you want with Steam inventory, you need an external backend.’ Much more confident, much more direct, and actually correct. Even when I challenged it on details, it stayed true to Steam’s actual behavior. The funny part is that all this happened the same day Anthropic announced skills. The first version of Skill Seekers shipped that same night.”

From there, Karaaslan kept iterating on the prototype, prompting Claude Code to extend it with web scraping and working around internal context limits until each new data source slotted in cleanly. “Until a certain point, my focus was on feeding in as many data sources as possible to build a richer, more complex skill: The need to download a dynamic website, or the need to scrape a codebase. Once I felt we’d covered almost all the major data sources and some of the more niche ones, I started backtracking to improve the system architecture. At that point I hit Claude’s internal limits again: This time around analyzing the whole codebase. So I fell back on my usual trick: I had Claude generate full class diagrams, package diagrams, and flow diagrams for the key systems. With those UMLs in hand, I could pinpoint exact pain points, architectural problems, and more. From there, I designed the new architecture end-to-end and replaced the live version module by module.”

The developer’s iterative approach led him to map all connections and flows within the project, and eventually unify everything with a single interface, ensuring key functions were connected and removing duplicate branches. He explains that he relies on three primary skills himself for design, implementation, and PR reviews. “I actually developed these skills side by side with Skill Seekers itself, based on my own observations along the way. And I just open-sourced my AI workflows too.”

“Using my design skill, I start by using a card that simply explains test-driven development, and from that, I generate UML diagrams of how that should work. I then review, accept changes, and create issues and tasks, then create a development report of any issues. After everything is complete, I will send the output to the PR review skill. In this way, I have full control over the project.”

Accelerating AI Learning by Modeling Human Learning

Karaaslan suggests that his approach building skills started in much the same way he, himself, learns new things. “Before using AI, [to learn new development topics] I would read documentation, take notes, and learn from example projects. At first, I mimicked this process.”

The developer took a similar approach to eventually making Skill Seekers a project capable of ingesting data from a variety of different formats. “To implement how I learn things by watching videos into Skill Seekers, I would get code from the screen, collect timelines, and copy transcripts and screenshots into Skill Seekers. I was just mimicking how I learn without AI.”

The developer notes that after the release of the project, he observed how others would use it, noting that users would want to review new additions. “People would want to adjust some stuff, add some stuff, remove some stuff. I wanted to automate the process, so I created a skill for that, and afterwards, we created the workflows, and gave the skill to that workflow. Exactly what we would do manually. It’s about mimicking, optimizing, and automating.”

What Skills Add for Developers

Karaaslan suggests a variety of development use cases have emerged from the project. “It started as a helper, but it has become a data management system for any kind of input that can be structured and reproduced, and with minimal cost because once you create a new skill config file, you can run that config again and again with a single command-line prompt, or just by telling Claude to do it directly via MCP.”

The developer lists a variety of use cases, including an integration partner that uses the project to power its marketing and helper bots with skills that ping codebases to answer user questions about them. An academic AI research startup uses the project to scan new research papers for usable skills that the team can then use internally.

“Like I said, it’s all just data that AI can understand. You can update to new versions easily and connect to your codebase in one click. So you can create your own tools, your own skills from your codebase. You can give it official documentation, your own implementation, examples, maybe some video, and it all gets merged into one big skill that includes everything.”

“After you create skills, you can add a workflow to create an agent and system calls. I use that feature often because you can control the context. It’s actually a helpful tool for context engineering.”

“In my workflows I have an automated skill that’s triggered every time I push to the main repo. Every time I want to do design work I have a ‘design repos’ skill that I create with Skill Seekers focusing on the architecture and API. Whenever I run my agent skill, it automatically uploads that skill, and every time I run it, I know I have the latest version of my architecture as knowledge within the context inside the agent.”

How to Think About Skills for Beginners

“Let’s say you’re not very fond of AI, and you just want to do your work as a developer. Developers always think in terms of architecture, how we do stuff, how we structure stuff. If you want to use some new back-end framework or a new language you’re not experienced with, you can stay focused on system architecture while Skill Seekers builds a skill that turns your agent into an expert on your tech stack. The best part is that Skill Seekers has an MCP, so you can just casually tell Claude to create skills for your project. No extra steps.”

“Let’s say we wanted to start a new project and it’s in an area we’re not familiar with. We might want to do some light experimentation before building because the cost of learning seems too high. Previously, we might avoid focusing too much on learning entirely new frameworks and languages. We would tend to just stay with what we know now.”

“You can tell Skill Seekers, ‘I want to do this project, I want to use this technology, I want to try this new framework, I want to try this new library,’ in casual, natural language. The MCP will trigger and generate the skill for the latest, top-of-the-line skill for you. You can just say, ‘I have an idea’ or ‘I want to try this functionality of this language or this library,’ and you start writing anything that you can test.”

“It’s not too different from being an engineering lead. You say, ‘I have a vision. I want to do this thing,’ and it creates a team for you that knows that field and executes on your vision. You have a problem, you have a solution, you have a tool, you have a limitation, you have an output. This is always the same.”

Real-World Use Cases for Skills

“The biggest use case I see is builders using skills to power support for their GitHub repos across the different language versions they need to maintain. Each version has its own skill, and every time they push an update, an automatic trigger refreshes the bot’s knowledge. Doing that by hand is incredibly time-consuming.”

Karaaslan’s most-used example comes from inside his own company, where the team builds with Unity. He started by setting up an internal skill marketplace so developers could share work with each other, then began authoring skill config files for the technologies the team uses day-to-day. It didn’t take long to spot the bottleneck. “Even with Skill Seekers doing the hard part, the surrounding loop was repetitive: Create a config, generate a skill, package it, and manually upload it to the marketplace. The friction was in everything around the generation step.”

The developer explains his process of layering in fixes, one at a time. “First, I added a shared config repo so anyone on the team could publish or pull a config through their own Skill Seekers install. Then, I added packaging and uploading to the marketplace as native functionality inside Skill Seekers itself. The flow worked end-to-end, but I hit a new wall. The volume of data I was sending Claude during the local enhancement step burned through my usage almost immediately.”

The fix was to stop tying Skill Seekers to any single model. “I made it agent-, CLI-, and LLM-agnostic. Now I trigger the whole pipeline from the command line: Scrape, restructure, combine sources, run the local AI enhancement step on whichever model I want, package, and push straight to our internal marketplace. I tend to use OpenCode with Kimi for the enhancement step. I tend to get more usage with it, and in my own tests, the skill quality came out on par with Claude.”

That decoupling is also where Karaaslan sees the project’s longer-term direction. “My vision for Skill Seekers is for it to become a universal data layer for AI systems. Model-agnostic, source-agnostic, pipeline-friendly. Whatever the input, whatever the agent on the other end, the skill is the same shape. That’s the point.”

This interview originally appeared on Heavybit.com.

Beyond simple, one-off tasks like sending an email on your behalf, we’re seeing increased interest in long-horizon tasks in the world of AI agents: Multi-step processes like designing net-new feature specs or performing full codebase refactors. As we’ve covered previously, a key issue in the current state of long horizon is the battle between context window size (the maximum amount of information/text/data/tokens a model can ingest in one prompt) and a model’s performance. Larger context windows let users feed in more context to provide more-explicit direction for models, but they also introduce context rot (hallucinations and other performance drops). For one-off tasks, it’s generally considered a best practice to keep your context window as small as possible to minimize rot (and latency). That isn’t stopping teams from running increasingly complex agentic jobs, which, as researcher Toby Ord writes, is potentially leading to pricier agentic bills on an hourly basis. Major US labs are focusing on long-horizon functionality this week with launches such as OpenAI’s ChatGPT 5.5, which “outperforms GPT‑5.4” on the company’s internal long-horizon benchmark; and Poolside’s Laguna XS.2 and M.1, which the vendor bills as expressly “built for long-horizon work.” So are major Chinese labs, such as Moonshot’s Kimi K2.6 family, which includes a variety of agentic functionality to support long-horizon coding; BABA’s Qwen3.6-27B, which offers “flagship-level coding in a 27B dense model”; and Xiaomi’s MiMo-V2.5-Pro, a 1.02T-parameter model with “significant” improvements in long-horizon coding over its predecessor V2-Pro. Every new model is impressive in its own right, but there are still multiple breaking points in today’s long-horizon agentic landscape, including context rot, memory, and orchestration (among others). Which probably means plenty of opportunities to build picks and shovels to fill the gaps.

Up Next: All the AI news that busy dev teams need to know

💡 Spotlight: Newly Launched AI Startups

- exe.dev: $35M from Amplify/Heavybit for developer AI compute
- Rilian: $17M from 8VC for agentic cybersecurity
- General Analysis: $10M from Altos for adversarial agentic sec
- Redpine: $7.9M from NordicNinja for AI datasets
- NudgeBee: $3M from Kalaari for AI cloud operations
- Copperhelm: $7M from TLV Partners for agentic cloud sec

🚀 New Models & Products Spotlight

- Qwen3.6-Max Preview: Early look at BABA’s next major model
- ChatGPT Images 2.0: OpenAI’s latest image model
- Deep Research & Deep Research Max: Gemini 3.1-based research models
- Nemotron 3 Nano Omni: NVDA’s new multimodal for doc and A/V analysis

💻 Development

[GH] stash: Persistent Agentic Memory
Postgres-based persistent memory layer for agents.

Benchmarking Agentic Inference Engines
This lightweight harness benchmarks agentic inference engines. Try it here.

Guide: How to Write a Good AGENTS.md File
AI engineer Slava Zhenylenko covers best practices.

Guide: How to Build Agents That Actually Make It to Prod
Anthropic’s guide covers how to best utilize API calls, CLI, and MCP.

🤔 Interesting AI Projects, Research, and Updates

Blog: Choosing Code vs. Markdown for Agents
Founders Vikram Seekranti and Joseph Gonzalez outline the tradeoffs of being prescriptive vs. being flexible with agents.

Are You Tokenmaxxing? Your Model Might Be Budgetmaxxing
Fun fact: AI models tend to run over their prescribed budgets, no matter how many times you tell them not to.

Guide: Recursive Learning Models
Founder Akshay Pachaar explains how RLMs address context rot.

Research: Prefill-as-a-Service
This report explores an alternative architecture that distributes KVCache in the Cloud with potentially less data center compute usage.

Video: How Foundation Models Are Trained and Served
If you missed the latest Dwarkesh Patel podcast, Reiner Pope explains the math behind modern inference.

💼 Hiring and Community

Startups Hiring This Week:
- Clinical Data Eng → Karina Health
- Senior Infra Eng → Bland

Mid-Markets Hiring This Week:
- Principal ML Eng → Aimpoint Digital
- Senior ML Eng → Iterable

Enterprises Hiring This Week:
- Senior ML Eng → GBG
- AI Eng → BAH

🏭 Industry News & Trends

GOOG Cloud Next Recap, $10-$40B Anthropic Investment
The search giant’s announced bringing Gemini to enterprise, new TPU chips, and agentic data infra. It will also cut a gigantic check for Claude.

AMZN: Compute Deals With META and Anthropic
Amazon signs a Graviton chip deal with META and will provide up to 5GW of compute to the Claude vendor.

TCEHY & BABA: $20B for DeepSeek → Deep 75% Price Cut
The Chinese tech giants make a massive investment in the viral model vendor, which announces a massive price cut to compete with US labs.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

Chinese Govt. Blocks $2B META + Manus Merger
Chinese regulators put a stop to the blockbuster merger, setting back META’s quest to gain traction in the agentic space.

White House Accuses China of Model Ripoff While Missing AI Deadline
The White House accuses China of using distillation on US models to train its own, while missing the April deadline on AI consumer protection policy.

MSFT Partners With Aussie Government to Invest $18B in AI Infra
The deal includes “digital infrastructure, AI supercomputing, and cloud capacity.”

GOOG Expands AI Access to Pentagon
Remember how the Pentagon isn’t friends with Anthropic anymore? Turns out, they recently made a new friend.

As we previously covered, Claude Code’s source code was leaked in March, and now, every AI vendor seems to be dropping hints, previews, or warnings about their next projects. OpenAI “tested” its next multimodal model GPT-image-2 under a variety of aliases on a number of benchmarking sites, and also seems to be trialing web browsing features in Codex. Gemini Enterprise’s new Agent tab suggests GOOG is looking to challenge Claude Cowork as an agentic offering sitting within its model product. MSFT is also apparently hinting at bringing OpenClaw-like functionality to Copilot 365, even as Anthropic just launched Claude for MS Word. And Anthropic itself is remaining tight-lipped about its potential world-beater Claude Mythos, a model that is reportedly so powerful it can detect complex software vulnerabilities, including decades-old issues in OpenBSD and FFmpeg. According to the vendor, Mythos’ ability to detect software vulnerabilities makes it simply too dangerous to release to the public at this time. One thing we know: All the hints and “quiet previews” are 100% accidental, and none of them are intended to be promotional tactics to drive demand for unreleased products. You can be sure of that.

Up Next: All the AI news that busy dev teams need to know

💡 Spotlight: Newly Launched AI Startups

- Trent AI: $13M from LocalGlobe for agentic sec
- Satark AI: $4M from Infynno for AI-driven cyber risk
- Provally: Undisclosed amount from BluePoint for AI-driven cyber

🚀 New Models & Products Spotlight

- Opus 4.7: With “notable improvement in advanced software engineering”
- GLM-5.1: Z.ai’s foray into long-horizon tasks
- Muse Spark: META’s multimodal model builds toward “personal superintelligence”
- Gemma 4: DeepMind’s new generation of Apache 2.0 open models
- Cursor 3: Now with new agentic features
- QWEN3.6-Plus: BABA’s latest multimodal is open for vibe coding
- MSFT’s new MAI models for transcription, voice, and images

💻 Development

Building Agentic Systems
Founder Ashpreet Bedi suggests that successful agentic execution requires systems for data, sec, interface, and infra as well.

How to Help Agents Remember
This guide from founder Akshay Pachaar outlines a framework to help agents retain persistent knowledge.

Improving Agents for Recurring Tasks
Engineering leader Will Larson outlines a prototype/refactor process that helped him make agents significantly faster and more reliable.

🤔 Interesting AI Projects, Research, and Updates

[GH] LLM Wiki - Karpathy’s Self-Building Wiki
The AI researcher’s latest project lets you set your LLM to build a continually-updated wiki of its own learning.

Research: SandMLE - 13x Cheaper ML Environments
META researchers propose a scaled-down agentic framework for ML engineering that performs better and cheaper.

Research: Meta-Harness - Automating Harness Engineering
Harnesses being what turn a LLM into an agent, this proposed framework offers an alternative to hand-coding harnesses.

How Feeding Coding Agents Research → +15% Faster Inference
ML engineer Alex Kim explains how feeding agents arxiv papers and other relevant materials made llama go faster.

💼 Hiring and Community

Startups Hiring This Week:
- AI Engineer → NeuBird AI
- Applied AI Engineer → Sygaldry

Mid-Markets Hiring This Week:
- Agent Engineer → TRM Labs
- AI Engineer → MONTICELLOAM

Enterprises Hiring This Week:
- Director, ML Eng → PINS
- Full-Stack Eng, RL → Anthropic

🏭 Industry News & Trends

Anthropic Acquires Coefficient Bio for $400M in Stock
Perhaps as a follow-up to its 2025 life sciences launch, the Claude vendor has acquired the AI drug discovery startup.

OpenAI Acquires Finance Startup Hiro for Undisclosed Amount
The acquisition follows the recent launch of GPT 5.4 Thinking, which had new features for finance use cases.

Allbirds Shoes Is Now…an AI Compute Company?
Didn’t snag a pair of Wool Runners? Why not rent some GPU cycles instead?

⚖️ Copyright, IP, Licensing, Regulation, and Safety

People Are Getting Madder at AI and Those Who Work With It
You probably heard about the Molotov cocktail attack (and shooting) on the home of OpenAI’s CEO. Had you heard about the Indianapolis city council member whose home was shot up after a data center vote?

Americans’ Concerns About Affordability Rivaled by AI Concerns
This report notes that Americans increasingly see AI job displacement as a cause for both social and economic concern.

Deep Dive: Interview with the founders of Unslop.xyz on AI coding’s newest bottleneck: Understanding all that slop code

It seems like it barely makes sense at all to sit down in front of a terminal and write your own code. Every wondrous AI coding agent out there can spin up thousands of lines of code as fast as a human programmer, if not faster. So the bottleneck for “developer productivity” isn’t the quantity of code anymore. Now, it’s code quality, or more specifically, being able to understand whether the code your AI tools just spun up is actually any good.

Below, the founders of Unslop.xyz, creator of the Noodles open-source visualization project, explain why visual annotations may be the first step in cutting through all those lines of AI-generated code and finding signal among the noise.

So, AI Generated Lots of Code. What Does any of it Do?

For years, the all-consuming measurement of effective software development has been “developer productivity,” the ability of a team to deliver as much high-quality code as quickly as possible. Thanks to advances in AI-powered code generation tools, delivering large quantities of code quickly is now less of an issue.

But there are new bottlenecks. As AI tools spin up millions of lines of seemingly “good enough” code, the bottleneck has shifted to reviewing it. With the cost of generating code already approaching zero, one of the biggest future challenges in software may simply be understanding it. As new engineers enter the workforce and existing ones move on to new projects, they will inherit entire codebases that were not written by human hands.

To help humanity navigate this new future, a team of graduate students (Amir Amangeldi, Felix Wang, and Allen Zhang) set to work on building Noodles, an open-source project that helps users understand the reasoning behind different code branches with visual abstract syntax tree (AST) parsing. Below, the team explains why they built the project and what comes next.

The Noodles project visualizes what each branch of a repo does with AI-generated annotation. Image courtesy Unslop

The New Bottleneck: Understanding AI-Generated Code

The Noodles team is blunt. “The fundamental problem [in AI coding today] is that parsing AI output is hard.” The three creators explain that each of them relies heavily on AI tools like Claude Code these days, but have found it increasingly difficult to find time to review every last line of AI-generated code.

“We felt that we needed a higher level of abstraction to review changes. So we thought: Why not create diagrams that visualize what the changes are at a higher level?” The Noodles project emerged as a way to represent that abstraction, essentially as an additional, visual layer that stays updated as the codebase changes.

“Understanding AI-generated code, that’s 100% the bottleneck,” says the team. “And when we say ‘understanding code,’ that entails reviewing, asking critical questions, making probing assumptions, and testing. Which is harder to do.” Admittedly, it’s possible to delegate code review-related tasks to agents now, but that also effectively delegates away the ability to understand AI-reviewed code, to say nothing of how detailed or granular the picture could or should be.

“The other bottleneck is: As you scale and have more output, perhaps you are able to grasp what’s happening. But how do you coordinate the different coding agents that are working for you at the same time? How do you keep track of the context of what’s happening at any given point on your behalf?” Beyond the immediate issue of understanding what a specific code branch does, there’s a second, broader gap in trying to understand the larger context across everything your agents are doing.

Using Deterministic Tooling for Visualization

The Noodles team admits that their initial spec, which utilized LLMs for visualization, didn’t quite work out as planned. “To be frank, the output was different every time, which became difficult to track. So we thought: How can we make this as deterministic as possible?”

The team landed on AST parsing as an alternative. “It’s fully deterministic. Of course, it’s not perfect, but it’s able to go through classes and functions, and understand what’s calling what consistently, and then represent that on a diagram.” The Noodles project was built to output a visual graph that displays connections across various functional nodes to explain what each connection means, and summarize changes at the PR and branch levels. It uses AI to group nodes together to arrive at higher-level diagrams as well as to enhance nodes and edges with helpful description.

The team notes that the project was otherwise engineered around simplicity, leading to three different node designations for input entry points, process points (encompassing either individual functions or aggregate processes), and outputs, utilizing Mermaid diagrams, which turned out to be more popular than D2 diagrams.

Real-World Use Cases (and Limits) of Visualization

The team notes that the project is seeing the most use in collaborative settings among teams where not everyone has been exposed to new code or changes. “For teams that are still using pull requests, [Noodles seems like an] entry point that’s helpful, specifically for large changes. That’s the point where people want to understand ‘what happened.’”

“And that’s why we have a free GitHub integration where you can integrate our unslop bot into your repository, and then analyze it, just like you would with a regular code review agent. Running an analysis spits out a link that you can click to see a diagram pertaining to the changes in this particular PR. So that’s the dominant use case.”

The team notes that in the grand scheme of things, onboarding is one of the most important use cases that requires more understanding. “Passing on the foundational understanding of what’s been done before is absolutely key, so for learning in general, there’s absolutely going to be space for that in education.”

“However, architectural diagrams on the cloud level are probably not going to go away. The further you zoom out for a complex system that has multiple different servers, databases, and whatnot…those types of diagrams are here to stay. But for code-level diagrams, it’s not as certain.”

“It’s really a question of how we can adapt fast-moving AI tools for humans, whether they’re developers or vibe coders or any other sort of human using AI. And there’s definitely [a place for] visualization, but I think the bigger problem is what level of detail is too much to ask a human. To us, that ‘level of detail’ comes down to the human judgment that needs to be applied.”

“AI can predict a lot of things. But the places where human judgment needs to be applied, that’s where some level of information representation needs to be passed on to humans. Whether it’s in the form of a notification, including text or a diagram that explains to the human what remains for them to decide. Whether it’s concise enough and delivers information in the right shape, at the right time, with the right level of scope. That’s really the problem we’re going after here.”

The team notes that, over conversations with its growing community, it has started to realize the limits of visual diagrams to help people understand code. “Folks are moving so fast with AI that hardly anyone has time to consistently stop and think about what’s happening. They want to continue moving as fast as AI. So, having the understanding is great, but it does slow them down.”

The team utilizes its own tool to inspect its codebase, but finds itself wanting to move faster and find bottlenecks and workarounds more quickly. “The core question we’re struggling with now is how to have that understanding while moving faster. We’re designing our next project around that question.”

How Devs, Tooling, and IDEs Will Change

“With the barrier to entry into building things becoming lower, anybody can build. So the expectations of a human interacting with these tools is going to be different too. And so, the hardcore engineering practices around what it takes to understand or debug are going to have to change. The type of human that’s engaged in engineering will be different.”

The team suggests that all the talk in the headlines of organizations flattening their hierarchies due to AI isn’t just talk. “All these roles of product managers, designers, and engineers collapsing into one role in fast-moving startups is a real trend. The developer tooling of the future is going to have to be much more approachable, much more simple.”

What will the IDE of the future look like? “We don’t see it being a bunch of files that you have to click through, because the sheer number of files is going to be so large. And it’s not clear that a terminal is good enough either, because the terminal is constrained in its functionality. We believe it will have to be a standalone user experience that is able to balance between the chatting, the browsing, and the visuals in a very simple, unified manner.”

“So yes, the IDE of the future is certainly going to borrow some aspects of all of the things that have come before, but it will have to fundamentally change.”

This interview originally appeared on Heavybit.com.

Thanks to the mega-popular coding model Claude Code, Anthropic had been on an unprecedented winning streak. Claude Code has reportedly gained massive commercial traction with paying customers while winning its first injunction against the Pentagon after the US military designated it a supply chain threat. The company has also been shipping high-profile features like scheduled tasks and computer use at a brisk clip. Then, just a few days ago, the source code for Claude Code was leaked to the general public by way of a non-private JS Map file in the most recent version. By now, you’ve heard the story: First, researcher Chaofan Shou detected the unprotected code and uploaded it in a ZIP file to social media. Where it was promptly downloaded a bazillion times. Despite Anthropic pulling down the npm package and issuing DMCA takedowns, coder Sigrid Jin, the world’s foremost Claude Code power user by token usage, was fast enough to pull the package and port it to Python. Within 24 hours, the coder had uploaded the Python port of Claude Code to GitHub, where it has become the fastest project in history to hit 100K stars (with a Rust version in the works as well). Due to the new project’s fundamentally different architecture, it’s not clear whether the “Claw Code” project is legally actionable for Anthropic. You can’t make this stuff up. And you can’t let your guard down on unsecured code packages, especially if you’re an industry-leading AI vendor.

And now: The bottom line in AI news for busy dev teams

💡 Spotlight: Newly Launched AI Startups

- Whirl AI: $8.9M from ICONIQ for AI-powered enterprise IT workflows
- Nomadic: $8.4M from TQ Ventures for physical AI data
- Huskeys: $8M from various investors for AI-powered firewall sec
- Pado AI: $6M from NovaWave for data center resource orchestration

🚀 New Models & Products Spotlight

- PrismML Bonsai 8B: 1-bit compression offers “10x intelligence density” on edge
- Qwen3.5-Omni: BABA’s latest omnimodal offering
- Composer 2: Cursor’s latest coding model offers significantly lower token pricing
- Gemini 3.1 Flash Live: GOOG DeepMind’s newest real-time voice model
- Veo 3.1 Lite: GOOG DeepMind’s lower-cost video gen model
- DynaEdit: GOOG DeepMind’s no-training-needed AI video editor

💻 Development

[GH] Pretext: AI-Friendly Front-End Library
Engineer Cheng Lou’s viral JS/TypeScript library can lay out websites without requiring reflow and DOM measurement.

[GH] skills: Specifically for Agentic Coding
Dev-specific (front end, full stack, mobile, etc.) skills for coding agents.

Blog: The Software Factory
Engineer Alexander Opalic’s overview the “software factory” model of development.

Top Contributor to OpenAI’s Latest Repo Is…Claude?
OpenAI’s parameter-golf project focuses on training small models. Guess who one of its top contributors turned out to be.

By the Way, GitHub Opted You Into Training Copilot
If you haven’t already, opt-out of GitHub feeding your projects to its AI here.

🤔 Interesting AI Projects, Research, and Updates

Can You Infer AI Vendors’ Product Roadmaps From Their Job Listings?
The researchers at Epoch AI gleaned a surprising amount of product intel after analyzing job listings at major frontier labs.

TurboQuant: GOOG’s New Compression Algo Reduces Memory Usage 6x
DeepMind’s new algorithm may reduce AI memory footprint up to 6x.

ARC-AGI-3 Benchmark Now Available
The latest edition of the AGI benchmark uses interactive reasoning environments to evaluate how smart agents are.

💼 Hiring and Community

Startups Hiring This Week:
- Applied AI Engineer → Splitero
- Applied AI Engineer → Luminai

Mid-Markets Hiring This Week:
- AI Engineer → Petlibro
- AI Engineer → Royalty Pharma

Enterprises Hiring This Week:
- AI Engineer → Aspen One
- Data Scientist → EXL

🏭 Industry News & Trends

OpenAI Raises $122B, Acquires Astral
The ChatGPT vendor acquires the Python devtool startup and raises the largest private funding round ever.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

Security Dominos: LiteLLM → Delve → Mercor
Well-regarded LLM proxy server LiteLLM was found to be trojan-horsing malware, despite having SOC2 certification. From Delve, which has been accused of offering users “fake compliance,” and most recently, AI recruiter Mercor was reportedly affected by the LiteLLM sec issue.

Deep Dive: Interview with founder Dexter Horthy on what’s needed to make AI agents mainstream for developers

While AI in development seems like a foregone conclusion, not every developer is using agents in their daily workflows. What’s missing? Trust? Consistently deterministic outcomes? Security and compliance? This week, we spoke with HumanLayer founder Dexter Horthy to discuss what’s needed to normalize agentic software development.

2025 was to be the year of AI agents, a prediction that may or may not have come true, depending on which people you ask. For a time, it seemed like the open-standard MCP would become the standard for orchestrating multiple autonomous AI agents (and perhaps it will), but not before it addresses its known security vulnerabilities.

For more perspective on putting agents into production for real-world engineering teams, we spoke with founder Dexter Horthy of HumanLayer to understand how agentic coding will evolve past its many questions about security, reliability, and token economics into full adoption into mainstream software development.

Evolving Agentic Products for Real-World Engineering Teams

Horthy acknowledges the breakneck pace at which agentic has evolved even in the past year. His own organization rolled out a “Research, Plan, Implement“ (RPI) framework to help engineers break complex coding tasks into discrete phases: research the codebase first, plan the approach, then implement, rather than throwing an entire ticket at a coding agent and hoping for the best.

“We rolled this out last year, but I tell people: ‘Look, I don’t know if we’ll even be using these same techniques in six months.’” Still, the founder notes that the RPI framework and his team’s original work on context engineering and 12-factor agents seemed to become much more relevant as coding agents improved by leaps and bounds in the back half of 2025.

“I wanted to make it clear that there was no ‘magic’ or ‘silver bullet.’ Under the hood, we were fundamentally helping people do better context engineering, and that’s what we’ve started productizing.”

Dexter Horthy discusses the importance of managing context windows in software engineering. Image courtesy AI Engineer World’s Fair

The Adoption Gap: When Expert Tools Meet Real Teams

The founder candidly admits that his team saw a consistent pattern when rolling out RPI across engineering organizations. Expert engineers who invested deeply in learning the tools started shipping like crazy, but when those experts handed the same tools to their teams, the results were inconsistent at best.

“We had people who loved what we did and became very good at it just by finding our stuff online, or sitting down and doing a workshop with us.” But when the early adopters tried to share the new knowledge and frameworks with their colleagues who weren’t as invested, those folks didn’t get the incredible results that were possible with a lot of time investment, and the adoption didn’t spread.

When Horthy’s team investigated, they found specific, diagnosable problems. The first was bad research: A skilled engineer would naturally break a ticket into targeted codebase questions, but most people would just paste the entire ticket into the research tool and get back opinions instead of facts.

“The problem is, if you tell the model what you’re building, you get opinions. The model starts making implementation decisions instead of proposing options to you around how to proceed.”

The second problem was bad plans. The team’s planning prompt had 85+ instructions, and buried inside were critical interactive steps like “present design options to the user” and “get feedback before writing the plan.” For about half of users, the agent would skip all of that and just hand back a finished plan with all decisions already made.

“When we investigated, we found the difference between good and bad results was a single line: ‘Work back and forth with me, starting with your open questions and outline before writing the plan.’ I found myself standing in workshops full of enterprise engineers saying, ‘Folks, yeah, here’s the software, but don’t forget to say the magic words.’ It was, quite frankly, embarrassing.”

The founder notes that this wasn’t the user’s fault. It was a tooling problem. His co-founder Kyle wrote a blog post citing research showing that frontier LLMs can only follow about 150 to 200 instructions with good consistency. Their planning prompt alone had 85 instructions. Add in the system prompt, tool definitions, and MCP servers, and the model was simply over budget.

“Solving these problems is actually still pretty hard. I don’t think any of this stuff has been commoditized yet, though maybe it will be. But that’s why we’re excited about building collaboration features and changing the SDLC by creating a Google Docs/Notion/Figma-like experience around coding agents.”

The Instruction Budget: Why Less Context Means Better Code

This diagnosis (that LLMs have a finite instruction budget) became the foundation for Horthy’s fix. As Geoff Huntley puts it, the less of the context window you use, the better results you’ll get. But it’s not just about stuffing in too much information. You can also overload the model with too many instructions.

Huntley’s Ralph Wiggum loop, an ultra-viral technique of wrapping a coding agent in a Bash while loop to periodically refresh its context, took off because it addressed the performance-eroding context rot that creeps in over successive runs. Horthy sees the lessons as more durable than the technique itself.

“The thing about Ralph is that we might not be using the same prompts in six months. It might not be the same technique in six months. This stuff is going to evolve. I think the most valuable thing about Ralph is the lessons that it teaches about context engineering.”

“If what you’re doing is grounded in understanding context engineering and the limitations of attention, you don’t actually have to do any of that complicated stuff. If you just take care of keeping your context window small, you can get really good results.”

Horthy suggests the broader conversation around Ralph ultimately reinforced a principle his team was already learning the hard way: giving the model fewer instructions, simpler tasks, and smaller context windows. “I don’t know if I actually recommend that people should use Ralph to write production software.”

“It’s more about understanding how it works and why it’s good and then applying those lessons for your workflow. For example, our implementer agent uses faster, smaller models for writing the code and running the tests, and then a bigger, smarter model spot checks changes, keeping context usage in the parent session low, allowing for resteering.”

Geoff Huntley’s viral Ralph loop is a clever hack, but does it represent the actual future of agentic coding? Image courtesy Geoff Huntley

Will Coding Agents Help Everyone Roll Their Own Everything?

While use cases for different verticals emerge frequently, AI’s strongest proof of concept still seems to be in code generation. Yet as users attempt to build in more multi-step long-horizon tasks with their coding agents, they continue to bump up against the limitations of long context windows, including rot and hallucinations.

“I think the thing that everyone will agree on is that you will always get better results if you use less of the context. And ‘using less of the context’ is a thing that you, the engineer, have to think about. How do you want to do this, and how do you fit that into your workflow?”

Will models and coding agents eventually hit an inflection point past which organizations stop relying on packaged software and start simply rolling their own? “I think we’re going to see a lot more hand-rolled versions of software. The 2010 ZIRP-era practice of ‘get a SaaS for everything’ is probably going to look very different in the coming years.”

“I got really excited a few years back when Klarna claimed it would drop Salesforce and Workday and just build it all themselves. Those are the two vendors that nobody ever fires! I think they ended up like walking that back a bit.” However, the founder suggests that orgs rolling their own internal tools may still be the right direction.

“I’m not going to say that the change will happen through vibe coding because I’m sure engineering will need to be done. There’s a difference between spending 20 years building a piece of software and building an internal version in a hurry. There are just things that happen when you spend 20 years on something that you will not get for a while no matter how much AI you point at it.”

Please Read the Code: Why Reviewing Plans Wasn’t Enough

Perhaps the most candid reversal in Horthy’s updated thinking concerns code review. In the original RPI framework, he advocated for reading plans instead of code, with the logic being that an engineer can’t easily review 2,000 lines of generated code, but can review a 200-line plan.

“I was wrong. I am humble enough to admit when I was wrong.” The founder explains that after six months of reviewing plans instead of code, the team had to rip out and replace large parts of their system. A 1,000-line plan tends to produce about 1,000 lines of code, so there was no actual reading “savings.” Worse, the implementation would sometimes diverge from the plan, forcing the reviewer to hold both in their head.

“The new advice is: Don’t read the plans. Please read the code. It’s the same amount of work, there may be surprises that cause drift between the plan and the code, and the code is the thing that actually ships.”

Horthy acknowledges the elephant in the room, namely high-profile open-source projects like Beads where allegedly nobody reads all the code. “These are very cool projects. But nobody gets paged at 3 a.m. if they’re broken. Nobody gets fined millions of dollars if it’s done wrong.”

The founder describes how this realization shaped the team’s approach to feedback loops. “How do you increase the odds that the code being reviewed is correct and needs no iteration?”

Rather than reviewing faster, the goal became aligning the agent before it writes code. The team now surfaces all codebase patterns that might apply to a problem and lets humans decide between old patterns that shouldn’t be used anymore and newer approaches that should inform future decisions.

“LLMs are really good at finding patterns and following them. In a way, this is all they do. They take some input, some text, and translate it. ‘Match the format of this code we found in the codebase in this other file over here.’”

Horthy also wrote a blog post about making testing feedback more context-efficient, giving the model deterministic signals about what’s broken rather than dumping entire test outputs into the context window. “No matter how big context windows get, you always get better results if you use less of them.”

“No matter how big context windows get, you always get better results if you use less of them.” -Dexter Horthy, Founder/HumanLayer

From RPI to QRSPI: Splitting the Monolith

Before going deep on coding agents, Horthy wrote 12 Factor Agents, which was arguably the first deep dive on “context engineering.” The paper’s core argument: Don’t use prompts for control flow. If you know what the workflow is, use actual control flow. Classify the input, then feed it to a series of smaller, more-focused prompts with fewer instructions and fewer actions to choose from.

The irony was not lost. “We got on stage and said ‘full-fat agents don’t work, build workflows and micro agents, use control flow for control flow.’ Then we turned around and wrote a giant monolithic 85-instruction prompt. It was time to drink our own Kool-Aid.”

So his team split it. Their original three-step workflow (Research, Plan, Implement) became a seven-step pipeline: Questions, Research, Design, Structure, Plan, Worktree, Implement, each step with fewer than 40 instructions.

The key innovation is where the human review now happens. Rather than reviewing a 1,000-line plan or 1,000 lines of code after the fact, the new workflow front-loads alignment into two short artifacts: a design discussion (~200 lines) and a structure outline (~2 pages).

The design discussion captures current state, desired end state, patterns found in the codebase, resolved decisions, and open questions. The structure outline maps out the order of phases and how to validate along the way.

“You’re forcing the agent to brain-dump everything it found, everything it wants to do, everything it thinks you want, and ask you questions about things it doesn’t know. You get to do brain surgery on the agent before you proceed downstream. 200 lines instead of 1,000. That’s leverage.”

The acronym didn’t work out so well (QRDSPWIP), so the team picked a subset and started calling it QRSPI.

The Future of Orchestration

Regarding the future of managing agents, Horthy cautions against making absolute judgements about whether one agentic technique or framework is uniformly “better” without considering the details.

What’s more important is knowing when to decompose. “Sometimes you hit a wall, so you decide to decompose your workflow and make it more reliable, more robust, but maybe a little bit less able to handle everything in the input space.”

“But once you get to know what the input space would be, you can give people a happy path that’s much more reliable. And then you can always still eject out of the situation and say, ‘Hey, something unexpected happened. Let’s see if the LLM can figure it out.’”

On the question of whether agentic frameworks could be declared uniformly ‘good’ or ‘bad,’ the founder concedes that using them like properly-specified tools in a loop, with proper expectations, can work well...provided you’re prepared to decompose longer workflows into smaller slices and manage context across each step. “These are tools in your arsenal. You need to know where on that autonomy slider you want to be.”

This interview originally appeared on Heavybit.com.

The world’s most valuable company is holding its annual tech conference, announcing entries into robotics and robotaxis, plus the DLSS 5 AI-enhanced graphics you’ve seen in those hilarious internet memes. However, NVDA has also provided updates on its Vera Rubin chip generation, which purports to offer 5x inference speeds and 10x lower inference costs (and will apparently power orbital data centers). On the hardware side, the company is looking to go beyond GPUs, unveiling the licensed Groq 3 language processing unit (LPU) optimized specifically for inference, as well as its BlueField-4 STX architecture for storage to address data bottlenecks as part of its proposed 5-layer AI stack: apps, models, infra, chips, and energy. NVDA is also pushing into agentic with NemoClaw, its guardrail-enabled take on the viral OpenClaw agent. But the most interesting development may have been a [purposely provocative] remark in CEO Jensen Huang’s keynote speech. Specifically, the executive suggested that tokens are becoming so crucial to AI operations that the future of engineering may involve comp packages that include tokens alongside salary. Will the future of hiring involve offers that come down to token allowances? (To say nothing of how engineers would try to game that type of system?)

Next Up: The bottom line in AI news for busy dev teams

💡 Spotlight: Newly Launched AI Startups

- AMI Labs: $1B from Cathay for Yann LeCunn’s world AI model startup
- Armadin: $189M from Accel for AI-native cyber
- Kai: $125M from Evolution for AI-powered IT/OT sec
- Standard Kernel: $20M from Jump for autonomous GPU kernel generation
- Niv-AI: $12M from Glilot data center power optimization
- Manifold: $8M from Costanoa for agentic risk management
- Crafting: $5.5M from Mischief for agentic engineering infra
- NeoLab AI: $5M from “undisclosed VCs” for AI infrastructure

🚀 New Models & Products Spotlight

- Small 4: Mistral’s latest small multimodal offering
- Nemotron 3 Super: NVDA’s newest heavy-duty open-weight model
- GPT 5.4 and 5.4 Pro plus GPT 5.4 Mini and 5.4 Nano: OpenAI’s latest
- GLM-5-Turbo: Z.ai’s latest model optimized for OpenClaw
- Gemini Embedding 2: GOOG’s first natively multimodal embedding model
- Agent 4: Replit’s new model emphasizes creative design
- Unsloth Studio: No-code Web platform for training/running open models
- Claude 4.6’s 1M Context Window: Opus and Sonnet both get the upgrade

💻 Development

[GH] OpenClaw-RL: Async RL
This async framework helps you train OpenClaw agents.

Guide: The Anatomy of an Agentic Harness
LangChain’s new guide explains how harnesses work with models to execute.

[GH] ContextHub: Fetch API Docs
Professor Ng’s latest project makes agents fetch API docs for you.

Reddit: Alternatives to Function Calling
Former Manus backend lead Morro Hsu unpacks how he used Unix-style commands instead of function calls to get better agentic performance.

🤔 Interesting AI Projects, Research, and Updates

LLM Architecture: A Gallery
Researcher Sebastian Raschka unpacks visual architecture maps of the major LLMs on the market.

Claude Uncovers 14 High-Severity Firefox Bugs
Turns out, high-severity sec threats lurk even in popular public software.

[GH] autoresearch: Karpathy’s small-scale autonomous LLM trainer
The AI researcher’s new project sets up training loops on a single GPU.

Blog: MCP Is (Not) Dead
Veteran developer Charles Chen lays out the case for CLI not actually killing off MCP, which will be a reality at the enterprise for some time.

💼 Hiring and Community

Startups Hiring This Week:
- AI Engineer → HumanSignal
- AI Engineer → StackOne

Mid-Markets Hiring This Week:
- Senior PM: AI Ops → MedVidi
- Applied AI Engineer → Kodiak

Enterprises Hiring This Week:
- UX Researcher → ROKU
- Computer Vision ML Engineer → AAPL

OpenAI’s Robotics Lead Departs
In the wake of OpenAI’s recent deal with the Pentagon, robotics lead Caitlin Kalinowski departs on safety concerns.

🏭 Industry News & Trends

META Acquires Moltbook
The Llama vendor has acquired the website on which OpenClaw agents (and more than a few human marketers disguised as such) convene.

OpenAI Acquires PromptFoo
The open-source eval startup will remain open source after the acquisition.

Shocking: AI Vendors Want Enterprise Customers
New enterprise offerings: Mistral’s training tool Forge, MSFT’s Copilot Cowork, and Perplexity’s new enterprise browser. Meanwhile, after its AI tech lead departs, BABA re-organizes its Qwen AI division to nab enterprise revenue.

⚖️ Copyright, IP, Licensing, Regulation, and Safety

AMZN Wins Court Order to Block Perplexity Shopping Agent
The once-viral browser company’s shopping agent has been locked out of AMZN’s popular e-commerce infrastructure.

Anthropic Sues Pentagon
After being classified a supply-chain risk, the Claude vendor has followed through on its plan to pursue legal action.