Sunny Bains

In distributed systems, reliability isn’t just about retries and durability, it’s about knowing who owns recovery. My latest post, based on the Coordinated Progress model I posted previously, explores how reliable triggers create responsibility boundaries and how those boundaries shape resilience, observability, and complexity. https://lnkd.in/diCmywnS

117

8 Comments

New blog post, "Symmetric ACP": https://lnkd.in/dDU7FDbq This post describes **SymmACP** -- a proposed extension to Zed Industries' Agent Client Protocol that lets you build AI tools like Unix pipes or browser extensions. Want a better TUI? Found some cool slash commands on GitHub? Prefer a different backend? With SymmACP, you can mix and match these pieces and have them all work together without knowing about each other.

18

Chunking and snippet extraction has been a huge focus lately - my latest blog dives into some of the work we've done on this to date, including support for a chunk rescorer in our semantic reranking retriever, as well as some useful ES|QL primitives to get more visibility into chunks and snippets. #elasticsearch #semanticreranking #snippets #chunks #chunking #esql https://lnkd.in/e6UD7iii

13

New blog post: A Fork in the Road: Deciding Kafka's Diskless Future. Kafka is getting serious about S3 and finds itself at an architectural crossroads that will shape its next decade. Several new KIPs (1150, 1176, 1183) aim to reduce replication costs across cloud availability zones, but the implications go far beyond networking cost. It’s a mistake to think of S3 as simply a cheaper disk or a networking cheat. Building on object storage opens the door to operational benefits such as elastic, stateless compute — something many modern analytics systems already exploit. In my latest post, I outline two competing future paths: 🔹 Evolutionary path: Reuse large parts of existing Kafka components to reduce code changes and long-term maintenance. 🔹 Revolutionary path: Separate stateless and stateful layers to realize the full operational benefits of disaggregated storage. The post examines the trade-offs of each path, how the current KIPs map to these two paths and poses a broader question: what should Kafka become? And what will keep it relevant in the decade ahead? https://lnkd.in/d4E2BAe4

149

12 Comments

Seeing quite a few discussions lately about Kafka/Iceberg integrations being "zero-copy" or not. I think this is largely missing the point. First, where I agree is that this integration should be "zero-effort" for users. Materializing a Kafka topic into an Iceberg table shouldn't require more than a click of a button. Queries should provide a uniform way for accessing the data in both a topic and the corresponding table. This is the stream/table duality, and it should Just Work™. Now, whether this requires to store the bytes of data once in a Kafka topic, and a second time elsewhere for table access, shouldn't really matter from a user perspective. I'd argue storing the data twice is actually a benefit, and in fact it's a pattern well established: it resembles the design of WAL and table files known from databases for decades. I don't think anyone ever complained about this structure in their RDBMS? Which makes sense, it's an implementation detail, opaque to users. But as it turns out, having log and table data separately is even more advantageous for the deconstructed database that is Kafka and Iceberg: you can have multiple readers of the same log (Kafka topic), materializing views in multiple destinations and systems optimized for specific use cases. Maybe multiple Iceberg tables with different projections (think PII), maybe an Iceberg table and a full-text index in Elasticsearch, maybe an... you catch my drift. Furthermore, the log is replayable, so you can recreate views if needed, or you can implement new use cases you didn't originally have in mind. All in all, I think "zero copy" is mostly a red herring. Sure, it can be an optimization for certain scenarios, but mostly it's a distraction from the immense value you get from combining Kafka and Iceberg when done the right (seamless) way.

127

26 Comments

In-memory databases are the architectural backbone of real-time AI. They’re a necessary advantage for memory-intensive workloads that rely on instant access to context. Modern systems like Redis blend durability, multi-model flexibility (JSON, vectors, time-series), and tiered storage to deliver the speed of memory with the reliability of disk. The result? Real-time analytics, instant personalization, and AI systems that actually feel intelligent. Here’s how we deliver the speed of memory with the reliability of disk: https://lnkd.in/gSm65kdB

49

A recent K8s podcast with Clayton Coleman and Rob Shaw is a fascinating listen: https://lnkd.in/gJmpX_DB Why do standard K8s approaches don't work with LLM inference? Unlike a normal web app, LLM generates text one token at a time, each one depending on the one that came before. The technique to remember the context of the conversation is called KV caching. Now regular K8s tools for balancing traffic, adding servers and managing resources at huge scale work really well, because the amount of required varies depending on the prompt. The tools the community created to solve this problem: 🛠️  vLLM - popular inference server that is very good at KV cache management. 🛠️  Inference Gateway - smarter load balancer that enables efficient traffic distribution. 🧩 The llm-d project brings these together in a ready-to-use stack, tested and benchmarked well-lit paths, smarter routing, intelligent work splitting to optimize time-to-first token, methods to run "Mixture-of-Experts" models across multiple servers

50

What I'm Reading This Week 📚 🤖 Vibe Code Reality Check Steve Krouse's piece argues that "vibe coding" where you "forget that the code even exists" creates legacy code nobody understands. Programming is fundamentally theory building, not just producing lines of code. This resonates - I recently spent hours trying to fix a frontend bug with Claude Code and Gemini CLI stuck in a dead loop of guesses. Two minutes of human inspection found the inefficient hook render. We still very much need the human (expert) in the loop. https://lnkd.in/g_Dt-_kz 📐 Mathematics as Code Dan Abramov's exploration of Lean shows how mathematicians can treat mathematics as code - breaking it into structures, theorems, and proofs that are statically checked and composable. His playful example of "haunted math" where 2=3 demonstrates how axioms shape mathematical reality. Always fascinating to see bleeding-edge programming language theory, especially when it bridges math and code in such elegant ways. https://lnkd.in/gebHeMSU 💻 Local LLMs Innovation Simon Willison highlights how his 2.5-year-old laptop can now write Space Invaders using GLM-4.5 Air and MLX. Open source models are rapidly pushing the frontier of what's possible on a chip. The future where we get GPT-4 level intelligence running on phones feels closer than ever and a powerful enabler for humanity. 🚀 JavaScript's Wild Decade Jamie Birch's comprehensive dive through JavaScript runtimes reveals a staggering ecosystem - from Node.js to Cloudflare Workers to React Native's Hermes. I remember when Apple had a strict no-JS-runtime policy for the App Store. We've come so far that every platform now has multiple JS engines optimized for specific use cases. The proliferation shows JavaScript's incredible adaptability across contexts. https://lnkd.in/g843sYBk 📊 Amazon's Dev Experience ROI Amazon's "Cost to Serve Software" framework quantified a 15.9% reduction in delivery costs through developer experience improvements. The key insight: giving developers back time and reducing toilesome work. For a company with 1,000 developers, a 15% improvement translates to $20M in cost avoidance. Good concrete ROI numbers on investing in developer productivity. https://lnkd.in/ghv5CTfa Which trend do you think will have the biggest impact on software development: local AI democratizing access, the continued JavaScript runtime explosion, or finally having concrete metrics for developer experience ROI? #AI #JavaScript #DeveloperExperience #Programming #LocalLLMs #VibeCoding #SoftwareDevelopment

9

2 Comments

Starting the year strong at AWS for .NET with .NET 10 support Lambda. A new feature we added for .NET 10 is support for writing C# file based Lambda functions. Would love to hear feedback on the new support. https://lnkd.in/gzpH_ST3

119

7 Comments

🚀 TOON vs JSON - How I Cut My LLM Token Usage (and Cost) by 30–60% I integrated TOON (Token-Oriented Object Notation) into my daily autonomous agent workflow, the same system that sets up my day’s action items automatically. Before TOON, that workflow consumed anywhere from 60K to 1M tokens per run, depending on the context. After converting all structured JSON data into TOON, the exact same logic now uses only 45K to 72K tokens. ✅ Same context. ✅ Same responses. ❌ Lower cost. That’s a massive 30–60% token reduction, just by changing the data format. So, what’s TOON? It’s a new, compact data format designed for LLM efficiency. Unlike JSON, which repeats keys, braces, and punctuation, TOON defines the structure once and just streams the values. The result? Far fewer tokens, faster processing, and cheaper model runs. Many of my engineering friends ask: “Isn’t TOON just like CSV?” Conceptually: 👉 TOON = CSV’s compactness + JSON’s structure + AI readability Here’s how they differ: JSON → General-purpose data exchange (APIs, configs) CSV → Simple, flat tabular data (spreadsheets, databases) TOON → Structured data optimized for AI token efficiency If your agents or pipelines frequently push JSON into LLMs, switching to TOON could save real money and make your workflows scale more efficiently. 🔗 Learn more: https://lnkd.in/dC4wnkmG #AI #MachineLearning #LLM #SoftwareEngineering #DataEngineering #Automation #TOON #JSON #TokenOptimization #AIInfra

458

70 Comments

Accidentally discovered a new way to work with GitHub Copilot today: write new tests based off test coverage reports. Copilot (or I) run this command to generate a report of all the uncovered lines in the current diff: pytest --cov --cov-report=xml && \ diff-cover coverage.xml --html-report coverage_report.html && \ open coverage_report.html Then Copilot reviews the report, summarizes the uncovered situations, and proposes new test cases to add, or if the code is truly unreachable, suggests deleting it. Just another reason why you should add diff-cover (or similar tool) to your projects!

115

4 Comments