Infrastructure Management

Working with multiple LLM providers, prompt engineering, and complex data flows requires thoughtful organization. A proper structure helps teams: - Maintain clean separation between configuration and code - Implement consistent error handling and rate limiting - Enable rapid experimentation while preserving reproducibility - Facilitate collaboration across ML engineers and developers The modular approach shown here separates model clients, prompt engineering, utils, and handlers while maintaining a coherent flow. This organization has saved many people countless hours in debugging and onboarding. Key Components That Drive Success Beyond folders, the real innovation lies in how components interact: - Centralized configuration through YAML - Dedicated prompt engineering module with templating and few-shot capabilities - Properly sandboxed model clients with standardized interfaces - Comprehensive caching, logging, and rate limiting Whether you're building RAG applications, fine-tuning foundation models, or creating agent-based systems, this structure provides a solid foundation to build upon. What project structure approaches have you found effective for your generative AI projects? I'd love to hear your experiences.

Grid Control Series: How grid frequency stays stable even when power consumption fluctuates. Curious? Let me explain below! 👇 This will be the first post of the grid control series which will cover grid control methods, why they are needed and what are the challenges within the actual energy transformation. One key aspect of each power grid system is a stable frequency. But how is it ensured that the frequency remains stable even when continuous load changes occur within the grid? The frequency of the power system depends directly on differences between the generated power and the consumed power. It can be imagined as a scale that, when there is an imbalance ➡️ the frequency will decrease if consumption is bigger than generation ➡️ the frequency will increase if consumption is lower than generation ➡️ Traditional Power Systems: In traditional power systems (Large power plants) the following mechanisms stabilize the frequency of the grid: 1️⃣ Dynamic load fluctuations are absorbed to a certain extent by the inertia of rotating masses and their stored kinetic energy. This natural inertia resists rapid frequency changes. 2️⃣ Frequency deviations are further stabilized by the provision of controllable reserve power, which is traded on the reserve power market. 3️⃣ For larger frequency deviations (e.g., ±200 mHz in Germany), inherent system functions of the power controllers like P(f) come into play. These are specified in standards (e.g., VDE AR-N-4110) in Germany and must be provided by every generation unit. ➡️ Modern Grid Approaches with Renewable Energies: As renewable and inverter-based generation increases, physical inertia decreases as they typically don't provide mechanical inertia like traditional generators. However, modern grid forming inverters combined with battery storage systems are able to emulate the inertia and thus, to stabilize the grid on dynamic load changes (1️⃣) by: ✅ Virtual Synchronous Machines (VSM) ✅ Virtual Inertia Emulation ✅ Droop Control In addtion, as in traditional approaches they are also able to participate in the reserve power market (2️⃣) as well to provide frequency control mechanisms like P(f) (3️⃣). This allows modern grids to maintain frequency stability even in low-inertia conditions. What are your main challenges in designing and controlling renewable energy systems in modern grids? #ControlSystemEngineering #GridStability #ActivePowerControl #InertiaEmulation #RenewableEnergy #PowerSystems #Simulation

���� How BYD Solve the Grid Nightmare of Megawatt Charging? Let's look closer to BYD’s new All-Liquid-Cooled Megawatt Charger, isn’t just about speed. It’s a masterclass in redefining charging infrastructure economics. 🔌🔋 ⚡ The "Impossible Math" Solved Traditional megawatt charging requires a 1,600kVA transformer ($$$$), brutal grid loads, and $$$ civil works. BYD’s system?   - Transformer Size Slashed: 315kVA (80% smaller!) → cuts grid strain and saves $40k/year in post-2030 utility fees.   - Cost Halved: Total station build drops from ~$70k to $15k (transformer + construction).   - Secret Sauce: Integrated 225kWh battery storage buffers grid demand, enabling 1MW charging with a fraction of the power draw.  🔋 Storage Meets Speed: The Killer Combo   - 5-Minute 400km Charge: Matches gas station speed, no swap stations needed.   - Grid-Friendly: Storage absorbs peak loads, avoiding costly grid upgrades.   - Profit Play: Off-peak charging + peak discharge turns stations into virtual power plants (VPPs).  🌍 Why This Will Go Viral  1. Scalability: Tiny footprint + low grid dependency = rapid nationwide rollout.   2. Policy Proof: Dodges post-2030 “basic electricity fee” traps (saves ~$4k/month per station).   3. Storage Gold Rush: Each charger needs a battery – 3M+ EVs in China alone could birth a $30B+ storage market (bigger than commercial & industrial ESS!).  📊 BYD vs. Traditional Chargers   Metric        BYD’s System | Legacy Megawatt Charger Transformer Size 315kVA | 1,600kVA Build Cost $15k | $50k+ Grid Impact   Low (storage-buffered) | High (direct grid pull) ROI Timeline    <3 years | 5–7 years 🔥 The Bigger Picture   “This isn’t just charging – it’s energy infrastructure democratization,” said Lian Yubo, BYD’s Engineering VP. With 4,000+ stations planned, BYD is turning every charger into a grid asset, not a liability.  💡 Question:   Could this model make standalone ESS projects obsolete?  #BYD #EnergyStorage #EVCharging #SmartGrid #Innovation  

Monthly Book Review: Two reads for building real AI systems (from architecture to agents) 📘 𝗟𝗟𝗠𝘀 𝗶𝗻 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 (conceptual, system-level view) In short, this one is about ‘how to think about LLMs in business systems’. It focuses on how LLMs are deployed and integrated into organizations - covering architecture, governance, scaling, evaluation, and real-world adoption patterns. I’d say it’s especially useful for shaping the mindset around frameworks and understanding how LLMs actually fit into enterprise infrastructure. 𝗪𝗵𝗮𝘁 𝘀𝘁𝗼𝗼𝗱 𝗼𝘂𝘁 𝘁𝗼 𝗺𝗲: -  Clear breakdowns of common architecture patterns (RAG, fine-tuning, deployment, governance, etc.) - Strong focus on integration with existing workflows and data systems - Practical discussion of risk, cost, and compliance trade-offs 𝗕𝗲𝘀𝘁 𝗳𝗼𝗿 (𝗶𝗺𝗼): ▪️Technical leads moving into architecture or management roles ▪️Engineers and managers who want to understand the full picture ▪️Non-technical leaders looking to understand how LLMs can fit into their current stack 📙 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁𝘀 𝗶𝗻 𝗣𝗿𝗮𝗰𝘁𝗶𝗰𝗲 (hands-on and builder-focused) This one’s much more practical and tutorial-style. You’ll learn how to build agentic systems that connect to tools, APIs, and external data sources. 𝗪𝗵𝗮𝘁 𝗶𝘁 𝗰𝗼𝘃𝗲𝗿𝘀: ➤ Step-by-step use of LangChain, LlamaIndex, and similar frameworks ➤ Multi-agent workflows, reasoning loops, and task execution ➤ Code examples that bring together planning, memory, and real-world orchestration 𝗠𝘆 𝘁𝗮𝗸𝗲: If you’re building anything agentic, this is a great one to keep on your desk. It does assume you’re already comfortable with ML foundations and some coding, but nothing very advanced. ***Both books are great, but serve different needs. You don’t need to read them in order, but if you plan to go through both, I’d start with LLMs in Enterprise and follow with AI Agents in Practice. It’s a natural flow from systems to agents. Hope this helps anyone exploring this space, would love to hear if you’ve read either, or if you’ve got others to recommend. 🔗Links to both books below (both first edition): ✔️ AI Agents in Practice by Valentina Alto https://packt.link/RIVbG ✔️ LLMs in Enterprise by Ahmed Menshawy and Mahmoud Fahmy https://packt.link/wu2d7 __________ For more on AI and learning materials, plz check my previous posts. I share my journey here. Join me and let's grow together. Alex Wang #aiagents #agenticai #enterpriseai #bussiness

🔌 Grid operators are implementing various strategies to manage the declining inertia caused by the increased penetration of variable generation (VG) resources, such as wind and solar. These strategies fall into three main categories: maintaining inertia, providing more response time, and enhancing fast frequency response. To maintain inertia, operators can ensure that a mix of synchronous generators is online to exceed critical inertia levels. Additionally, synchronous renewable energy sources and synchronous condensers can be deployed to provide inertia. To provide more response time, operators can reduce contingency sizes and adjust underfrequency load shedding (UFLS) settings. Finally, enhancing fast frequency response involves leveraging load resources, extracting wind kinetic energy, and dispatching inverter-based resources to improve the grid's ability to respond to frequency changes. 🍃 Extracted wind kinetic energy refers to the capability of wind turbines to provide fast frequency response (FFR) by utilising the kinetic energy stored in their rotating blades. This approach can be particularly effective in addressing the challenges posed by declining inertia in power systems with high wind penetration. By extracting kinetic energy, wind turbines can respond rapidly to frequency deviations, thereby helping to stabilise the grid. This method can be used in conjunction with other resources to enhance overall system reliability and maintain frequency within acceptable limits. 💡 High deployment of variable generation (VG) resources can be effectively managed by combining extracted kinetic energy from wind turbines and increasing output from curtailed wind plants. The figure below illustrates that when these two strategies are combined, they significantly mitigate frequency decline. The simulation shows that relying solely on extracted kinetic energy results in frequency falling below UFLS (underfrequency load shedding), while using only FFR barely avoids UFLS. However, when both methods are applied together, the frequency decline is minimal, demonstrating that these approaches can serve as viable alternatives to traditional inertia and primary frequency response from conventional generators. #gridmodernization #stability #gridforming #powerelectronics #renewables #cleanenergy #solidstate

I think Red Hat’s launch of 𝗹𝗹𝗺-𝗱 could mark a turning point in 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗔𝗜. While much of the recent focus has been on training LLMs, the real challenge is scaling inference, the process of delivering AI outputs quickly and reliably in production. This is where AI meets the real world, and it's where cost, latency, and complexity become serious barriers. 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗶𝘀 𝘁𝗵𝗲 𝗡𝗲𝘄 𝗙𝗿𝗼𝗻𝘁𝗶𝗲𝗿 Training models gets the headlines, but inference is where AI actually delivers value: through apps, tools, and automated workflows. According to Gartner, over 80% of AI hardware will be dedicated to inference by 2028. That’s because running these models in production is the real bottleneck. Centralized infrastructure can’t keep up. Latency gets worse. Costs rise. Enterprises need a better way. 𝗪𝗵𝗮𝘁 𝗹𝗹𝗺-𝗱 𝗦𝗼𝗹𝘃𝗲𝘀 Red Hat’s llm-d is an open source project for distributed inference. It brings together: 1. Kubernetes-native orchestration for easy deployment 2. vLLM, the top open source inference server 3. Smart memory management to reduce GPU load 4. Flexible support for all major accelerators (NVIDIA, AMD, Intel, TPUs) AI-aware request routing for lower latency All of this runs in a system that supports any model, on any cloud, using the tools enterprises already trust. 𝗢𝗽𝘁𝗶𝗼𝗻𝗮𝗹𝗶𝘁𝘆 𝗠𝗮𝘁𝘁𝗲𝗿𝘀 The AI space is moving fast. New models, chips, and serving strategies are emerging constantly. Locking into one vendor or architecture too early is risky. llm-d gives teams the flexibility to switch tools, test new tech, and scale efficiently without rearchitecting everything. 𝗢𝗽𝗲𝗻 𝗦𝗼𝘂𝗿𝗰𝗲 𝗮𝘁 𝘁𝗵𝗲 𝗖𝗼𝗿𝗲 What makes llm-d powerful isn’t just the tech, it’s the ecosystem. Forged in collaboration with founding contributors CoreWeave, Google Cloud, IBM Research and NVIDIA and joined by industry leaders AMD, Cisco, Hugging Face, Intel, Lambda and Mistral AI and university supporters at the University of California, Berkeley, and the University of Chicago, the project aims to make production generative AI as omnipresent as Linux. 𝗪𝗵𝘆 𝗜𝘁 𝗠𝗮𝘁𝘁𝗲𝗿𝘀 For enterprises investing in AI, llm-d is the missing link. It offers a path to scalable, cost-efficient, production-grade inference. It integrates with existing infrastructure. It keeps options open. And it’s backed by a strong, growing community. Training was step one. Inference is where it gets real. And llm-d is how companies can deliver AI at scale: fast, open, and ready for what’s next.

"One of the key ways to make energy systems more reliable is by maximizing flexibility — improving how well the system can adapt in real time to changes in supply and demand. The more flexible the system, the better it can handle sudden demand spikes in the event of extreme weather, such as cold snaps or heat waves, or respond to supply disruptions such as plant outages. Improving flexibility includes upgrading aging infrastructure. Much of the U.S. grid was built decades ago under different demand patterns. Modernizing the grid — by updating substations and transmission equipment, deploying advanced sensors and incorporating advanced transmission technologies (ATTs), for example — can reduce failure rates during extreme heat and cold. These technologies help operators detect problems quicker, reroute power if equipment is damaged and restore service fast. Modernization not only improves reliability but also reduces expensive emergency interventions and lowers long-term maintenance costs. Increasing grid capacity, both through deployment of ATTs and building regional and interregional transmission lines, can reduce the risk of a local weather event turning into a widespread outage. Creating a more interconnected grid allows regions to share power during shortages. Having this greater transmission capacity also help keep prices down by allowing lower-cost electricity to reach areas facing higher demand. Demand-side management options can help ease pressure on the system during extreme weather events. These include encouraging customers and large users to reduce or shift electricity use during peak periods in exchange for lower bills or leveraging distributed energy resources to help prevent shortages. Systems that rely too much on a single fuel are more vulnerable to disruption. Diversification across energy sources and technologies helps reduce the risk of issues related to fuel shortages, infrastructure failures and localized weather impacts. Finally, policy is also critical. It’s vital that incentives are properly aligned with modern needs for flexibility and preparedness. This can help utilities make system investments that really work in extreme weather and minimize costs to consumers in both the short and the long run." Kelly Lefler World Resources Institute https://lnkd.in/e5syqXQp

Modern data center strategy has become a strategic differentiator in the AI era. Leaders can no longer rely on hybrid-by-default environments shaped by fragmented cloud, colocation, and on-premises decisions. Instead, a deliberate, hybrid-by-design approach is now essential to scale innovation, manage risk, and enhance value across cloud, on-premises, colocation, and edge.    In our latest Deloitte perspective (https://deloi.tt/4rkttVw), my colleagues Lou DiLorenzo, Jagjeet Gill, Heather Rangel, and I outline practical steps for leaders driving this shift, including:    🟢 Intentional workload placement based on latency, control, data sovereignty, economics, and resiliency needs 🟢 Strategic segmentation of AI-intensive workloads to manage compute, power, and cooling demands 🟢 Transparent economics that tie infrastructure cost to business value 🟢 Built-in governance across hybrid environments through standardized controls and automation The goal is not incremental modernization, but intentional architecture that turns complexity into advantage and enables resilient, responsible AI at scale.    Proud of our team's work in helping organizations build forward-thinking data center strategies and leading our hybrid infrastructure managed services, led by Erin Abbey, Rahul Bajpai, Micah Bible, Megan Ellis, Christian Grant, Kelly Marchese, Nicholas Merizzi, and Myke Miller. Let me know if building a hybrid-by-design strategy is top of mind for your organization in 2026; would love to connect! 

Building Redundancy with 3 Routers: DHCP + OSPF in Action In networking, redundancy isn’t just a buzzword—it’s the backbone of reliability. Imagine a setup with three routers working together, where even if one fails, communication still flows seamlessly across the network. In this lab, I explored how to: - Configure DHCP on the routers to automatically assign IP addresses to end devices. - Implement OSPF (Open Shortest Path First) as the dynamic routing protocol, ensuring all routers know the best path to reach every network. - Build redundancy so no single point of failure breaks communication between the LANs. 💡 Key takeaway: Redundancy + OSPF = a resilient network where devices stay connected, and IP management is automated through DHCP. Would love to hear how others are approaching redundancy in their lab setups—are you relying more on OSPF, EIGRP, or experimenting with HSRP/VRRP for failover? #Networking #Cisco #OSPF #DHCP #Redundancy #LearningInPublic

If you’re building anything with LLMs, your system architecture matters more than your prompts. Most people stop at “call the model, get the output.” But LLM-native systems need workflows, blueprints that define how multiple LLM calls interact, how routing, evaluation, memory, tools, or chaining come into play. Here’s a breakdown of 6 core LLM workflows I see in production: 🧠 LLM Augmentation Classic RAG + tools setup. The model augments its own capabilities using: → Retrieval (e.g., from vector DBs) → Tool use (e.g., calculators, APIs) → Memory (short-term or long-term context) 🔗 Prompt Chaining Workflow Sequential reasoning across steps. Each output is validated (pass/fail) → passed to the next model. Great for multi-stage tasks like reasoning, summarizing, translating, and evaluating. 🛣 LLM Routing Workflow Input routed to different models (or prompts) based on the type of task. Example: classification → Q&A → summarization all handled by different call paths. 📊 LLM Parallelization Workflow (Aggregator) Run multiple models/tasks in parallel → aggregate the outputs. Useful for ensembling or sourcing multiple perspectives. 🎼 LLM Parallelization Workflow (Synthesizer) A more orchestrated version with a control layer. Think: multi-agent systems with a conductor + synthesizer to harmonize responses. 🧪 Evaluator–Optimizer Workflow The most underrated architecture. One LLM generates. Another evaluates (pass/fail + feedback). This loop continues until quality thresholds are met. If you’re an AI engineer, don’t just build for single-shot inference. Design workflows that scale, self-correct, and adapt. 📌 Save this visual for your next project architecture review. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg