Optimizing Technology Spending

You're a #CTO. Your board asks: "What's our ROI on AI coding tools?" Your answer: "40% of our code is AI-generated!" They respond: "So what? Are we shipping faster? Are customers happier?" Most CTOs are measuring AI impact completely wrong. Here's what some are tracking: - Percentage of AI-generated code - Developer hours saved per week - Lines of code produced - AI tool adoption rates These metrics are like measuring how fast your assembly line workers attach parts while ignoring whether your cars actually start. Here's what you SHOULD measure instead: 1. Delivered business value 2. Customer cycle time 3. Development throughput 4. Quality and reliability 5. Total cost of delivery (not just development) 6. Team satisfaction Software development isn't a typing competition—it's a complex system. If AI makes your developers 30% faster but your deployment takes 2 weeks and QA adds another week, your customer delivery improves by maybe 7%. You've speed up the wrong part. The solution: A/B test your teams. Give half your teams AI tools, measure business outcomes over 2-3 release cycles. Track what customers actually experience, not how much developers produce. Companies that measure business impact from AI will pull ahead. Those measuring vanity metrics will wonder why their expensive tools aren't moving the needle. Stop measuring how much code AI generates. Start measuring how much faster you deliver value to customers. What are you actually measuring? And is it moving your business forward? -> Follow me for more about building great tech organizations at scale. More insights in my book "All Hands on Tech"

You don't need a 2 trillion parameter model to tell you the capital of France is Paris. Be smart and route between a panel of models according to query difficulty and model specialty! New paper proposes a framework to train a router that routes queries to the appropriate LLM to optimize the trade-off b/w cost vs. performance. Overview: Model inference cost varies significantly: Per one million output tokens: Llama-3-70b ($1) vs. GPT-4-0613 ($60), Haiku ($1.25) vs. Opus ($75) The RouteLLM paper propose a router training framework based on human preference data and augmentation techniques, demonstrating over 2x cost saving on widely used benchmarks. They define the problem as having to choose between two classes of models: (1) strong models - produce high quality responses but at a high cost (GPT-4o, Claude3.5) (2) weak models - relatively lower quality and lower cost (Mixtral8x7B, Llama3-8b) A good router requires a deep understanding of the question’s complexity as well as the strengths and weaknesses of the available LLMs. Explore different routing approaches: - Similarity-weighted (SW) ranking - Matrix factorization - BERT query classifier - Causal LLM query classifier Neat Ideas to Build From: - Users can collect a small amount of in-domain data to improve performance for their specific use cases via dataset augmentation. - Can expand this problem from routing between a strong and weak LLM to a multiclass model routing approach where we have specialist models(language vision model, function calling model etc.) - Larger framework controlled by a router - imagine a system of 15-20 tuned small models and the router as the n+1'th model responsible for picking the LLM that will handle a particular query at inference time. - MoA architectures: Routing to different architectures of a Mixture of Agents would be a cool idea as well. Depending on the query you decide how many proposers there should be, how many layers in the mixture, what the aggregate models should be etc. - Route based caching: If you get redundant queries that are slightly different then route the query+previous answer to a small model to light rewriting instead of regenerating the answer

Too many AI strategies are being built around the technology instead of the business challenges they should solve. The real value of AI comes when it is directly tied to your goals. I have arrived at seven lessons on how to align your AI strategy directly with your business goals: 1. Start with the "why," not the "what." Before discussing models or tools, ask what business problem you need to solve. It could be speeding up product development, or cutting operational costs. Let that answer be your guide. 2. Think in terms of business outcomes. Measure AI success by its impact on metrics like revenue growth or employee productivity not by technical accuracy. 3. Build a cross-functional team. AI can't live solely in the IT department. Include leaders from all relevant departments from day one to ensure the strategy serves the entire business. 4. Prioritize quick wins to build momentum. Identify a few small, high-impact projects that can deliver results quickly. This builds organizational confidence and makes people ready to take on larger initiatives. 5. Invest in data foundations. The best AI strategy will fail without clean and well-governed data. A disciplined approach to data quality is non-negotiable. 6. Focus on change management. Technology is the easy part. Prepare your people for new workflows and equip them with the skills to work alongside AI effectively. 7. Create a feedback loop. An AI strategy is not a one-time plan. Continuously gather feedback from users and analyze performance data to adapt and refine your approach. The goal is to make AI a part of how you achieve your objectives, not a separate project. #AIStrategy #BusinessGoals #DigitalTransformation #Leadership #ArtificialIntelligence

Why do so many legal technology implementations fail to deliver their promised value? Too often, legal teams rush to adopt the latest tools without first understanding their actual pain points. Here are the critical steps that separate successful implementations from costly failures: 📊 Start with Discovery, Not Solutions Map your current workflows meticulously. Track how long tasks take, where errors occur, and what frustrates your team most. 🎯 Set Measurable Goals Replace vague aspirations like "improve efficiency" with concrete targets: -Reduce contract turnaround by 30% -Eliminate 50% of manual compliance errors -Increase client intake capacity by 25% These specific metrics give you clear success criteria and help demonstrate ROI to stakeholders. 👥 Embrace Change Management Technology fails when people resist it. Appoint enthusiastic "technology champions" who can provide peer support and bridge the gap between IT and daily users. Their grassroots advocacy often proves more effective than top-down mandates. 🔄 Pilot, Learn, Iterate Test solutions with a small group for 6-8 weeks before full rollout. That same legal department reduced their NDA processing time to 1.5 hours and cut errors by 80% during their pilot. These wins built momentum for broader adoption. Remember: legal technology adoption is about solving real problems, not chasing innovation for its own sake. #legaltech #innovation #law #business #learning

Over the last 18 months, the FinOps Foundation has seen a dramatic shift in the scope of spending that #FinOps practices manage beyond public cloud. We first explored this anticipated shift in the second edition of Cloud FinOps (pg. 401) where we shared a vision for how we expected the scope of FinOps to expand: to a world where FinOps practices are integrating costs beyond public cloud – from SaaS, to licensing, datacenter, and private cloud – for a more complete picture of cost to drive value-based decision-making across a broader scope of spending. In recent surveys, we are seeing upwards of 70% of practitioners now extending their practice beyond public cloud to other types of technology spend. To reflect this reality, the FinOps Foundation Technical Advisory Council has approved a new element in the FinOps Framework to capture the segments associated with the different types of technology cost and usage data FinOps Practitioners are managing: FinOps Scope. Read more in the new Insights article on the expanded scope of FinOps: https://lnkd.in/gPH3vQEn In some cases, especially for companies “born in the cloud,” FinOps teams are the only technology cost management team in the organization. In other cases, FinOps Practitioners are working alongside Allied Personas  (ITAM/ITSM/ITFM/TBM/SAM). But in all cases, FinOps’ success in managing cloud spending has the business asking “Can FinOps keep doing what you’re doing for cloud, AND also do it for X?” While other disciplines report on cost at a chargeback level, they do this for a monthly and quarterly roll-up of financial reporting at the general ledger level. FinOps, by contrast, is leveraging extremely granular cost and usage data at levels for all stakeholders, from engineering, to architecture, to product, to finance, and to executives, enabling them to: - Make information available outside of traditional silos to empower Personas across the organization, beyond Leadership – not just the CFO and CIO. - Enable timely decision-making about technology investment choices in “fixed” and variable Scopes. - Enable collaboration between technology and business teams at the engineering and product level. - Enable Cost Aware Product Decisions by bringing cost considerations earlier into the product development lifecycle. - Optimize, modernize, and automate to create consistency and iteratively improve technology usage and cost. Applying FinOps Capabilities to additional Scopes of spending gives businesses more comprehensive visibility into their technology costs. The goal for organizations is to understand and optimize the cost of offering each individual product or service. The first step is to get complete visibility into the cost of a product or service by pulling together all types of costs associated with delivering it... Read more in the new Insights article on the expanded scope of FinOps: https://lnkd.in/gPH3vQEn

If you’re an AI engineer trying to optimize your LLMs for inference, here’s a quick guide for you 👇 Efficient inference isn’t just about faster hardware, it’s a multi-layered design problem. From how you compress prompts to how your memory is managed across GPUs, everything impacts latency, throughput, and cost. Here’s a structured taxonomy of inference-time optimizations for LLMs: 1. Data-Level Optimization Reduce redundant tokens and unnecessary output computation. → Input Compression:  - Prompt Pruning, remove irrelevant history or system tokens  - Prompt Summarization, use model-generated summaries as input  - Soft Prompt Compression, encode static context using embeddings  - RAG, replace long prompts with retrieved documents plus compact queries → Output Organization:  - Pre-structure output to reduce decoding time and minimize sampling steps 2. Model-Level Optimization (a) Efficient Structure Design → Efficient FFN Design, use gated or sparsely-activated FFNs (e.g., SwiGLU) → Efficient Attention, FlashAttention, linear attention, or sliding window for long context → Transformer Alternates, e.g., Mamba, Reformer for memory-efficient decoding → Multi/Group-Query Attention, share keys/values across heads to reduce KV cache size → Low-Complexity Attention, replace full softmax with approximations (e.g., Linformer) (b) Model Compression → Quantization:  - Post-Training, no retraining needed  - Quantization-Aware Training, better accuracy, especially <8-bit → Sparsification:  - Weight Pruning, Sparse Attention → Structure Optimization:  - Neural Architecture Search, Structure Factorization → Knowledge Distillation:  - White-box, student learns internal states  - Black-box, student mimics output logits → Dynamic Inference, adaptive early exits or skipping blocks based on input complexity 3. System-Level Optimization (a) Inference Engine → Graph & Operator Optimization, use ONNX, TensorRT, BetterTransformer for op fusion → Speculative Decoding, use a smaller model to draft tokens, validate with full model → Memory Management, KV cache reuse, paging strategies (e.g., PagedAttention in vLLM) (b) Serving System → Batching, group requests with similar lengths for throughput gains → Scheduling, token-level preemption (e.g., TGI, vLLM schedulers) → Distributed Systems, use tensor, pipeline, or model parallelism to scale across GPUs My Two Cents 🫰 → Always benchmark end-to-end latency, not just token decode speed → For production, 8-bit or 4-bit quantized models with MQA and PagedAttention give the best price/performance → If using long context (>64k), consider sliding attention plus RAG, not full dense memory → Use speculative decoding and batching for chat applications with high concurrency → LLM inference is a systems problem. Optimizing it requires thinking holistically, from tokens to tensors to threads. Image inspo: A Survey on Efficient Inference for Large Language Models ---- Follow me (Aishwarya Srinivasan) for more AI insights!

I found a way to triple eCommerce revenue without writing hundreds of articles. Instead, by focusing on these big needle-movers (which most brands skip)... We grew a client’s monthly traffic by 115%... and monthly revenue 198% in just 9 months. Here's how: (full eCommerce SEO crash course)👇 #1: Personalize and Streamline the Shopping Experience • Add product recommendations based on user behavior • Show “complete the look” bundles • Enable cart-saving and wishlists • Simplify checkout with guest options, pre-filled forms, one-click purchase, and collapsible stages #2: Leverage Seasonal Search Trends Ask ChatGPT: “Suggest seasonal keywords for [your niche].” Validate demand in Google Trends. Build content 2 months before interest spikes. This allows you to pre-position for surges in buyer intent. #3 Micro-Moments Strategy There are 4 key decision-making moments in the buyer’s journey. "I want to know" content (researching): • Blog posts answering product questions • FAQ pages about product care/maintenance • Explainer videos showing product features "I want to go" content (for physical stores): • About page with clear location info • Contact page with embedded map • Updated Google Business Profile with photos and reviews "I want to do" content (learning how to use): • How-to guides for using your products • Assembly instructions with clear visuals • Video tutorials showing product in action • Downloadable user manuals "I want to buy" content (ready for purchase): • Product descriptions that highlight benefits • Category pages that compare similar products • Customer reviews and testimonials • Clear pricing and availability info #4 Faceted Navigation Strategy Faceted nav lets users filter by size, color, price, etc. But done wrong, it’ll bloat your index with duplicate URLs. Here’s how to implement it properly: • Use buttons or <input>—not <a href> • Add canonical tags on filtered pages → point to main category page • For high-potential filtered URLs (e.g. “blue running shoes”), create internal links to them • Use AJAX so filters don’t generate new URLs • Remove noindex/nofollow/robots.txt blocks for URLs you want indexed WordPress? Use WP Grid Builder. WooCommerce? Follow the official SEO filtering guide. Shopify? Enable Storefront Filtering. #5 Schema Markup Strategy Schema helps Google understand your content and display rich results. When your listing takes up more real estate and draws the eye, users are more likely to click. Use these two schema types: • Product Schema: includes ratings, reviews, price, availability • BreadcrumbList Schema: helps Google understand your site structure Use ChatGPT to generate your schema fast, then validate it on validator(.)schema(.)org before uploading. Our client’s result after implementing this strategy? • Organic traffic: +115% (12.8K to 27.6K sessions) • Monthly revenue: +198% ($10.2K to $30.6K) • Keywords in top 10: +36% (2,005 to 2,737)

Having a lot of data isn’t the same thing as having high-value data. If you’re having a hard time explaining that to executive leaders, try a different approach. Teach them how to put a dollar value on the business’s data. Every curated dataset creates new opportunities for the business, and that’s the connection between data and profit. The simplest data valuation method is called ‘With & Without’. The business thinks that every dataset creates the same value, so I run an early experiment to disprove that assumption. I turn off access to datasets that stakeholders believe are high value and wait for the complaints to roll in. In most cases, no one notices. Three months later, I propose putting the dataset into cold storage. Business leaders push back, saying their teams would grind to a halt without access to those datasets. I tell them about the experiment. Now I can start a rational conversation about connecting data to use cases and putting a dollar value on each dataset. Data doesn’t create value for two reasons: 1️⃣ It’s incomplete. The data required to support the use case isn’t being gathered holistically. Sometimes that’s an accessibility issue. Other times, the use case, workflow, and outcomes aren’t understood well enough to know what data is necessary. 2️⃣ It lacks context. Data points aren’t enough to support use cases. Context about the process, product, person, intent, and outcome is required. Until data is gathered contextually, its value creation is limited. Connecting datasets with opportunities creates the justification for changing how the business gathers and leverages data. Putting a dollar value on contextual datasets quantifies the ROI of information architecture and engineering initiatives. That’s the shortest path to getting budget and buy-in. Quantify value in terms that business leaders care about and show them a clear connection with outcomes they believe are essential.

Look at this fascinating chart from the Bureau of Labor Statistics tracking price changes from 2000 to 2022. It’s striking how certain goods and services have soared in cost—like hospital services, college tuition, and textbooks—while items such as TVs, toys, and software have become dramatically more affordable. This divergence often results from how easily technology can boost productivity. Consumer electronics, for example, benefit from rapid innovation and economies of scale. By contrast, labor-intensive sectors like healthcare and education have been harder to automate, causing costs to balloon over time. Artificial intelligence stands to change this dynamic. Machine learning and other AI tools can: 1. Automate Repetitive Tasks: From diagnostic screenings in healthcare to administrative work in higher education, AI has the potential to free up human time for high-impact tasks. 2. Enhance Efficiency: Data-driven insights can reduce waste, optimize operations, and drive down expenses—particularly in service-heavy industries. 3. Expand Access: AI-powered solutions (telehealth, online courses, intelligent tutoring systems) might increase supply and improve affordability for services that have traditionally been expensive and difficult to scale. Implications for Leaders and Professionals: • Opportunity to Innovate: As AI adoption grows in cost-heavy sectors, organizations that embrace it strategically can deliver higher-quality services at lower prices. • Skill Shifts: Tasks in project management, data analysis, and AI oversight will become even more critical to ensuring that technology actually improves outcomes rather than just cutting costs. • Future Competition: Startups and incumbents alike will be racing to apply AI in these traditionally high-cost areas, creating a competitive edge for first movers. Ultimately, charts like this remind us of how unevenly technology affects costs—and how AI offers new ways to tackle price inflation in essential services. If we harness it responsibly, we just might help bend those red lines back downward…

Is your Shopify store a maze of apps? Simplify to amplify. Less is more when it comes to optimizing for conversion! Imagine walking into a store with products scattered everywhere. Confusing, right? Your online store isn't so different. Too many apps can clutter your digital shelves, slow down your site, and frustrate potential customers. Here's how to streamline your Shopify store for maximum impact: ✔ Audit your apps → List all the apps you're currently using. → Identify which ones are truly essential. → Remove the ones that don’t directly contribute to your core goals: conversion and customer satisfaction. ✔ Evaluate functionality over quantity → Opt for apps that provide multiple functions. → This reduces complexity and improves site speed. ✔ Test site speed regularly → Use tools like Google PageSpeed Insights. → Monitor how each app impacts your speed. → Prioritize apps that are lightweight and optimized. ✔ Continuous review → The #ecommerce landscape evolves rapidly. → Regularly reassess your app suite to ensure you're aligned with the latest trends and customer expectations. Remember, it's not about how many apps you have. It's about having the right ones. Simplify your digital storefront to amplify your sales. What app has made the biggest impact on your #Shopify store?