Business Intelligence Consulting

Explore top LinkedIn content from expert professionals.

  • View profile for Yassine Mahboub

    Data Engineer @ Deloitte | Azure & Fabric | CDMP®

    41,822 followers

    📌 MS Fabric Breakdown # 1: Architecture (How to Build a BI Solution with Microsoft Fabric) Since Fabric’s release in 2023, a lot of Power BI-centric organizations are moving their entire data stack into Fabric. And honestly it makes perfect sense. If you're already spending thousands on Power BI licensing, why not unify your entire data architecture under one platform? You get a single environment for: → Ingestion → Storage (Lakehouse & Warehouse) → Modeling → Dashboarding All in one place and you solve most of your data silos problems. In this first post of the Fabric series, I’m sharing a high-level BI architecture you can use as a framework for your implementation: 1️⃣ 𝐊𝐧𝐨𝐰 𝐘𝐨𝐮𝐫 𝐏𝐫𝐢𝐨𝐫𝐢𝐭𝐢𝐞𝐬 Before diving into technical planning, align your BI goals with real business needs. Ask yourself: → Which departments need reporting the most right now? → What KPIs are critical to track in the short term? → Who are the key stakeholders and decision-makers? This helps you focus your resources and deliver impact from day one. 2️⃣ 𝐈𝐝𝐞𝐧𝐭𝐢𝐟𝐲 𝐘𝐨𝐮𝐫 𝐃𝐚𝐭𝐚 𝐒𝐨𝐮𝐫𝐜𝐞𝐬 Now it’s time to map where your data lives. Some typical examples: ⤷ SQL Server & on-prem databases ⤷ ERPs (SAP, Oracle, etc.) ⤷ SaaS platforms (Salesforce, HubSpot, Stripe, etc.) ⤷ Excel files & manual spreadsheets And remember to prioritize what’s valuable to the business. Don’t waste time and resources ingesting data no one uses. 3️⃣ 𝐌𝐚𝐩 𝐎𝐮𝐭 𝐘𝐨𝐮𝐫 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 Once you know your sources, you need to design the data platform. There are many schools of thought on how to build modern BI architecture. But one of the most practical and scalable is the Medallion Architecture, built across three layers: 1) Bronze Layer: Raw data in its original format 2) Silver Layer: Cleaned, structured, and business-ready tables 3) Gold Layer: Modeled datasets optimized for reporting In Fabric, you can easily orchestrate everything using Data Factory (equivalent of ADF if you're familiar with Azure Ecosystem) and store in OneLake, using Lakehouses and Warehouses depending on your use case. 4️⃣ 𝐂𝐫𝐞𝐚𝐭𝐞 𝐃𝐚𝐭𝐚 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐬 𝐟𝐨𝐫 𝐒𝐞𝐥𝐟-𝐒𝐞𝐫𝐯𝐢𝐜𝐞 Your final goal is not just storage. It's decision-making. You can build reusable semantic models with KPIs and business logic defined upstream (vs PBI Desktop). This enables business users to explore, visualize, and analyze data without engineering support. Power BI then becomes a front-end for exploration not just a reporting tool. Next in the series: We’ll break down each layer (Bronze → Silver → Gold) with practical examples and tips and how Power BI fits within this architecture. 📥 Save this post if you’re planning to implement Fabric. #MicrosoftFabric #PowerBI #BusinessIntelligence

  • View profile for Aishwarya Pani

    Senior Data+AI Engineer @ EY | Helping 100K+ Professionals Break Into Data Engineering 🚀 | Azure | Databricks | AI | 4x Microsoft Certified | 3x Databricks Certified | Career Coach | Paid Brand Collaborations

    147,026 followers

    𝗧𝗵𝗲 𝘀𝗵𝗶𝗳𝘁 𝗳𝗿𝗼𝗺 𝗱𝗮𝘁𝗮 𝘄𝗮𝗿𝗲𝗵𝗼𝘂𝘀𝗲𝘀 𝘁𝗼 𝗹𝗮𝗸𝗲-𝗰𝗲𝗻𝘁𝗿𝗶𝗰 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲𝘀 𝗶𝘀𝗻’𝘁 𝗮 𝘁𝗿𝗲𝗻𝗱. It’s survival. Modern data teams aren’t struggling because of 𝘁𝗼𝗼𝗹𝘀 — they’re struggling because of 𝘀𝗰𝗮𝗹𝗲, 𝘀𝗽𝗲𝗲𝗱, 𝗮𝗻𝗱 𝗰𝗼𝗺𝗽𝗹𝗲𝘅𝗶𝘁𝘆. And that’s where architecture decisions quietly make or break platforms. The real question is not: ❌ 𝗪𝗵𝗮𝘁’𝘀 𝗽𝗼𝗽𝘂𝗹𝗮𝗿 𝗿𝗶𝗴𝗵𝘁 𝗻𝗼𝘄? ✅ 𝗪𝗵𝗮𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗳𝗶𝘁𝘀 𝘁𝗵𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺? Here’s how I usually think about architecture choices ⬇️ 𝗠𝗲𝗱𝗮𝗹𝗹𝗶𝗼𝗻 (𝗕𝗿𝗼𝗻𝘇𝗲–𝗦𝗶𝗹𝘃𝗲𝗿–𝗚𝗼𝗹𝗱) A disciplined, layered approach that brings order to chaos. Works best when data quality, incremental loads, and reliability matter more than speed alone. 𝗗𝗮𝘁𝗮 𝗠𝗲𝘀𝗵 Domain-driven ownership with federated governance. Extremely powerful — but only when ownership, standards, and contracts are real (not aspirational). 𝗧𝗿𝗮𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗗𝗮𝘁𝗮 𝗪𝗮𝗿𝗲𝗵𝗼𝘂𝘀𝗲 Still undefeated for analytics-heavy workloads. Fast SQL, complex joins, historical reporting, and a clear single source of truth. From 𝗟𝗮𝗺𝗯𝗱𝗮, 𝗞𝗮𝗽𝗽𝗮, 𝗗𝗮𝘁𝗮 𝗩𝗮𝘂𝗹𝘁, 𝗟𝗮𝗸𝗲𝗵𝗼𝘂𝘀𝗲 and beyond — every architecture exists because someone hit a real limitation and had to solve it. There is no 𝗯𝗲𝘀𝘁 architecture. There is only 𝗳𝗶𝘁. A few lessons from building and supporting real platforms: • Architecture should follow business constraints, not hype cycles • Hybrid approaches (Lakehouse, Mesh + Medallion) are far more common than “pure” models • The best designs reduce pipeline fragility — not just query latency If you’re choosing today, ask yourself: – Need strong governance with modern BI? → Medallion – Query performance and historical analytics matter most? → Data Warehouse – Large org with clear domain ownership? → Data Mesh Good architecture doesn’t show off. It quietly enables 𝘀𝗰𝗮𝗹𝗲, 𝘁𝗿𝘂𝘀𝘁, 𝗮𝗻𝗱 𝗰𝗵𝗮𝗻𝗴𝗲. 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝗶𝘀 𝘁𝗵𝗲 𝗹𝗼𝗻𝗴-𝘁𝗲𝗿𝗺 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻 𝘆𝗼𝘂 𝗺𝗮𝗸𝗲 𝗼𝗻 𝗱𝗮𝘆 𝗼𝗻𝗲. Curious — 𝗵𝗼𝘄 𝗱𝗼 𝘆𝗼𝘂 𝗱𝗲𝗰𝗶𝗱𝗲 𝘄𝗵𝗶𝗰𝗵 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 𝘁𝗼 𝘂𝘀𝗲 𝗶𝗻 𝘆𝗼𝘂𝗿 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀? What worked, and what didn’t? ♻️ 𝗥𝗲𝗽𝗼𝘀𝘁 𝗶𝗳 𝘁𝗵𝗶𝘀 𝘄𝗮𝘀 𝗵𝗲𝗹𝗽𝗳𝘂𝗹. 🔔 𝗙𝗼𝗹𝗹𝗼𝘄 Aishwarya Pani 𝗳𝗼𝗿 𝗺𝗼𝗿𝗲 𝗶𝗻𝘀𝗶𝗴𝗵𝘁𝘀 𝗼𝗻 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴.

  • View profile for Aditya Singh Rathore

    Data engineer @GOD l Winner @GSSoC’25 |Fabric certified Data Engineer | 5 x Azure certified |Data engineering top voice 💡

    3,616 followers

    Most data engineers argue about tools. Elite ones argue about architecture. And the scary part? Most teams don’t realize they picked the wrong one until 18 months later. I’ve reviewed 200+ data systems in the last 3 years. 90% weren’t failing because of bad engineering. They were failing because the architecture didn’t match: • Org structure • Data volume • Latency expectations • Team maturity Same 3 architectures keep showing up. But only 1 in 5 engineers truly know when to use which. Here’s the real breakdown 👇 🕸️ 𝗗𝗔𝗧𝗔 𝗠𝗘𝗦𝗛 — “𝗔𝘂𝘁𝗼𝗻𝗼𝗺𝘆 𝗮𝘁 𝗦𝗰𝗮𝗹𝗲” Domain-owned data products. Decentralized responsibility. Federated governance. ✅ 𝗨𝘀𝗲 𝘄𝗵𝗲𝗻: • You have multiple mature domain teams • Data ownership conflicts are slowing delivery • Central data team has become a bottleneck ❌ 𝗔𝘃𝗼𝗶𝗱 𝗶𝗳: • Your teams aren’t technically mature • You lack strong governance standards Mesh without discipline = distributed chaos. 🏛️ 𝗗𝗔𝗧𝗔 𝗪𝗔𝗥𝗘𝗛𝗢𝗨𝗦𝗘 — “𝗖𝗼𝗻𝘁𝗿𝗼𝗹 & 𝗖𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝗰𝘆” Centralized. Structured. Trusted. ETL → Curated models → BI layer. ✅ 𝗨𝘀𝗲 𝘄𝗵𝗲𝗻: • You need reliable reporting • Executives care about one version of truth • Compliance & governance matter ❌ 𝗔𝘃𝗼𝗶𝗱 𝗶𝗳: • You need ultra-low latency ML pipelines • Teams require rapid experimentation • Warehouses optimize for trust, not speed. 🥇 𝗠𝗘𝗗𝗔𝗟𝗟𝗜𝗢𝗡 — “𝗦𝗽𝗲𝗲𝗱 𝘄𝗶𝘁𝗵 𝗦𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲” Bronze → Silver → Gold. Batch + streaming. AI-ready pipelines. ✅ 𝗨𝘀𝗲 𝘄𝗵𝗲𝗻: • You need ML/AI at scale • Raw + refined data must coexist • Real-time insights matter ❌ 𝗔𝘃𝗼𝗶𝗱 𝗶𝗳: • You don’t actually need layered complexity • Not every company needs Gold tables on day one. 🎯 𝗧𝗛𝗘 𝗥𝗘𝗔𝗟 𝗩𝗘𝗥𝗗𝗜𝗖𝗧 There is no “best” architecture. There is only: • The one your org can execute well • The one your teams can sustain • The one aligned with your business velocity Pick wrong — and you’ll spend 2 years refactoring pipelines instead of building value. Pick right — and architecture becomes invisible. And that’s the goal. If this made you rethink your stack, repost ♻️ Because someone in your org is about to make this decision. 💬 What’s your company running on — Mesh, Warehouse, Medallion… or a hybrid nobody admits? 👀 #DataEngineering #DataArchitecture #DataMesh #Lakehouse #DataWarehouse #BigData #Analytics #MLOps #TechLeadership

  • View profile for Sumit Gupta 📊

    100K community | Top 5 Data/AI creator | Author/Keynote Speaker | Ex-Notion, Snowflake, Dropbox | EB1A | GDE

    57,531 followers

    𝐈𝐟 𝐲𝐨𝐮𝐫 𝐝𝐚𝐭𝐚 𝐬𝐭𝐫𝐚𝐭𝐞𝐠𝐲 𝐬𝐭𝐚𝐫𝐭𝐬 𝐰𝐢𝐭𝐡 "𝐰𝐡𝐢𝐜𝐡 𝐭𝐨𝐨𝐥," 𝐢𝐭'𝐬 𝐚𝐥𝐫𝐞𝐚𝐝𝐲 𝐰𝐫𝐨𝐧𝐠. It should start with which problem. Mesh, Fabric, and Lakehouse get pitched like rival platforms. They're not. They answer three completely different questions. Use this to decide 👇 𝐈𝐬 𝐲𝐨𝐮𝐫 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐩𝐞𝐨𝐩𝐥𝐞? → 𝐃𝐚𝐭𝐚 𝐌𝐞𝐬𝐡 → Central data team is the bottleneck → Domains can't move fast enough → Need: domain ownership, data-as-product, federated governance, SLA contracts 𝐈𝐬 𝐲𝐨𝐮𝐫 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐢𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧? → 𝐃𝐚𝐭𝐚 𝐅𝐚𝐛𝐫𝐢𝐜 → Data scattered across cloud, SaaS, on-prem → Manual lineage, policy gaps, slow discovery → Need: active metadata, knowledge graph, virtualization, AI/ML augmentation, real-time discovery 𝐈𝐬 𝐲𝐨𝐮𝐫 𝐩𝐫𝐨𝐛𝐥𝐞𝐦 𝐬𝐭𝐨𝐫𝐚𝐠𝐞? → 𝐃𝐚𝐭𝐚 𝐋𝐚𝐤𝐞𝐡𝐨𝐮𝐬𝐞 → Lake too messy, warehouse too rigid → Duplicating data for BI vs ML → Need: ACID transactions, Delta/Iceberg/Hudi, BI + ML on one copy, streaming + batch unified 𝐓𝐡𝐞 𝐬𝐡𝐢𝐟𝐭: Stop asking "which architecture?" Start asking "which bottleneck?" The right answer often involves all three, but in the right order. What's your biggest data bottleneck right now? Follow Sumit Gupta for more such insights!!

  • View profile for Bhausharani Machireddy

    Senior Data Engineer | Data Modeler | Data Governance | Analyst | Big Data & Cloud Specialist | SQL, Python, Scala, Spark | Azure, AWS, GCP | Snowflake, Databricks, Fabric

    6,232 followers

    𝐃𝐚𝐭𝐚 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐏𝐡𝐢𝐥𝐨𝐬𝐨𝐩𝐡𝐢𝐞𝐬: 𝐈𝐧𝐦𝐨𝐧, 𝐊𝐢𝐦𝐛𝐚𝐥𝐥, 𝐃𝐚𝐭𝐚 𝐕𝐚𝐮𝐥𝐭 & 𝐌𝐞𝐝𝐚𝐥𝐥𝐢𝐨𝐧 As organizations scale their data platforms, choosing the right architecture becomes a strategic differentiator. From traditional EDWs to modern lakehouse patterns, each framework offers unique strengths in quality, agility, auditability, and real-time processing. 💡 Key philosophies shaping today’s data platforms 🔹 Inmon – The Corporate Powerhouse A disciplined, enterprise-wide approach focused on governance, consistency, and historical reporting. Still a strong fit for regulated and compliance-driven environments. 🔹 Kimball – The Business-First Rebel Dimensional modeling that prioritizes speed, business accessibility, and rapid ROI with conformed dimensions and self-service insights. 🔹 Data Vault – The Historian’s Architecture A resilient, audit-friendly model designed for traceability, change management, and complex lineage across modern data ecosystems. 🔹 Medallion – The Lakehouse Reality A modular, layer-based approach enabling real-time analytics, ML readiness, and scalable ingestion in cloud-native platforms. 📊 Where these models shine • Inmon → Enterprise governance & financial reporting • Kimball → Dashboards, BI, & quick business value • Data Vault → Compliance, historical audit, and complex integration • Medallion → Streaming, ML workloads, and unified lakehouse architectures 📈 Industry-wide data engineering trends • Hybrid modeling (Vault + Lakehouse + Dimensional) becoming the norm • Data quality, lineage, and governance shifting left into pipelines • Real-time and near-real-time architectures accelerating enterprise demand • AI-driven automation influencing modeling, validation, and monitoring As Senior Data Engineers, our responsibility is to pick the architecture that aligns with business outcomes — not just follow a methodology. The future lies in blending these patterns to build scalable, governed, and AI-ready data ecosystems. #DataEngineering #DataArchitecture #Inmon #Kimball #DataVault #MedallionArchitecture #Lakehouse #BI #DataQuality #Analytics

  • View profile for Igor G.

    Sr AVP, Technical Data Strategy Leader for Data and AI

    7,632 followers

    Data modelling in a #Bronze#Silver#Gold architecture is not just about moving data through layers. It is about applying the right model at the right stage. Bronze should preserve data as-is. Minimal transformation. Full lineage. Strong auditability. Silver is where real architecture begins. This is the layer for cleansing, standardization, conformance, survivorship rules, and business entities such as Customer, Product, Supplier, Policy, and Location. In practice, this is where 3NF or Data Vault patterns often create the reusable foundation for downstream use cases. Gold is where data is shaped for consumption. Dimensional models, star schemas, and curated wide tables make analytics, reporting, and AI use cases easier to consume. My view: #MDM is the cornerstone of the whole medallion architecture. Why? Because without MDM, Silver becomes a staging area for conflicting definitions, and Gold becomes a polished layer of inconsistent business meaning. MDM brings: • trusted business keys • standardized reference data • survivorship rules • hierarchy management • shared definitions for core entities across domains That means Bronze captures reality, Silver reconciles it, and Gold publishes it with confidence. So if you are designing a modern lakehouse or medallion platform, do not position MDM as an optional side capability. Position it as the semantic and operational foundation that turns layered data into trusted business data. Medallion architecture organizes data by quality. MDM ensures the meaning stays consistent across every layer. That is what makes the architecture scalable for BI, governance, and AI. #DataArchitecture #DataModeling #MedallionArchitecture #MDM #MasterDataManagement #DataGovernance #Lakehouse #Analytics #AI #EnterpriseArchitecture EXL Data Management https://lnkd.in/ebndkk77

  • View profile for Peter K.

    Principal AI & Data Architect | GenAI Enterprise Strategist | Databricks, Snowflake & Azure | Architecting Data Foundations for LLM, RAG & Enterprise Agents

    2,746 followers

    Redefining Medallion Layers: Integrating Databricks with Microsoft Fabric Modern analytics frequently rely on both Azure Databricks and Microsoft Fabric. Integrating them expands the traditional Bronze, Silver, and Gold Medallion architecture into a highly scalable logical framework. Crucially, these platforms do not compete—they complement each other to eliminate operational friction and reduce costs. The Roles: The Engine and The Window To manage storage layers across both platforms, you must understand their distinct roles: * Databricks is the "Engine," designed for data engineering, machine learning at scale, and heavy distributed processing. * Fabric is the "Window," bringing data closer to the business via OneLake and native Power BI integration. * Avoid the common mistake of treating them as substitutes or replicating Databricks structures inside Fabric. * Zero-Copy Layering Strategy Instead of physically moving data, a connected architecture uses Fabric OneLake "Shortcuts" (which act like symbolic links) to maintain logical layers without duplication: * Pre-Bronze (Ingestion): Fabric Data Factory orchestrates external data into ADLS for Databricks to pick up. * Bronze (Raw): The raw entry point. Data is converted into Delta Lake for ACID transactions and time travel. No data changes are allowed here. * Silver (Enriched): Bronze data is cleansed, conformed, and deduplicated to provide a standardized "Enterprise view." * Gold (Curated):** Enriched data is structured and optimized for fast analytics, dashboards, and business consumption. * Seamless Access and Governance In this architecture, data physically remains in Databricks while being consumed in Fabric: * Mirroring Azure Databricks Unity Catalog into Fabric allows workloads to read managed data via shortcuts. * Power BI accesses Databricks tables using Direct Lake, drastically improving performance. * Unity Catalog remains the single source of truth, ensuring unified security without duplication. This approach ensures your Medallion architecture can scale endlessly—using Databricks for the heavy transformations and Fabric for unified visualization. #DataStrategy #DataGovernance #MedallionArchitecture #DataArchitecture

  • View profile for Jaswindder Kummar

    Engineering Director | Cloud, Platform Engineering & AI Transformation | Building Secure, Scalable and High-Performing Technology Organizations

    25,799 followers

    𝐃𝐚𝐭𝐚 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 𝐓𝐡𝐚𝐭 𝐀𝐜𝐭𝐮𝐚𝐥𝐥𝐲 𝐌𝐚𝐭𝐭𝐞𝐫𝐬: 𝟔 𝐏𝐚𝐭𝐭𝐞𝐫𝐧𝐬 𝐒𝐡𝐚𝐩𝐢𝐧𝐠 𝐌𝐨𝐝𝐞𝐫𝐧 𝐃𝐚𝐭𝐚 𝐒𝐲𝐬𝐭𝐞𝐦𝐬 Data warehouse vs data lake was yesterday's debate. Today's decision is between six architectures each solving a different problem. Here is what matters and when to use each. 𝟏. 𝐋𝐀𝐊𝐄𝐇𝐎𝐔𝐒𝐄 • Combines data lakes and warehouses with open formats and separate compute.  • Serves as the standard backbone for modern analytics. • Sources to Ingestion to Storage + Table Format to BI + AI Pick Lakehouse when you need both analytics and AI on the same data without maintaining two separate systems. This is the default starting point for most modern data teams. 𝟐. 𝐒𝐓𝐑𝐄𝐀𝐌𝐈𝐍𝐆-𝐅𝐈𝐑𝐒𝐓 • Built around continuous data streams instead of batch processing.  • Powers real-time use cases like personalization, fraud detection, and AI. • Event Sources to Stream Bus to Processing to Apps Pick Streaming-First when batch processing is too slow. If your business decisions depend on data that is minutes or seconds old not hours this is your architecture. 𝟑. 𝐃𝐀𝐓𝐀 𝐌𝐄𝐒𝐇 • Promotes decentralized ownership where teams manage their own data products.  • Blends organizational structure with data architecture. • Domain Sources to Data Products to Platform to Consumers Pick Data Mesh when your organization is large enough that centralized data teams have become a bottleneck. Each domain owns its data as a product. This is an organizational shift as much as a technical one. 𝟒. 𝐀𝐈-𝐍𝐀𝐓𝐈𝐕𝐄 • Designed specifically for machine learning and generative AI systems.  • Supports low-latency inference with consistent online and offline data. • Sources to Feature Store to Models to Applications Pick AI-Native when ML and GenAI are your primary workloads. The feature store ensures training and serving use the same data eliminating the train-serve skew that breaks models in production. 𝟓. 𝐂𝐎𝐌𝐏𝐎𝐒𝐀𝐁𝐋𝐄 𝐒𝐓𝐀𝐂𝐊 • Separates storage, compute, and serving for flexibility.  • Enables using multiple tools without heavy vendor dependence. • Sources to Storage to Compute to APIs Pick Composable Stack when you want to avoid vendor lock-in and need the flexibility to swap tools at any layer. Best for teams with strong engineering talent who want maximum control. 𝟔. 𝐑𝐄𝐕𝐄𝐑𝐒𝐄 𝐄𝐓𝐋 • Moves processed data back into operational platforms.  • Bridges the gap between insights and real-world actions. • Warehouse to Transform to Sync to Apps Pick Reverse ETL when your analytics insights are stuck in dashboards nobody checks. This pattern pushes data from your warehouse back into CRM, marketing, and operational tools where teams actually work. Which architecture pattern is your team building on? ♻️ Repost this to help your network get started ➕ Follow Jaswindder for more #DataArchitecture #DataEngineering #DevOps

  • View profile for Sandeep Kachru

    Building Agami · The Context Layer for Trusted Enterprise Data · ex Google, Meta, Airtable

    2,147 followers

    "Semantic model" search interest hit an all-time high recently. Five years ago this was a niche BI term. Today it is the central architecture decision for every team building an AI data agent. A quick history, because the idea is older than most people realize. 1/ 1990s: GUI hell - BusinessObjects, Cognos Framework Manager, Microsoft Analysis Services. Locked to one vendor's reporting suite. 2/ 2012: First glimmer of hope - Looker reinvents the category with LookML. Semantics as version-controlled code, in git. 3/ 2023: Nirvana, we thought - dbt ships MetricFlow. Metric definitions finally live in the same repo as the transforms producing them. Every generation solved the same problem. When "revenue" can mean six different things across six dashboards, somebody has to write down what it means once. Why it's suddenly everywhere again: turns out LLMs are like humans here. Very good at writing SQL. Bad at knowing which of those six revenues your CFO uses. The semantic model is the bridge. Every serious AI data agent in the market is now built on top of one. There is no shortcut. What the major vendors are shipping: → Databricks: Unity Catalog Metric Views → Snowflake: Semantic Views → Looker: LookML → Tableau: Published Data Sources, plus Waii post-acquisition → Power BI: Tabular Model and DAX → dbt Labs: Semantic Layer with MetricFlow → Cube: Cube Schema Unfortunately, none of these formats talk to each other. Switching vendors is no longer just a data migration. It is a migration of every metric definition, every join, every business rule your team built over years. The Open Semantic Interchange (OSI) is the answer the industry just agreed on. v1 spec finalized in January under Apache 2.0. The architecture is hub-and-spoke: author your semantic model once in OSI, export to whatever vendor format you need. So, if you are picking a stack right now and don't want to re-litigate this decision in 18 months: author in OSI and save in your repo. Transform into the vendor-specific format and load to the tools that need one. At Agami, we consume OSI natively. Point us at the file. There is no Agami semantic model schema and no translation tax. Full post linked in comments.

Explore categories