Jordi Bartual Paris

Metadata architecture is key to successful data catalog implementations. Don’t think of architecture as just documentation. Good metadata architecture defines all the components needed to achieve a goal and how they connect, both internally and externally. For a data catalog, that means understanding: • What metadata exists and where • How it flows across systems • How teams and other software interact with it (the most important) Architecture also communicates structure and expected behavior to all stakeholders, ensuring that everyone (business, IT, compliance, analytics, etc.) shares the same understanding. Without good metadata architecture, a catalog is just a tool. With it, it has the potential to become a connected, operating ecosystem. Note: This post was influenced by my recent visit to the Frank Lloyd Wright house with Louis Tran.

33

1 Kommentar

Production-like testing ≠ testing in production 🧨 One is a DataOps best practice The other is basically an extreme sport. This is the next article in my 10 Forgotten Data Engineering Tasks series and it’s all about why data testing needs more than just lower environments. https://lnkd.in/eJsBXNbe

52

9 Kommentare

Processing flight data at scale ✈️📊 See how one declarative pipeline built with Lakeflow + PySpark can: -Stream live ADS-B data -Pre-aggregate metrics with materialized views -Run reliably at production scale — up to hundreds of millions of events/day

14

Due to the increased importance of context, we need to reprioritize certain Data Governance efforts and objectives towards ensuring that data leads to actual context-rich decisions and actions. Having clear responsibilities on (structured and unstructured) data sources, the metadata layer, the semantic layer, the knowledge layer and the market place layer is key, and we need to ensure that there standards and rules around how to connect data objects to the business context. We should build on the proven principles of domain ownership and transparency, and enrich it now with the additionally needed skills and tasks to ensure we bring data products into the context-rich world. Let me know if you need any help. #datagovernance #context #tiankaistuff

282

36 Kommentare

Another Big challenge in the CDO table: how manage & Governance the unstructure Data for AIgen at scale (for get about the marvelous POC with a local RAG). Interesting post about how the medallion Arq. And Governance data principles help us to avoid the problem. https://lnkd.in/dpmxT8CC

24

1 Kommentar

43% of architects want to see how their peers are already applying AI in their EA workflows. In our latest Illuminate webinar, we opened up Ardoq Labs to show real AI experiments in action, from diagram-to-data transformation to conversational architecture. 📽️ Watch the on-demand session today: https://lnkd.in/eUNmjzu6

7

❓ What makes the ideal Agentic AI ready Knowledge Graph ❓ 💥 One that exposes a well-modeled, cohesive schema or ontology in a machine-readable, easy-to-consume format. This article explains why, and outlines the other capabilities of your knowledge graph you need to expose. #AgenticAI #MCP #AI #KnowledgeGraph #GraphDatabase https://lnkd.in/e8Q2_Bvb

16

1 Kommentar

The MCP Architecture Stack That Enables Autonomous SoR Agents Building enterprise AI agents is hard. Semantic accuracy over messy data. Business context across tribal knowledge. Grounding in actual system state rather than hallucinated responses. Solve all that, and there's another problem: they still can't act. Detection: ✓ (hard) Diagnosis: ✓ (hard) Recommendations: ✓ (hard) Execution: ✗ (harder) The last mile makes everything more complicated. Each agent needs custom integration code for every system it touches. CRMs, ERPs, billing, inventory - all with different APIs, authentication, error handling. N × M complexity that explodes as agent count grows. Model Context Protocol addresses this with standardized integration architecture. Instead of point-to-point chaos, MCP creates a protocol layer. Agents expose capabilities. Systems expose actions. Complexity shifts from N × M to N + M. Collections Agents escalate high-risk accounts automatically across systems. Inventory Agents trigger replenishment orders without human relay. Sales Ops Agents coordinate pricing adjustments in real-time. The progression: Read operations → Write with approval → Autonomous execution The value: 100x speed, consistent process every time, platform effects that compound The safety: Rate limits, approval thresholds, semantic constraints, rollback capabilities This is the architectural shift from AI advisors to AI operators. From agentic systems to autonomous enterprises. Here is the new post on my medium: https://lnkd.in/eYg4G4H2

20