Rishi Bhatnagar

Are you struggling to get a data governance program off the ground? Are you having difficulty getting your business stakeholders aligned around specific data definitions and quality standards? Or, perhaps you struggle to get your customers fully engaged in data stewardship activities? If yes, I have some advice. This advice comes from the school of hard knocks - having been in several situations where I wrestled with all the issues highlighted above. Here is my advice: for early stage governance efforts, wherever possible, look to use 𝐭𝐡𝐢𝐫𝐝 𝐩𝐚𝐫𝐭𝐲 𝐫𝐞𝐟𝐞𝐫𝐞𝐧𝐜𝐞 𝐝𝐚𝐭𝐚 as a source of 'truth'. Why? ✅ One of the greatest values of third party data, especially if you buy from a reputable company, is that it comes 'pre-governed'. Your vendor does the hard work of defining and enforcing governance standards for that data. ✅ Third party data is generally easy to 'defend', especially when it comes from a reputable vendor. So, when your internal customers look at a report and ask 'where did this come from?', it's easy to explain what they are seeing when it comes from a reputable vendor in the space. ✅ Your 'time to market' of any data management solution that relies on a source of truth, like MDM, will be significantly reduced. Instead of spending days in governance committee meetings debating definitions and quality standards (assuming you can even get people to attend), you can be focused on solving specific business problems. ✅ Your need to steward any data provided by a third party should be negligible - assuming the governance standards of the vendor well-align to your existing (or expected) governance policies. These are some of the many benefits of using third party data, but with the benefits, come costs. Often, these costs can be significant. Worse yet, its impossible for a given data buyer to know if what they are paying for third party data aligns to industry standards, or if similar quality products exist on the market at a similar price. This lack of transparency on the price and quality of data providers is exactly the problem that my friends at Blue Street Data exist to solve. So, if you see the value in third party data as an effective tool to accelerate your data governance efforts - and need help in the process of engaging a potential vendor - then I recommend you check out the buyers guide linked below. It provides you detailed guidance on every step of the process to acquire third party data, and will help ensure that you are getting the right product, from the right vendor, and the right price. Check it out! #thirdpartydata #referencedata #datagovernance https://lnkd.in/eHCAsU9s

28

1 Comment

When should you strive for gathering data directly, vs using third-party data sources? Or even giving up a dataset entirely? This is the data angle behind Stratechery's latest analysis. (Go check them out, they do great in-depth analysis of tech strategy) https://lnkd.in/guBTYdTK Google gave up a strategic position when it stopped paying to be the default maps app. Apple built a maps app that was terrible at first but has grow in capability and adoption. For an advertising company, data from real-time consumer behavior through a maps app is valuable! Why would they do that? Google did, however, continue to pay $20 billion a year to be the default search app. Someone did the math and determined the overall payment to competitive company wasn't worth it the valuable data they would be getting. In Datateer's world, we see also that nothing comes without a price. Sometimes vendors even charge to open up access to an API--for a product our customers are already paying for! Additional charges like this give companies pause, because every cost adds up and affects the total ROI

5

1 Comment

The organizations that will execute on AI will be the organizations that have the capability to scale quickly as any AI innovation happens. Open AI just announced GPT 5.2, and not only is Databricks the only platform that already supports it, but we used our benchmark to measure it against enterprise tasks.

23

1 Comment

wrapping up a 6mo project with a large client in the enterprise manufacturing space this week. we migrated materialized Snowflake views to a new dbt Labs project: - 322 staging files - 71 intermediate tables - 35 mart tables - 7 incremental tables I used the opportunity to lean into new AI code dev tools: - Anthropic's claude code within Cursor - nao Labs (YC X25) the client was ~18k person org. im proud of the rate at which we got things done despite the friction we faced in (2) areas. (1) our end product was affected by compromises on quality to appease the client who wanted things a certain way / didn't prioritize certain dev standards, in lieu of faster code deployments (lots of small things e.g. no lint checks in CI jobs, bypassed code approvals). a consensus on compromised quality is toxic to a project. It’s a silent agreement that discourages individual contributors from pushing a task closer to perfection, once they know their colleagues won’t be. It kills motivation. The collective end result is haphazard work and tech debt. (2) client-side analyst/engineers became defensive over recommendations and hard to work with. in these situations I try to stay firm on my position while being mindful / proactive addressing personal feelings at stake. work is emotional and people are sensitive. I think its better to have an uncomfy chat about feelings hurt than allow hidden resentments to fester eg 'im sorry you feel belittled by this choice....im pushing for X to achieve Y but can see how its causing Z and thats not cool'. vibes impact quality always learning. headed to asia for the next 2mo. stay tuned massive thanks to my consulting colleagues. (i wish you could add spotify songs to linkedin posts 😅)

53

6 Comments

#TalkToYourData solutions don't solve the problem of humans actually understanding the results, being able to put them in context, or know how to behave in response to the answers. However, full agentic AI solutions continue to get us closer. One good way to accelerate learning (for humans) is to maintain some best practices and something like a #PromptHub where you keep track of, share, and continuously improve the prompts you use for specific goals. I used to do this with SQL in an enterprise wiki. Here's an example: https://lnkd.in/gs4tXJpc

7

Ascend just outscored the big 3 hyperscalers and the entire Modern Data Stack in Dresner's Data Engineering report. And honestly? It feels pretty good. Not because of the ranking itself—but because of what it represents. Here's what this really means: 🔹 Data teams are tired of tools that create more problems than they solve 🔹 The demand for actual automation (not just hype) is real 🔹 Engineers want to build cool stuff, not babysit broken pipelines The gap is closing. Two years ago, everyone started talking about data engineering automation. Few delivered it. Today, teams that embrace real automation are moving 10x faster than those stuck with manual processes. This recognition validates something we've believed from day one: → Data engineering doesn't have to be painful → Your best engineers shouldn't spend time on toil → Automation should actually work (revolutionary, I know) The Ascend team has been obsessed with making data engineering delightful. Seeing that work recognized alongside industry leaders? That's the team delivering on a mission. But we're just getting started. The future belongs to data teams that automate the grunt work and focus on what actually moves the needle. What's the biggest time-waster your data team deals with that should just... not exist? #DataEngineering #Automation #TeamAscend

88

6 Comments

Adrian always tells it like it is...most AI-enabled supply chain startups/existing companies skip over the data quality issues when selling new solutions, then have to spend significant implementation time cleaning up data quality issues--you think we all would have learned by now... https://lnkd.in/gMJbmmQu

31

5 Comments

As someone operating in both Snowflake and Databricks environments, I’m energized by Snowflake’s vision shared at the summit last week — and just as excited to see what Databricks unveils at their Summit this week. For those of us bridging structured + unstructured, BI + ML, this cross-platform momentum is real and valuable. 🚀 Reflections from #SnowflakeSummit 2025 The big picture laid out by Snowflake’s leadership was bold and forward-looking: a unified platform where governed data, agentic AI, and transactional + analytical apps come together seamlessly. Sridhar Ramaswamy’s keynote along with Sam Altman framed it best: “There’s no AI strategy without a data strategy.” A truism. Key takeaways that stood out to me: 🤖Agentic AI in action Snowflake Intelligence and Cortex Agents bring a new paradigm—AI agents that retrieve, reason, and act—all within Snowflake’s secure data perimeter. 🧠AI for everyone Tools like AISQL and the Data Science Agent lower the barrier to insight. Prompt your data like you prompt ChatGPT—no Python required. ⚙️Smarter infrastructure Adaptive Compute and Gen2 Warehouses optimize cost and performance automatically—bringing efficiency without complexity. 🧾Unified workloads With PostgreSQL stack coming soon, Snowflake is ready to handle OLTP and analytics on the same platform. 🔐Governance at the core Semantic modeling, masking, and secure data sharing are not afterthoughts—they’re foundational to the future of responsible AI. #SnowflakeSummit #Databricks #AIstrategy #DataCloud #AgenticAI #AISQL #ProductLeadership #DataPlatform #Governance #CortexAgents

109

4 Comments

You think you’ve migrated. But 80% of your business logic is still stuck in that Teradata box no one dares to touch. Now you have two sources of truth. Snowflake is live. Pipelines in products look slick. Cloud migration in progress. But here’s the uncomfortable truth Half your data is still stuck in Oracle, SAP, or that Teradata box no one dares to touch. You didn’t modernize. You just built a duct-taped mess hybrid trap, legacy + cloud Hybrid environments look harmless at first. But duplicating logic, moving everything with no clear priortization, is a dead game. Running duplicated systems is not goal. It's a phase. I’ve seen it across dozens of companies: → Two data teams are maintaining duplicate pipelines. → Two budgets paying to process the same data. → And one VP still asking, “Which number do we trust?” Every month you delay decommissioning legacy systems, you’re paying double. That’s your “hybrid tax.” And most execs don’t even know they’re paying it. This includes 1. Double Spend. Duplicated Licenses, compute, ETL jobs. You’re not saving by “phasing it out slowly.” 2. Compounded Complexity. Your engineers are maintaining SQL and shell scripts, dbt and old Informatica Engineers are babysitting legacy. That’s why your best people quit. 3. Broken Trust. When half your dashboards pull from the cloud and half from legacy, Nobody believes the data. Self-service, compliance reporting, AI become impossible. 4. Lost Speed & Innovation. You can’t run AI or scale insights when half your business logic still lives on an old system. This isn’t about technology. It’s about focus and ruthless simplification. 1. Quantify the Hybrid Tax. Audit total cost of ownership, storage, ETL, maintenance, labor. Show the CFO the overlap. 2. Decouple Before You Decommission. Inventory every report, pipeline, and dashboard touching legacy systems. Then kill them one by one, not “all at once.” Move only the highest impact that relevant now not 12-18 months. The goal isn’t “all in the cloud.” The goal is no dual dependencies. 3. Redesign for Reuse. Hybrid environments expose one ugly truth: Your data models are too brittle to scale, usless and 89% deleted. Lock in your business definitions (semantic layer). Build modular, reusable data products. Document everything. 4. Redefine “Done.” “Cloud migration complete” isn’t done. Replicating everything into the cloud is just duplication at scale. The goal isn’t to move all your tables. The goal is to translate what matters, retire what doesn’t, and prove ROI at each step. 5. Institutionalize the Reset. Every month, review how much of your data still touches legacy. Everything should be priortized based on impact x complexity from the data strategy What % of workloads still depend on legacy? What costs can we retire next? and so on Have seen companies depend on 3 cloud systems + legacy, everything just amplified the chaos.

49

5 Comments

Good news for ALL customers trying to create semantic models & the context layer for their AI tools (like Snowflake Intelligence/Databricks Genie/ and others) to work. We are Snowflake native on their marketplace & have started customer deployments. 🚀 AI needs more than metadata. It needs: - Semantic relationships between entities, - Business definitions in natural language,  - Measure definitions and calculations logic, & - Domain specific context and terminology - the enterprise brain - Agentic systems to understand data structures Creating all this takes 3-6 months/ sometimes longer - stalling AI adoption. At ekai, we produce all the artifacts that downstream AI applications need: data catalogs, business glossaries, metrics definitions, lineage maps, validation rules. The entire context layer, generated and maintained automatically. And we can serve it. Shahid Azim Patricia Geli Beth Porter David Berlin Kate Landmann Lucky Byas Yaser Najafi Ryan Lieber Niels ter Keurs Anahita Tafvizi Eddie Blackwell, MBA, MSLC Wayne Wilson

50

Just spent two energizing weeks at the Snowflake Summit and Databricks Data + AI Summit, and the message across both summits was clear: there is no AI strategy without a data strategy. At Anomalo, we’ve long believed that AI's potential starts with trust in your data. This year, it’s been inspiring to see that belief echoed so strongly across the industry. Some key takeaways that stood out to me: Agentic AI is coming to the Enterprise Both Snowflake and Databricks are going all-in on AI agents, signaling a major shift in how enterprise apps will be built and interacted with. Tools like Cortex Agents and Agent Bricks highlight how fast this space is maturing. I was especially impressed by the many advancements in Agent Bricks and how seamlessly it integrates with the rest of the Databricks platform. Data is becoming more accessible to the business From Snowflake’s Semantic Views to Databricks’ Unity Metrics, there’s a clear push to give business teams governed, accessible, and trustworthy data so they can build, explore, and activate AI without writing code or navigating raw tables. This goes even further with tools like Snowflake Intelligence and Databricks Genie that harness the power of AI to allow business users to analyze and query data simply through natural language. Data Quality Is no longer optional AI is accelerating, and clean data is more essential than ever. As Databricks’ CEO Ali Ghodsi pointed out, AI still needs human oversight, and none of it works without trustworthy data. At Anomalo, we’re proud to help enterprises trust their data every step of the way. We integrate tightly with platforms like Snowflake and Databricks so that your quality checks work out of the box no matter how your stack evolves. We’re doubling down on our mission to help enterprises: * Monitor the health of structured and unstructured data automatically * Detect issues before they impact dashboards,  models, or workflows * Empower business teams with a no-code UI and the context to resolve problems quickly AI will change everything—but only if we trust the data it’s built on. Let’s make that future possible

89

1 Comment

🚨Snowflake’s new Query_Insights just raised the bar for query diagnostics. For data teams juggling performance, cost, and complexity, this release adds much-needed visibility where it counts most, inside the query execution path. Here’s what’s now flagged automatically: - Exploding joins (nested & non-nested) - Joins without conditions - Unselective or inapplicable filters - Remote spillage - Search optimization usage …and more. Each signal is tied to a QUERY_ID and surfaced directly in the UI. No more post-mortems at the EXPLAIN plan level. Read more about it 🐷 #Snowflake #QueryInsights #ExplodingJoin #DataEngineering #CloudWarehouse #RootCauseAnalysis #FinOps #PerformanceTuning

13