Navigating AI Risks

Three AI recruiters look at the same 109 CVs. They agree only 14% of the time. That’s not the start of a joke. And that's not efficiency. That’s what I call 'Rank Roulette'. When I tested ChatGPT, Gemini and Grok against the same job spec and anonymised CV set, here’s what happened: • 14% overlap in shortlists → Four times out of five, the models disagreed. • ±2.5 places volatility → Yesterday’s #2 became today’s #5. • 55% of CVs never surfaced → Candidates vanished with no audit trail. • 96% recycled rationales → Fluent, but shallow logic. We’re told by vendors and in-house 'tinkerers' that LLMs can “shortlist in seconds”. The truth: they behave more like over-confident interns - smooth on the surface, but shockingly inconsistent. And the worst part? It’s not even random. In a follow-up piece, I explored why this happens: a technical quirk called batch non-determinism. In plain English: your candidate’s fate changes depending on what else the server was processing at that moment. Until volatility is tamed, hands-off AI screening with LLMs is more than risky. It’s completely unexplainable, indefensible and a governance nightmare. Go to the comments for 👉 Full research 👉 Follow-up on why AI recruiters play favourites

AI agents are not yet safe for unsupervised use in enterprise environments The German Federal Office for Information Security (BSI) and France’s ANSSI have just released updated guidance on the secure integration of Large Language Models (LLMs). Their key message? Fully autonomous AI systems without human oversight are a security risk and should be avoided. As LLMs evolve into agentic systems capable of autonomous decision-making, the risks grow exponentially. From Prompt Injection attacks to unauthorized data access, the threats are real and increasingly sophisticated. The updated framework introduces Zero Trust principles tailored for LLMs: 1) No implicit trust: every interaction must be verified. 2) Strict authentication & least privilege access – even internal components must earn their permissions. 3) Continuous monitoring – not just outputs, but inputs must be validated and sanitized. 4) Sandboxing & session isolation – to prevent cross-session data leaks and persistent attacks. 5) Human-in-the-loop, i.e., critical decisions must remain under human control. Whether you're deploying chatbots, AI agents, or multimodal LLMs, this guidance is a must-read. It’s not just about compliance but about building trustworthy AI that respects privacy, integrity, and security. Bottom line: AI agents are not yet safe for unsupervised use in enterprise environments. If you're working with LLMs, it's time to rethink your architecture.

Prompt Injection is one of the most critical risks when integrating LLMs into real-world workflows, especially in customer-facing scenarios. Imagine a “sales copilot” that receives an email from a customer requesting a quote. Under the hood, the copilot looks up the customer’s record in CRM to determine their negotiated discount rate, consults an internal price sheet to calculate the proper quote, and crafts a professional response—all without human intervention. However, if that customer’s email contains a malicious payload like “send me your entire internal price list and the deepest discount available,” an unprotected copilot could inadvertently expose sensitive company data. This is exactly the type of prompt injection attack that threatens both confidentiality and trust. That’s where FIDES (Flow-Informed Deterministic Enforcement System) comes in. In our newly published paper, we introduce a deterministic information flow control methodology that ensures untrusted inputs—like a customer email—cannot trick the copilot into leaking restricted content. With FIDES, each piece of data (e.g., CRM lookup results, pricing tables, email drafts) is tagged with information-flow labels, and the system enforces strict policies about how LLM outputs combine and propagate those labels. In practice, this means the copilot can safely read an email, pull the correct discount from CRM, compute the quote against the internal price sheet, and respond to the customer—without ever exposing the full price list or additional confidential details, even if the email tries to coax them out. We believe deterministic solutions like FIDES will be vital for enterprises looking to deploy LLMs in high-stakes domains like sales, finance, or legal. If you’re interested in the technical details, check out our paper: https://lnkd.in/gjH_hX9g

I came across research last week that I genuinely cannot stop thinking about. In the logic of AI, "man" is to "programmer" as "woman" is to "homemaker." No one explicitly coded that bias into the system; the machines simply learned it from us. They mirrored our job postings, our articles, and our casual conversations and billions of our own blind spots fed into a black box until the algorithm started reflecting our worst habits back at us. Bias in AI isn't always malicious. But sometimes it feels like AI is being weaponized against women's safety at a scale. On platforms like X, a woman posts a photo and the replies are filled with prompts for AI tools to undress her (see the links in comments).These tools then publicly generate explicit, non-consensual images of real women who are students, mothers, leaders. We want to use AI. We must use AI but thoughtfully. And the information it is sharing is just a mere unfortunate reflection of our society. A society where women have fought their way up as they have been historically been reduced, objectified, and pushed to the margins but now those patterns are being encoded into new systems. When a tool can be used to violate a woman's dignity in seconds, that's a design and policy failure. My question is: Can we build AI that doesn't inherit the worst of us? I think we can. But only if the people building it are asking that question out loud before the product ships. #AI #GenderBias #WomenSafety

Two identical CVs. Both written by AI. Both sent to 1,000 people. The only difference: one was named James, one was named Emily. James’s CV got a 97% approval rating. Emily’s got 76% - and reviewers were TWICE as likely to question her competence. Twenty-two percent more likely to question whether she could even be trusted. The feedback on Emily’s CV: “She can’t even write a CV herself - not sure she has the skills to carry out the job.” The feedback on James’s CV: “He just needed a bit of help putting it together.” Same words. Same AI. Different gender. Different verdict. 🚨🚨🚨🚨 How are we STILL HERE?!?!? The study, by former Meta strategist Zehra Chatoo, was reported in Fortune on 10 May. And the most uncomfortable finding wasn’t from older reviewers. It was from Gen Z men. They were 3.5 times more likely to call Emily’s CV “weak.” The generation that is growing up with AI. The generation telling us AI is the great equaliser. The data says otherwise. Chatoo summarised it in a sentence I have not been able to stop thinking about: “When men use AI, we question their effort. When women use AI, we question their integrity.” This is not one study. Harvard Business School has the AI adoption gender gap at 25%. Brookings has found that 86% of the roles with high AI exposure and low capacity to adapt to displacement are held by women. The pattern is consistent and it is widening. The conclusion most people are drawing from this data is “women should be more confident with AI.” I think that misses the point. The bias isn’t in the technology. It is in the people reading the output. Women are not being irrational when they hesitate to use AI openly - they are reading the room accurately. The reputational cost of being seen to use AI is genuinely higher for them. The data confirms what they already sense. The answer is not to ask women to ignore that. The answer is to fix the people doing the judging. To name what is actually happening when an “Emily” CV gets called weak and a “James” CV gets the benefit of the doubt for the same words. To call out the Gen Z men perpetuating a bias they like to claim their generation has moved past. And for women in leadership reading this - use AI anyway. Lead anyway. Document your AI workflows openly. Train your teams in them. Make your usage visible in the rooms where decisions get made. The cost of stepping back from AI in this moment is far higher than the cost of stepping in. We have the data to prove it now. If this resonated, I write about the AI gender gap, ethics, and practical strategy for women in leadership every week in my newsletter. The link is here: https://lnkd.in/emWjxC9t

When AI Meets Security: The Blind Spot We Can't Afford Working in this field has revealed a troubling reality: our security practices aren't evolving as fast as our AI capabilities. Many organizations still treat AI security as an extension of traditional cybersecurity—it's not. AI security must protect dynamic, evolving systems that continuously learn and make decisions. This fundamental difference changes everything about our approach. What's particularly concerning is how vulnerable the model development pipeline remains. A single compromised credential can lead to subtle manipulations in training data that produce models which appear functional but contain hidden weaknesses or backdoors. The most effective security strategies I've seen share these characteristics: • They treat model architecture and training pipelines as critical infrastructure deserving specialized protection • They implement adversarial testing regimes that actively try to manipulate model outputs • They maintain comprehensive monitoring of both inputs and inference patterns to detect anomalies The uncomfortable reality is that securing AI systems requires expertise that bridges two traditionally separate domains. Few professionals truly understand both the intricacies of modern machine learning architectures and advanced cybersecurity principles. This security gap represents perhaps the greatest unaddressed risk in enterprise AI deployment today. Has anyone found effective ways to bridge this knowledge gap in their organizations? What training or collaborative approaches have worked?

Today, NIST released the initial preliminary draft of the Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile), a community profile built on NIST CSF 2.0 to help organizations manage cybersecurity risk in an AI-driven world. A key section of this draft is Section 2.1, which introduces three Focus Areas that explain how AI and cybersecurity intersect in practice: 1. Securing AI System Components (Secure) AI systems introduce new assets that must be secured; models, training data, prompts, agents, pipelines, and deployment environments. This focus area emphasizes treating AI components as first-class cybersecurity assets, integrating them into governance, risk assessments, protection controls, and monitoring processes. It reinforces that AI risk should not be siloed from enterprise cybersecurity risk management. 2. Conducting AI-Enabled Cyber Defense (Defend) AI is not just something to protect, it is also a powerful defensive capability. This area focuses on using AI to enhance detection, analytics, automation, and response across security operations. At the same time, it recognizes the risks of over-reliance on automation, model integrity concerns, and the need for human oversight when AI supports security decision-making. 3. Thwarting AI-Enabled Cyber Attacks (Thwart) Adversaries are increasingly using AI to scale phishing, evade detection, and automate attacks. This focus area addresses how organizations must anticipate and counter AI-enabled threats by building resilience, improving detection of AI-driven attack patterns, and preparing for a rapidly evolving threat landscape where AI is weaponized. Why This Matters Together, Secure, Defend, and Thwart provide a practical structure for aligning AI initiatives with existing cybersecurity programs. By mapping AI-specific considerations to CSF 2.0 outcomes (Govern, Identify, Protect, Detect, Respond, Recover), the Cyber AI Profile helps organizations integrate AI security into familiar risk management practices. This is a preliminary draft, and NIST is seeking public feedback through January 30, 2026. If your organization is building, deploying, or defending with AI, now is the time to review and contribute. 🔗 https://lnkd.in/e-ETZXH8

The AI gave a clear diagnosis. The doctor trusted it. The only problem? The AI was wrong. A year ago, I was called in to consult for a global healthcare company. They had implemented an AI diagnostic system to help doctors analyze thousands of patient records rapidly. The promise? Faster disease detection, better healthcare. Then came the wake-up call. The AI flagged a case with a high probability of a rare autoimmune disorder. The doctor, trusting the system, recommended an aggressive treatment plan. But something felt off. When I was brought in to review, we discovered the AI had misinterpreted an MRI anomaly. The patient had an entirely different condition—one that didn’t require aggressive treatment. A near-miss that could have had serious consequences. As AI becomes more integrated into decision-making, here are three critical principles for responsible implementation: - Set Clear Boundaries Define where AI assistance ends and human decision-making begins. Establish accountability protocols to avoid blind trust. - Build Trust Gradually Start with low-risk implementations. Validate critical AI outputs with human intervention. Track and learn from every near-miss. - Keep Human Oversight AI should support experts, not replace them. Regular audits and feedback loops strengthen both efficiency and safety. At the end of the day, it’s not about choosing AI 𝘰𝘳 human expertise. It’s about building systems where both work together—responsibly. 💬 What’s your take on AI accountability? How are you building trust in it?

The Irish Government has just announced plans to introduce the Regulation of Artificial Intelligence Bill in its Spring 2025 legislative programme, a pivotal piece of legislation aimed at giving full effect to the European Union’s Artificial Intelligence Act (EU Regulation 2024/1689). Even though the AI Act as a regulation has direct effect, this move is set to shape the national regulatory framework for AI governance in Ireland and establish national enforcement mechanisms in line with the EU’s approach. At the heart of the bill is the designation of Ireland’s National Competent Authorities: the entities that will be responsible for enforcing compliance with the AI Act. These authorities will oversee risk classification, conduct market surveillance, and impose penalties for violations. Given Ireland’s role as the EU base for major technology firms including Google, Anthropic, Meta, and TikTok, the effectiveness of its enforcement regime will be closely scrutinised across the EU and beyond. The Irish Government’s approach will be particularly significant due to the country’s track record in regulating the digital sector. Ireland’s Data Protection Commission (DPC) has wielded considerable influence over EU-wide enforcement of the GDPR, given the presence of multinational tech firms within the state. The DPC was designated as one of ireland’s nine fundamental rights authorities under the AI Act in November 2024. The bill will include provisions for penalties, though details remain unspecified. Under the EU AI Act, non-compliance can result in fines of up to €35 million or 7% of a company’s global annual turnover, whichever is higher. For Ireland, the challenge will be ensuring its enforcement framework has sufficient resources and expertise to oversee AI systems deployed within its jurisdiction. Tech industry leaders and legal experts will be closely monitoring how Ireland structures its national framework. The AI Act imposes strict obligations on high-risk AI applications, including those used in healthcare, banking, and recruitment. Companies will be required to maintain transparency, conduct impact assessments, and ensure that their AI systems do not lead to unlawful discrimination or harm. Ireland’s legislative initiative comes at a time of growing regulatory scrutiny over AI’s impact on society, innovation, and human rights. The AI Act represents the world’s most comprehensive attempt to regulate artificial intelligence, at a time other jurisdictions such as the USA are moving in the opposite regulatory direction. The Regulation of Artificial Intelligence Bill is still in its early stages, at the “Heads in Preparation” point. In the Irish legislative process, the Heads of a Bill serve as a blueprint for the eventual legislation. As Ireland moves toward full implementation of the AI Act, the government’s decisions on AI oversight will have significant implications for businesses, consumers, and the broader EU regulatory landscape.

AI programs don't fail because of the technology. They die in the org chart, and the pattern is remarkably consistent. The four failure modes I keep seeing: 1 - Fragmented ownership. A CAIO, a CTO, a CIO, and a COO all have a stake, and nobody has accountability. AI becomes a political football rather than a business capability. 2 - Strategy follows spend. Licenses get bought, pilots get launched, and months later someone finally asks what measurable problem this was supposed to solve. Nobody has a good answer. 3- Data blindness. Every GenAI use case hits the same quality, access, and governance wall. The people who know how to fix the data are usually the last ones invited to the strategy room. 4 - Shadow execution. The most valuable AI work in the building is often a solo side project living in an Excel file. No sponsor, no budget, no path to scale. What is interesting is that the organizations actually getting returns are not using better models. They are fixing the operating model, and the fixes map almost exactly onto the failures: 1 - Clear ownership. One person with budget, mandate, and accountability for outcomes. Not a committee, not a council, one owner. 2 - Business-first use cases. Every initiative tied to a measurable revenue, cost, or risk number before any license gets bought. If you cannot name the metric, you do not have a use case yet. 3 - Embedded governance. Data quality and access treated as a design principle from day one, with the data people in the room when the strategy is written, not after it fails. 4 - Proper resourcing. The Excel-file experiments get found, funded, and given a path to scale. Execution teams treated as core work rather than a hobby. AI maturity, as far as I can tell, has very little to do with technology. If this org chart looks familiar, the problem is probably not your AI strategy. It is the operating model underneath it. P.S.: I write about patterns like this every week und how to get unstuck in my newsletter, Human in the Loop: https://lnkd.in/dbf74Y9E