AI in Cybersecurity

AI is not failing because of bad ideas; it’s "failing" at enterprise scale because of two big gaps: 👉 Workforce Preparation 👉 Data Security for AI While I speak globally on both topics in depth, today I want to educate us on what it takes to secure data for AI—because 70–82% of AI projects pause or get cancelled at POC/MVP stage (source: #Gartner, #MIT). Why? One of the biggest reasons is a lack of readiness at the data layer. So let’s make it simple - there are 7 phases to securing data for AI—and each phase has direct business risk if ignored. 🔹 Phase 1: Data Sourcing Security - Validating the origin, ownership, and licensing rights of all ingested data. Why It Matters: You can’t build scalable AI with data you don’t own or can’t trace. 🔹 Phase 2: Data Infrastructure Security - Ensuring data warehouses, lakes, and pipelines that support your AI models are hardened and access-controlled. Why It Matters: Unsecured data environments are easy targets for bad actors making you exposed to data breaches, IP theft, and model poisoning. 🔹 Phase 3: Data In-Transit Security - Protecting data as it moves across internal or external systems, especially between cloud, APIs, and vendors. Why It Matters: Intercepted training data = compromised models. Think of it as shipping cash across town in an armored truck—or on a bicycle—your choice. 🔹 Phase 4: API Security for Foundational Models - Safeguarding the APIs you use to connect with LLMs and third-party GenAI platforms (OpenAI, Anthropic, etc.). Why It Matters: Unmonitored API calls can leak sensitive data into public models or expose internal IP. This isn’t just tech debt. It’s reputational and regulatory risk. 🔹 Phase 5: Foundational Model Protection - Defending your proprietary models and fine-tunes from external inference, theft, or malicious querying. Why It Matters: Prompt injection attacks are real. And your enterprise-trained model? It’s a business asset. You lock your office at night—do the same with your models. 🔹 Phase 6: Incident Response for AI Data Breaches - Having predefined protocols for breaches, hallucinations, or AI-generated harm—who’s notified, who investigates, how damage is mitigated. Why It Matters: AI-related incidents are happening. Legal needs response plans. Cyber needs escalation tiers. 🔹 Phase 7: CI/CD for Models (with Security Hooks) - Continuous integration and delivery pipelines for models, embedded with testing, governance, and version-control protocols. Why It Matter: Shipping models like software means risk comes faster—and so must detection. Governance must be baked into every deployment sprint. Want your AI strategy to succeed past MVP? Focus and lock down the data. #AI #DataSecurity #AILeadership #Cybersecurity #FutureOfWork #ResponsibleAI #SolRashidi #Data #Leadership

The 10 AI Threats Quietly Putting Enterprises at Risk What most companies get wrong about AI security? Thinking it’s just a “tech problem.” It’s not. It’s a behavior problem. Enterprise AI is no longer just answering questions. It’s making decisions. Triggering actions. Accessing sensitive systems. And that changes everything. Here’s the part many teams underestimate: AI doesn’t need to be hacked… It just needs to be misguided. And the impact looks exactly like a breach. Here are 10 AI security threats every enterprise should be thinking about: Prompt Injection Attacks ↳ AI follows malicious instructions → data leaks or wrong actions Data Poisoning ↳ Bad data in training = corrupted outputs at scale Model Inversion ↳ Attackers pull sensitive data from responses Sensitive Data Leakage ↳ Poor context control exposes confidential info API Key & Credential Theft ↳ One stolen key = full system access Unauthorized Tool Invocation ↳ AI triggers actions it shouldn’t even have access to Supply Chain Vulnerabilities ↳ Third-party models can introduce hidden risks Model Drift ↳ AI silently becomes unreliable over time Excessive Autonomy ↳ Agents act beyond boundaries → real-world damage Compliance Violations ↳ AI outputs break regulations without warning What actually protects you isn’t just better models. It’s better control. • Input and output guardrails • Dataset validation pipelines • Access control and tool restrictions • Continuous monitoring • Human-in-the-loop for critical decisions Because here’s the reality: The more powerful your AI becomes… The smaller your margin for error gets. The companies that win with AI won’t be the fastest. They’ll be the most controlled. If you’re deploying AI today Are you treating it like a smart assistant… or like a potential insider with access to everything? Share it with your network. 📌 Follow Marcel Velica for more insights on AI, security, and real-world strategies. If you want short daily thoughts, quick threat observations, and real-time discussions, follow me on X as well →https://x.com/MarcelVelica

The National Institute of Standards and Technology (NIST) has released a draft of its “Cybersecurity Framework Profile for Artificial Intelligence” (open for public comment until Jan 30, 2026) to help organizations think about how to strategically adopt AI while addressing emerging cybersecurity risks that stem from AI’s rapid advance. Building on the #NIST Cybersecurity Framework 2.0, the Cyber AI Profile translates well-established risk management concepts into AI-specific cybersecurity considerations, offering a practical reference point as organizations integrate AI into critical systems and confront AI-enabled threats. The Cyber AI Profile centers on three focus areas: • Securing AI systems: identifying cybersecurity challenges when integrating AI into organizational ecosystems and infrastructure. • Conducting AI-enabled cyber defense: identifying opportunities to use AI to enhance cybersecurity, and understanding challenges when leveraging AI to support defensive operations. • Thwarting AI-enabled cyberattacks: building resilience to protect against new AI-enabled threats. The Profile complements existing NIST frameworks (CSF, AI RMF, RMF) by prioritizing AI-specific cybersecurity outcomes rather than creating a standalone regime.

Today, NIST released the initial preliminary draft of the Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile), a community profile built on NIST CSF 2.0 to help organizations manage cybersecurity risk in an AI-driven world. A key section of this draft is Section 2.1, which introduces three Focus Areas that explain how AI and cybersecurity intersect in practice: 1. Securing AI System Components (Secure) AI systems introduce new assets that must be secured; models, training data, prompts, agents, pipelines, and deployment environments. This focus area emphasizes treating AI components as first-class cybersecurity assets, integrating them into governance, risk assessments, protection controls, and monitoring processes. It reinforces that AI risk should not be siloed from enterprise cybersecurity risk management. 2. Conducting AI-Enabled Cyber Defense (Defend) AI is not just something to protect, it is also a powerful defensive capability. This area focuses on using AI to enhance detection, analytics, automation, and response across security operations. At the same time, it recognizes the risks of over-reliance on automation, model integrity concerns, and the need for human oversight when AI supports security decision-making. 3. Thwarting AI-Enabled Cyber Attacks (Thwart) Adversaries are increasingly using AI to scale phishing, evade detection, and automate attacks. This focus area addresses how organizations must anticipate and counter AI-enabled threats by building resilience, improving detection of AI-driven attack patterns, and preparing for a rapidly evolving threat landscape where AI is weaponized. Why This Matters Together, Secure, Defend, and Thwart provide a practical structure for aligning AI initiatives with existing cybersecurity programs. By mapping AI-specific considerations to CSF 2.0 outcomes (Govern, Identify, Protect, Detect, Respond, Recover), the Cyber AI Profile helps organizations integrate AI security into familiar risk management practices. This is a preliminary draft, and NIST is seeking public feedback through January 30, 2026. If your organization is building, deploying, or defending with AI, now is the time to review and contribute. 🔗 https://lnkd.in/e-ETZXH8

The AI cybersecurity race is here. Today, the Google Threat Intelligence Group released our latest AI Threat Tracker. Here’s the reality: adversaries are deploying highly coordinated, AI-augmented operations at scale. To build a resilient enterprise, security leaders recognize that protecting the AI pipeline is what ultimately unlocks the confidence to scale it. Here are three findings from this latest intelligence: - First AI-developed zero-day: We identified a zero-day exploit (a 2FA bypass) where the adversary likely used an AI model to assist in discovering and weaponizing the vulnerability. The script contained clear indicators of AI generation, including a hallucinated CVSS security score. Our discovery likely prevented its use in a planned mass exploitation event. - Autonomous malware: We're tracking PROMPTSPY, a new Android backdoor designed to autonomously navigate a victim’s device UI and actively block uninstallation attempts. - AI supply chain attacks: Adversaries are increasingly targeting AI software dependencies, such as LiteLLM, to compromise build environments and extract cloud credentials. In this landscape, manual defense fails. When adversaries use automation, defense must move at machine speed. At Google, we are tipping the scale back to the defender: by deploying agentic cyber defense—like Big Sleep and CodeMender—we are finding and patching vulnerabilities before they can be exploited. We are using AI to build software that is secure by design, even as we continue to defend the massive landscape of legacy code the world relies on today. Read the full GTIG AI Threat Tracker report here: https://lnkd.in/gn6UHXaV 

Leveraging this new OpenAI real time translator to phish via phone calls in the target’s preferred language in 3…2… So far, AI has been used for believable translations in phishing emails — E.g. my Icelandic customers are seeing a massive increase in phishing in their language in 2024. Before only 350,000 or so people comfortably spoke Icelandic correctly, now AI can do it for the attacker. We’re going to see this real time translation tool increasingly used to speak in the target’s preferred language during phone call based attacks. These tools are easily integrated into the technology we use to spoof caller ID, place calls, and voice clone. Now, in any language. Educate your team & family + friends. Make sure folks know: - AI can voice clone - AI can real time translate to speak in any language - Caller ID is easily spoofed with or without AI tools - AI tools will increase in believability Example AI voice clone/spoof example here: https://lnkd.in/gPMVDBYC Will this AI be used for good? Sure! Real time translations are quite useful for people, businesses, & travel. We still need to educate folks on how AI is currently use to phish people & how real time AI translations will increase scams across (previous) language barriers. *What can we do to protect folks from attackers using AI to trick?* - Educate first: make sure folks around you know it’s possible for attackers to use AI to voice clone, deepfake video and audio (in real time during calls) - Be politely paranoid: encourage your team and community to use 2 methods of communication to verify someone is who they say they are for sensitive actions like sending money, data, access, etc. For example, if you get a phone call from your nephew saying he needs bail money now, contact him a different way before sending money to confirm it’s an authentic request - Passphrase: consider using a passphrase with your loved ones to verify identity in emergencies (e.g. your sister calls you crying saying she needs $1,500 urgently ask her to say the passphrase you agreed upon together or contact with another communication method before sending money)

AI agents have security problems that most companies are not ready for. Not because agents are dangerous by default. Because agents need access. - To databases. - Customer records - Personnel files - Financial data. - Internal policies. - Workflows. - Permissions. That means the data layer is no longer just storage. It is becoming the new security boundary. This is the part many enterprise AI strategies miss. If your AI agent can query the data, summarize the data, reason over the data, and act on the data, then security cannot only sit in the application layer. It has to live where the data lives. That is why Oracle’s latest AI Database security push is interesting. The message is simple: Secure at the source. Secure at speed. Secure through resilience. Offer security, patching and upgrade tools at no cost or deeply discounted to get started fast. In other words: Protect the data directly. Patch faster than attackers move. Recover quickly when things go wrong. Get into an accelerated data protection cycle. That is the new AI security model. https://lnkd.in/gSynh72J The next AI questions are not just: “How many agents can we deploy?” It is: “Is our data secure enough to survive them? “Do we have a strategy to stop rogue agents and shadow agents at the source?” and  “Can our architecture scale to sustain hundreds, thousands or hundreds of thousands of AI agents?”

When AI Meets Security: The Blind Spot We Can't Afford Working in this field has revealed a troubling reality: our security practices aren't evolving as fast as our AI capabilities. Many organizations still treat AI security as an extension of traditional cybersecurity—it's not. AI security must protect dynamic, evolving systems that continuously learn and make decisions. This fundamental difference changes everything about our approach. What's particularly concerning is how vulnerable the model development pipeline remains. A single compromised credential can lead to subtle manipulations in training data that produce models which appear functional but contain hidden weaknesses or backdoors. The most effective security strategies I've seen share these characteristics: • They treat model architecture and training pipelines as critical infrastructure deserving specialized protection • They implement adversarial testing regimes that actively try to manipulate model outputs • They maintain comprehensive monitoring of both inputs and inference patterns to detect anomalies The uncomfortable reality is that securing AI systems requires expertise that bridges two traditionally separate domains. Few professionals truly understand both the intricacies of modern machine learning architectures and advanced cybersecurity principles. This security gap represents perhaps the greatest unaddressed risk in enterprise AI deployment today. Has anyone found effective ways to bridge this knowledge gap in their organizations? What training or collaborative approaches have worked?

🚨 AI Privacy Risks & Mitigations Large Language Models (LLMs), by Isabel Barberá, is the 107-page report about AI & Privacy you were waiting for! [Bookmark & share below]. Topics covered: - Background "This section introduces Large Language Models, how they work, and their common applications. It also discusses performance evaluation measures, helping readers understand the foundational aspects of LLM systems." - Data Flow and Associated Privacy Risks in LLM Systems "Here, we explore how privacy risks emerge across different LLM service models, emphasizing the importance of understanding data flows throughout the AI lifecycle. This section also identifies risks and mitigations and examines roles and responsibilities under the AI Act and the GDPR." - Data Protection and Privacy Risk Assessment: Risk Identification "This section outlines criteria for identifying risks and provides examples of privacy risks specific to LLM systems. Developers and users can use this section as a starting point for identifying risks in their own systems." - Data Protection and Privacy Risk Assessment: Risk Estimation & Evaluation "Guidance on how to analyse, classify and assess privacy risks is provided here, with criteria for evaluating both the probability and severity of risks. This section explains how to derive a final risk evaluation to prioritize mitigation efforts effectively." - Data Protection and Privacy Risk Control "This section details risk treatment strategies, offering practical mitigation measures for common privacy risks in LLM systems. It also discusses residual risk acceptance and the iterative nature of risk management in AI systems." - Residual Risk Evaluation "Evaluating residual risks after mitigation is essential to ensure risks fall within acceptable thresholds and do not require further action. This section outlines how residual risks are evaluated to determine whether additional mitigation is needed or if the model or LLM system is ready for deployment." - Review & Monitor "This section covers the importance of reviewing risk management activities and maintaining a risk register. It also highlights the importance of continuous monitoring to detect emerging risks, assess real-world impact, and refine mitigation strategies." - Examples of LLM Systems’ Risk Assessments "Three detailed use cases are provided to demonstrate the application of the risk management framework in real-world scenarios. These examples illustrate how risks can be identified, assessed, and mitigated across various contexts." - Reference to Tools, Methodologies, Benchmarks, and Guidance "The final section compiles tools, evaluation metrics, benchmarks, methodologies, and standards to support developers and users in managing risks and evaluating the performance of LLM systems." 👉 Download it below. 👉 NEVER MISS my AI governance updates: join my newsletter's 58,500+ subscribers (below). #AI #AIGovernance #Privacy #DataProtection #AIRegulation #EDPB

AI is changing the economics and speed of cyberattacks. What once took threat actors days or weeks can now happen in minutes: automated reconnaissance, AI-assisted exploit development, credential targeting, lateral movement, and highly personalized phishing at scale. This is why Palo Alto Networks believes so strongly in the concept of autonomous resilience. The traditional model of security operations: fragmented tools, manual escalation paths, and human-speed response cycles - was not designed for machine-speed threats. Autonomous resilience means building security architectures that can continuously reduce exposure, validate trust, and contain threats in real time. What does that look like in practice? 🔸 Minimize attack surface Continuously identify and remediate exposed assets, misconfigurations, vulnerable APIs, and unmanaged cloud resources before attackers can weaponize them. For example, AI-driven exposure management can detect an internet-facing development environment created outside policy and trigger automated remediation immediately. 🔸 Secure every identity Trust must extend beyond employees to machine identities, workloads, APIs, and AI agents. This means enforcing least privilege, adaptive access controls, and continuous identity validation to stop credential misuse and token theft before attackers gain persistence. 🔸 Defend the software supply chain AI-assisted attacks increasingly target CI/CD pipelines, open-source dependencies, and code repositories. Organizations need runtime protections, code integrity validation, and automated policy enforcement to prevent manipulated code from reaching production environments. 🔸 Constrain blast radius Zero Trust architectures become even more critical in an AI-driven threat landscape. Microsegmentation, continuous inspection, and behavioral analytics help prevent attackers from moving laterally across environments once initial access is achieved. 🔸 Detect and respond in real time Security teams cannot rely on analysts manually correlating thousands of alerts. AI-driven SOC operations can automatically prioritize incidents, enrich telemetry, isolate compromised assets, and initiate containment workflows within minutes — dramatically reducing operational fatigue and response time. The outcome is not “fully autonomous security.” The outcome is resilient organizations that can adapt, contain, and recover faster in an increasingly automated threat environment. Cybersecurity is evolving from reactive defense into continuous operational resilience. The organizations preparing for that shift now will be far better positioned for what comes next.