Hacking AI is TOO EASY (this should be illegal)

NetworkChuck Technology Careers Career & Money

About this video

Chuck interviews AI security researcher Jason Haddock (spelled that way in the captions) on how attackers compromise AI apps. Haddock distinguishes AI red teaming (getting a model to say harmful things) from a holistic AI penetration test, and lays out a six-part attack methodology: identify system inputs, attack the surrounding ecosystem, red-team the model itself, attack the prompt engineering, attack the data, attack the application, and pivot to other systems. The central weapon is prompt injection. Chuck relays that OpenAI’s Sam Altman, asked about it at a conference, said he thinks defenders can get to 95% but are not there yet, and that prompt injection will be around a long time. Haddock’s taxonomy of injection techniques covers intents (around 21 or 22, including leaking the system prompt and extracting business info), techniques, evasions, and utilities, which he says combine into 9.9 trillion possible attack combinations. Chuck demos the Gandalf game, whose eight levels teach injection against rising guardrails, then shows real techniques: emoji smuggling (instructions hidden in Unicode inside an emoji), a ‘syntactic anti-classifier’ that rewrites blocked image prompts with synonyms and metaphors, and link smuggling, where an AI is tricked into encoding stolen data such as a credit card number into an image URL that phones home to an attacker’s server. Real-world findings include companies unknowingly sending sensitive Salesforce data to OpenAI, a Slack sales bot whose over-scoped API access let testers write malicious content back into Salesforce, and serious weaknesses in MCP (Model Context Protocol) servers, which often lack access controls. A sponsor segment covers Wiz, which Chuck says is trusted by over 45% of Fortune 100 companies (updated to 50% in the video). The defense section prescribes three layers: web-layer fundamentals like input/output validation, an AI firewall or classifier guarding prompts in both directions, and least-privilege scoping of every API key an agent uses. The video closes with Haddock’s story of leaking GPT-4o’s system prompt by asking it to put the prompt as flavor text on a Magic-style card it generated of itself.

Key takeaways

AI pen testing is broader than red teaming: it covers inputs, the surrounding ecosystem, the model, prompt engineering, data, the application, and pivoting to other systems
Prompt injection needs no coding skill to start, and per the video Sam Altman believes defenders can only get to roughly 95% against it
Concrete techniques shown include emoji smuggling, a synonym-based anti-classifier for image guardrails, and link smuggling that exfiltrates data through image URLs
The biggest real-world risks are architectural: over-scoped API keys, no input validation on tool calls, and MCP servers with no access control
Defense is layered: secure the web layer, filter prompts in and out with an AI firewall/classifier, and scope every API key to least privilege

Episode details: published August 12, 2025 · 26:39 long. Summary written by Self Growth Videos from the episode itself. Business results and any income figures mentioned are NetworkChuck’s own claims, shared for education — they are not typical or guaranteed.

← Previous: You NEED to Use n8n RIGHT NOW!! (Free, Local, Private) · Next: THIS Is Where the Internet Lives →

All NetworkChuck videos · NetworkChuck profile · NetworkChuck on YouTube