DevSecOps

DevOps, on paper, is a set of practices that merges software development and IT operations into one continuous loop instead of two handoffs — CI/CD pipelines, infrastructure as code, automated testing, and monitoring feedback, aimed at shipping changes faster and more reliably by dissolving the wall between the people writing code and the people running it. Nobody hands a build to "ops" and waits; the same team that writes the code owns it in production.

Most orgs don't dissolve that wall, they hire for it: a "DevOps engineer" or "SRE" role sits next to frontend and backend engineers as its own specialization, and a dedicated DevOps or platform team owns deployment pipelines and tooling for everyone else. That's not a failure to implement DevOps correctly so much as it is the shape most companies actually settle into once they're past a certain size — the question isn't whether a dedicated team exists, it's what role that team plays for everyone else.

Prompt injection comes in two shapes. The direct kind is a user simply talking a model into ignoring its own rules — in November 2023, a former X employee told a Chevrolet dealership's chatbot to agree with anything the customer said and to close every reply with a line declaring the offer legally binding, then asked for a $76,000 Chevy Tahoe for $1. The bot complied exactly as instructed, and the exchange went viral within days.

The indirect kind is harder to see coming, because the attacker never talks to the model at all. A 2023 paper by Greshake et al., “More Than You've Asked For”, showed that hidden instructions planted inside a webpage, document, or email — invisible to a human reader, sitting in an HTML comment or white-on-white text — get treated by an LLM-powered agent exactly like the user's own instructions once that content lands in the context window. In their demonstration, a browsing agent that was only asked to summarize a page ended up exfiltrating the user's data to an attacker-controlled endpoint, without the user ever writing a malicious prompt themselves.

Both cases point at the same underlying limit: a language model has no hard boundary between "instructions" and "data." Everything in the context window is just tokens, so any external content an agent is asked to read — a support ticket, a PDF, a GitHub issue, a search result — is an opportunity for someone else's instructions to ride in disguised as content. No amount of system-prompt wording fully closes that gap, because the model is being asked to both follow instructions and faithfully process arbitrary text in the same pass.

That's the case for curated knowledge storage: instead of letting a retrieval-augmented system pull freely from the open web or unreviewed user submissions at query time, maintain a vetted, access-controlled corpus with an ingestion pipeline that reviews and sanitizes content before it ever becomes retrievable, and tracks provenance per chunk so a bad entry can be traced and pulled. This doesn't make prompt injection impossible — a compromised internal document is still a document — but it collapses the attack surface from "anything on the internet the agent might fetch" down to "content that already passed your own review," which is a boundary an engineering team can actually defend.

Pickle-based model files are dangerous because deserializing one runs Python code as a side effect — a malicious .pt file downloaded from a public model hub can execute on your machine the moment torch.load() touches it, no separate exploit required. Safetensors removes that specific mechanism: its format has no executable payload path, so loading one cannot run code.

What it doesn't remove is trust in the weights themselves. Trail of Bits' 2024 Sleepy Pickle research showed how a pickle file's own deserialization step can be hijacked to patch a model's weights in memory right after it loads — inserting a backdoor, tampering with outputs, or exfiltrating input data — all without changing a single byte the model was actually trained on. A self-propagating variant, Sticky Pickle, copies the same payload forward into every model version downstream of it. Safetensors closes the exact code-path Sleepy Pickle relies on, which is real progress, but a model file — in any format — is still a black box of numbers. Nothing in the file format checks whether those numbers were trained to misbehave on a specific trigger phrase or image before you ever run inference on it.

AI workloads run on GPUs, and GPUs run through drivers that operate as privileged kernel modules — a very different trust boundary than a CPU-only web service. In 2024, Wiz Research disclosed CVE-2024-0132, a critical (CVSS 9.0) time-of-check-to-time-of-use flaw in the NVIDIA Container Toolkit that let a malicious container image escape entirely and gain full read access to the host filesystem, then pivot to root through the container runtime socket. Wiz estimated over 35% of cloud environments were exposed. The fix that shipped in 2024 was later found incomplete, and a follow-up bypass surfaced in 2025 — underscoring that a kernel-facing driver stack is a moving target, not a one-time patch.

In a shared GPU cloud, that isn't an abstract risk: a tenant's container escaping through the GPU driver lands directly on a host that may also be running other customers' training jobs, credentials, and model weights.

A container is not a VM. Docker containers share the host's kernel through namespaces and cgroups, which means isolation is only as strong as that shared kernel's attack surface — a single kernel exploit (Dirty Pipe, Dirty COW, and their successors) can walk straight through every container on the box. AI workloads make this worse in a specific way: GPU passthrough typically requires elevated device access and capabilities that ordinary web-app containers never need, which is exactly the access the NVIDIA Container Toolkit flaw above abused. The more privileged a container has to be to do its job, the less the word "container" is actually buying you in terms of isolation.

It compounds further at the image layer: a typical ML Docker image pulls in a CUDA base image, a Python environment, and dozens of pip and conda packages, each with their own transitive dependencies that nobody on the team has individually audited. "It's containerized" reads as a security boundary; in practice it's a large, fast-changing dependency tree running with more host access than most engineers realize.

AI infrastructure concentrates an unusual amount of value into a small number of files: a trained model's weights can represent years of compute spend and proprietary data, compressible to a size that fits on a USB drive. In January 2026, a federal jury convicted former Google engineer Linwei Ding on economic espionage and trade secret theft charges — the first AI-related espionage conviction in the U.S. — after he uploaded over 1,000 files describing Google's AI data-center hardware and software stack to a personal cloud account while secretly running an AI company in China. No exotic exploit was involved; an authorized employee with legitimate access simply copied what he already had access to.

That's the uncomfortable part of AI-infrastructure security: access control, audit logging, and data-loss prevention on internal systems matter as much as anything covered above, because the people most capable of walking out with a model or its training infrastructure are the ones already inside the perimeter.

• Supply-chain packages. PyPI and the Hugging Face Hub have both hosted typosquatted or outright malicious packages disguised as popular ML libraries — the same lesson as the pipeline story above, running at open-source speed.
• Secrets sprawl in notebooks. Jupyter notebooks routinely get shared, committed, or copied with cloud credentials and API keys hardcoded in a cell — a habit web developers mostly trained out of themselves years ago, but common in ML workflows.
• Model stealing via API access. An attacker with nothing but query access to a hosted model can systematically extract enough input/output pairs to train a functional copy of it, turning an API rate limit into the only real defense.
• Training-data and RAG poisoning. Anything a model learns from or retrieves at inference time is an injection point — a poisoned document in a retrieval index can steer output just as effectively as a compromised dependency can steer code.
• Multi-tenant GPU memory residue. Shared GPU clusters need to fully wipe device memory between tenants; a GPU that hands off VRAM still holding a previous customer's activations or weights is a data leak with no network traffic to detect it.

SSL was Netscape's original protocol for encrypting traffic between a browser and a server, first shipped as SSL 2.0 in 1995 and replaced within a year by SSL 3.0 after 2.0 turned out to have serious design flaws. SSL 3.0 lasted much longer, but it too is broken: the 2014 POODLE attack showed how to force a connection to downgrade to SSL 3.0 and then exploit its padding scheme to decrypt data byte by byte. Every version of SSL is now formally prohibited under PCI DSS, and no modern browser will negotiate it.

The catch is naming, not technology: “SSL certificate” and “SSL/TLS” remain the common shorthand for what is actually TLS running underneath. That legacy phrasing still shows up in vendor contracts and compliance questionnaires today, which is worth knowing before assuming a document that mentions “SSL” is describing something insecure.

TLS is SSL's successor, standardized by the IETF starting with TLS 1.0 in 1999. TLS 1.0 and 1.1 were formally deprecated in 2020–2021 after years of known weaknesses (BEAST among them), and PCI DSS has required TLS 1.2 or higher since 2018. TLS 1.2, from 2008, is still acceptable when configured with strong cipher suites and no legacy CBC-mode padding exposure — but it is easy to misconfigure into an insecure state, because it still permits those legacy options.

TLS 1.3 (2018) is the current standard and removes that footgun entirely: it drops every legacy cipher suite, mandates forward secrecy through ephemeral key exchange on every connection, and cuts the handshake down to one round trip (with an optional 0-RTT mode). The practical takeaway is that a valid certificate says nothing about whether the server behind it still allows an old TLS version, a weak cipher, or improper certificate validation — encryption in transit is a configuration discipline that has to be checked, not a box that gets checked once.

Every layer discussed above assumes the CPU, firmware, and silicon underneath it can be trusted to do what the algorithm says. Hardware-based encryption is what happens when that assumption itself gets tested — TPMs, hardware security modules, and secure enclaves that keep keys and crypto operations off the general-purpose CPU entirely, plus a separate discipline of verifying that a chip's own design has no exploitable flaw baked into it. That distinction matters because a defect at the microarchitecture or silicon level, like an undocumented debug path or a timing side channel, survives every operating system patch and every TLS upgrade running above it. Nothing in software can fix a mistake that shipped in the die.

Cycuity is one of the companies built specifically around that problem: static and dynamic security verification tooling (its Radix product line) that checks a chip's RTL design and firmware for known weakness classes and traces information flow through the hardware before it ever tapes out, rather than after it ships. The approach traces back to hardware-security research from UC San Diego's Kastner Research Group (Kastner Lab), the same lab known for pioneering gate-level information-flow tracking as a way to catch hardware vulnerabilities on paper instead of in production.

These two get confused constantly because both are used to justify the same encryption and access-control work, but they answer different questions for different audiences.

In practice: HIPAA asks whether PHI was adequately protected and whether that can be defended in a federal audit. SOC 2 asks whether an independent auditor observed a company's controls actually operating as described over a period of months. A company can hold a clean SOC 2 report and still not be HIPAA-eligible without a signed Business Associate Agreement and PHI-specific controls — they are not tiers of the same ladder, they are answers to two different questions.

The costs below are drawn from public reporting on fines, settlements, and remediation spend, then divided by ten to make the numbers comparable across very different incidents. Treat the per-year figures as an illustration of scale, not an audited actuarial number.

• The Trans-Siberian pipeline explosion (1982, disputed). Per the memoirs of Thomas Reed, a former Air Force secretary on Reagan's National Security Council, industrial-control software bound for the Soviet Union was quietly compromised somewhere along its supply chain before it ever reached the Trans-Siberian gas pipeline's SCADA system — a logic bomb built into legitimate-looking Canadian software, not something stolen or tampered with after the fact. The software ran correctly for months before deliberately mis-setting pump speeds and valve pressures until the line ruptured in what Reed described as the largest non-nuclear explosion ever recorded, visible from orbit, in June 1982. The account has never been independently confirmed — other former intelligence officials attribute the actual blast to a Soviet engineer overpressuring the line after finding a leak, not sabotage. If Reed's version is accurate, it is one of the earliest known software supply-chain attacks, and the lesson only gets sharper with time: compromising software before it reaches its target required a state intelligence apparatus and a physical diversion in 1982. It is far easier today, when a single compiled dependency can pull in dozens of transitive packages nobody on the team has actually read. The 2024 XZ Utils backdoor — years spent building trust as a maintainer before slipping remote-code-execution into a compression library used by OpenSSH across most Linux distributions — is the same playbook running at open-source speed. Think twice before reaching for that convenient open-source package; a logic bomb can sit compiled and dormant inside it, waiting for the moment it does the most damage.
• Unencrypted satellite backhaul (2025). The UC San Diego study above found live, unencrypted traffic from T-Mobile, Walmart, Intelsat/Panasonic in-flight systems, banks, and U.S. and Mexican military and police channels — passively interceptable by anyone with about $800 of gear, for as long as the misconfiguration existed. No theft was confirmed because this was a research disclosure, not an attack, but that is the point: passive satellite eavesdropping leaves no trace, so the honest cost isn't a settlement figure, it's years of unmeasured exposure across military, banking, and retail traffic. Unmeasured risk is the most expensive kind, precisely because no one can price it in advance.
• Heartbleed (2014). A bug in OpenSSL's TLS heartbeat extension let anyone pull up to 64KB of raw server memory per request — private keys, session cookies, credentials — from roughly 17% of the internet's TLS-secured servers. Banks, governments, and major platforms had to mass-reissue certificates and force password resets within days. One widely cited estimate put global remediation near $500 million, normalizing to about $50 million a year in ongoing OpenSSL-ecosystem auditing and cert-rotation discipline that traces directly back to one line of missing bounds checking.
• Equifax (2017). An unpatched Apache Struts vulnerability exposed Social Security numbers, birth dates, and addresses for 147 million people. Congressional hearings and executive departures followed, along with a reputational hit to a company whose entire business is trust in data handling. Combined settlement, legal, and remediation costs are commonly cited near $1.4 billion — roughly $140 million a year normalized — enough to fully fund a Fortune 500 security organization for a decade, spent instead on a breach that already happened.
• Capital One (2019). A misconfigured web application firewall allowed a server-side request forgery attack that reached AWS credentials and pulled data on 106 million customers out of S3 storage. An $80 million OCC fine plus roughly $190 million in customer settlements put the total near $270 million, or about $27 million a year normalized — for a single cloud configuration mistake.