AI Is In Your Stack. Is Anyone Managing the Risk?
Table of Contents
Published by Nate Olson — Founder | Fractional IT Director & Virtual CIO, N.O. IT Strategy LLC
There’s a conversation happening in every boardroom and leadership meeting right now.
How do we get AI into the business?
It’s the right question.
But most organizations are skipping the one that should come first.
What are we actually bringing in?
What Happened This Week
On March 24, 2026, a Python package called LiteLLM was compromised.
LiteLLM is an open-source Python library and proxy server that provides a unified interface to call over 100 different Large Language Models (LLMs). It simplifies AI development and is used by NASA, Netflix, Stripe, NVIDIA, and tens of thousands of other organizations. It sits between applications and every major AI provider, OpenAI, Anthropic, Google, AWS , routing credentials and API keys through a single proxy.
It has 97 million downloads a month, that’s 3.4 Million downloads a day.
Present in 36% of cloud environments.
For a few hours yesterday, installing it was enough to silently hand over every API key, cloud credential, SSH key, and sensitive configuration file on the machine.
You didn’t need to import it. You didn’t need to call it. It ran the second it existed in the environment.
The attackers didn’t break down the front door. They compromised a security scanning tool first. Used those credentials to access LiteLLM’s publishing pipeline. Pushed malicious code directly to the package registry.
The malicious versions (1.82.7 and 1.82.8) were live on PyPI for roughly 3 hours. Published around 10:39 UTC, quarantined by PyPI by around 11:25 to 13:00 UTC depending on the version.
That’s actually what makes it so alarming. Three hours was enough time for it to get pulled into thousands of environments, because LiteLLM is downloaded 3.4 million times per day. At that download velocity, even a 3 hour window is massive exposure.
Five ecosystems compromised in two weeks. Each breach providing the credentials to unlock the next one.
The only reason the damage wasn’t worse?
The malware was so poorly written it crashed machines. A developer noticed. That was the entire defense.
If the code had been cleaner, nobody notices for weeks. Maybe months.
This Is Not a Developer Problem
Most business leaders read a story like this and assume it belongs to the engineering team.
It doesn’t.
This is a governance problem.
The organizations most exposed right now are not the ones with weak developers. They are the ones where AI adoption is happening fast, at every level, with no structured oversight of what’s being introduced, how it’s being configured, or what it can access.
That’s a leadership gap.
And it’s more common than most organizations want to admit.
What AI Governance Actually Means
AI governance is not about slowing down adoption.
It’s about being deliberate.
For most small to mid-market organizations, it comes down to four questions leadership should be able to answer:
1. What AI tools are running in our environment right now?
Not just the ones IT approved. The ones being used across departments. The plugins, the integrations, the tools someone installed because it made their job easier.
Most organizations can’t answer this question. That’s the problem.
2. What does each tool have access to?
AI tools and packages frequently sit in privileged positions in the technology stack. They touch credentials, data, APIs, and configuration files. The access a tool has been granted matters as much as the tool itself.
3. How are our AI credentials being managed?
API keys for AI providers are sensitive secrets. They need to be treated with the same discipline as any other critical credential. Rotated regularly. Stored securely. Revoked when no longer needed.
Most organizations are not doing this.
4. Who owns the decision when something goes wrong?
When a compromised package exfiltrates credentials from a developer’s machine at 10:39 in the morning, who gets the call? Who makes the decisions? Who is accountable?
If the answer is unclear, the organization is not ready for the risk it’s already carrying.
The Speed Problem
The companies deploying AI the fastest right now have the least visibility into what’s underneath it.
That’s not an opinion.
That’s the pattern playing out across every organization that is prioritizing speed to market over structural readiness.
The dependency chains underneath modern AI tooling are complex. A developer installs one package. That package pulls in twelve others. One of those twelve has a transitive dependency nobody chose and nobody reviewed.
That’s how LiteLLM ended up on a developer’s machine through a Cursor plugin they didn’t even know was installed.
Speed is a competitive advantage.
But speed without governance is a liability you haven’t discovered yet.
What Leadership Should Do Now
This doesn’t require a full security overhaul.
It requires ownership.
- Conduct an inventory of AI tools in use across the organization, not just in IT
- Establish a simple approval process before new AI tools are introduced
- Ensure AI API credentials are being stored, managed, and rotated properly
- Assign clear ownership for AI-related risk and incident response
- Build AI governance into your technology road map, not as an afterthought
None of this is technically complex.
All of it requires someone in the room whose job it is to see it coming.
The Leadership Gap in AI Adoption
The risk profile of AI tooling is different from traditional software.
It changes faster. The dependency chains are less transparent. The credentials it touches are high value. And the ecosystem is new enough that security standards are still catching up.
That’s exactly the kind of environment where experienced, independent IT leadership pays for itself.
AI governance and emerging technology risk management is a core component of the vCIO and Fractional IT Director engagements at N.O. IT Strategy. Not a future add-on. Not a separate project. Built into how strategic IT leadership is delivered from day one.
Because the organizations that get this right aren’t the ones that moved the fastest.
They’re the ones that moved with someone in the room who knew what they were walking into.
N.O. IT Strategy LLC provides independent fractional IT leadership for growing organizations. No vendor incentives. No product bias. Just honest guidance.
strategy@noitstrategy.com | 458.262.5571 | noitstrategy.com