AI Coding Fake Packages: Who Checks the Name Before an Agent Installs It?
AI coding tools can invent package names that attackers later register. Learn how South African teams can verify dependencies before they enter trusted builds.
An AI coding assistant suggests a package that appears to solve a developer’s problem. The name looks credible. The agent installs it, the tests pass and the new feature works. Yet nobody has checked whether the package came from the project they intended to trust.
Sometimes the name the AI suggested did not exist when it first appeared in an answer. If an attacker registers that name and publishes malicious code under it, a future recommendation can lead a developer straight to the attacker’s package. That is the risk behind AI coding fake packages: a plausible suggestion can become a software supply chain decision before anyone examines its source.
I use coding agents myself. They save enough time that I would not want to return to the old workflow. That makes the small decisions they take on my behalf more important. An agent that can suggest a dependency, install it and continue building has moved the decision about trust much earlier in the development process.
Main Takeaways
- AI coding tools sometimes recommend package names that do not exist in the expected software registry.
- An attacker may register a repeatedly suggested name and wait for a developer or agent to install it.
- Before a new dependency enters a trusted build, check its exact name, publisher, history, scripts, other dependencies and permissions.
- Give coding agents a restricted workspace and an approved route for adding packages. Make the review meaningful at the point where external code enters the project.
How does an AI tool invent a software package?
Developers routinely use third-party packages to avoid writing every component from scratch. A JavaScript developer might install one from npm; a Python developer might use PyPI. Those registries contain useful code, but a package name tells you very little about who published it or what it will do when installed.
A package hallucination occurs when an AI coding tool recommends a package that is absent from the expected registry. It may blend the names of real libraries, draw on outdated information or produce a name that simply sounds right for the task.
Researchers presenting at USENIX Security 2025 tested code-generating models and found average hallucinated-package rates of at least 5.2% for the commercial models and 21.7% for the open-source models in their experiments. They identified more than 205,000 unique invented package names. Those figures describe the models and testing conditions in the study; they do not mean that one in five packages installed by a business is malicious. They establish that invented names are a repeatable problem.
The security risk emerges when someone else can publish code under one of those names. An attacker could identify a name that AI tools repeatedly recommend, register it and wait. The next developer who trusts the recommendation may install code the attacker controls. This has become known as slopsquatting.
The package does not need to look obviously suspicious. If it appears in a routine change to a project’s dependency list and the feature passes its tests, a reviewer may focus on the working feature rather than the identity of the new supplier of code.
How can a fake suggestion become a real incident?
Imagine a South African development team adding a reporting feature to an internal application. A coding agent finds a reference to a specialist file format and recommends a library with a convincing name. It installs the dependency, writes the integration and reports that the tests have passed.
This is an illustrative scenario, not a reported Netsurit incident. Several outcomes are possible. A malicious install script might run immediately on a developer’s machine or build server. Alternatively, the package might execute harmful code only when the application imports it. Depending on the access available in that environment, it could attempt to read source files, credentials or configuration, or connect to an external service.
Real software supply chain incidents show why the installation step deserves care. Unit 42 has documented malicious package and build-tool activity involving install scripts and attempts to collect secrets. That research demonstrates the consequences of trusting hostile code in a development environment; it is separate from the USENIX study of invented AI package names.
The chain has several links: a model suggests a name, someone publishes code under it, an agent or developer retrieves it, and the code runs somewhere trusted. A team can interrupt that chain without giving up the coding agent. The key is to treat the agent’s recommendation as a proposal that still needs verification.
What should a developer check before installing a package?
Start with the exact package name in the expected registry. Then find the project’s own documentation and confirm that it links to that same package. A similar name or a polished registry description is insufficient.
Look at the publisher and project history. Who maintains the package? How long has it existed? Does its repository show meaningful development and releases? Has ownership changed recently? A brand-new registry entry claiming to represent a long-established tool calls for investigation.
Next, examine what will run. Install and post-install scripts may execute before a developer has used the package in their application. Review what the package brings in through its own dependencies and whether its behaviour fits its stated purpose. Test an unfamiliar package in a restricted environment before it reaches a workstation or build pipeline with valuable credentials.
Finally, record why the dependency was selected and which version was approved. Months later, the team should be able to explain why the application relies on that package. A software bill of materials helps identify what shipped, but it cannot establish that a malicious package was safe to install in the first place.
These checks matter especially when the agent has access to cloud credentials, customer repositories or a continuous integration pipeline. The same package can present very different risks depending on where it runs.
What authority should a coding agent have?
An agent assigned to a reporting feature needs enough access to do the job. It does not automatically need every repository the organisation owns, production credentials or permission to introduce new third-party code into a trusted build.
A project-specific workspace and limited credentials reduce the consequences if an agent makes a poor dependency choice. An approved package source, private mirror or review process can prevent a newly suggested public package from entering the build before somebody has checked it. Those controls also need upkeep: an approved package can change, and a previously trusted maintainer account can be compromised.
The review point should be clear. Let an agent draft code and run tests in a restricted environment. Require a separate decision when it wants to add an external dependency to the trusted build or deploy a change. If developers must approve every harmless command, they will start clicking through prompts without reading them. Put their attention where a decision has lasting consequences.
There is another route to the same installation problem. An attacker can place instructions in a README, issue description or web page that the agent reads while working. The instruction might tell the agent to install a particular package. OWASP advises treating such repository and external content as untrusted input when an AI coding agent processes it. A hallucinated package name and a malicious instruction are different mechanisms, but either can lead an agent to fetch unwanted code.
This is one reason the security of an agent depends on more than the model. Its tools, permissions, sources of information and execution environment determine what happens after a questionable suggestion. Netsurit’s article on AI agent platform decisions explores how those architectural choices affect an organisation long after the first experiment.
Will these checks slow developers down?
Some checks take time, and that time should be spent on the decisions that carry real risk. Coding agents reduce the effort of producing a feature. They do not remove the need to understand a dependency that the organisation may support for years.
The practical answer is to gather routine evidence automatically. A pipeline can flag a package that is new to the organisation, highlight install scripts, show publisher changes and compare the dependency against approved sources. A developer or security reviewer can then concentrate on the few decisions that require judgement.
South African teams do not need a special version of this threat for it to matter locally. They use the same public registries, development tools and cloud build processes as teams elsewhere. A malicious dependency introduced during development can affect an application wherever its users and data are located.
The question I would ask of any team adopting coding agents is simple: who verifies the external code the agent chooses, and at what point does that verification happen? If the answer is “when someone notices it in the pull request”, the review may already be too late for a package that executed during installation.
Our whitepaper, AI Code You Can Trust, is a useful next read for teams examining how AI-assisted development fits into their software security practices.
Frequently asked questions
Can AI coding tools really invent package names?
Yes. AI coding tools can generate package names that appear legitimate but do not exist in official software registries such as npm or PyPI. This is known as package hallucination. It can happen when AI models combine existing library names, rely on outdated information or generate plausible names without verifying their existence.
Attackers can exploit this behaviour by registering invented package names and publishing malicious code. Developers and coding agents that install these packages without verification may unintentionally introduce security threats into their applications. Always confirm the exact package name, publisher and official repository before installation.
Is every new software package suspicious?
No. New software packages are not automatically malicious, but they require additional verification before being introduced into a development environment. Legitimate developers regularly publish new libraries to solve emerging technical challenges.
However, newly registered packages may lack an established maintenance history, independent security reviews or reliable publisher information. Development teams should examine the package's source repository, maintainer identity, dependencies, installation scripts and stated functionality. Packages with limited history or unclear ownership should undergo further review before being approved for production use.
Does pinning a dependency solve the problem?
No. Dependency pinning helps ensure that applications use specific package versions, reducing unexpected changes during installation or deployment. However, it does not guarantee that the selected package is trustworthy.
If a malicious package is installed initially, pinning its version simply preserves that compromised dependency. Teams should verify package authenticity before approval, maintain lockfiles, use trusted registries and implement automated vulnerability scanning. Regular dependency reviews are also necessary because previously trusted packages can become compromised through malicious updates or maintainer account takeovers.
Can an AI coding agent install packages safely in a sandbox?
Yes. A properly configured sandbox can reduce the risks associated with installing unfamiliar packages by isolating their execution from sensitive systems, credentials and production environments.
However, sandboxing does not automatically make a package safe. Development teams should restrict network access, limit file permissions, avoid exposing production credentials and monitor installation scripts. Agents should operate within project-specific workspaces and use approved package sources. Any new dependency should undergo verification before being introduced into trusted builds or production systems.
What is the most useful security rule for a development team using AI coding agents?
The most useful rule is to verify every new external dependency before allowing it to execute in a trusted development or production environment.
AI coding agents should be permitted to recommend packages and generate code, but introducing third-party dependencies should require an approved verification process. This includes confirming package identity, examining publisher history, reviewing installation scripts and checking permissions. Automated security checks can reduce manual effort, while human approval should focus on decisions involving unfamiliar code, sensitive credentials or production access.
What is slopsquatting in AI-assisted software development?
Slopsquatting is a software supply chain attack in which malicious actors register package names that AI coding tools are likely to invent or recommend. These names may appear credible because they resemble legitimate software libraries or match a developer's technical requirements.
When an AI assistant recommends one of these packages, a developer or autonomous coding agent may install it without verifying its authenticity. The malicious package could then execute harmful scripts, attempt to steal credentials or compromise the development environment. Organisations can reduce this risk through package verification, approved registries, restricted agent permissions and dependency security reviews.
How can developers identify malicious or fake software packages?
Developers can identify potentially malicious packages by checking their exact names against official registries, verifying publisher identities and reviewing their source repositories. Warning signs include recently created packages impersonating established libraries, unexplained ownership changes, suspicious installation scripts and dependencies unrelated to the package's stated purpose.
Additional checks should include examining release histories, analysing requested permissions and using software composition analysis tools to detect known vulnerabilities. Unfamiliar packages should be tested in restricted environments before receiving access to sensitive systems. Automated checks combined with developer review provide stronger protection than relying on package names or download counts alone.
How can organisations prevent AI-generated dependency attacks?
Organisations can reduce AI-generated dependency attacks by establishing security controls around package selection, installation and deployment. Coding agents should operate with limited permissions and obtain dependencies through approved package sources or controlled internal registries.
Development pipelines should verify package authenticity, scan dependencies for known vulnerabilities and flag suspicious installation scripts or publisher changes. Teams should maintain software bills of materials, review newly introduced dependencies and restrict access to production credentials. Regular security training and clear approval responsibilities help ensure that AI-assisted development improves productivity without weakening software supply chain security.
The decision hidden inside the suggestion
An AI agent can find a useful library in seconds. By naming it, the agent has not established a chain of trust from the publisher to your application. Somebody still needs to decide whether that package belongs in your development environment and, eventually, in the software your organisation runs.
Netsurit helps organisations bring security into their technology decisions and operations. If your team is introducing coding agents, speak to Netsurit about your development environment and security controls before a package recommendation becomes an unexamined installation.
