AI Coding Fake Packages: Who Checks the Name Before an Agent Installs It?
AI coding tools can invent package names that attackers later register. Learn how South African teams can verify dependencies before they enter trusted builds.
An AI coding assistant suggests a package that appears to solve a developer’s problem. The name looks credible. The agent installs it, the tests pass and the new feature works. Yet nobody has checked whether the package came from the project they intended to trust.
Sometimes the name the AI suggested did not exist when it first appeared in an answer. If an attacker registers that name and publishes malicious code under it, a future recommendation can lead a developer straight to the attacker’s package. That is the risk behind AI coding fake packages: a plausible suggestion can become a software supply chain decision before anyone examines its source.
I use coding agents myself. They save enough time that I would not want to return to the old workflow. That makes the small decisions they take on my behalf more important. An agent that can suggest a dependency, install it and continue building has moved the decision about trust much earlier in the development process.
Main Takeaways
- AI coding tools sometimes recommend package names that do not exist in the expected software registry.
- An attacker may register a repeatedly suggested name and wait for a developer or agent to install it.
- Before a new dependency enters a trusted build, check its exact name, publisher, history, scripts, other dependencies and permissions.
- Give coding agents a restricted workspace and an approved route for adding packages. Make the review meaningful at the point where external code enters the project.
How does an AI tool invent a software package?
Developers routinely use third-party packages to avoid writing every component from scratch. A JavaScript developer might install one from npm; a Python developer might use PyPI. Those registries contain useful code, but a package name tells you very little about who published it or what it will do when installed.
A package hallucination occurs when an AI coding tool recommends a package that is absent from the expected registry. It may blend the names of real libraries, draw on outdated information or produce a name that simply sounds right for the task.
Researchers presenting at USENIX Security 2025 tested code-generating models and found average hallucinated-package rates of at least 5.2% for the commercial models and 21.7% for the open-source models in their experiments. They identified more than 205,000 unique invented package names. Those figures describe the models and testing conditions in the study; they do not mean that one in five packages installed by a business is malicious. They establish that invented names are a repeatable problem.
The security risk emerges when someone else can publish code under one of those names. An attacker could identify a name that AI tools repeatedly recommend, register it and wait. The next developer who trusts the recommendation may install code the attacker controls. This has become known as slopsquatting.
The package does not need to look obviously suspicious. If it appears in a routine change to a project’s dependency list and the feature passes its tests, a reviewer may focus on the working feature rather than the identity of the new supplier of code.
How can a fake suggestion become a real incident?
Imagine a South African development team adding a reporting feature to an internal application. A coding agent finds a reference to a specialist file format and recommends a library with a convincing name. It installs the dependency, writes the integration and reports that the tests have passed.
This is an illustrative scenario, not a reported Netsurit incident. Several outcomes are possible. A malicious install script might run immediately on a developer’s machine or build server. Alternatively, the package might execute harmful code only when the application imports it. Depending on the access available in that environment, it could attempt to read source files, credentials or configuration, or connect to an external service.
Real software supply chain incidents show why the installation step deserves care. Unit 42 has documented malicious package and build-tool activity involving install scripts and attempts to collect secrets. That research demonstrates the consequences of trusting hostile code in a development environment; it is separate from the USENIX study of invented AI package names.
The chain has several links: a model suggests a name, someone publishes code under it, an agent or developer retrieves it, and the code runs somewhere trusted. A team can interrupt that chain without giving up the coding agent. The key is to treat the agent’s recommendation as a proposal that still needs verification.
What should a developer check before installing a package?
Start with the exact package name in the expected registry. Then find the project’s own documentation and confirm that it links to that same package. A similar name or a polished registry description is insufficient.
Look at the publisher and project history. Who maintains the package? How long has it existed? Does its repository show meaningful development and releases? Has ownership changed recently? A brand-new registry entry claiming to represent a long-established tool calls for investigation.
Next, examine what will run. Install and post-install scripts may execute before a developer has used the package in their application. Review what the package brings in through its own dependencies and whether its behaviour fits its stated purpose. Test an unfamiliar package in a restricted environment before it reaches a workstation or build pipeline with valuable credentials.
Finally, record why the dependency was selected and which version was approved. Months later, the team should be able to explain why the application relies on that package. A software bill of materials helps identify what shipped, but it cannot establish that a malicious package was safe to install in the first place.
These checks matter especially when the agent has access to cloud credentials, customer repositories or a continuous integration pipeline. The same package can present very different risks depending on where it runs.
What authority should a coding agent have?
An agent assigned to a reporting feature needs enough access to do the job. It does not automatically need every repository the organisation owns, production credentials or permission to introduce new third-party code into a trusted build.
A project-specific workspace and limited credentials reduce the consequences if an agent makes a poor dependency choice. An approved package source, private mirror or review process can prevent a newly suggested public package from entering the build before somebody has checked it. Those controls also need upkeep: an approved package can change, and a previously trusted maintainer account can be compromised.
The review point should be clear. Let an agent draft code and run tests in a restricted environment. Require a separate decision when it wants to add an external dependency to the trusted build or deploy a change. If developers must approve every harmless command, they will start clicking through prompts without reading them. Put their attention where a decision has lasting consequences.
There is another route to the same installation problem. An attacker can place instructions in a README, issue description or web page that the agent reads while working. The instruction might tell the agent to install a particular package. OWASP advises treating such repository and external content as untrusted input when an AI coding agent processes it. A hallucinated package name and a malicious instruction are different mechanisms, but either can lead an agent to fetch unwanted code.
This is one reason the security of an agent depends on more than the model. Its tools, permissions, sources of information and execution environment determine what happens after a questionable suggestion. Netsurit’s article on AI agent platform decisions explores how those architectural choices affect an organisation long after the first experiment.
Will these checks slow developers down?
Some checks take time, and that time should be spent on the decisions that carry real risk. Coding agents reduce the effort of producing a feature. They do not remove the need to understand a dependency that the organisation may support for years.
The practical answer is to gather routine evidence automatically. A pipeline can flag a package that is new to the organisation, highlight install scripts, show publisher changes and compare the dependency against approved sources. A developer or security reviewer can then concentrate on the few decisions that require judgement.
South African teams do not need a special version of this threat for it to matter locally. They use the same public registries, development tools and cloud build processes as teams elsewhere. A malicious dependency introduced during development can affect an application wherever its users and data are located.
The question I would ask of any team adopting coding agents is simple: who verifies the external code the agent chooses, and at what point does that verification happen? If the answer is “when someone notices it in the pull request”, the review may already be too late for a package that executed during installation.
Our whitepaper, AI Code You Can Trust, is a useful next read for teams examining how AI-assisted development fits into their software security practices.
Frequently asked questions
Can AI coding tools really invent package names?
Yes. Published research has observed coding models recommending names that were absent from the relevant package registry. A credible-looking name does not establish that a package exists or comes from a trustworthy publisher.
Is every new package suspicious?
No. New packages can be legitimate. Their limited history does mean a team has less evidence to assess, so the publisher, code, scripts and reason for using the package deserve closer review.
Does pinning a dependency solve the problem?
Pinning helps control which version is installed later. It does not make the first version chosen by the agent safe. Verify the package before its initial installation and continue monitoring it afterward.
Can an agent install packages safely in a sandbox?
A restricted sandbox can limit the impact of an installation if its file access, credentials and network destinations are controlled. The dependency still needs a separate review before it enters a trusted build or production environment.
What is the most useful rule for a development team?
Make every new external dependency an explicit review event. Support that review with registry and publisher checks, automated evidence gathering and a clear approval path.
The decision hidden inside the suggestion
An AI agent can find a useful library in seconds. By naming it, the agent has not established a chain of trust from the publisher to your application. Somebody still needs to decide whether that package belongs in your development environment and, eventually, in the software your organisation runs.
Netsurit helps organisations bring security into their technology decisions and operations. If your team is introducing coding agents, speak to Netsurit about your development environment and security controls before a package recommendation becomes an unexamined installation.
