Prove Security: Secure Code, Supply Chain and the Toolchain as Attack Surface

This chapter treats secure development when AI takes part in it: what studies measured about the security of generated code, how dependencies, secrets and licences behave when a tool proposes them, which classes of attack reach a coding assistant or agent, why the toolchain is itself a Tier 0 asset, and how a provenance record lets an owner show where code and dependencies came from. It concerns D4 Prove, because the question is what evidence shows that a change is safe to run, and D5 Steward, because the owner who answers for the change in production must be able to name what it contains and stop it from outside the pipeline that shipped it. Chapter 9 treated the general case of proving a change with an oracle independent of the generator. This chapter takes the part of proof that concerns an adversary: someone who writes the text the tool reads, publishes the package the tool suggests, or steals the credential the tool can reach. It is defensive throughout. Attacks are described by class and technique at the level needed to design controls and tests, and none is described in a form that could be repeated.

Figure 10.1: The toolchain trust boundary. Untrusted input reaches the agent only through policy points that sit outside it, the pull request passes gates that the change cannot alter, and a stop path outside the pipeline is held by named owners.

The argument has four steps. Generated code is untrusted input and is proved like any other. The supply chain gains an entry point in the tool's suggestions, so the control moves from the developer's judgement to a proxy that every request must pass. The tools read text that adversaries can write and hold credentials that adversaries want, so controls outside the model contain them. And the evidence is a record, kept with the code, that a supervisor or an incident responder can read. The conclusions are the chapter's own and are collected in the Summary.

What studies measured

Pearce and colleagues, in 2021, asked GitHub Copilot to complete code in 89 scenarios and produced 1,689 programs; they found approximately 40 per cent of them to be vulnerable (228, Pearce et al., 2021). The study measured a 2021 version of one tool on scenarios its authors chose; it says nothing about the tools of 2030 and does not compare the output with code written by people.

Two user studies compared people with and without an assistant. Perry and colleagues reported that participants with access to an AI assistant, based on the model codex-davinci-002, wrote significantly less secure code than those without access, and were more likely to believe that they had written secure code; participants who trusted the assistant less, and who engaged more with the wording and format of their prompts, produced code with fewer vulnerabilities (34, Perry et al., 2023). Sandoval and colleagues ran a study with 58 participants who implemented a singly linked "shopping list" structure in C, and reported that those with an assistant produced critical security bugs at a rate no greater than 10 per cent more than the control (235, Sandoval et al., 2023). The studies differ in task, language and population, and both used tools current in 2022. The reading is that the effect depends on the task and on how the person uses the tool, and that confidence is not evidence of security.

Veracode reported in 2025 that code from over 100 large language models, in Java, JavaScript, Python and C#, introduced risky security flaws in 45 per cent of its tests (229, Veracode, 2025). It is vendor research with the vendor's own measure: a signal that the finding has not disappeared with newer models, not a rate to apply to one's own repositories.

Three limits apply to all four sources. They measure code at the moment of generation, not what survives review and scanning in a pipeline. They judge vulnerabilities by the study's standard, not by exploitability in a particular service. And their evidence expires: a result for a tool of 2021 or 2022 can be neither carried over to the tools of 2030 nor dismissed, because no study of the tools in use at Nordvik Region or Hansa Mutual exists. An organisation that wants a number for its own code measures it.

Measuring your own code

The method fits in a quarter. Take merged changes from a defined period, classify each by whether an accepted suggestion or an agent's pull request contributed (the provenance record below supplies the classification), run the same static analysis rules over both groups, and report counts as well as rates, by class of code, with the period, the rule set and the date. The result is a claim with scope, method, date and outcome, and it expires like the adoption claims of Chapter 20.

Hansa Mutual has the beginning of such a measurement. The sample of 40 merged pull requests of August 2030 found 27 per cent of lines in the claims product line traceable to an accepted suggestion (AVR-HM-2030-01). That measured traceability, not security. The extension is to run the group's static analysis rules over the same sample and over a comparison sample of equal size, by service class, and to record what was found and when. Until then the group has no evidence about the security of its generated code, and no one should say that it does.

Dependencies and package hallucination

A coding tool proposes imports. Some of the names it proposes do not exist. Spracklen and colleagues analysed 576,000 code samples generated in two programming languages from two prompt datasets by 16 code-generating models, and found that on average at least 5.2 per cent of the packages suggested by commercial models and at least 21.7 per cent of those suggested by open-source models were hallucinated, with 205,474 unique hallucinated package names (234, Spracklen et al., 2025). The authors treat this as a threat to the software supply chain. The mechanism is the one that follows: if a name that a tool proposes is unregistered, anyone can register it, and a developer or an agent that installs it without checking has put the registrant's code into the build.

The class has neighbours, and the controls are the same for all: a name that does not exist (hallucination), one close to a real name (typosquatting), a private name that a public registry can shadow (dependency confusion), a real package whose maintainer's account is taken over, and a real package whose new version is malicious. All end with code from an unreviewed source running in the build or in production. The defence does not depend on spotting the tool's mistake. It depends on the path a name must travel before it is installed.

OWASP's list of risks for LLM applications names this area LLM03 Supply Chain, and LLM06 Excessive Agency for an agent with the power to install and run things (230, OWASP, 2025). AI Platform and Security, Chapter 16, treated models and datasets as supply-chain components with verifiable provenance (163, Nordqvist, 2026); this chapter treats the dependencies that the generated code itself pulls in.

Controls

Five controls hold the line, and each sits outside the tool.

1. A proxy that serves only approved registries and approved names. Developers, agents and the pipeline resolve packages only through it, and direct egress to public registries is denied. Both organisations have such a proxy: Nordvik Region's platform team runs a package proxy that serves only approved registries, and Hansa Forge runs one for the group.
2. A human approval for every new dependency. A name that has not been in the repository before is a decision, with a named approver, a reason and a record. An update to an existing, approved package at patch level is a different class, treated below.
3. A lock file with hashes, and builds that refuse to run without it. The build installs what the lock file says and fails if a resolved package differs.
4. A waiting period for new versions of existing packages. A version published hours ago has had no time for the community, a scanner or a registry to find a problem. The setting is the book's own recommendation, not a measured optimum; the seven days in the example below are illustrative and tuned against patch velocity.
5. A denial that is an event. When the proxy refuses a name, the refusal is logged with the run that asked and goes to a named owner.

The fifth control is the lesson of an episode at Hansa Mutual. In March 2030 a coding agent in the Mode 3 pilot proposed a dependency whose name did not exist in the registry, and the Hansa Forge package proxy refused it (FND-HM-2030-0012, illustrative). The control worked. A review of agent runs then found that two further hallucinated names had been proposed and rejected in the previous month, and no one had been told. The proxy had been preventing harm and producing evidence, and the evidence had no reader. The remedy is not a better proxy. It is a route from each denial to an owner who counts them, looks at the names, and tells the team that owns the agent's mandate when the rate changes. A rising count is a finding about the agent's prompts, context or model, and a name that appears repeatedly is a name that someone may register.

The Level 3 Bounded dependency class

Hansa Mutual's delegation profile has one Level 3 Bounded class: patch-level dependency updates in repositories outside the security-critical and payment paths, when all pipeline checks pass and the update is on the approved list, merged by the pipeline after a human approval of the class, not of each update (MND-HM-DEV-01, DLG-HM-DEV-01). Chapter 8 treats the mandate. This chapter states the conditions on which the class is safe, because it is where a compromised maintainer's account would reach production fastest. The proxy admits only approved names; the waiting period applies; the checks include the independent tests of Chapter 9 and a scan of the new version against vulnerability and malware feeds; and the class approval carries a stop condition, written as a number, that moves the class to Level 2 Supervised when crossed. Without them the class is a gap in the gate. With them it is a measured delegation, and AI Operations, Chapter 4, gave the form: errors consume a budget (170, Nordqvist, 2026).

Secrets and licences

Two kinds of leakage follow from putting a tool between a developer and the code: secrets leave the organisation in prompts, and secrets enter repositories in generated code. Licences are the third matter, and a smaller one for this chapter.

Secrets in prompts and in code

In March 2030 a consultant on Nordvik Region's case platform pasted a connection string for a test database into an unapproved public chat assistant while asking for help with a SQL view (INC-NR-2030-0071, illustrative). The platform team rotated the credential within 40 minutes, and no data access from outside was found. The framework agreement NR-2025-102, under which about 22 consultant developers work inside the Region's teams, had no clause on AI tools. The route was a Mode 1 assistant outside the approved list, so no gateway saw the prompt; the person was a consultant, so the Region's rules did not bind him by contract; and recovery depended on someone acting within the hour, which is a good result and not a control.

The controls address the three. A secret filter at the AI gateway catches strings that match credential patterns before a prompt leaves, which works only for tools behind the gateway; the Region's Mode 2 pilot NR-DEV-001 calls its self-hosted model directly and has no gateway, so it has neither the filter nor the log. A pre-commit and pipeline secret scan catches the other direction, a credential that a person or a tool has written into a file. Short-lived credentials, issued per task and per environment, limit what a leaked string is worth; a test database connection string that expires in an hour is a smaller incident than one that lasts for a year. And a clause in the supplier agreement, with a right to audit, binds consultants and suppliers to the approved tools.

GitGuardian reported that in 2024 it detected 23,770,171 new hardcoded secrets in public GitHub repositories, and that public repositories using GitHub Copilot had a leakage rate of 6.4 per cent against a baseline in which at least 4.61 per cent of the 69.6 million scanned public repositories contained a secret (237, GitGuardian, 2025). The page gives no selection method for the Copilot subset and no sample size, and the source sells secret-detection products. The figures show that leakage into public repositories is common and that Copilot-using repositories had a higher rate in that data; they do not show that the tool caused the difference. The reading that survives is sufficient for design: secrets reach repositories in volume, and a scan on every push is the control, whichever tool wrote the line.

Prompts are also records. Whether prompts to coding tools are records under Nordvik Region's public-access and archives rules is an open question for the Region's archivist and legal counsel. A prompt log is both a security control, because it shows what left the organisation, and a record with retention, access and personal-data duties and a works-council interest where it identifies the person who typed; Chapter 22 returns to the last point.

Licences

Chapter 26 treats intellectual property, including contamination and its detection. Here it is enough to fix where the facts are kept: the provenance record has a licence field for every dependency, and a change that adds a licence outside the approved list stops at the same gate as a vulnerable one. The record does not assert that a snippet is free of licensed code, because no pipeline check can establish that with certainty, and a record that claimed to would be worse than one that said "not asserted".

Prompt injection and excessive agency

A coding assistant reads text, and some of the text is written by people who are not the developer. An issue in a tracker, the README of a dependency, a web page the agent fetched, a comment in a file, the description a tool server gives of its own tools, a rules file in a repository that a contractor controls: all of it can contain text that reads as an instruction to the model, and a model has no reliable way to tell an instruction from the developer from an instruction inside the data. Greshake and colleagues described this class as indirect prompt injection, in which instructions placed in data that the model is likely to retrieve compromise an LLM-integrated application without direct access to it, and demonstrated it against systems that included code-completion engines (236, Greshake et al., 2023). OWASP lists it as LLM01 Prompt Injection (230, OWASP, 2025), and MITRE ATLAS, the public knowledge base of adversary tactics, techniques and procedures targeting AI systems, is the shared vocabulary for the techniques, in the way MITRE ATT&CK is for conventional intrusions (25, MITRE, n.d.).

The series concluded that prompt injection is a property of the architecture and not a patchable defect, so controls sit outside the model (AI Platform and Security, Chapter 16 (163, Nordqvist, 2026), and AI Red Team, Chapter 13 (164, Nordqvist, 2026)). AI Blue, Chapter 15, stated the consequence for the defenders' own agents, that the controls bound suppression, action and disclosure but not misdirection (168, Nordqvist, 2026). What such controls cannot bound is a suggestion made subtly wrong by what the model read, and there review and the independent oracle of Chapter 9 remain necessary.

What the agent can be made to do is what its permissions allow

The measure of the risk is the power the agent holds when it reads the wrong text. An injected instruction to an assistant that can only suggest text in an editor costs the developer a bad suggestion. The same instruction to an agent that can run commands, read the environment, call the network and open pull requests can cost the organisation a credential, a dependency or a change to a gate. OWASP calls the second case Excessive Agency (LLM06). The reported Replit incident, in which a coding agent reportedly executed destructive commands during a code freeze, illustrates the class (33, AI Incident Database, 2025); the catalogue record lists press reports and no primary post-incident report, so no detail is drawn from it.

The design response is a list of limits, none of which depends on the model behaving well.

• Classify inputs by who can write them. A repository file written by the team is different from an issue anyone can open, and different again from a page fetched from the open web. The context an agent receives carries that label, and the policy that governs its actions can differ by label.
• Give the agent no secrets that it does not need for the task. Task-scoped, short-lived credentials, issued outside the agent's runtime, and no ambient access to the developer's environment variables or credential stores.
• Deny egress by default. An agent that cannot reach arbitrary hosts cannot send a file to one. The proxy and the gateway are the only routes.
• Separate reading from acting. An action that leaves the sandbox, installs a new dependency, changes a workflow or touches a credential needs an approval from a named person, and the approval screen shows the action, not the model's explanation of it.
• Keep the gate out of reach. The agent cannot change its own mandate, permissions, credentials, test oracles, review rules or pipeline gates. This is one of the Level 4 prohibitions of the delegation ladder, and the controls above are what make the prohibition real: a rule that depends on the agent's cooperation is not a rule.

Testing the limits

An untested limit is declared, not verified, in the vocabulary of AI Threat, Chapter 30, where a control is a defence only when a recent test has shown that it holds (165, Nordqvist, 2026). The test is an exercise under written rules of engagement (RoE-*), run against the organisation's own repositories and sandboxes, with a scope, a stop path outside the agent and a named white-team owner. A tester places benign, marked test content in a source the agent reads, in a repository created for the purpose; the question is whether the agent stayed inside its mandate and the policy point refused what fell outside. The result is a record: what was planted, where, at what input label, what the agent did, which control acted, and whether a re-test passed. A finding closes when the re-test passes and not before (AI Blue, Chapter 11). The record names the technique by its ATLAS identifier, the class of source and the outcome, and holds no injection text. It ends in a defensive improvement: a control that held, with the date, or a gap and the pull request that closes it.

The toolchain as Tier 0

The toolchain is the set of systems that turn a developer's intent into running software: the editor and its extensions, the agent and its sandbox, the gateway, the package proxy, the repository platform and its workflows, the build service, the signing keys and the release identities. AI Operations, Chapter 18, concluded that the operator's own AI is part of the estate it operates and a Tier 0 asset, with the broadest credentials and the best map of the organisation (170, Nordqvist, 2026), and AI Blue, Chapter 15, reached the same conclusion for the security operations centre. The developer's toolchain belongs in the class, with one difference: code is made there, so a compromise reaches every system the code reaches.

An incident of August 2025 shows the shape. The Nx project's advisory records that a pull-request-title validation workflow interpolated the title into a shell command and ran with elevated permissions; that an attacker used the weakness to reach the release pipeline, which held the npm publishing token; and that malicious versions of nx and several related packages were published on 26 August 2025, and the nx versions were removed from npm the same evening (31, Nx project (nrwl), 2025). A post-install script scanned the file system for files and credentials and posted the encoded results to a new repository under the victim's own account. The advisory also reproduces, in an appendix, a prompt inside the malicious script that instructs a file-search agent to list the paths of configuration and environment files. Two of the advisory's points matter here. The entry point was a workflow that ran with write access on content an outsider controlled, a weakness that has nothing to do with AI; and the editor extension that checked the version installed the latest one, so opening an editor could trigger the code. The book's reading is that a developer's machine with credentials, installed command-line tools and an agent that can run them has a larger blast radius than one without, which is a reason to scope what the agent can read and not a reason to forbid the tools.

Identities and keys

The design question is which identity can do what, written down. Four kinds matter.

• The developer's identity. It reviews and approves, and it should not hold publishing or signing rights as a matter of course.
• The agent's identity. It is separate from the developer's, scoped to a task and a repository, short-lived, and without publishing, signing or workflow-editing rights.
• The build identity. It runs in the hosted build service, can read the repository and write an artefact, and is the only identity that can sign provenance.
• The release identity. It promotes a signed artefact into an environment, and the pipeline holds it, not a person's laptop and not an agent.

No one identity combines authoring, approving and releasing. The signing keys and the gate configuration sit where the agent and the change cannot reach them, and a change to a workflow, a policy file, a code-ownership rule or a test oracle is routed to a named code owner by the code-ownership rule itself.

The stop path

The stop path must work when the pipeline does not. Named owners can revoke the agent's credentials, disable the gateway route for coding tools, freeze the release identity and block a package at the proxy, through a mechanism outside the pipeline and the agent. AI Operations, Chapter 17, set the tests: independence, dependence and stop (170, Nordqvist, 2026). A stop path never drilled is declared, not verified.

Provenance

Provenance is the record of where a piece of software came from: who or what wrote it, which dependencies it contains, how it was built, and what checked it. The record of a change that AI took part in is the evidence for the stewardship tests of Chapter 4. Name is answered by the owner field. Understand is supported by the list of what the change contains. Revert is supported by the build record that says which artefact to restore. The family of records is PRV-*, and it is a record in version control beside the code, generated by the pipeline wherever possible and completed by a person where not.

What a provenance record contains

A provenance record holds seven things: the subject (service, change record, pull request); the origin of the code (whether an assistant or an agent contributed, under which mandate and run record, with which model, hosted where, and the basis of the claim, because a commit tag is a declaration and a gateway log is a record; where nothing recorded the origin the field says "not asserted"); the dependencies (name, version, registry, who added it, the proxy's decision, licence, hash); the checks (secret, static, dependency and licence scans, the independent oracle VER-* and the review record RVW-*, each with result and date); the build (hosted service, commit, SLSA build level); the people (the owner OWN-* and the named approvers); and a reference to the bill of materials.

The bill of materials is a separate machine-readable file; the record points to it. CycloneDX, standardised as ECMA-424, catalogues software and hardware parts, services, dependencies, security flaws, cryptographic assets and machine-learning models (238, Ecma, 2025), so one file can describe a service's dependencies and the model it calls.

The levels that SLSA defines

SLSA, the Supply-chain Levels for Software Artifacts specification, defines a build track with levels. At Build L1 provenance exists: a consistent build process and provenance describing how the artefact was built, distributed to consumers; the specification says that level is trivial to bypass or forge and protects against mistakes. At Build L2 builds run on a hosted build platform, not an individual's workstation, with provenance tied to that platform by a digital signature that downstream verification must check. At Build L3 builds are hardened, so that runs cannot influence one another and the provenance signing secret is not accessible to user-defined build steps (231, OpenSSF, n.d.). The version read here is 1.0, and the site names a later version as current, so the citing record states the version it targets. The series claims for SLSA only a graded vocabulary for the statement that an artefact was built by this platform from this source. It does not claim that code built at L3 is secure.

NIST's Secure Software Development Framework, Special Publication 800-218, sets out recommended secure development practices (232, Souppaya et al., 2022), and the community profile in Special Publication 800-218A adds practices for developing generative AI and dual-use foundation models (233, Booth et al., 2024). The first is the baseline for the code the organisation writes. The second addresses building models, the subject of AI Platform and Security, not the use of a coding assistant.

Where the law asks for it

The Cyber Resilience Act, Regulation (EU) 2024/2847, requires in Annex I, Part II, point (1), that manufacturers of products with digital elements identify and document vulnerabilities and components contained in the products, including by drawing up a software bill of materials in a commonly used and machine-readable format covering at the very least the top-level dependencies. Article 13(5) requires manufacturers to exercise due diligence when integrating components sourced from third parties so that those components do not compromise the cybersecurity of the product, including free and open-source components that were not made available on the market in the course of a commercial activity (242, European Union, 2024). The Regulation applies from 11 December 2027, and Article 14 on reporting from 11 September 2026, as its final provisions state. Whether Hansa Mutual's mobile app and broker plug-in are products with digital elements, whether the group is a manufacturer and how the Region's published components stand are questions for Chapter 24. The expected consequence is observable in the direction of the instrument: where an organisation is a manufacturer, a machine-readable bill of materials and a due-diligence trail for components will be required, and a team that already keeps a provenance record for every change has the trail.

NIS 2, Directive (EU) 2022/2555, requires in Article 21(2) that the measures of essential and important entities include, among others, supply chain security, including security-related aspects concerning the relationships between each entity and its direct suppliers or service providers (point (d)), and security in network and information systems acquisition, development and maintenance, including vulnerability handling and disclosure (point (e)) (3, European Union, 2022). The text speaks of measures, not tools. The implication is that the proxy, the scans, the record and the stop path are evidence a supervisor can ask for under those points; how point (e) reaches a nearshore partner's developers is a question for Chapter 24, and scope depends on national transposition, for Nordvik Region the Swedish Cybersäkerhetslag (2025:1506) (11, Sveriges riksdag & Regeringskansliet, 2025).

A provenance record

The following record is illustrative. It shows a change to the customer portal and broker interface SVC-HM-005 opened by a coding agent under MND-HM-DEV-02, with one new dependency and one patch update, approved by two people. Every identifier and value is fictional.

provenance_record: PRV-HM-2030-0014
status: illustrative
subject:
  service: SVC-HM-005
  change_record: CHG-HM-2030-0388
  pull_request: claims-portal#2187
origin:
  contribution: agent_opened_pull_request
  mandate: MND-HM-DEV-02
  run_record: RUN-HM-2030-0231
  model: {name: hansa-code-m, version: "2030-09", host: frankfurt, route: hansa-forge-gateway}
  claim_basis: gateway_log            # a commit tag alone would be "declared"
  human_authored_share: not_asserted
dependencies:
  - {name: statement-pdf-lib, version: "3.2.0", change: added, registry: hansa-forge-mirror,
     proxy_decision: allowed_after_human_approval, licence: Apache-2.0, sha256: "<from-lockfile>"}
  - {name: json-schema-check, version: "5.1.4", change: patch_update, registry: hansa-forge-mirror,
     proxy_decision: allowed_class_approval, licence: MIT, age_days_at_merge: 9}
checks:
  secret_scan: {result: pass, date: 2030-11-04}
  static_analysis: {ruleset: group-baseline-2030-q4, result: pass, date: 2030-11-04}
  dependency_and_licence_scan: {result: pass, date: 2030-11-04}
  independent_oracle: {record: VER-HM-2030-0044, written_by: test_engineer_not_author_of_change}
  review: {record: RVW-HM-2030-0388, human_approvals: 2, ai_comments_counted: 0}
build:
  service: hansa-forge-build
  commit: "<sha>"
  slsa_build_level_target: "L2 (specification v1.0)"
  provenance_signed_by: build_identity    # not reachable from the agent or the developer
  bill_of_materials: "sbom/SVC-HM-005-3.14.0.cdx.json"
people:
  owner: OWN-HM-2030-0031
  approvers: [claims_portal_tech_lead, test_engineer]
revert:
  previous_artefact: "SVC-HM-005-3.13.2"
  who_can_revert: [claims_portal_team, hansa_forge_release_owner]

The record makes a claim about origin only as far as a log supports it, says nothing about the share of code a human wrote, and leaves the AI's comments out of the count of approvals. A reader can ask any of the three stewardship questions of it. The independent oracle is a record written by someone other than the author of the change, and the record does not treat a model's own tests as one.

Controls in practice

This section turns the chapter into one practical guide, the controls and degrees of delegation by mode and class, and the position of the two organisations.

Practical guide: Writing the controls for a coding agent

Purpose. Establish, before a coding agent receives a mandate and when it changes, the controls that keep untrusted input, packages and secrets from making the agent a route for an adversary, and the evidence that they work.

Roles. The tech lead who owns the mandate, the platform engineer who runs the proxy, the gateway and the build service, the security architect or the person who holds that duty, a tester or quality engineer who owns the oracle, the data protection officer where prompts may contain personal data, and a works-council or staff representative where logs identify people.

Inputs. The draft mandate manifest (Chapter 8), the delegation profile, the list of repositories and their classification, the list of registries, the identity inventory of the toolchain, and the last drill record.

Steps.

1. List what the agent reads and label each source by who can write it: the team, the organisation, a supplier, anyone.
2. List what the agent can do and strike everything the task does not need.
3. Issue a separate identity with task-scoped, short-lived credentials from outside its runtime, and remove ambient credentials from the sandbox.
4. Route packages through the proxy, deny direct egress, set the waiting period and the rule that a new name needs a named approver.
5. Put the secret filter and the log at the gateway, and agree with the data protection officer and the representatives how long logs are kept, who reads them and how they are reported.
6. Place the gates, signing keys, code-ownership rule and mandate beyond the agent's reach.
7. Write the stop path: who can revoke, freeze and block, how they are reached out of hours, and when it is drilled.
8. Write the exercise under rules of engagement that tests the limits by class, with a re-test date, and add the toolchain section to the mandate manifest.

Artefact. The toolchain section of the mandate manifest. For MND-HM-DEV-02 it could read as follows (illustrative).

mandate: MND-HM-DEV-02
section: toolchain_controls
identity:
  agent_identity: separate_from_developer
  credentials: {scope: task_and_repository, lifetime_minutes: 60, issued_by: hansa-forge, in_runtime: false}
  forbidden_rights: [publish, sign, edit_workflow, edit_policy, edit_mandate, edit_test_oracle]
inputs:
  labels: {team_written: trusted_as_data, supplier_written: untrusted, public_issue_or_web: untrusted}
  untrusted_input_may_trigger: [read, propose]   # never run, install or send
packages:
  resolve_only_via: hansa-forge-mirror
  direct_egress: deny
  new_name: {human_approval: required, approver_role: tech_lead}
  new_version_min_age_days: 7                    # illustrative; tuned against patch velocity
  on_unknown_name: {action: deny, notify: [mandate_owner, hansa_forge_platform]}
secrets:
  gateway_filter: enabled
  pre_commit_and_pipeline_scan: required
actions_needing_named_approval: [install_new_dependency, change_workflow, call_new_host, touch_credential]
stop_path:
  holders: [mandate_owner, hansa_forge_release_owner, security_on_call]
  mechanism: outside_pipeline
  last_drill: 2030-10-14
  next_drill_due: 2031-01-12
exercise: {rules_of_engagement: RoE-HM-2030-DEV-01, retest_due: 2031-01-31}

Across the usage modes

The risk in this chapter follows the usage mode more than the model, as AI Red Team, Chapter 6, concluded: in Modes 1 to 3 the tool acts with a person's identity and reach, and in Mode 5 it would act with an identity of its own (164, Nordqvist, 2026). The table sets out the main security exposure for each mode of a coding tool and the control that answers it.

Mode	Coding example	Main security exposure	Control that sits outside the tool
Mode 0 Ambient AI	Line completion in the IDE; a supplier's own code assistance in managed tooling	Suggested imports and weak patterns accepted without a decision; origin unrecorded; supplier tooling invisible	Proxy and lock file; pre-commit scan; provenance field "not asserted"; contract right to know
Mode 1 Conversational assistants	Chat in a browser or approved enterprise tier	Secrets and code pasted out of the organisation (INC-NR-2030-0071)	Approved list; gateway filter where routed; supplier clause; short-lived credentials
Mode 2 Embedded workplace assistants	IDE assistants NR-DEV-001, HM-DEV-001	Context sent to a model; suggestions that add dependencies; no gateway where the model is called directly	Route through the gateway; telemetry by team; proxy; review
Mode 3 Agentic work tools	Agents that edit files, run commands, install packages, open pull requests	Reading untrusted text with real permissions; excessive agency; installing a hallucinated name	Separate identity; egress deny; named approval for outward actions; stop path
Mode 4 Built AI applications	HM-DEV-002 test-drafting assistant	Tests that share the generator's blind spots; context drawn from repositories	Oracle written by someone other than the change's author; mutation checks
Mode 5 Autonomous production agents	None for coding in either organisation	Would act with its own identity on production	Not used; the rule that no coding agent merges or deploys on its own stands

Across the degrees of delegation

The degrees of delegation set the ceilings. The table below gives the ceilings of DLG-NR-DEV-01 and DLG-HM-DEV-01 for the classes that touch security, and the condition that makes the ceiling safe. The concrete profile governs; a ceiling is not a licence.

Work class	Nordvik Region (DLG-NR-DEV-01)	Hansa Mutual (DLG-HM-DEV-01)	Security condition
Generate	Level 1 Assisted	Level 2 Supervised outside the security-critical and payment paths; Level 1 Assisted inside them	Named developer reviews code in authentication, authorisation, cryptography and payment; scans pass
Refactor	Level 1 Assisted	Level 2 Supervised	Independent oracle; no change to gates in the same pull request
Test	Level 1 Assisted for drafts, never the oracle for the same change	Level 1 Assisted for drafts; oracle by someone other than the author	Oracle independent of the generator
Review	Level 0 Advisory	Level 1 Assisted; no AI comment counts as a human approval	A finding from an AI reviewer is a lead, not a clearance
Deploy	Level 0	Level 0	The pipeline deploys; signing and release identities are out of reach of any tool
Dependency update, patch level	Not delegated	Level 3 Bounded, outside the security-critical and payment paths, by class approval (MND-HM-DEV-01)	Approved names only, waiting period, scans, stop condition
Change to the agent's mandate, permissions, credentials, test oracles, review rules or pipeline gates	Not delegated	Not delegated	Level 4 is never used; a named owner changes them

Level 4 Autonomous is not used for any of these. The controls above are how its prohibitions (production without an unalterable gate, security-critical code without a named reviewer, the agent's own mandate, permissions, credentials, oracles, review rules and gates) are enforced and not merely stated.

Nordvik Region and Hansa Mutual

Nordvik Region has about 25 people who write code as their main work, about 22 consultant developers inside its teams, supplier teams whose code it does not review, and 96 repositories of which 31 per cent have an owner who has left or changed team (OWN-NR-2030-00). The platform team runs the package proxy and the two golden paths; coding tools are not behind the AI gateway, and the Mode 2 pilot calls the self-hosted model directly. The secret in the prompt is the Region's lesson, and its follow-up is four decisions for the CIO and the head of Applications and Integration, each to be recorded as an AI Decision Record: to route the pilot through the gateway before any extension beyond 31 December 2030; to propose an AI-tools clause for the next call-off under NR-2025-102; to add a secret scan to both golden paths; and to start the provenance record for the Region's published e-service components, where the open licence makes a bill of materials useful to the public as well as to the Region. The union branch's IT representative has asked to negotiate before the pilot is extended, and a prompt log that identifies a person is the kind of tool MBL-NR-IT-2027-01 asks the Region to negotiate before switching on; MBA-NR-2030-03 is the place. The Region has no Mode 5 agent, so its main exposure is Modes 1 and 2 and the supplier's code.

Hansa Mutual has about 290 developers, about 190 external developers, 62 delivery teams, 410 services and about 120 engineers using agentic tools through the Hansa Forge gateway, in a sandbox, with pull requests and no merge to the protected branch. The package that did not exist is the group's lesson: the proxy held and the denial had no reader. The follow-up is a route from every denial to the mandate owner, Stefan Brückner for MND-HM-DEV-02, and to Hansa Forge under Lukas Frey, with a monthly count; the toolchain section above; and the extension of the August 2030 sample to a security comparison. The Level 3 Bounded dependency class stands only on the conditions stated, and a rise in denials or a failed seeded test moves it to Level 2 Supervised. Alster Managed Services' own code assistance for the mainframe batch is a Mode 0 blind spot, and the provenance field for such changes says "not asserted" until the contract grants the right to know. Where the group is a manufacturer under the Cyber Resilience Act, the record is the start of the due-diligence trail that Article 13(5) asks for (Chapter 24). Under the draft annex KBV-IT-2024-DEV-01 telemetry is reported by team and never per named developer, and the same rule applies to the prompt log and to denial counts.

Summary

This chapter concerned D4 Prove and D5 Steward. The studies of AI-generated code measured particular tools in 2021 and 2022 on particular tasks and disagree: roughly 40 per cent of 1,689 programs vulnerable in one, less secure code with more confidence in a user study, and no more than 10 per cent more critical bugs in another, with a vendor's 2025 report of risky flaws in 45 per cent of its tests. The book's own conclusion is that generated code is untrusted input, that the effect depends on the task and on how the person uses the tool, and that an organisation that wants a number for its own code measures it, with a method, a date and a result.

The supply chain gains an entry point in the tool's proposals. The defence is the path a name must travel: a proxy of approved registries, a named approval for every new dependency, a lock file, a waiting period and a denial that reaches an owner. Secrets leave in prompts and enter in code, and the controls are a gateway filter, a scan on every push, short-lived credentials and a supplier clause. Prompt injection is architectural, so controls sit outside the model, permissions set the cost of being steered, and exercises under rules of engagement test the limits and end in a re-test. The toolchain is a Tier 0 asset whose identities, keys, workflows and stop path are kept apart from the agent and from the change, and PRV-* records give the owner what the stewardship tests need, claiming origin only as far as a log supports. The Cyber Resilience Act (Annex I Part II(1), Article 13(5)) and NIS 2 (Article 21(2)(d) and (e)) ask for the same content in other words, with scope left to Chapter 24.

Observable: package hallucination, secrets in public repositories and compromise of build pipelines are documented now. Expected: the Cyber Resilience Act applies from 11 December 2027, and supervisors will ask for bills of materials and due-diligence trails. Speculation: that later generations of tools remove these classes unaided; no evidence supports it, and the author's projections are kept to Chapters 29 and 33. Chapter 11 turns from proof to release and asks which gate a change cannot alter and who can revert it outside the pipeline.

Sources

1. Spracklen, J., Wijewickrama, R., Sakib, A. H. M. N., Maiti, A., Viswanath, B. & Jadliwala, M. (2025). We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs. arXiv:2406.10279. https://arxiv.org/abs/2406.10279 (234, Spracklen et al., 2025)
2. Sandoval, G., Pearce, H., Nys, T., Karri, R., Garg, S. & Dolan-Gavitt, B. (2023). Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants. arXiv:2208.09727. https://arxiv.org/abs/2208.09727 (235, Sandoval et al., 2023)
3. Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T. & Fritz, M. (2023). Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv:2302.12173. https://arxiv.org/abs/2302.12173 (236, Greshake et al., 2023)
4. GitGuardian (2025). The State of Secrets Sprawl 2025. https://www.gitguardian.com/state-of-secrets-sprawl-report-2025 (237, GitGuardian, 2025)
5. Ecma International (2025). ECMA-424: CycloneDX Bill of materials specification, 2nd edition. https://ecma-international.org/publications-and-standards/standards/ecma-424/ (238, Ecma, 2025)
6. MITRE. MITRE ATLAS (Adversarial Threat Landscape for AI Systems): atlas-data repository README. https://github.com/mitre-atlas/atlas-data (25, MITRE, n.d.)
7. Nx project (nrwl) (2025). Security advisory GHSA-cxm3-wv7p-598c. https://github.com/nrwl/nx/security/advisories/GHSA-cxm3-wv7p-598c (31, Nx project (nrwl), 2025)
8. Perry, N., Srivastava, M., Kumar, D. & Boneh, D. (2023). Do Users Write More Insecure Code with AI Assistants? CCS '23; arXiv:2211.03622. (34, Perry et al., 2023)
9. AI Incident Database (2025). Incident 1152: LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze. (33, AI Incident Database, 2025)
10. Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B. & Karri, R. (2021). Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions. arXiv:2108.09293. (228, Pearce et al., 2021)
11. Veracode (2025). October 2025 Update: GenAI Code Security Report. (229, Veracode, 2025)
12. OWASP Gen AI Security Project (2025). OWASP Top 10 for LLM Applications 2025. (230, OWASP, 2025)
13. Open Source Security Foundation and Linux Foundation. SLSA specification. (231, OpenSSF, n.d.)
14. Souppaya, M., Scarfone, K. & Dodson, D. (2022). Secure Software Development Framework (SSDF) Version 1.1. NIST SP 800-218. (232, Souppaya et al., 2022)
15. Booth, H., Souppaya, M., Vassilev, A., Ogata, M., Stanley, M. & Scarfone, K. (2024). Secure Software Development Practices for Generative AI and Dual-Use Foundation Models: An SSDF Community Profile. NIST SP 800-218A. (233, Booth et al., 2024)
16. European Parliament and Council of the European Union. Regulation (EU) 2024/2847 (Cyber Resilience Act). (242, European Union, 2024)
17. European Parliament and Council of the European Union. Directive (EU) 2022/2555 (NIS 2 Directive). (3, European Union, 2022)
18. Sveriges riksdag, Regeringskansliet. Cybersäkerhetslag (2025:1506). (11, Sveriges riksdag & Regeringskansliet, 2025)
19. Nordqvist, G. (2026). AI Platform and Security. (163, Nordqvist, 2026)
20. Nordqvist, G. (2026). AI Red Team. (164, Nordqvist, 2026)
21. Nordqvist, G. (2026). AI Blue. (168, Nordqvist, 2026)
22. Nordqvist, G. (2026). AI Operations. (170, Nordqvist, 2026)
23. Nordqvist, G. (2026). AI Threat. (165, Nordqvist, 2026)