[FEAT] Resolve Key Vault-Backend Environment References and Update .env_example

[Victor Valbuena (ValbuenaVC)]

Description

This PR adds Key Vault-backed dotenv bootstrapping while preserving PyRIT's existing python-dotenv workflow and list-shaped env_akv_ref configuration. It also updates .env_example, which has drifted out of sync from the rest of the library.

To use the keyvault:

env_akv_ref:
  - https://my-vault.vault.azure.net/secrets/pyrit-env
env_akv_strict: true

Every URL in env_akv_ref is treated as a bootstrap dotenv document and loaded in list order. A bootstrap document can use native ${NAME} interpolation and complete-value references to scalar secrets in the same vault:

ENDPOINT="https://example.openai.azure.com/openai/v1"
MODEL="${PYRIT_MODEL}"
API_KEY="kv:https://my-vault.vault.azure.net/secrets/api-key"
PINNED_KEY="kv:https://my-vault.vault.azure.net/secrets/api-key/version-id"

kv: is canonical; akv:, azure_key_vault:, and env_akv_ref: remain compatibility aliases. Child-secret values are terminal and are not interpreted as further references.

Source Precedence

Sources load sequentially in this order:

1. Existing process environment
2. Key Vault bootstrap documents in env_akv_ref order
3. Explicit env_files in order, or the defaults:
   • ~/.pyrit/.env
   • ~/.pyrit/.env.local

Every source uses python-dotenv's standard override=True and ${NAME} interpolation behavior. Later assignments override earlier assignments, and interpolation follows assignment and file order. If no Key Vault or local files are configured, PyRIT continues with the existing process environment.

Key Vault child references are resolved only in bootstrap documents. A kv: value in a local file remains literal. Bootstrap references are fetched before later bootstrap documents or local files load, so a reference may be fetched even when a later source overrides it.

Loading intentionally retains historical non-transactional dotenv behavior. Each source updates os.environ as it loads; if a later source or child-secret lookup fails, earlier assignments remain.

Key Vault Validation

References must occupy the complete value and use a full HTTPS secret URL from the bootstrap document's vault. Versionless URLs retrieve the latest version; versioned URLs pin a version. Short names, cross-vault references, arbitrary hosts, credentials, ports, queries, fragments, and malformed secret paths are rejected before client creation.

Supported vault DNS suffixes are:

• .vault.azure.net
• .vault.azure.cn
• .vault.usgovcloudapi.net

env_akv_strict applies only to bootstrap documents:

• true rejects malformed or valueless entries before that document is loaded.
• false warns, removes invalid entries, and loads valid assignments.
• Empty assignments such as NAME= and child secrets containing an empty string remain valid.
• Local files retain python-dotenv's permissive parsing.

Key Vault clients use an asynchronous retry policy with up to three retries and exponential backoff. Bootstrap, authentication, authorization, transport, malformed-document, and child-secret failures raise KeyVaultInitializationException, preserve the original cause, and remain ValueError-compatible.

Bootstrap documents remain in memory and are never written to disk. Child secrets are not cached; each complete-value reference performs a Key Vault read.

.env_example Updates

.env_example has been updated. Keys referencing azure endpoints are now prefixed with AZURE across PyRIT. Additionally, redundant or stale entries have been removed. See #2229 (comment) for an example.

Tests and Documentation

Expanded setup coverage verifies list-shaped configuration, source precedence, assignment-order interpolation, strict and non-strict parsing, terminal child values, versioned URLs, same-vault restrictions, sovereign-cloud endpoints, pre-authentication host rejection, non-transactional failures, retry configuration, exception chaining, and async resource cleanup.

Updated .pyrit_conf_example and the getting-started guide to document the same behavior.

Local Validation

• Setup unit tests: 160 passed
• Ruff: passed

The full unit suite, integration tests, Playwright/E2E tests, documentation build, and JupyText checks were not rerun locally after this simplification.

[Copilot]

Pull request overview

Adds Azure Key Vault-backed environment bootstrapping with recursive reference resolution, precedence handling, warnings, and documentation.

Changes:

• Resolves env:, kv:, aliases, and escaped literals.
• Adds environment-source validation and AKV/local-file precedence.
• Expands tests and configuration documentation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
pyrit/setup/initialization.py	Implements AKV loading and reference resolution.
tests/unit/setup/test_initialization.py	Tests environment initialization behavior.
doc/getting_started/pyrit_conf.md	Documents loading precedence and AKV references.
.pyrit_conf_example	Updates example AKV configuration guidance.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

tests/unit/setup/test_initialization.py:380

• This patch target is no longer called by initialize_pyrit_async, leaving the output assertion dependent on any real default environment files. Patch _resolve_environment_files instead so unrelated local files cannot add output or trigger reference resolution.

    @mock.patch("pyrit.setup.initialization._load_environment_files", return_value=True)

pyrit/setup/initialization.py:537

• Direct callers using the former list-shaped env_akv_ref reach .strip() here and get AttributeError, rather than the deliberate ValueError used by ConfigurationLoader. Validate the runtime type before calling string methods so this public API rejects legacy values consistently.

        if not env_akv_ref.strip():

pyrit/setup/initialization.py:439

• This merge is case-sensitive even on Windows. For example, ambient Path=old plus a winning PATH=new leaves both keys, so env:Path returns the ambient exact match and violates the documented merged-source precedence. Normalize keys on Windows while applying values last.

    reference_environment = {**ambient_environment, **values}

tests/unit/setup/test_initialization.py:372

• initialize_pyrit_async no longer calls _load_environment_files, so this patch is inert and the test can read real ~/.pyrit files (and even resolve their Key Vault references). Patch the resolver now used by initialization to keep the unit test isolated.

This issue also appears on line 380 of the same file.

    @mock.patch("pyrit.setup.initialization._load_environment_files", return_value=True)

doc/getting_started/pyrit_conf.md:174

• This row contradicts both the implementation and the earlier AKV precedence section: when env_files is omitted, initialization loads both .env and .env.local after the bootstrap. Remove the claim that only .env.local is loaded.

| Omitted or `null` | Load default `~/.pyrit/.env` and `~/.pyrit/.env.local`, or only `.env.local` after an AKV root |

[hannahwestra25]

why do we have < > now ?

[Victor Valbuena (ValbuenaVC)]

It's how .env_example was before. Do you want to change it?

[hannahwestra25]

why is the default 500 ?

[Victor Valbuena (ValbuenaVC)]

It's somewhat arbitrary, since PyritException requires a status code and 500 seemed like a reasonable default (key vault not reachable for some reason). Do you think it should be changed and/or do you think KeyVaultInitializationException should exist?

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

.env_example:294

• This Azure OpenAI TTS base URL is missing the required /openai segment. OpenAITTSTarget._get_provider_examples() expects https://{resource}.openai.azure.com/openai/v1, so copying this example will fail endpoint validation or send requests to the wrong path.

AZURE_OPENAI_TTS_ENDPOINT2 = "<https://xxxxx.openai.azure.com/v1>"

pyrit/setup/initialization.py:208

• This still accepts malformed/nonexistent Azure vault hosts such as one-character names, leading/trailing hyphens, and consecutive hyphens. Azure vault names are 3–24 characters, start with a letter, end with an alphanumeric character, and cannot contain consecutive hyphens; without those checks, URLs that should fail preflight proceed to credential/client creation.

    valid_vault_name = 1 <= len(vault_name) <= 63 and all(
        char.isascii() and (char.isalnum() or char == "-") for char in vault_name
    )

.env_example:55

• Dotenv preserves the angle brackets inside these quoted values, so this becomes the literal endpoint <https://api.openai.com/v1>, which is not a valid URL. The same pattern now appears on every endpoint assignment in this file; remove the < and > from all actual URL values.

PLATFORM_OPENAI_CHAT_ENDPOINT="<https://api.openai.com/v1>"

[Richard Lundeen (rlundeen2)]

There's a lot to read through, so apologies if this has been addressed.

But I want to make sure we don't lose debuggability and that I can easily tell which targets we have. Right now, I use .env ~weekly to help debug myself and others.

E.g. what is the default adversarial model? What is configured? Where is the default open ai target referencing? Where is X pointing at? Etc

Right now I do a lot of that with .env. In theory we could download a .env and I could use it the same way and it could reference key vault secrets. I'm worried if all of .env is obstructed, I won't be able to see what's configured. e.g. what is the adversarial model? Or how do I configure for another target when not in the GUI?

There might be answers to this. But if they're aren't, we may want to download a .env to help even see which targets are available (and have that be able to reference keyvault secrets)

[Victor Valbuena (ValbuenaVC)]

There's a lot to read through, so apologies if this has been addressed.

But I want to make sure we don't lose debuggability and that I can easily tell which targets we have. Right now, I use .env ~weekly to help debug myself and others.

E.g. what is the default adversarial model? What is configured? Where is the default open ai target referencing? Where is X pointing at? Etc

Right now I do a lot of that with .env. In theory we could download a .env and I could use it the same way and it could reference key vault secrets. I'm worried if all of .env is obstructed, I won't be able to see what's configured. e.g. what is the adversarial model? Or how do I configure for another target when not in the GUI?

There might be answers to this. But if they're aren't, we may want to download a .env to help even see which targets are available (and have that be able to reference keyvault secrets)

Fwiw, one of the goals of this PR is to avoid having users keep an .env on disk. This feels like a possible anti-pattern when we have registries and CLI utilities to find components, but feel free to disagree.

I see what you mean though and I think we can fix this by adding a save to disk flag that saves the new .env (without actual secrets, just references to kv secrets) to disk. This seems like the best of both worlds and for debugging lets you investigate line-by-line.

[Copilot]

Pull request overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

.env_example:191

• This primary TTS section still defines the old OPENAI_TTS_* names, while both TTS target registrations now read AZURE_OPENAI_TTS_* (pyrit/setup/initializers/targets.py:360-371). Filling this section therefore does not configure either registered TTS target; reconcile these names and the duplicate Azure definitions later in the file.

OPENAI_TTS_ENDPOINT1 = "<https://xxxxx.openai.azure.com/openai/v1>"
OPENAI_TTS_MODEL1 = "tts"
OPENAI_TTS_UNDERLYING_MODEL1 = "tts"

OPENAI_TTS_ENDPOINT2 = "<https://xxxxx.openai.azure.com/v1>"

pyrit/setup/initialization.py:106

• The PR description promises that bootstrap documents remain in memory and are never written to disk, but this public option intentionally persists them, including any literal credentials in the bootstrap document. Either remove the write-to-disk feature or update the stated contract and scope so this security-sensitive behavior is explicitly reviewed.

        env_akv_write_env (bool): If True, save fetched bootstrap documents with unresolved
            child references to ``~/.pyrit/.env``. Defaults to False.

.env_example:32

• The angle brackets are literal dotenv value characters, not Markdown delimiters, so copying this example produces endpoints such as <https://...> that URL clients will reject. This pattern occurs throughout the newly updated endpoint values; remove the < and > wrappers everywhere in this file.

AZURE_OPENAI_GPT4O_ENDPOINT="<https://xxxx.openai.azure.com/openai/v1>"

.env_example:140

• These aliases are interpolated before PLATFORM_OPENAI_CHAT_ENDPOINT and PLATFORM_OPENAI_CHAT_MODEL are assigned at lines 318–320. Because python-dotenv resolves in assignment order and does not revisit earlier values, both aliases become empty when this file is loaded. Move source definitions before their aliases (also for the response, realtime, image, and TTS forward references below) or move the aliases after the sources.

OPENAI_CHAT_ENDPOINT=${PLATFORM_OPENAI_CHAT_ENDPOINT}
OPENAI_CHAT_MODEL=${PLATFORM_OPENAI_CHAT_MODEL}

.env_example:172

• This primary image section still defines the old OPENAI_IMAGE_*2 names, while TargetConfig now reads AZURE_OPENAI_IMAGE_*2 (pyrit/setup/initializers/targets.py:349-352). A user following the file's instruction to fill only this section will not configure openai_image_platform; reconcile these names and the duplicate Azure definitions later in the file.

This issue also appears on line 187 of the same file.

OPENAI_IMAGE_ENDPOINT2 = "<https://xxxxx.openai.azure.com/openai/v1>"
OPENAI_IMAGE_MODEL2 = "dall-e-3"
OPENAI_IMAGE_UNDERLYING_MODEL2 = "dall-e-3"