feat(config): add --extra_config_url to merge external .pr_agent.toml#2406
feat(config): add --extra_config_url to merge external .pr_agent.toml#2406kiennt2 wants to merge 7 commits into
Conversation
Review Summary by QodoAdd external configuration URL support for shared defaults
WalkthroughsDescription• Add --extra_config_url CLI flag and env var for external config merging • Support HTTP(S) URLs, file:// paths, and local filesystem sources • Implement secure config loading with 1 MB size limit and 10s timeout • Maintain proper precedence: external config < repo-local < wiki < env vars • Add comprehensive test suite covering resolver, CLI, and precedence scenarios Diagramflowchart LR
CLI["CLI Flag<br/>--extra_config_url"]
ENV["Environment Var<br/>PR_AGENT_EXTRA_CONFIG_URL"]
RESOLVE["_resolve_extra_config_to_file<br/>HTTP/File/Local"]
APPLY["_apply_settings_from_file<br/>Merge via custom_merge_loader"]
REPO["Repo-local<br/>.pr_agent.toml"]
FINAL["Final Settings<br/>Repo overrides External"]
CLI --> RESOLVE
ENV --> RESOLVE
RESOLVE --> APPLY
APPLY --> FINAL
REPO --> FINAL
File Changes1. pr_agent/cli.py
|
Code Review by Qodo
1.
|
|
Persistent review updated to latest commit 248358b |
|
Persistent review updated to latest commit 6cba663 |
|
Persistent review updated to latest commit 6f9db0b |
|
Persistent review updated to latest commit 422a07a |
|
Persistent review updated to latest commit 1886406 |
|
Persistent review updated to latest commit 2bc9179 |
|
Preparing review... |
1 similar comment
|
Preparing review... |
Motivation
.pr_agent.tomltoday is resolved from one of: the repo's default branch, the wiki, or apr-agent-settingsrepo one level above the target repo. None of these scale cleanly to:What this PR adds
A new CLI flag
--extra_config_url(and matchingPR_AGENT_EXTRA_CONFIG_URLenv var /CONFIG.EXTRA_CONFIG_URLsetting) that points to an additional.pr_agent.toml. The file is resolved at the start ofapply_repo_settings()and merged through the same secure loader used for repo-local config — before the git provider is constructed and before the repo-local file is applied. Repo-local config still overrides the extra file on conflicting
keys, so the new layer acts as defaults.
Accepted sources
https://…/http://…— fetched via stdliburllib, written to a tempfile, cleaned up after mergefile:///path/to/shared.toml— local filesystem (non-localhost netloc preserved, percent-encoded paths URL-decoded)file://Authentication for private endpoints
Optional
PR_AGENT_EXTRA_CONFIG_AUTH_HEADERenv var, formatted as<HeaderName>: <value>:A malformed header (no
:) is dropped and a warning is logged so misconfiguration is diagnosable rather than silent.Usage
Security
The external file is loaded through the same
custom_merge_loader.pyused for repo-local config, so:includes,preload, customloaders,dynaconf_include, …) are rejected byvalidate_file_securityexactly as for any other config source.http,https,fileschemes are accepted; anything else is dropped with a warning.finallyblock after the merge, success or failure.[openai],[gitlab], etc.can't leak into CI logs.
CONFIG.EXTRA_CONFIG_URLis rejected with a warning rather than raising fromurlparse.Implementation notes
pr_agent/cli.py— registers--extra_config_url, defaults to the env var, stashes the value onCONFIG.EXTRA_CONFIG_URLso the loader picks it up.pr_agent/settings/configuration.toml— declaresextra_config_url=""under[config]so the option appears in the canonical list.pr_agent/git_providers/utils.py— adds_safe_url_for_log(URL sanitiser),_resolve_extra_config_to_file(source)(URL/path → local file, handlesfile://host + percent-decoding viaurl2pathname, HTTP(S) fetch withoptional auth header and bounded size/timeout), and
_apply_settings_from_file(path, label)(merges sections intoget_settings(), includes the same DynaconfTypeErrorfallback used by the repo-settings loader). Wired intoapply_repo_settings()before the git provider is constructed so provider initialisers (e.g.GitLabProviderreadingGITLAB.PERSONAL_ACCESS_TOKEN) see merged values, and before the repo-settings block so repo-local alwaysoverrides the extra file.
urllibandtempfilefrom stdlib.GITEA.REPO_SETTINGingitea_provider.py), generalised across all providers.Tests
tests/unittest/test_extra_config_url.py— 41 tests, all green:file:///file://localhost/…/ percent-encodedfile:/// missing path / unsupported scheme / empty input / non-string input; HTTP fetch end-to-end against a realhttp.serverthread (boundto port 0, no race); auth header injection from env var; missing-auth → 401 → skipped; HTTP 404 → skipped; >1 MB body → rejected; malformed auth header warns and proceeds; URL with embedded credentials never appears in any log
line;
_safe_url_for_logstrips userinfo and query.the post-merge log line; loader falls back when Dynaconf rejects the hardened kwargs.
apply_repo_settings(6) with a_FakeGitProvider: repo-local wins over extra on shared keys; extra-only keys survive; repo-only keys survive; works with empty repo settings; unreachable extra URLdoesn't block repo settings; extra config is merged before the git provider is constructed.
Run:
pytest tests/unittest/test_extra_config_url.py -v # 41 passedDocs
docs/docs/usage-guide/configuration_options.md— adds an External configuration URL section covering usage, accepted source shapes, auth header, full precedence chain, and security/limits. Bumps the intro list from 3 → 4mechanisms and extends the precedence sentence.
Backwards compatibility
Fully backwards compatible. With no
--extra_config_urland noPR_AGENT_EXTRA_CONFIG_URL/CONFIG.EXTRA_CONFIG_URLset,apply_repo_settings()behaves exactly as before.Changed files
pr_agent/cli.pypr_agent/git_providers/utils.pypr_agent/settings/configuration.tomldocs/docs/usage-guide/configuration_options.mdtests/unittest/test_extra_config_url.py