Skip to content

THREESCALE-10236 Custom CA certificate support for operator deployment#1181

Draft
borisurbanik wants to merge 9 commits into
3scale:masterfrom
borisurbanik:THREESCALE-10236
Draft

THREESCALE-10236 Custom CA certificate support for operator deployment#1181
borisurbanik wants to merge 9 commits into
3scale:masterfrom
borisurbanik:THREESCALE-10236

Conversation

@borisurbanik

Copy link
Copy Markdown
Contributor

WIP

The 3scale operator connects to the 3scale Admin API (and, via the Tenant controller, the Master API) to reconcile capabilities CRs (Backend, Product, Application, ActiveDoc, etc.). These connections use HTTPS.

In environments where 3scale is deployed with internal or self-signed CA, the operator's HTTP client rejects the connection because the CA is not in the system trust store. The existing workarounds — the insecure_skip_verify annotation and patching the operator Subscription to mount a CA via SSL_CERT_FILE (Red Hat Solution 7049968) — are either insecure or fragile (overwritten on upgrades).

Customers need a way to provide a custom CA bundle so the operator can trust internal CAs without disabling verification.

Sources of 3scale connections (each with different credential configuration):

  1. Explicit providerAccountRef: the CR references a Secret containing adminURL and token.
  2. Default threescale-provider-account secret: a well-known Secret name. Same schema as source 1 but discovered by convention.
  3. Local APIManager: the operator discovers an APIManager CR in the namespace and derives the admin URL and token from system-seed.
  4. Tenant controller / Master API: the Tenant CR references a masterCredentialsRef secret (defaulting to system-seed) containing master API credentials.

Implements openspec task ca-client-configuration.

- Add PortaClientFromURLWithTLSConfig to pkg/controller/helper/threescale_api.go:
  when tlsConfig is non-nil it is used directly as the transport TLS config;
  when nil, delegates to PortaClientFromURL unchanged.
- Add three test cases to threescale_api_test.go covering: nil+insecureSkipVerify=false
  rejects untrusted cert, custom CA succeeds, nil+insecureSkipVerify=true accepts cert.
…e API calls

- Add PortaClientWithTLSConfig convenience wrapper to pkg/controller/helper/threescale_api.go
- Add CAProvider field to all 11 capabilities reconciler structs
- Replace PortaClient call sites in 9 controllers with CAProvider.TLSConfig() + PortaClientWithTLSConfig
- Rewrite TenantReconciler.setupPortaClient to use url.Parse + PortaClientFromURLWithTLSConfig
- Construct a single shared *CAProvider in main.go and inject into all 11 reconcilers
- Add per-controller unit tests verifying invalid CA bundle surfaces as a status condition
Replace the one-shot loaded flag with a ResourceVersion-keyed cache.
TLSConfig re-parses only when the ConfigMap version changes; the Get is
served from the controller-runtime in-memory cache so there is no extra
network cost per reconcile.

Thread ctx through the helper methods in each capabilities controller so
the Get can be cancelled with the reconcile context.

Adds TestCAProvider_AutoReload (no Reload needed after cm update) and
TestCAProvider_CacheHit (same pointer returned on cache hit).
Introduce cacheEntry struct holding config, cachedErr and
resourceVersion together. All three fields are now read and written
as a single value under the mutex, eliminating the possibility of a
partial read across the three fields.

loadFromConfigMap becomes a pure function returning a cacheEntry
instead of a method that mutates shared state while the caller holds
the write lock.

Remove concurrency tests that used time.Sleep-bounded loops;
drop the now-unused sync and time imports.
@openshift-ci

openshift-ci Bot commented Jun 25, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@codecov-commenter

codecov-commenter commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.33333% with 74 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.27%. Comparing base (ade0ed0) to head (8937032).
⚠️ Report is 21 commits behind head on master.

Files with missing lines Patch % Lines
controllers/configuration/ca_bundle_watcher.go 0.00% 61 Missing ⚠️
main.go 0.00% 7 Missing ⚠️
controllers/capabilities/tenant_controller.go 50.00% 1 Missing and 1 partial ⚠️
controllers/capabilities/backend_controller.go 50.00% 1 Missing ⚠️
...ollers/capabilities/developeraccount_controller.go 50.00% 1 Missing ⚠️
...ntrollers/capabilities/developeruser_controller.go 50.00% 1 Missing ⚠️
controllers/capabilities/product_controller.go 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1181      +/-   ##
==========================================
+ Coverage   42.67%   48.27%   +5.60%     
==========================================
  Files         204      206       +2     
  Lines       20899    21010     +111     
==========================================
+ Hits         8919    10143    +1224     
+ Misses      11212     9861    -1351     
- Partials      768     1006     +238     
Flag Coverage Δ
unit 48.27% <33.33%> (+5.60%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
apis/apps/v1alpha1 (u) 63.56% <ø> (+4.27%) ⬆️
apis/capabilities/v1alpha1 (u) 40.35% <ø> (+36.84%) ⬆️
apis/capabilities/v1beta1 (u) 29.78% <ø> (+9.56%) ⬆️
controllers (i) 23.40% <55.49%> (+14.09%) ⬆️
pkg (u) 64.03% <96.42%> (+1.01%) ⬆️
Files with missing lines Coverage Δ
controllers/capabilities/activedoc_controller.go 32.25% <100.00%> (+32.25%) ⬆️
controllers/capabilities/application_controller.go 59.32% <100.00%> (+9.60%) ⬆️
...rollers/capabilities/applicationauth_controller.go 38.46% <100.00%> (+13.60%) ⬆️
.../capabilities/custompolicydefinition_controller.go 36.53% <100.00%> (+36.53%) ⬆️
...lers/capabilities/proxyconfigpromote_controller.go 45.25% <100.00%> (+11.67%) ⬆️
controllers/configuration/tls_config.go 100.00% <100.00%> (ø)
pkg/controller/helper/threescale_api.go 86.20% <100.00%> (+4.38%) ⬆️
controllers/capabilities/backend_controller.go 16.76% <50.00%> (+16.76%) ⬆️
...ollers/capabilities/developeraccount_controller.go 25.95% <50.00%> (+25.95%) ⬆️
...ntrollers/capabilities/developeruser_controller.go 24.57% <50.00%> (+24.57%) ⬆️
... and 4 more

... and 39 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@borisurbanik borisurbanik force-pushed the THREESCALE-10236 branch 2 times, most recently from c7f659f to 030c98c Compare June 28, 2026 21:40
…ndleWatcher

- Define HTTPClientSource interface in pkg/reconcilers for controllers to
  depend on without importing pkg/controller/helper
- CABundleWatcher now owns and serves a single cached *http.Client via
  atomic.Pointer; CA bundle parse errors are emitted as Warning events and
  logged rather than propagated into capability controllers
- Replace two-step TLSConfig()+PortaClientWithTLSConfig() pattern in all
  10 capability controllers with GetHTTPClient()+PortaClientFromAccount();
  insecureSkipVerify stays as a per-CR concern in the porta client constructors
…onfig

Remove the HTTPClientSource interface and its injection into all 10 capability
controllers.  Instead, CABundleWatcher writes a shared *tls.Config via
package-level accessors in controllers/configuration, and the porta client
constructors read it on every call to build a fresh *http.Client.

No shared client is ever stored on a struct field or passed as a parameter;
each reconcile invocation gets a client that reflects the CA bundle at call
time without any cross-reconcile shared state.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants