helm install --create-namespace fails with Capsule tenant even with cluster-admin clusterRole

[maybeclever]

Bug description

Running helm install --create-namespace as a Capsule tenant owner fails with a 403 Forbidden on namespace-scoped resources, even though the tenant owner has cluster-admin as their clusterRole.

Creating the namespace with kubectl works fine.

How to reproduce

Steps to reproduce the behavior:

1. Provide the Capsule Tenant YAML definitions
   `apiVersion: capsule.clastix.io/v1beta2
   kind: Tenant
   metadata:
   name: my-tenant
   spec:
   owners:

• kind: Group
  name: my-tenant
  clusterRoles:
  • cluster-admin
  • capsule-namespace-deleter
    forceTenantPrefix: true`

2. Run as a tenant owner
   helm upgrade --install my-release ./my-chart \ --namespace my-tenant-test \ --create-namespace

Expected behavior

Helm creates the namespace and installs the chart successfully.

Logs

Error: Unable to continue with install: could not get information about the resource ServiceAccount "my-release-sa" in namespace "my-tenant-test": serviceaccounts "my-release-sa" is forbidden: User "..." cannot get resource "serviceaccounts" in API group "" in the namespace "my-tenant-test"

If applicable, please provide logs of capsule.

In a standard stand-alone installation of Capsule,
you'd get this by running kubectl -n capsule-system logs deploy/capsule-controller-manager.

Additional context

• Capsule version: 0.12.4
• Helm version: v3.19.0
• Kubernetes version: EKS 1.31

[prometherion]

Could you give us a sample Chart you're testing?

I suspect it's related to the eventually consistent RBAC objects created upon Namespace creation, this failing: if you issue a second time the Helm install command, does it exit successfully?

[maybeclever]

thanks for your help @prometherion, i edited the issue with more info.
Initially the chart is an internal project that deploys many resources.
I made a sample chart that deploys:

• ServiceAccount
• Deployment
• Service
  And still the same issue.

a second helm install doesnt pass, as the namespace is not created.

[maybeclever]

any help, is this a known bug or a missconfiguration on my side ? couldn't find a similar issue.

[oliverbaehler]

Probably misconfiguration. Just from the logs i understand that you are installing the Chart as ServiceAccount. Is the User a Tenant Owner? Are you sing SSA with Helm?

[maybeclever]

Hello, thanks for your reply !
No SSA : Helm version is 3.19.0
If i rekon i am not installing the chart as a ServiceAccount — the identity is an IAM role mapped to Kubernetes groups via EKS Access Entry:
here is our authentication flow:

Tenant's AWS account assumes Aws IAM role in the cluster account via aws eks get-token
That role is registered as an EKS Access Entry, mapping it to Kubernetes groups ["k8s-read", "my-tenant] — no namespace binding, groups only
The kubeconfig server points to Capsule Proxy (not the EKS API directly)

ATTRIBUTE   VALUE 
Username  arn:aws:sts::xxxx:assumed-role/CrossAccountEKSRoleEKSGetTokenAuth
Groups      [k8s-read my-tenant system:authenticated]

[prometherion]

I think the creation of the Namespace and namespace-scoped resources in the same process won't work: when a Namespace is created, we delegate the controller to create all the required RBAC objects, and this could take time. If Helm tries to create the namespace-scoped objects before the Capsule operator has installed the required RBAC, it will, of course, fail.

However, it's weird you're not getting at least Namespace created: could it be related to the error cannot get resource "serviceaccounts" in API group "" in the namespace "my-tenant-test"?

[maybeclever]

as mentionned, the serviceaccount is a resource being created by the chart. It seems that Helm, since it reconciliates resources, checks first if the resource exist, and in this case when doing a "GET" on a non existant namespace, its getting the deny.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[aminmr]

I reproduced this on a clean kind cluster and traced the root cause. Since this is tagged blocked-needs-validation, here is a complete copy-pasteable reproduction, plus the diagnosis and a control test. I have a fix working against capsule-proxy and will link the PR.

TL;DR: the bug is real and reproduces 100% of the time (not intermittently). The namespace is never created at all — Helm aborts before it gets that far. The reason is that a tenant owner can only ever receive 403 (never 404) for a resource in a namespace that doesn't exist yet, which breaks every "GET, create it if 404" client.

---

Full reproduction

Prereqs: kind, kubectl, helm, docker, openssl.

1. Cluster + Capsule

kind create cluster --name capsule-test          # kindest/node:v1.36.1 (k8s 1.36)

helm repo add jetstack https://charts.jetstack.io
helm repo add projectcapsule https://projectcapsule.github.io/charts
helm repo update

helm install cert-manager jetstack/cert-manager \
  -n cert-manager --create-namespace --set crds.enabled=true --wait

helm install capsule projectcapsule/capsule \
  -n capsule-system --create-namespace --version 0.13.9 --wait

helm install capsule-proxy projectcapsule/capsule-proxy \
  -n capsule-system --version 0.13.7 --wait

2. Tenant (verbatim from this issue)

kubectl apply -f - <<'EOF'
apiVersion: capsule.clastix.io/v1beta2
kind: Tenant
metadata:
  name: my-tenant
spec:
  owners:
  - kind: Group
    name: my-tenant
    clusterRoles:
    - cluster-admin
    - capsule-namespace-deleter
  forceTenantPrefix: true
EOF

3. Tenant-owner identity

An x509 client cert signed by the cluster CA. Note the two O= values — besides the tenant owner group my-tenant, the identity also needs the Capsule users group (projectcapsule.dev, the default CapsuleConfiguration.spec.users), which is what carries the capsule-namespace-provisioner ClusterRole allowing cluster-scoped namespace creation. Without it the owner cannot create namespaces at all, which is a different failure.

docker cp capsule-test-control-plane:/etc/kubernetes/pki/ca.crt ./ca.crt
docker cp capsule-test-control-plane:/etc/kubernetes/pki/ca.key ./ca.key

openssl genrsa -out jane.key 2048
openssl req -new -key jane.key -out jane.csr \
  -subj "/CN=jane-tenant-owner/O=my-tenant/O=projectcapsule.dev"
openssl x509 -req -in jane.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out jane.crt -days 30

4. Point a kubeconfig at capsule-proxy

kubectl -n capsule-system port-forward svc/capsule-proxy 9001:9001 &

cat > jane-kubeconfig.yaml <<EOF
apiVersion: v1
kind: Config
clusters:
- {name: p, cluster: {server: "https://127.0.0.1:9001", insecure-skip-tls-verify: true}}
contexts:
- {name: c, context: {cluster: p, user: u}}
current-context: c
users:
- {name: u, user: {client-certificate: $PWD/jane.crt, client-key: $PWD/jane.key}}
EOF

kubectl --kubeconfig jane-kubeconfig.yaml auth whoami
# Username  jane-tenant-owner
# Groups    [my-tenant projectcapsule.dev system:authenticated]

This matches @maybeclever's setup (EKS Access Entry → groups → capsule-proxy).

5. Minimal chart

mkdir -p my-chart/templates
cat > my-chart/Chart.yaml <<'EOF'
apiVersion: v2
name: my-chart
version: 0.1.0
EOF
cat > my-chart/templates/serviceaccount.yaml <<'EOF'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-release-sa
EOF

6. Reproduce

helm install my-release ./my-chart --kubeconfig jane-kubeconfig.yaml \
  --namespace my-tenant-test --create-namespace

Actual (every run):

Error: INSTALLATION FAILED: unable to continue with install: could not get information about
the resource ServiceAccount "my-release-sa" in namespace "my-tenant-test": serviceaccounts
"my-release-sa" is forbidden: User "jane-tenant-owner" cannot get resource "serviceaccounts"
in API group "" in the namespace "my-tenant-test"

I ran this 25× on Helm v3.19.0 (the version in the report) and 10× on Helm v4.2.2 — 0 successes. Confirm the namespace was never created:

kubectl get ns my-tenant-test     # Error from server (NotFound)

7. Control test — proves it's the ordering, not permissions

Same identity, same chart, but create the namespace first:

kubectl --kubeconfig jane-kubeconfig.yaml create namespace my-tenant-test
sleep 5
helm install my-release ./my-chart --kubeconfig jane-kubeconfig.yaml -n my-tenant-test
# STATUS: deployed

The tenant owner's permissions are fine. Only the ordering is broken.

---

Root cause

Two separate problems; the second is the one suspected earlier in this thread.

1. A tenant owner can never get 404 for a missing namespace — only 403. This is the deterministic failure.

Kubernetes authorizes namespace-scoped requests using RoleBindings that live inside the target namespace. A namespace that doesn't exist yet can't contain one, so a tenant owner probing a resource there always gets 403 Forbidden. A cluster-admin — authorized by a cluster-scoped binding, independent of the namespace existing — gets a clean 404 Not Found for the identical request.

Helm does a pre-flight "does this resource already exist?" GET for each chart resource before creating the target namespace. Logging every request through the proxy during a failing run:

GET /version
GET /api/v1/namespaces/my-tenant-test/secrets?labelSelector=...   (release history; tolerated)
GET /api, /apis, /openapi/v3                                       (discovery)
GET /api/v1/namespaces/my-tenant-test/serviceaccounts/my-release-sa  -> 403, fatal, abort

There is no namespace-create request at all. Helm sees the 403, treats it as a genuine permission error, and gives up before the step that would have created the namespace. Any client using the standard "GET, create if 404" pattern hits this — kubectl apply, Argo CD, Terraform.

This also explains why kubectl create ns "works" while Helm doesn't: kubectl isn't doing a create-if-missing probe.

2. Owner RBAC is provisioned asynchronously after creation. This is the race @prometherion suspected, and it's real — just secondary.

Once creation does proceed, namespace creation triggers a full reconcile of the parent Tenant (capsule/tenants in internal/controllers/tenant/manager.go), which runs collectRBAC → reconcileNamespaces → metadata → quotas → limitranges → networkpolicies, and only then syncRoleBindings (internal/controllers/tenant/rolebindings.go). Measured on this cluster:

Event	Wall clock
POST /api/v1/namespaces returns 201	+298 ms
first owner RoleBinding appears	+368 ms (+70 ms after the client was told "created")
second RoleBinding appears	+401 ms

So for ~70–100 ms the client holds a namespace it owns with no RBAC inside it. Helm's next call after creating the namespace can land in that window.

---

Note on fixing it

RoleBindings are namespaced, so they cannot exist before the namespace does — there will always be some window between "namespace persisted" and "owner RBAC live", and an admission webhook can't close it either (webhooks run before the object is persisted). A change in the operator could shrink the window but not remove it, which would turn a deterministic failure into a flaky one.

Both problems are fixable in capsule-proxy, which already sits synchronously in the request path, and with no new RBAC. I have that working — 15/15 on Helm 3.19.0 and 10/10 on Helm 4.2.2, with cross-tenant isolation verified intact. PR: projectcapsule/capsule-proxy#1082 — happy to adjust the approach if you'd prefer it elsewhere. @prometherion