Problem
manage.sh destroy fails at Stage 2 (intra-cluster) when the gameservers namespace contains Agones fleets/game servers that were deployed outside of Terraform (via deploy-ncat-fleets.sh or deploy-stk-fleets.sh).
Terraform tries to delete the namespace, but Agones fleet controllers keep pods alive, causing the namespace deletion to hang until it hits the 5-minute context deadline.
Steps to Reproduce
- Deploy the full stack with
manage.sh deploy
- Deploy game server fleets with
deploy-ncat-fleets.sh
- Run
manage.sh destroy
- Stage 2 hangs on
kubernetes_namespace.this["gameservers"]: Still destroying... and eventually fails with context deadline exceeded
Root Cause
The deploy-ncat-fleets.sh script creates Agones Fleet resources in the gameservers namespace. These are not managed by Terraform, so Terraform doesn't know to delete them before removing the namespace. The Agones controller keeps game server pods alive via its own reconciliation loop, preventing the namespace from being deleted.
Possible Solutions
- Add fleet cleanup to
manage.sh destroy: Before running terraform destroy on intra-cluster, run kubectl delete fleets --all -n gameservers on both clusters to remove manually-deployed fleets.
- Add a pre-destroy hook in Terraform: Add a
null_resource with a when = destroy provisioner that cleans up fleets in the namespace before Terraform attempts namespace deletion.
- Increase Terraform timeout: Less ideal, but increasing the namespace deletion timeout would at least allow the namespace finalizer script (which already exists) more time to clear finalizers.
Affected Files
scripts/manage.sh (destroy flow)
Problem
manage.sh destroyfails at Stage 2 (intra-cluster) when thegameserversnamespace contains Agones fleets/game servers that were deployed outside of Terraform (viadeploy-ncat-fleets.shordeploy-stk-fleets.sh).Terraform tries to delete the namespace, but Agones fleet controllers keep pods alive, causing the namespace deletion to hang until it hits the 5-minute context deadline.
Steps to Reproduce
manage.sh deploydeploy-ncat-fleets.shmanage.sh destroykubernetes_namespace.this["gameservers"]: Still destroying...and eventually fails withcontext deadline exceededRoot Cause
The
deploy-ncat-fleets.shscript creates Agones Fleet resources in thegameserversnamespace. These are not managed by Terraform, so Terraform doesn't know to delete them before removing the namespace. The Agones controller keeps game server pods alive via its own reconciliation loop, preventing the namespace from being deleted.Possible Solutions
manage.sh destroy: Before runningterraform destroyon intra-cluster, runkubectl delete fleets --all -n gameserverson both clusters to remove manually-deployed fleets.null_resourcewith awhen = destroyprovisioner that cleans up fleets in the namespace before Terraform attempts namespace deletion.Affected Files
scripts/manage.sh(destroy flow)