Skip to content
rackctlrackctlrackctl

Runbook

A provision is unattended for most of its ~45 minutes, but the first ten are worth doing deliberately. This is the sequence.

1. Preflight

Section titled “1. Preflight”
Terminal window
aws sso login --profile <your-profile>
rackctl check

With no cluster yet, check runs the pre-spend set: the tools are present, your AWS identity resolves and matches cloud.accountId, and nothing is left over from a previous attempt. Don’t proceed until it’s clean — every downstream phase assumes all of it.

2. Review the config

Section titled “2. Review the config”

Open your rackctl.yaml and check the three things that hurt most when wrong:

  • cloud.accountId — the exact 12-digit account you intend.
  • cloud.region — where everything lands.
  • environment — and, for production, that cluster.endpointPublicAccess is false.

3. Plan

Section titled “3. Plan”
Terminal window
rackctl plan -c rackctl.yaml

Read it. The plan lists every phase and the commands it will run. This is the last checkpoint before real resources exist. If anything looks off — a wrong repo, an unexpected addon — stop and fix the config, not the running provision.

4. Apply

Section titled “4. Apply”
Terminal window
rackctl apply -c rackctl.yaml --tui

--tui gives a live progress view. Let it run. The long poles are the cluster phase (EKS control plane creation) and addon convergence (ArgoCD reconciling the catalog). If it stalls in the cluster phase, check quotas.

If a phase fails, the engine rolls back completed phases in reverse. To stop that and inspect the state instead:

Terminal window
rackctl apply -c rackctl.yaml --no-clean-on-failure

5. Verify

Section titled “5. Verify”
Terminal window
rackctl check

With a cluster up, check also asserts the invariants of a provisioned platform — that it is reachable, and that ArgoCD’s applications are present and syncing. Then spot-check directly:

Terminal window
kubectl get nodes
kubectl -n argocd get applications

6. Hand off

Section titled “6. Hand off”

Core platform is up and reconciling. Day-2 work moves to the portal (enable it with controlPlane.portal: true). rackctl stays for lifecycle only.

Ongoing

Section titled “Ongoing”

Upgrade

Section titled “Upgrade”
Terminal window
rackctl apply -c rackctl.yaml   # apply syncs the catalog fork and re-applies

Pulls the latest addon catalog and bumps the operator; ArgoCD reconciles the rest.

Tear down

Section titled “Tear down”
Terminal window
rackctl destroy -c rackctl.yaml            # plan
rackctl destroy -c rackctl.yaml            # reverse teardown (confirms first)

Troubleshooting

Section titled “Troubleshooting”
Symptom Likely cause Fix
check reports missing tools A prerequisite isn’t on PATH Install it
aws identity unavailable SSO session expired aws sso login --profile <profile>
Preflight stops on account mismatch cloud.accountId ≠ resolved account Fix the id or switch profiles
Cluster phase hangs on capacity Unapproved vCPU quota Check Service Quotas; wait for the increase
Agent platform up but models missing Bedrock family not enabled Enable it in the Bedrock console for the region
Invalid config before any work A validation rule failed Read the error — all failures are listed at once
Rollback itself fails Stuck finalizer / dependency in use Clear the blocker, then re-run or rackctl destroy

See also

Section titled “See also”