project-connor/docs/DEPLOYMENT.md

# Deployment Runbook — k3s on Vultr

Deploys `dist/` as a static site on the existing k3s cluster at **65.20.110.165**, served at
**https://www.apexturf.es** (with `apexturf.es` redirecting to it), using a self-hosted **Gitea** instance
at **https://git.apexturf.es** as the container registry.

You run every command below yourself (via `!` in Claude Code or your own terminal) — nothing here is
executed automatically. Each phase assumes the previous one succeeded; verify before moving on.

**Assumptions/placeholders to double check before you start:**
- You have root SSH access to `65.20.110.165` already (this box was provisioned outside this session).
- k3s was installed with its default bundled **Traefik** ingress controller (verify in Phase 1).
- DNS for `apexturf.es` is managed somewhere you can add A records (any registrar/DNS host works —
  instructions below are provider-agnostic).
- Let's Encrypt contact email is set to `hello@albertabril.com` in
  `deploy/k8s/cert-manager/cluster-issuer.yaml` — change it there if you'd rather use another address.
- Gitea owner/username is assumed to be `apexadmin` in examples below — swap in whatever you actually pick.

---

## Phase 0 — Local tools

```bash
# You already have Docker. Confirm kubectl + helm are installed:
kubectl version --client
helm version

# If either is missing:
brew install kubectl helm
```

## Phase 1 — Get kubectl talking to the cluster

```bash
# Pull the kubeconfig off the node (adjust the ssh user if it's not root):
scp root@65.20.110.165:/etc/rancher/k3s/k3s.yaml ~/.kube/apexturf-k3s.yaml

# The file points at 127.0.0.1 by default — repoint it at the real IP.
# (macOS/BSD sed needs the empty '' after -i; drop it on Linux.)
sed -i '' 's/127\.0\.0\.1/65.20.110.165/' ~/.kube/apexturf-k3s.yaml
chmod 600 ~/.kube/apexturf-k3s.yaml

export KUBECONFIG=~/.kube/apexturf-k3s.yaml
kubectl get nodes
kubectl get pods -n kube-system   # confirm a traefik pod is running
```

> Add `export KUBECONFIG=~/.kube/apexturf-k3s.yaml` to your shell profile (`~/.zshrc`) so it's set in new
> terminals — every command below assumes it's exported.

**If `kubectl get nodes` times out:** the Vultr firewall group attached to this instance likely doesn't
allow inbound 6443 from your IP. Add a rule for it (and for 80/443, needed later for Let's Encrypt) in the
Vultr dashboard → your instance → Firewall.

## Phase 2 — DNS

At whatever DNS host manages `apexturf.es`, add three **A** records, all pointing at `65.20.110.165`:

| Host                | Type | Value           |
|---------------------|------|-----------------|
| `apexturf.es` (`@`) | A    | 65.20.110.165   |
| `www.apexturf.es`   | A    | 65.20.110.165   |
| `git.apexturf.es`   | A    | 65.20.110.165   |

Verify propagation before moving on (can take minutes to hours depending on TTL/provider):

```bash
dig +short apexturf.es
dig +short www.apexturf.es
dig +short git.apexturf.es
```

## Phase 3 — cert-manager + Let's Encrypt

```bash
# Check https://cert-manager.io/docs/installation/ for the current latest version —
# this pins a known-good one, but a newer release is likely available:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml

kubectl -n cert-manager rollout status deploy/cert-manager
kubectl -n cert-manager rollout status deploy/cert-manager-webhook

kubectl apply -f deploy/k8s/cert-manager/cluster-issuer.yaml
kubectl get clusterissuer
```

Both `letsencrypt-prod` and `letsencrypt-staging` should show `READY=True`. Every Ingress below requests
`letsencrypt-prod` directly; if you want to dry-run against staging first (untrusted cert, but validates
the whole HTTP-01 chain without touching the production rate limit), temporarily change the
`cert-manager.io/cluster-issuer` annotation to `letsencrypt-staging` on an Ingress, confirm it issues, then
switch it back.

## Phase 4 — Gitea (registry + git host)

```bash
helm repo add gitea-charts https://dl.gitea.com/charts/
helm repo update

helm upgrade --install gitea gitea-charts/gitea \
  --namespace gitea --create-namespace \
  -f deploy/k8s/gitea/values.yaml \
  --set gitea.admin.password='PICK-A-STRONG-PASSWORD-HERE'

kubectl -n gitea rollout status deploy/gitea
kubectl -n gitea get certificate   # wait for READY=True on the git.apexturf.es cert
```

Then:
1. Visit **https://git.apexturf.es**, log in as `apexadmin` / the password you set above.
2. **Change that password** (Settings → Account) — the one above only exists in your shell history/values
   override now, not committed anywhere.
3. Settings → Applications → **Generate New Token**, scopes `read:package` + `write:package` (this is what
   `docker login`/`docker push` will use — don't reuse your account password for that). Save the token
   somewhere safe; Gitea only shows it once.

## Phase 5 — Build & push the site image

From the project root (not `deploy/` — the Dockerfile expects repo-root as build context):

```bash
docker login git.apexturf.es -u apexadmin   # paste the access token as the password when prompted

docker build -f deploy/Dockerfile -t git.apexturf.es/apexadmin/apex-turf-web:latest .
docker push git.apexturf.es/apexadmin/apex-turf-web:latest
```

Confirm it landed: on Gitea, go to your user (or org) → **Packages** tab, you should see
`apex-turf-web:latest`.

## Phase 6 — Deploy the site

```bash
# 1. Namespace first
kubectl apply -f deploy/k8s/apex-turf/namespace.yaml

# 2. Let the cluster pull from Gitea's registry — same token as Phase 4/5
kubectl create secret docker-registry gitea-registry \
  --namespace apex-turf \
  --docker-server=git.apexturf.es \
  --docker-username=apexadmin \
  --docker-password='PASTE-THE-ACCESS-TOKEN-HERE' \
  --docker-email=hello@albertabril.com

# 3. Point the Deployment at your actual Gitea owner (default in the file is GITEA_OWNER)
sed -i '' 's/GITEA_OWNER/apexadmin/' deploy/k8s/apex-turf/deployment.yaml

# 4. Apply everything else
kubectl apply -f deploy/k8s/apex-turf/deployment.yaml
kubectl apply -f deploy/k8s/apex-turf/service.yaml
kubectl apply -f deploy/k8s/apex-turf/ingress.yaml
kubectl apply -f deploy/k8s/apex-turf/redirect-ingress.yaml
```

## Phase 7 — Verify

```bash
kubectl -n apex-turf get pods,svc,ingress,certificate
kubectl -n apex-turf rollout status deploy/apex-turf-web

curl -I https://www.apexturf.es      # expect 200
curl -I https://apexturf.es          # expect 301 -> https://www.apexturf.es/
```

Open **https://www.apexturf.es** in a browser and confirm the site loads with a valid padlock.

### Troubleshooting

- **Certificate stuck `False`/pending**: `kubectl -n apex-turf describe certificate apex-turf-web-tls` and
  `kubectl -n apex-turf describe challenge` — almost always DNS not propagated yet, or port 80 blocked by
  the Vultr firewall (HTTP-01 needs 80 reachable from the internet, not just 443).
- **`ImagePullBackOff`**: check `kubectl -n apex-turf describe pod <pod>` — usually the `gitea-registry`
  secret's token is wrong/expired, or `GITEA_OWNER` in `deployment.yaml` doesn't match the actual
  image path you pushed to.
- **Redirect not firing**: `kubectl -n apex-turf get middleware` should list `redirect-to-www`; if Traefik
  isn't picking it up, confirm the Ingress annotation matches
  `<namespace>-<middleware-name>@kubernetescrd` exactly (`apex-turf-redirect-to-www@kubernetescrd`).

## Redeploying after changes to `dist/`

```bash
docker build -f deploy/Dockerfile -t git.apexturf.es/apexadmin/apex-turf-web:latest .
docker push git.apexturf.es/apexadmin/apex-turf-web:latest
kubectl -n apex-turf rollout restart deployment/apex-turf-web
kubectl -n apex-turf rollout status deployment/apex-turf-web
```