turbovault-app/TODO.md

TurboVault TODO

Fix Gitea Ingress for Large Container Pushes

Issue: Docker push to Gitea registry times out on large layers (~250MB)

Error:

Package registry API internal error: 500 unexpected EOF

Root Cause: Ingress controller (nginx/traefik) in front of Gitea has timeout limits that prevent large uploads from completing.

Symptoms:

• Small layers push fine (already exists)
• Large gem layer (12f753f9ec10, ~250MB) times out after ~1 minute
• Gitea logs show: PATCH /v2/ryan/turbovault-app/blobs/uploads/... elapsed 3275.4ms ... 500 unexpected EOF

Solution: Update Gitea Ingress

Find your Gitea ingress configuration and add these annotations:

For NGINX Ingress Controller:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gitea
  namespace: tools  # or wherever gitea is
  annotations:
    # Allow unlimited upload size
    nginx.ingress.kubernetes.io/proxy-body-size: "0"
    
    # Increase timeouts to 10 minutes
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    
    # Enable chunked uploads
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"

For Traefik Ingress:

apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: gitea-buffering
  namespace: tools
spec:
  buffering:
    maxRequestBodyBytes: 0  # Unlimited
    memRequestBodyBytes: 2097152  # 2MB in memory, rest to disk

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gitea
  namespace: tools
  annotations:
    traefik.ingress.kubernetes.io/router.middlewares: tools-gitea-buffering@kubernetescrd

Steps to Fix:

1. Find current Gitea ingress:

   kubectl get ingress -n tools
   kubectl get ingress gitea -n tools -o yaml
   

2. Identify ingress controller:

   kubectl get ingressclass
   # Check which controller: nginx, traefik, etc.
   

3. Update ingress with appropriate annotations (see above)

4. Apply changes:

   kubectl apply -f <your-gitea-ingress>.yaml
   

5. Test push:

   docker push gitea.kazcloud.dev/ryan/turbovault-app:v1.0.0
   

Alternative: Increase Gitea Service Timeouts

If using a LoadBalancer or NodePort directly:

Edit Gitea's app.ini:

[server]
LFS_MAX_FILE_SIZE = 0
HTTP_PORT = 3000

[packages]
ENABLED = true
CHUNKED_UPLOAD_PATH = /tmp/package-upload
LIMIT_TOTAL_OWNER_SIZE = -1
LIMIT_SIZE_CONTAINER = -1

Then restart Gitea pod.

---

Current Workaround

Using GitHub Container Registry (ghcr.io) for now:

• Image: ghcr.io/ryankazokas/turbovault-app:latest
• All k8s manifests updated to use ghcr.io
• GitHub Actions workflow configured
• Works perfectly, no timeout issues

Once Gitea ingress is fixed, can switch back by updating:

• k8s/deployment.yaml
• k8s/migrate-job.yaml
• .github/workflows/build-and-deploy.yml
• scripts/update-deployment.sh

---

Priority: Low (ghcr.io works fine for now)
Complexity: Medium (depends on ingress controller setup)
Benefit: Full control over container registry on your infrastructure