# DEPLOYMENT PLAN - COMPLETE STEP-BY-STEP

**READ THIS ENTIRE DOCUMENT BEFORE EXECUTING ANY COMMANDS**

This deployment plan contains EVERY SINGLE COMMAND needed to deploy the Firefrost Knowledge Engine.

**Execution time:** 10-15 hours (spread across 2-3 sessions recommended)

**Recommended breaks:**
- After Phase 2 (Infrastructure deployed)
- After Phase 3 (Automation configured)
- After Phase 5 (Testing complete)

---

## 📋 EXECUTION CHECKLIST

**Track your progress - check boxes as you complete each phase:**

- [ ] Phase 0: Stop AnythingLLM
- [ ] Phase 1: Install Nginx and SSL
- [ ] Phase 2: Deploy Docker Stack
- [ ] Phase 3: Configure Dify
- [ ] Phase 4: Setup n8n Workflows
- [ ] Phase 5: Configure Discord Integration
- [ ] Phase 6: Setup Git Integration
- [ ] Phase 7: Configure Monitoring
- [ ] Phase 8: User Onboarding
- [ ] Phase 9: Testing and Verification
- [ ] Phase 10: Backup Automation
- [ ] Phase 11: Final Cleanup

---

## ⚠️ CRITICAL SAFETY RULES

**BEFORE EVERY PHASE:**
1. Read the entire phase before executing
2. Have rollback plan ready
3. Jack's health takes absolute priority - pause if needed
4. If tired, STOP and continue later
5. If unsure, STOP and review documentation

**DURING EXECUTION:**
- Copy commands ONE AT A TIME
- Verify success before proceeding
- Document any errors immediately
- Take screenshots of successful completions

**EMERGENCY STOP:**
- If something breaks: STOP immediately
- Do NOT try to fix on the fly
- Document what happened
- Review troubleshooting guide
- Consider rollback if needed

---

## PHASE 0: STOP ANYTHINGLLM

**Estimated time:** 5 minutes

### Step 0.1: SSH to TX1

**Command:**
```bash
ssh root@38.68.14.26
```

**Expected:** Successful login to TX1

**Verification:**
```bash
hostname
```

**Expected output:** `TX1` or similar

---

### Step 0.2: Locate AnythingLLM Installation

**Command:**
```bash
cd /opt/anythingllm
```

**If directory doesn't exist, try:**
```bash
find / -name "anythingllm" -type d 2>/dev/null
```

**Expected:** Directory found (location may vary)

---

### Step 0.3: Stop AnythingLLM

**Command:**
```bash
docker-compose down
```

**Expected output:**
```
Stopping anythingllm_...
Removing anythingllm_...
Removing network anythingllm_default
```

**Verification:**
```bash
docker ps | grep anything
```

**Expected:** No output (no containers running)

---

### Step 0.4: Verify Port 3001 is Free

**Command:**
```bash
sudo lsof -i :3001
```

**Expected:** No output (port is free)

**If port still in use:**
```bash
# Find the process
sudo lsof -i :3001

# Kill it if necessary (use PID from above)
sudo kill -9 <PID>
```

---

### Step 0.5: Document Current State

**Command:**
```bash
docker ps -a
docker images
```

**Save this output** - shows what was running before deployment

---

## PHASE 1: INSTALL NGINX AND SSL

**Estimated time:** 30 minutes

### Step 1.1: Update Package Lists

**Command:**
```bash
apt-get update
```

**Expected:** Package lists updated successfully

---

### Step 1.2: Install Nginx

**Command:**
```bash
apt-get install nginx -y
```

**Expected:** Nginx installed successfully

**Verification:**
```bash
nginx -v
```

**Expected output:** `nginx version: nginx/X.X.X`

---

### Step 1.3: Install Certbot

**Command:**
```bash
apt-get install certbot python3-certbot-nginx -y
```

**Expected:** Certbot installed successfully

**Verification:**
```bash
certbot --version
```

**Expected output:** `certbot X.X.X`

---

### Step 1.4: Verify DNS Propagation

**⚠️ CRITICAL: Do NOT proceed until DNS is fully propagated**

**Command (run from TX1):**
```bash
dig codex.firefrostgaming.com +short
dig n8n.firefrostgaming.com +short
```

**Expected output:**
```
38.68.14.26
38.68.14.26
```

**If NOT showing 38.68.14.26:**
- STOP deployment
- Wait for DNS propagation
- Check https://dnschecker.org
- Do NOT proceed until globally propagated

---

### Step 1.5: Stop Nginx Temporarily

**Command:**
```bash
systemctl stop nginx
```

**Why:** Certbot needs port 80 for standalone verification

**Verification:**
```bash
systemctl status nginx
```

**Expected:** Shows "inactive (dead)"

---

### Step 1.6: Generate SSL Certificates

**⚠️ CRITICAL: Replace email address with your actual email**

**Command:**
```bash
certbot certonly --standalone \
  -d codex.firefrostgaming.com \
  -d n8n.firefrostgaming.com \
  --email codex@firefrostgaming.com \
  --agree-tos \
  --non-interactive
```

**Expected output:**
```
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/codex.firefrostgaming.com/fullchain.pem
Key is saved at: /etc/letsencrypt/live/codex.firefrostgaming.com/privkey.pem
```

**If fails with DNS error:**
- DNS not propagated - WAIT
- Check DNS at https://dnschecker.org
- Try again in 1 hour

**If fails with other error:**
- Check firewall (port 80 must be open)
- Check logs: `journalctl -u certbot -n 50`
- Review TROUBLESHOOTING.md

---

### Step 1.7: Verify Certificates Created

**Command:**
```bash
ls -l /etc/letsencrypt/live/codex.firefrostgaming.com/
```

**Expected files:**
- cert.pem
- chain.pem
- fullchain.pem
- privkey.pem
- README

**If files missing:** Certificate generation failed, review logs

---

### Step 1.8: Set Certificate Permissions

**Command:**
```bash
chmod 644 /etc/letsencrypt/live/codex.firefrostgaming.com/fullchain.pem
chmod 600 /etc/letsencrypt/live/codex.firefrostgaming.com/privkey.pem
```

**Why:** Nginx needs read access, private key should be restricted

---

### Step 1.9: Start Nginx

**Command:**
```bash
systemctl start nginx
```

**Verification:**
```bash
systemctl status nginx
```

**Expected:** Shows "active (running)"

---

### Step 1.10: Enable Nginx Auto-Start

**Command:**
```bash
systemctl enable nginx
```

**Expected:** Nginx will start automatically on server reboot

---

### Step 1.11: Test Nginx Default Page

**Command:**
```bash
curl -I http://38.68.14.26
```

**Expected:**
```
HTTP/1.1 200 OK
Server: nginx
```

---

## PHASE 2: DEPLOY DOCKER STACK

**Estimated time:** 45 minutes

### Step 2.1: Create Deployment Directory

**Command:**
```bash
mkdir -p /opt/firefrost-codex
cd /opt/firefrost-codex
```

**Verification:**
```bash
pwd
```

**Expected output:** `/opt/firefrost-codex`

---

### Step 2.2: Create docker-compose.yml

**⚠️ CRITICAL: This file must be EXACT - no typos**

**Command:**
```bash
nano docker-compose.yml
```

**Paste this EXACT content:**

```yaml
services:
  # --- DATABASE & CACHE ---
  db:
    image: postgres:15-alpine
    restart: always
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: postgres
      POSTGRES_DB: dify
    volumes:
      - ./volumes/db/data:/var/lib/postgresql/data
    networks:
      - firefrost-net

  redis:
    image: redis:6-alpine
    restart: always
    volumes:
      - ./volumes/redis/data:/data
    networks:
      - firefrost-net

  # --- DIFY CORE ---
  dify-api:
    image: langgenius/dify-api:latest
    restart: always
    environment: &dify-env
      # Database
      DB_USERNAME: postgres
      DB_PASSWORD: ${DB_PASSWORD}
      DB_HOST: db
      DB_DATABASE: dify
      # Redis
      REDIS_HOST: redis
      REDIS_DB: 0
      # Vector Store
      VECTOR_STORE: qdrant
      QDRANT_HOST: qdrant
      QDRANT_PORT: 6333
      # Ollama (Connecting to host)
      OLLAMA_API_BASE_URL: http://host.docker.internal:11434
      # Security
      SECRET_KEY: ${DIFY_SECRET_KEY}
    depends_on:
      - db
      - redis
    networks:
      - firefrost-net
    extra_hosts:
      - "host.docker.internal:host-gateway"

  dify-worker:
    image: langgenius/dify-api:latest
    restart: always
    environment: *dify-env
    entrypoint: /bin/bash /entrypoint.sh worker
    depends_on:
      - dify-api
    networks:
      - firefrost-net

  dify-web:
    image: langgenius/dify-web:latest
    restart: always
    ports:
      - "127.0.0.1:3000:3000"
    networks:
      - firefrost-net

  # --- VECTOR STORE ---
  qdrant:
    image: qdrant/qdrant:latest
    restart: always
    ports:
      - "127.0.0.1:6333:6333"
    volumes:
      - ./volumes/qdrant/storage:/qdrant/storage
    networks:
      - firefrost-net

  # --- AUTOMATION & GIT ENGINE ---
  n8n:
    image: n8nio/n8n:latest
    restart: always
    ports:
      - "127.0.0.1:5678:5678"
    environment:
      - N8N_HOST=n8n.firefrostgaming.com
      - N8N_PORT=5678
      - N8N_PROTOCOL=https
      - NODE_FUNCTION_ALLOW_EXTERNAL=fs,path,child_process
      # Git Identity for Commits
      - GIT_USER_NAME=${GIT_USER_NAME}
      - GIT_USER_EMAIL=${GIT_USER_EMAIL}
    volumes:
      - ./volumes/n8n:/home/node/.n8n
      - ./git-repos:/data/git-repos
      - ~/.ssh:/home/node/.ssh:ro
    networks:
      - firefrost-net
    extra_hosts:
      - "host.docker.internal:host-gateway"

networks:
  firefrost-net:
    driver: bridge
```

**Save and exit:** Ctrl+O, Enter, Ctrl+X

---

### Step 2.3: Create .env File

**⚠️ CRITICAL: Replace ALL placeholder values with your actual values**

**Command:**
```bash
nano .env
```

**Paste and MODIFY with your actual values:**

```bash
# --- DATABASE SECRETS ---
# Generate with: openssl rand -base64 32
DB_PASSWORD=REPLACE_WITH_YOUR_GENERATED_PASSWORD

# --- DIFY CONFIGURATION ---
# Generate with: openssl rand -base64 42
DIFY_SECRET_KEY=REPLACE_WITH_YOUR_GENERATED_SECRET
DIFY_API_KEY=will_be_set_after_dify_setup

# --- GIT IDENTITY ---
GIT_USER_NAME=Firefrost Codex AI
GIT_USER_EMAIL=codex@firefrostgaming.com

# --- DISCORD CONFIGURATION ---
DISCORD_WEBHOOK_CODEX_ALERTS=https://discord.com/api/webhooks/YOUR_WEBHOOK_HERE
DISCORD_WEBHOOK_SYSTEM_CRITICAL=https://discord.com/api/webhooks/YOUR_WEBHOOK_HERE
MICHAEL_DISCORD_ID=YOUR_DISCORD_USER_ID_HERE

# --- KNOWLEDGE BASE IDS ---
# These will be set after creating datasets in Dify
DIFY_DATASET_ID_MAIN=will_be_set_later
DIFY_DATASET_ID_POKEROLE=will_be_set_later
```

**Save and exit:** Ctrl+O, Enter, Ctrl+X

**⚠️ DOUBLE CHECK:** 
- DB_PASSWORD is set (not the placeholder)
- DIFY_SECRET_KEY is set (not the placeholder)
- Discord webhooks are YOUR webhook URLs
- Michael's Discord ID is YOUR user ID

---

### Step 2.4: Create Directory Structure

**Command:**
```bash
mkdir -p volumes/db/data
mkdir -p volumes/redis/data
mkdir -p volumes/qdrant/storage
mkdir -p volumes/n8n
mkdir -p git-repos
```

**Verification:**
```bash
ls -la volumes/
```

**Expected:** Directories created (db, redis, qdrant, n8n)

---

### Step 2.5: Pull Docker Images

**⚠️ This will download ~2-3 GB of images**

**Command:**
```bash
docker-compose pull
```

**Expected:** Images download successfully
- postgres:15-alpine
- redis:6-alpine
- langgenius/dify-api:latest
- langgenius/dify-web:latest
- qdrant/qdrant:latest
- n8nio/n8n:latest

**Time:** 5-10 minutes depending on connection

**If fails:** Check internet connection, try again

---

### Step 2.6: Start Docker Stack

**⚠️ CRITICAL MOMENT - Stack deployment**

**Command:**
```bash
docker-compose up -d
```

**Expected output:**
```
Creating network "firefrost-codex_firefrost-net"
Creating firefrost-codex_db_1
Creating firefrost-codex_redis_1
Creating firefrost-codex_qdrant_1
Creating firefrost-codex_dify-api_1
Creating firefrost-codex_dify-worker_1
Creating firefrost-codex_dify-web_1
Creating firefrost-codex_n8n_1
```

**Time:** 30-60 seconds for all containers to start

---

### Step 2.7: Verify All Containers Running

**Command:**
```bash
docker-compose ps
```

**Expected:** ALL services show "Up" status
```
NAME                    STATE
db                      Up
redis                   Up
qdrant                  Up
dify-api                Up
dify-worker             Up
dify-web                Up
n8n                     Up
```

**If ANY show "Exit" or "Restarting":**
```bash
# Check logs for that service
docker-compose logs <service_name>
```

**Common issues:**
- db: Password not set in .env
- dify-api: Can't connect to database (wait 30 seconds, check again)
- n8n: Permission issues with volumes

---

### Step 2.8: Wait for Services to Initialize

**⚠️ IMPORTANT: Services need time to initialize**

**Command:**
```bash
sleep 60
```

**Why:** PostgreSQL needs to create databases, Dify needs to initialize schema

---

### Step 2.9: Check Dify API Health

**Command:**
```bash
curl http://127.0.0.1:3000
```

**Expected:** HTML response from Dify (NOT error)

**If connection refused:** Services still starting, wait another 30 seconds

---

### Step 2.10: Check n8n Health

**Command:**
```bash
curl http://127.0.0.1:5678
```

**Expected:** HTML response from n8n

---

### Step 2.11: Check Qdrant Health

**Command:**
```bash
curl http://127.0.0.1:6333/
```

**Expected:** JSON response with Qdrant version

---

### Step 2.12: Verify Ollama Connection

**Command:**
```bash
docker exec -it $(docker ps -qf "name=dify-api") curl http://host.docker.internal:11434/api/version
```

**Expected:** JSON response with Ollama version

**If fails:** Ollama not running on host, start it

---

## PHASE 3: CONFIGURE NGINX REVERSE PROXY

**Estimated time:** 20 minutes

### Step 3.1: Create Nginx Configuration

**Command:**
```bash
nano /etc/nginx/sites-available/firefrost-codex.conf
```

**Paste this EXACT content:**

```nginx
# Define rate limiting zones
limit_req_zone $binary_remote_addr zone=codex_limit:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=30r/s;

# Redirect all HTTP traffic to HTTPS
server {
    listen 80;
    listen [::]:80;
    server_name codex.firefrostgaming.com n8n.firefrostgaming.com;
    
    return 301 https://$host$request_uri;
}

# Dify (Codex) Server Block
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name codex.firefrostgaming.com;

    # SSL Configuration
    ssl_certificate /etc/letsencrypt/live/codex.firefrostgaming.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/codex.firefrostgaming.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Security Headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";
    add_header X-XSS-Protection "1; mode=block";

    # Allow large document uploads
    client_max_body_size 100M;

    # Apply standard rate limiting
    limit_req zone=codex_limit burst=20 nodelay;

    # Error Pages
    error_page 502 /502.html;
    location = /502.html {
        root /var/www/html;
        internal;
    }

    location / {
        proxy_pass http://127.0.0.1:3000;
        
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # WebSocket Support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        # Timeouts
        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
    }
}

# n8n Server Block
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name n8n.firefrostgaming.com;

    # SSL Configuration
    ssl_certificate /etc/letsencrypt/live/codex.firefrostgaming.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/codex.firefrostgaming.com/privkey.pem;

    # Security Headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";

    client_max_body_size 50M;

    # Webhooks (public access)
    location /webhook/ {
        limit_req zone=webhook_limit burst=50 nodelay;
        proxy_pass http://127.0.0.1:5678;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Main Editor Interface
    location / {
        # Optional: Lock down to your home IP
        # allow YOUR.HOME.IP.ADDRESS;
        # deny all;

        proxy_pass http://127.0.0.1:5678;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # WebSocket Support
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        # n8n SSE
        proxy_buffering off;
        proxy_cache off;
    }
}
```

**Save and exit:** Ctrl+O, Enter, Ctrl+X

---

### Step 3.2: Create Custom 502 Error Page

**Command:**
```bash
nano /var/www/html/502.html
```

**Paste this content:**

```html
<!DOCTYPE html>
<html>
<head>
    <title>Codex Offline</title>
    <style>
        body { 
            font-family: sans-serif; 
            text-align: center; 
            padding: 50px; 
            background: #1a1a1a; 
            color: #fff; 
        }
        h1 { color: #4CAF50; }
    </style>
</head>
<body>
    <h1>🔥❄️ Firefrost Codex is Restarting</h1>
    <p>The AI engine is currently running an automated update or self-healing routine.</p>
    <p>Please try again in 3 minutes.</p>
    <p>If this persists, Michael has already been notified via Discord.</p>
</body>
</html>
```

**Save and exit:** Ctrl+O, Enter, Ctrl+X

---

### Step 3.3: Enable Nginx Configuration

**Command:**
```bash
ln -s /etc/nginx/sites-available/firefrost-codex.conf /etc/nginx/sites-enabled/
```

**Verification:**
```bash
ls -l /etc/nginx/sites-enabled/
```

**Expected:** Symlink to firefrost-codex.conf exists

---

### Step 3.4: Test Nginx Configuration

**⚠️ CRITICAL: Test before reloading**

**Command:**
```bash
nginx -t
```

**Expected output:**
```
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
```

**If fails:**
- Check for typos in configuration
- Check certificate paths
- Review error messages carefully

---

### Step 3.5: Reload Nginx

**Command:**
```bash
systemctl reload nginx
```

**Verification:**
```bash
systemctl status nginx
```

**Expected:** Shows "active (running)" with no errors

---

### Step 3.6: Test HTTPS Access to Dify

**Command (from your local machine, NOT TX1):**
```
Open browser: https://codex.firefrostgaming.com
```

**Expected:** Dify setup page loads with valid SSL certificate

**If fails:**
- Check Nginx logs: `sudo tail -f /var/log/nginx/error.log`
- Verify Docker containers running: `docker-compose ps`
- Check firewall: `sudo ufw status`

---

### Step 3.7: Test HTTPS Access to n8n

**Command (from your local machine):**
```
Open browser: https://n8n.firefrostgaming.com
```

**Expected:** n8n setup page loads with valid SSL certificate

---

**END OF DEPLOYMENT-PLAN.md PART 1**

**Continue to DEPLOYMENT-PLAN-PART-2.md for remaining phases...**

💙🔥❄️
