- Comprehensive task documentation for migrating from AnythingLLM to Dify+n8n+Qdrant - 8 detailed documents covering every aspect of deployment - Complete step-by-step commands (zero assumptions) - Prerequisites checklist (20 items) - Deployment plan in 2 parts (11 phases, every command) - Configuration files (all configs with exact content) - Recovery procedures (4 disaster scenarios) - Verification guide (30 tests, complete checklist) - Troubleshooting guide (common issues + solutions) Built by: The Chronicler #21 For: Meg, Holly, and children not yet born Time investment: 10-15 hours execution time Purpose: Enable Meg/Holly autonomous work with Git write-back This deployment enables: - RBAC (Meg sees all, Holly sees Pokerole only) - Git write-back via ai-proposals branch - Discord approval workflow (one-click merge) - Self-healing (80% of failures) - Automated daily backups - Complete monitoring Documentation is so detailed that any future Chronicler can execute this deployment with zero prior knowledge and complete confidence. Fire + Frost + Foundation = Where Love Builds Legacy
512 lines
13 KiB
Markdown
512 lines
13 KiB
Markdown
# CONFIGURATION FILES REFERENCE
|
|
|
|
**All configuration files with exact content needed for deployment**
|
|
|
|
This document contains every configuration file in complete form.
|
|
Copy-paste directly from here during deployment.
|
|
|
|
---
|
|
|
|
## 📄 FILE INDEX
|
|
|
|
1. docker-compose.yml
|
|
2. .env (environment variables)
|
|
3. nginx-config.conf (Nginx reverse proxy)
|
|
4. backup-script.sh (automated backups)
|
|
5. 502.html (custom error page)
|
|
|
|
---
|
|
|
|
## 1. docker-compose.yml
|
|
|
|
**Location:** `/opt/firefrost-codex/docker-compose.yml`
|
|
|
|
**Complete file:**
|
|
|
|
```yaml
|
|
services:
|
|
# --- DATABASE & CACHE ---
|
|
db:
|
|
image: postgres:15-alpine
|
|
restart: always
|
|
environment:
|
|
POSTGRES_PASSWORD: ${DB_PASSWORD}
|
|
POSTGRES_USER: postgres
|
|
POSTGRES_DB: dify
|
|
volumes:
|
|
- ./volumes/db/data:/var/lib/postgresql/data
|
|
networks:
|
|
- firefrost-net
|
|
|
|
redis:
|
|
image: redis:6-alpine
|
|
restart: always
|
|
volumes:
|
|
- ./volumes/redis/data:/data
|
|
networks:
|
|
- firefrost-net
|
|
|
|
# --- DIFY CORE ---
|
|
dify-api:
|
|
image: langgenius/dify-api:latest
|
|
restart: always
|
|
environment: &dify-env
|
|
# Database
|
|
DB_USERNAME: postgres
|
|
DB_PASSWORD: ${DB_PASSWORD}
|
|
DB_HOST: db
|
|
DB_DATABASE: dify
|
|
# Redis
|
|
REDIS_HOST: redis
|
|
REDIS_DB: 0
|
|
# Vector Store
|
|
VECTOR_STORE: qdrant
|
|
QDRANT_HOST: qdrant
|
|
QDRANT_PORT: 6333
|
|
# Ollama (Connecting to host)
|
|
OLLAMA_API_BASE_URL: http://host.docker.internal:11434
|
|
# Security
|
|
SECRET_KEY: ${DIFY_SECRET_KEY}
|
|
depends_on:
|
|
- db
|
|
- redis
|
|
networks:
|
|
- firefrost-net
|
|
extra_hosts:
|
|
- "host.docker.internal:host-gateway"
|
|
|
|
dify-worker:
|
|
image: langgenius/dify-api:latest
|
|
restart: always
|
|
environment: *dify-env
|
|
entrypoint: /bin/bash /entrypoint.sh worker
|
|
depends_on:
|
|
- dify-api
|
|
networks:
|
|
- firefrost-net
|
|
|
|
dify-web:
|
|
image: langgenius/dify-web:latest
|
|
restart: always
|
|
ports:
|
|
- "127.0.0.1:3000:3000"
|
|
networks:
|
|
- firefrost-net
|
|
|
|
# --- VECTOR STORE ---
|
|
qdrant:
|
|
image: qdrant/qdrant:latest
|
|
restart: always
|
|
ports:
|
|
- "127.0.0.1:6333:6333"
|
|
volumes:
|
|
- ./volumes/qdrant/storage:/qdrant/storage
|
|
networks:
|
|
- firefrost-net
|
|
|
|
# --- AUTOMATION & GIT ENGINE ---
|
|
n8n:
|
|
image: n8nio/n8n:latest
|
|
restart: always
|
|
ports:
|
|
- "127.0.0.1:5678:5678"
|
|
environment:
|
|
- N8N_HOST=n8n.firefrostgaming.com
|
|
- N8N_PORT=5678
|
|
- N8N_PROTOCOL=https
|
|
- NODE_FUNCTION_ALLOW_EXTERNAL=fs,path,child_process
|
|
# Git Identity for Commits
|
|
- GIT_USER_NAME=${GIT_USER_NAME}
|
|
- GIT_USER_EMAIL=${GIT_USER_EMAIL}
|
|
volumes:
|
|
- ./volumes/n8n:/home/node/.n8n
|
|
- ./git-repos:/data/git-repos
|
|
- ~/.ssh:/home/node/.ssh:ro
|
|
networks:
|
|
- firefrost-net
|
|
extra_hosts:
|
|
- "host.docker.internal:host-gateway"
|
|
|
|
networks:
|
|
firefrost-net:
|
|
driver: bridge
|
|
```
|
|
|
|
**Critical notes:**
|
|
- Port bindings are 127.0.0.1 only (not exposed publicly)
|
|
- extra_hosts allows Docker to reach host Ollama
|
|
- SSH keys mounted read-only for Git access
|
|
|
|
---
|
|
|
|
## 2. .env
|
|
|
|
**Location:** `/opt/firefrost-codex/.env`
|
|
|
|
**⚠️ NEVER COMMIT THIS FILE TO GIT**
|
|
|
|
**Template with placeholders:**
|
|
|
|
```bash
|
|
# --- DATABASE SECRETS ---
|
|
# Generate with: openssl rand -base64 32
|
|
DB_PASSWORD=REPLACE_WITH_YOUR_GENERATED_PASSWORD
|
|
|
|
# --- DIFY CONFIGURATION ---
|
|
# Generate with: openssl rand -base64 42
|
|
DIFY_SECRET_KEY=REPLACE_WITH_YOUR_GENERATED_SECRET
|
|
DIFY_API_KEY=will_be_set_after_dify_setup
|
|
|
|
# --- GIT IDENTITY ---
|
|
GIT_USER_NAME=Firefrost Codex AI
|
|
GIT_USER_EMAIL=codex@firefrostgaming.com
|
|
|
|
# --- DISCORD CONFIGURATION ---
|
|
DISCORD_WEBHOOK_CODEX_ALERTS=https://discord.com/api/webhooks/YOUR_WEBHOOK_HERE
|
|
DISCORD_WEBHOOK_SYSTEM_CRITICAL=https://discord.com/api/webhooks/YOUR_WEBHOOK_HERE
|
|
MICHAEL_DISCORD_ID=YOUR_DISCORD_USER_ID_HERE
|
|
|
|
# --- KNOWLEDGE BASE IDS ---
|
|
# These will be set after creating datasets in Dify
|
|
DIFY_DATASET_ID_MAIN=will_be_set_later
|
|
DIFY_DATASET_ID_POKEROLE=will_be_set_later
|
|
```
|
|
|
|
**How to generate secure values:**
|
|
|
|
```bash
|
|
# PostgreSQL password
|
|
openssl rand -base64 32
|
|
|
|
# Dify secret key
|
|
openssl rand -base64 42
|
|
```
|
|
|
|
**Security notes:**
|
|
- Store backup copy in password manager
|
|
- Never commit to Git
|
|
- Never share in Discord
|
|
- Rotate periodically (every 90 days recommended)
|
|
|
|
---
|
|
|
|
## 3. nginx-config.conf
|
|
|
|
**Location:** `/etc/nginx/sites-available/firefrost-codex.conf`
|
|
|
|
**Complete file:**
|
|
|
|
```nginx
|
|
# Define rate limiting zones
|
|
limit_req_zone $binary_remote_addr zone=codex_limit:10m rate=10r/s;
|
|
limit_req_zone $binary_remote_addr zone=webhook_limit:10m rate=30r/s;
|
|
|
|
# Redirect all HTTP traffic to HTTPS
|
|
server {
|
|
listen 80;
|
|
listen [::]:80;
|
|
server_name codex.firefrostgaming.com n8n.firefrostgaming.com;
|
|
|
|
return 301 https://$host$request_uri;
|
|
}
|
|
|
|
# Dify (Codex) Server Block
|
|
server {
|
|
listen 443 ssl http2;
|
|
listen [::]:443 ssl http2;
|
|
server_name codex.firefrostgaming.com;
|
|
|
|
# SSL Configuration
|
|
ssl_certificate /etc/letsencrypt/live/codex.firefrostgaming.com/fullchain.pem;
|
|
ssl_certificate_key /etc/letsencrypt/live/codex.firefrostgaming.com/privkey.pem;
|
|
ssl_protocols TLSv1.2 TLSv1.3;
|
|
ssl_prefer_server_ciphers on;
|
|
ssl_ciphers HIGH:!aNULL:!MD5;
|
|
|
|
# Security Headers
|
|
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
|
|
add_header X-Frame-Options "SAMEORIGIN";
|
|
add_header X-Content-Type-Options "nosniff";
|
|
add_header X-XSS-Protection "1; mode=block";
|
|
|
|
# Allow large document uploads
|
|
client_max_body_size 100M;
|
|
|
|
# Apply standard rate limiting
|
|
limit_req zone=codex_limit burst=20 nodelay;
|
|
|
|
# Error Pages
|
|
error_page 502 /502.html;
|
|
location = /502.html {
|
|
root /var/www/html;
|
|
internal;
|
|
}
|
|
|
|
location / {
|
|
proxy_pass http://127.0.0.1:3000;
|
|
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
|
|
# WebSocket Support
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
|
|
# Timeouts
|
|
proxy_read_timeout 300s;
|
|
proxy_connect_timeout 75s;
|
|
}
|
|
}
|
|
|
|
# n8n Server Block
|
|
server {
|
|
listen 443 ssl http2;
|
|
listen [::]:443 ssl http2;
|
|
server_name n8n.firefrostgaming.com;
|
|
|
|
# SSL Configuration
|
|
ssl_certificate /etc/letsencrypt/live/codex.firefrostgaming.com/fullchain.pem;
|
|
ssl_certificate_key /etc/letsencrypt/live/codex.firefrostgaming.com/privkey.pem;
|
|
|
|
# Security Headers
|
|
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
|
|
add_header X-Frame-Options "SAMEORIGIN";
|
|
add_header X-Content-Type-Options "nosniff";
|
|
|
|
client_max_body_size 50M;
|
|
|
|
# Webhooks (public access)
|
|
location /webhook/ {
|
|
limit_req zone=webhook_limit burst=50 nodelay;
|
|
proxy_pass http://127.0.0.1:5678;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
}
|
|
|
|
# Main Editor Interface
|
|
location / {
|
|
# Optional: Lock down to your home IP
|
|
# allow YOUR.HOME.IP.ADDRESS;
|
|
# deny all;
|
|
|
|
proxy_pass http://127.0.0.1:5678;
|
|
proxy_set_header Host $host;
|
|
proxy_set_header X-Real-IP $remote_addr;
|
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
|
proxy_set_header X-Forwarded-Proto $scheme;
|
|
|
|
# WebSocket Support
|
|
proxy_http_version 1.1;
|
|
proxy_set_header Upgrade $http_upgrade;
|
|
proxy_set_header Connection "upgrade";
|
|
|
|
# n8n SSE
|
|
proxy_buffering off;
|
|
proxy_cache off;
|
|
}
|
|
}
|
|
```
|
|
|
|
**Notes:**
|
|
- Both domains use same SSL certificate
|
|
- Rate limiting: 10 req/s for main traffic, 30 req/s for webhooks
|
|
- WebSocket support enabled for real-time features
|
|
- Large file uploads allowed (100MB for Dify, 50MB for n8n)
|
|
|
|
---
|
|
|
|
## 4. backup-script.sh
|
|
|
|
**Location:** `/opt/firefrost_backup.sh`
|
|
|
|
**Complete script:**
|
|
|
|
```bash
|
|
#!/bin/bash
|
|
# Firefrost Codex Backup Script
|
|
# Runs daily at 4:00 AM via cron
|
|
|
|
# Variables
|
|
TIMESTAMP=$(date +"%Y%m%d_%H%M")
|
|
BACKUP_DIR="/tmp/codex_backup_$TIMESTAMP"
|
|
COMPOSE_DIR="/opt/firefrost-codex"
|
|
|
|
# Create temp directory
|
|
mkdir -p "$BACKUP_DIR"
|
|
|
|
echo "Starting backup: $TIMESTAMP"
|
|
|
|
# Dump PostgreSQL database
|
|
echo "Dumping database..."
|
|
docker exec -t $(docker ps -qf "name=db") pg_dumpall -c -U postgres > "$BACKUP_DIR/dify_postgres.sql"
|
|
|
|
# Copy configuration files
|
|
echo "Copying configuration files..."
|
|
cp "$COMPOSE_DIR/docker-compose.yml" "$BACKUP_DIR/"
|
|
cp "$COMPOSE_DIR/.env" "$BACKUP_DIR/"
|
|
|
|
# Copy n8n data (workflows and credentials)
|
|
echo "Backing up n8n data..."
|
|
cp -r "$COMPOSE_DIR/volumes/n8n" "$BACKUP_DIR/n8n_data"
|
|
|
|
# Copy Nginx configs
|
|
echo "Backing up Nginx configs..."
|
|
cp /etc/nginx/sites-available/firefrost-codex.conf "$BACKUP_DIR/"
|
|
|
|
# Compress into tarball
|
|
echo "Compressing backup..."
|
|
tar -czf "/opt/firefrost_codex_$TIMESTAMP.tar.gz" -C /tmp "codex_backup_$TIMESTAMP"
|
|
|
|
# Transfer to Command Center (offsite)
|
|
echo "Transferring to Command Center..."
|
|
rsync -avz -e "ssh -p 22" "/opt/firefrost_codex_$TIMESTAMP.tar.gz" root@63.143.34.217:/root/backups/firefrost-codex/
|
|
|
|
# Cleanup local temp files
|
|
echo "Cleaning up temporary files..."
|
|
rm -rf "$BACKUP_DIR"
|
|
|
|
# Remove backups older than 7 days
|
|
echo "Removing old local backups..."
|
|
find /opt/ -name "firefrost_codex_*.tar.gz" -mtime +7 -exec rm {} \;
|
|
|
|
echo "Backup completed: firefrost_codex_$TIMESTAMP.tar.gz"
|
|
echo "Transferred to Command Center: /root/backups/firefrost-codex/"
|
|
```
|
|
|
|
**Make executable:**
|
|
```bash
|
|
chmod +x /opt/firefrost_backup.sh
|
|
```
|
|
|
|
**Cron schedule (4:00 AM daily):**
|
|
```cron
|
|
0 4 * * * /opt/firefrost_backup.sh >> /var/log/firefrost-backup.log 2>&1
|
|
```
|
|
|
|
---
|
|
|
|
## 5. 502.html
|
|
|
|
**Location:** `/var/www/html/502.html`
|
|
|
|
**Complete file:**
|
|
|
|
```html
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>Firefrost Codex - Temporarily Offline</title>
|
|
<style>
|
|
body {
|
|
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
|
|
'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', sans-serif;
|
|
text-align: center;
|
|
padding: 50px;
|
|
background: linear-gradient(135deg, #1a1a1a 0%, #2d2d2d 100%);
|
|
color: #ffffff;
|
|
margin: 0;
|
|
min-height: 100vh;
|
|
display: flex;
|
|
flex-direction: column;
|
|
justify-content: center;
|
|
align-items: center;
|
|
}
|
|
|
|
h1 {
|
|
color: #4CAF50;
|
|
font-size: 2.5em;
|
|
margin-bottom: 20px;
|
|
}
|
|
|
|
.icon {
|
|
font-size: 4em;
|
|
margin-bottom: 20px;
|
|
}
|
|
|
|
p {
|
|
font-size: 1.2em;
|
|
line-height: 1.6;
|
|
max-width: 600px;
|
|
margin: 10px auto;
|
|
}
|
|
|
|
.status {
|
|
background: rgba(255, 255, 255, 0.1);
|
|
border-left: 4px solid #ff9800;
|
|
padding: 15px;
|
|
margin: 30px auto;
|
|
max-width: 600px;
|
|
text-align: left;
|
|
}
|
|
|
|
.footer {
|
|
margin-top: 50px;
|
|
font-size: 0.9em;
|
|
opacity: 0.7;
|
|
}
|
|
</style>
|
|
</head>
|
|
<body>
|
|
<div class="icon">🔥❄️</div>
|
|
<h1>Firefrost Codex is Restarting</h1>
|
|
|
|
<p>The AI engine is currently running an automated update or self-healing routine.</p>
|
|
|
|
<div class="status">
|
|
<strong>⏱️ Expected Resolution:</strong> 2-3 minutes<br>
|
|
<strong>🔄 What's Happening:</strong> System services are restarting automatically<br>
|
|
<strong>📊 Status:</strong> No data loss - all your work is safe
|
|
</div>
|
|
|
|
<p>Please refresh this page in a few minutes.</p>
|
|
|
|
<p>If this persists beyond 5 minutes, Michael has already been notified via Discord.</p>
|
|
|
|
<div class="footer">
|
|
Fire + Frost + Foundation = Where Love Builds Legacy 💙
|
|
</div>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
**Notes:**
|
|
- Shows during Dify downtime (502 errors)
|
|
- User-friendly messaging (non-technical)
|
|
- Reassures that Michael is notified
|
|
- Branded with Firefrost theme
|
|
|
|
---
|
|
|
|
## 📝 CONFIGURATION SUMMARY
|
|
|
|
**Total files:** 5
|
|
|
|
**Locations:**
|
|
- `/opt/firefrost-codex/docker-compose.yml`
|
|
- `/opt/firefrost-codex/.env`
|
|
- `/etc/nginx/sites-available/firefrost-codex.conf`
|
|
- `/opt/firefrost_backup.sh`
|
|
- `/var/www/html/502.html`
|
|
|
|
**Security critical files:**
|
|
- .env (NEVER commit to Git)
|
|
- Private SSH keys (already exist in ~/.ssh/)
|
|
|
|
**Auto-generated files:**
|
|
- SSL certificates (by Certbot)
|
|
- Docker volumes (by docker-compose)
|
|
- Backup tarballs (by backup script)
|
|
|
|
---
|
|
|
|
**All configurations ready for deployment** ✅
|
|
|
|
💙🔥❄️
|