383 lines
11 KiB
Bash
Executable File
383 lines
11 KiB
Bash
Executable File
#!/bin/bash
|
|
# =============================================================================
|
|
# FIREFROST DOCUMENTATION GAP FIX SCRIPT
|
|
# Created: February 9, 2026
|
|
# Purpose: Fix all documentation gaps identified in comprehensive audit
|
|
# =============================================================================
|
|
|
|
set -e
|
|
cd ~/firefrost-work/firefrost-operations-manual
|
|
|
|
echo "=========================================="
|
|
echo "FIREFROST DOCUMENTATION GAP FIX"
|
|
echo "Started: $(date)"
|
|
echo "=========================================="
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# GAP 1: Create phase0-dismantling.md
|
|
# -----------------------------------------------------------------------------
|
|
echo ""
|
|
echo "[1/5] Creating docs/phase0-dismantling.md..."
|
|
|
|
cat > docs/phase0-dismantling.md << 'PHASE0EOF'
|
|
# Phase 0: Infrastructure Dismantling & Vanilla Reset
|
|
|
|
**Date:** February 7, 2026
|
|
**Status:** COMPLETE
|
|
**Purpose:** Document what was removed and why during the Phase 0 vanilla reset
|
|
|
|
---
|
|
|
|
## Executive Summary
|
|
|
|
On February 7, 2026, we dismantled the "Frostwall Protocol v1.0" - a complex GRE tunnel architecture that was causing more problems than it solved. This document preserves the technical details for future reference and explains the strategic decision to rebuild from a "vanilla baseline."
|
|
|
|
---
|
|
|
|
## What Was Dismantled
|
|
|
|
### Command Center (63.143.34.217)
|
|
|
|
**GRE Tunnels Removed:**
|
|
- gre-nc1 - Tunnel to NC1 Charlotte (192.168.20.1/30)
|
|
- gre-tx1 - Tunnel to TX1 Dallas (192.168.10.1/30)
|
|
|
|
**Processes Killed:**
|
|
- 68 leaked tunnel-related processes
|
|
- master_restore.sh background processes
|
|
- reboot_audit.sh background processes
|
|
|
|
**Cron Jobs Disabled:**
|
|
- master_restore.sh - Auto-restore tunnel configuration
|
|
- reboot_audit.sh - Tunnel health monitoring
|
|
|
|
**iptables Rules Cleaned:**
|
|
- All GRE-related NAT rules
|
|
- All tunnel routing rules
|
|
- Reset to default firewall policy
|
|
|
|
### NC1 Charlotte (216.239.104.130)
|
|
|
|
**GRE Tunnel Removed:**
|
|
- gre-cc - Tunnel to Command Center
|
|
- Tunnel IP: 192.168.20.2/30
|
|
- Peer: 63.143.34.217
|
|
|
|
### TX1 Dallas (38.68.14.26)
|
|
|
|
**GRE Tunnel Removed:**
|
|
- gre-tx1 - Tunnel to Command Center
|
|
- Tunnel IP: 192.168.10.2/30
|
|
- Secondary IP on tunnel: 38.68.14.188/32 (Billing Portal routing)
|
|
- Peer: 63.143.34.217
|
|
|
|
---
|
|
|
|
## Why It Was Removed
|
|
|
|
### Problem 1: CosmicGuard Double-Encapsulation
|
|
The original Charlotte node was behind CosmicGuard DDoS protection, which automatically creates GRE tunnels. Running our tunnel over their tunnel created double encapsulation and MTU issues.
|
|
|
|
### Problem 2: Protocol 47 Blocking
|
|
Upstream carrier was black-holing Protocol 47 (GRE) on 38.x IP ranges. Required migration to 216.239.104.x range.
|
|
|
|
### Problem 3: Complexity vs. Benefit
|
|
Constant connectivity issues, difficult troubleshooting, 68+ leaked processes, midnight emergencies.
|
|
|
|
### Problem 4: Maintenance Burden
|
|
With Michael's health and family planning goals, midnight pages were unsustainable.
|
|
|
|
---
|
|
|
|
## The Decision: Vanilla Reset
|
|
|
|
**Philosophy:** "Start from a clean baseline and rebuild properly."
|
|
|
|
**Future Plan (Phase 1):**
|
|
- Design simplified DDoS protection
|
|
- Cloudflare Spectrum or simplified GRE (decision pending)
|
|
- Focus on reliability over complexity
|
|
|
|
---
|
|
|
|
## Lessons Learned
|
|
|
|
1. Complexity has a cost - Every added layer is a potential failure point
|
|
2. Health matters - Infrastructure should support life, not consume it
|
|
3. Document before dismantling - This document preserves institutional knowledge
|
|
4. Vanilla baseline enables iteration - Easier to build correctly from scratch
|
|
5. Provider relationships matter - Breezehost's Jon Beard was crucial
|
|
|
|
---
|
|
|
|
## Revision History
|
|
|
|
| Version | Date | Changes |
|
|
|---------|------|---------|
|
|
| 1.0 | 2026-02-09 | Initial documentation (retroactive) |
|
|
|
|
PHASE0EOF
|
|
|
|
echo "Created docs/phase0-dismantling.md"
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# GAP 2: Update architecture-decisions.md
|
|
# -----------------------------------------------------------------------------
|
|
echo ""
|
|
echo "[2/5] Updating docs/architecture-decisions.md..."
|
|
|
|
cat >> docs/architecture-decisions.md << 'ARCHEOF'
|
|
|
|
---
|
|
|
|
## Decision: Management Services on TX1 (Not Command Center)
|
|
|
|
**Date:** February 9, 2026
|
|
**Decision:** Deploy Phase 0.5 management services on TX1 Dallas instead of Command Center
|
|
**Status:** IMPLEMENTED
|
|
|
|
### Rationale
|
|
1. TX1 has 32 vCPU, 256GB RAM - currently 99% idle
|
|
2. Simpler networking without Command Center routing
|
|
3. Keeps Command Center clean for future Frostwall v2.0 DDoS protection
|
|
4. Gitea migration to TX1 was seamless - proven success
|
|
|
|
### Command Center Future Role
|
|
Reserved for Phase 1 DDoS protection (GRE hub, Cloudflare integration)
|
|
|
|
---
|
|
|
|
## Known Limitation: NC1 and TX1 Cannot Communicate Directly
|
|
|
|
**Date:** February 9, 2026
|
|
**Status:** PERMANENT INFRASTRUCTURE CONSTRAINT
|
|
|
|
NC1 Charlotte and TX1 Dallas are in different Breezehost datacenters with no direct routing.
|
|
|
|
### Impact
|
|
- Uptime Kuma on TX1 cannot monitor NC1 services
|
|
- NC1 game servers excluded from TX1-based monitoring
|
|
- Cross-datacenter communication requires public internet
|
|
|
|
### Acceptance
|
|
This is standard for multi-datacenter hosting and does not affect normal operations.
|
|
|
|
---
|
|
|
|
## Decision: Three-Tier Documentation Architecture
|
|
|
|
**Date:** February 9, 2026
|
|
**Decision:** Replace single BookStack with three-tier system
|
|
|
|
| Tier | Technology | Domain | Access |
|
|
|------|------------|--------|--------|
|
|
| PUBLIC | MkDocs | docs.firefrostgaming.com | Anyone |
|
|
| SUBSCRIBERS | Wiki.js + NextCloud | subscribers.firefrostgaming.com | Paid |
|
|
| STAFF | Wiki.js | staff.firefrostgaming.com | Staff |
|
|
|
|
### Rationale
|
|
- Security boundaries per tier
|
|
- Git-native public docs (MkDocs)
|
|
- UI-friendly private docs (Wiki.js for Meg)
|
|
- Appropriate tools for each use case
|
|
|
|
ARCHEOF
|
|
|
|
echo "Updated docs/architecture-decisions.md"
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# GAP 3: Update pterodactyl-extensions-plan.md
|
|
# -----------------------------------------------------------------------------
|
|
echo ""
|
|
echo "[3/5] Updating docs/pterodactyl-extensions-plan.md..."
|
|
|
|
cat >> docs/pterodactyl-extensions-plan.md << 'EXTEOF'
|
|
|
|
---
|
|
|
|
## License Inventory (Updated February 9, 2026)
|
|
|
|
### Already Installed
|
|
| Extension | Vendor | Status |
|
|
|-----------|--------|--------|
|
|
| Blueprint | - | INSTALLED |
|
|
| PteroStats | Elurym | INSTALLED |
|
|
| Subdomain Manager | CorwinDev | INSTALLED |
|
|
|
|
### Purchased - Awaiting Installation
|
|
| Extension | Vendor | Expires | Status |
|
|
|-----------|--------|---------|--------|
|
|
| Node Usage Status | Velta Studios | Never | NOT INSTALLED |
|
|
| Modpack Installer | Arnaud Lier | Never | NOT INSTALLED |
|
|
| Citadel Theme (Paymenter) | willos themes | Jan 26, 2027 | NOT INSTALLED |
|
|
|
|
### Installation Priority (Phase 0.6)
|
|
1. Node Usage Status - Monitor NC1/TX1 resources
|
|
2. Modpack Installer - One-click modpack deployment
|
|
3. Citadel Theme - Professional billing appearance
|
|
|
|
EXTEOF
|
|
|
|
echo "Updated docs/pterodactyl-extensions-plan.md"
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# GAP 4: Create mkdocs-deployment.md
|
|
# -----------------------------------------------------------------------------
|
|
echo ""
|
|
echo "[4/5] Creating docs/mkdocs-deployment.md..."
|
|
|
|
cat > docs/mkdocs-deployment.md << 'MKDOCSEOF'
|
|
# MkDocs Deployment Documentation
|
|
|
|
**Service:** 3/5 in Phase 0.5
|
|
**Domain:** docs.firefrostgaming.com
|
|
**Server:** TX1 Dallas (38.68.14.26)
|
|
**Deployed:** February 9, 2026
|
|
**Status:** OPERATIONAL
|
|
|
|
---
|
|
|
|
## Service Overview
|
|
|
|
MkDocs with Material theme provides the PUBLIC documentation tier for Firefrost Gaming.
|
|
|
|
## Technical Specifications
|
|
|
|
| Component | Value |
|
|
|-----------|-------|
|
|
| Software | MkDocs + Material Theme |
|
|
| Location | TX1 Dallas (38.68.14.26) |
|
|
| Domain | docs.firefrostgaming.com |
|
|
| SSL | Let's Encrypt (auto-renewal) |
|
|
|
|
## Why MkDocs (Not BookStack)
|
|
|
|
- Simple (static HTML vs database)
|
|
- Git-native (markdown in Gitea)
|
|
- Minimal resources
|
|
- Perfect for public docs
|
|
|
|
BookStack/Wiki.js reserved for SUBSCRIBER and STAFF tiers.
|
|
|
|
## Revision History
|
|
|
|
| Version | Date | Changes |
|
|
|---------|------|---------|
|
|
| 1.0 | 2026-02-09 | Initial deployment documentation |
|
|
|
|
MKDOCSEOF
|
|
|
|
echo "Created docs/mkdocs-deployment.md"
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# GAP 5: Update INDEX.md
|
|
# -----------------------------------------------------------------------------
|
|
echo ""
|
|
echo "[5/5] Updating docs/INDEX.md..."
|
|
|
|
cat > docs/INDEX.md << 'INDEXEOF'
|
|
# Firefrost Gaming Documentation Index
|
|
|
|
**Last Updated:** February 9, 2026 (Post-Audit)
|
|
|
|
---
|
|
|
|
## START HERE
|
|
|
|
### THE MASTER DOCUMENT
|
|
**FIREFROST-PROJECT-SCOPE-V2.md** - Single source of truth
|
|
|
|
### ESSENTIAL CONTEXT
|
|
- session-handoff.md - Current status for Claude
|
|
- workflow-guide.md - How Michael & Claude work together
|
|
- design-bible-v1.1.md - Visual identity & branding
|
|
- path-philosophy.md - Fire vs Frost philosophy
|
|
|
|
---
|
|
|
|
## OPERATIONAL DOCUMENTATION
|
|
|
|
### Infrastructure & Deployment
|
|
| Document | Status |
|
|
|----------|--------|
|
|
| gitea-deployment.md | Complete |
|
|
| uptime-kuma-deployment.md | Complete |
|
|
| mkdocs-deployment.md | Complete |
|
|
| phase0-dismantling.md | Complete |
|
|
|
|
### Architecture & Decisions
|
|
- architecture-decisions.md - Technical decisions
|
|
- pterodactyl-extensions-plan.md - Extension licenses
|
|
|
|
### Automation
|
|
- automation/USAGE.md - Daemon and queue system
|
|
|
|
### Business & Community
|
|
- subscription-tiers-final.md
|
|
- awakened-gateway.md
|
|
- discord-structure-complete.md
|
|
- luckperms-structure.md
|
|
|
|
### Social & Marketing
|
|
- firefrost-social-strategy.md
|
|
- megs-social-setup-guide.md
|
|
- visual-assets-guide.md
|
|
|
|
### Planning
|
|
- firefrost-master-implementation-plan.md
|
|
- firefrost-shopping-list.md
|
|
|
|
### Relationships
|
|
- provider-communications.md - Breezehost archive
|
|
|
|
---
|
|
|
|
## CURRENT STATUS
|
|
|
|
Phase 0.5: 60% complete (3/5 services)
|
|
- Gitea: DEPLOYED
|
|
- Uptime Kuma: DEPLOYED
|
|
- MkDocs: DEPLOYED
|
|
- Wiki.js: PLANNED
|
|
- NextCloud: PLANNED
|
|
|
|
### Known Limitations
|
|
- NC1 and TX1 cannot communicate directly (different datacenters)
|
|
|
|
---
|
|
|
|
**Fire + Frost = Where Passion Meets Precision**
|
|
|
|
INDEXEOF
|
|
|
|
echo "Updated docs/INDEX.md"
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Commit all changes
|
|
# -----------------------------------------------------------------------------
|
|
echo ""
|
|
echo "=========================================="
|
|
echo "Committing all changes to Git..."
|
|
echo "=========================================="
|
|
|
|
git add docs/phase0-dismantling.md
|
|
git add docs/architecture-decisions.md
|
|
git add docs/pterodactyl-extensions-plan.md
|
|
git add docs/mkdocs-deployment.md
|
|
git add docs/INDEX.md
|
|
|
|
git commit -m "Fix documentation gaps from comprehensive audit
|
|
|
|
NEW: phase0-dismantling.md, mkdocs-deployment.md
|
|
UPDATED: architecture-decisions.md, pterodactyl-extensions-plan.md, INDEX.md
|
|
|
|
Audit: February 9, 2026 - All 5 gaps fixed"
|
|
|
|
git push
|
|
|
|
echo ""
|
|
echo "=========================================="
|
|
echo "DOCUMENTATION GAP FIX COMPLETE"
|
|
echo "Finished: $(date)"
|
|
echo "=========================================="
|