115 lines
3.3 KiB
Bash
115 lines
3.3 KiB
Bash
#!/bin/bash
|
|
# Add TX1 Crisis Status to session-handoff.md
|
|
# February 11, 2026 - Critical infrastructure issue discovered
|
|
|
|
cd ~/firefrost-work/firefrost-operations-manual
|
|
|
|
cat >> docs/session-handoff.md << 'EOF'
|
|
|
|
---
|
|
|
|
## 🚨 CRITICAL: TX1 GAME SERVERS CRISIS (FEB 11, 2026 - 5:00 PM)
|
|
|
|
### **THE DISCOVERY:**
|
|
|
|
While diagnosing Holly's FoundryVTT access issues (50% success rate), discovered:
|
|
|
|
**❌ ALL 6 TX1 GAME SERVERS ARE DOWN**
|
|
|
|
### **ROOT CAUSE:**
|
|
|
|
**Pterodactyl Panel misconfiguration:**
|
|
- All TX1 servers configured with IP: `38.68.14.188` (Billing VPS)
|
|
- TX1's actual IPs: `38.68.14.26-30` (game servers)
|
|
- Docker cannot bind to IP that doesn't exist on machine
|
|
- Result: All servers fail to start (Exit Code 128)
|
|
|
|
### **AFFECTED SERVERS:**
|
|
|
|
1. Stoneblock 4 (wrong: 38.68.14.188:25565)
|
|
2. Reclamation (wrong: 38.68.14.188:25567)
|
|
3. Society: Sunlit Valley (wrong: 38.68.14.188:25568)
|
|
4. Vanilla 1.21.11 (wrong: 38.68.14.188:25571)
|
|
5. All The Mons (wrong: 38.68.14.188:25572)
|
|
6. FoundryVTT (wrong: 38.68.14.188:30000)
|
|
|
|
### **TX1 VERIFIED IPs:**
|
|
```
|
|
38.68.14.26 (game servers)
|
|
38.68.14.27 (game servers)
|
|
38.68.14.28 (game servers)
|
|
38.68.14.29 (game servers)
|
|
38.68.14.30 (game servers)
|
|
74.63.218.202 (Code-Server - already working)
|
|
74.63.218.203 (unused)
|
|
74.63.218.204 (unused)
|
|
74.63.218.205 (unused)
|
|
```
|
|
|
|
### **FIX IN PROGRESS:**
|
|
|
|
**Step 1: Allocate Port Ranges (IN PROGRESS)**
|
|
- For each IP (38.68.14.26-30):
|
|
- Minecraft: 25565-25580 (16 ports)
|
|
- Hytale: 5520-5521 (2 ports)
|
|
- FoundryVTT: 30000 (only on .26)
|
|
|
|
**Step 2: Assign Correct Allocations**
|
|
- Stoneblock 4 → 38.68.14.26:25565
|
|
- Reclamation → 38.68.14.27:25565
|
|
- Society → 38.68.14.28:25565
|
|
- Vanilla → 38.68.14.29:25565
|
|
- All The Mons → 38.68.14.30:25565
|
|
- FoundryVTT → 38.68.14.26:30000
|
|
|
|
**Step 3: Restart All Servers**
|
|
|
|
**Step 4: Add Game Servers to Uptime Kuma**
|
|
- CRITICAL LESSON: "I would have known earlier"
|
|
- We monitor VPS infrastructure but NOT game servers
|
|
- This created blind spot
|
|
- All 12 game servers (TX1 + NC1) need monitoring
|
|
|
|
### **CURRENT STATUS (5:30 PM CST):**
|
|
|
|
- 🔧 Allocating port ranges in Pterodactyl Panel
|
|
- ⏳ Awaiting allocation completion
|
|
- 📋 Next: Assign to servers, restart, monitor
|
|
|
|
### **LESSONS LEARNED:**
|
|
|
|
1. **Monitoring Gap:** Game servers not in Uptime Kuma = delayed crisis detection
|
|
2. **Allocation Verification:** Should have verified IPs during initial setup
|
|
3. **Complete Testing:** Node running ≠ servers running
|
|
4. **User-Facing Monitoring:** Monitor what USERS interact with, not just infrastructure
|
|
|
|
### **PRIORITY AFTER FIX:**
|
|
|
|
✅ Add all 12 game servers to Uptime Kuma
|
|
✅ Verify allocations match actual server IPs
|
|
✅ Test each server individually
|
|
✅ Document proper allocation process
|
|
|
|
---
|
|
|
|
**Crisis Discovered:** February 11, 2026 5:00 PM CST
|
|
**Status:** Fix in progress (allocating port ranges)
|
|
**ETA:** 30-60 minutes to full recovery
|
|
|
|
EOF
|
|
|
|
cd ~/firefrost-work/firefrost-operations-manual
|
|
git add docs/session-handoff.md
|
|
git commit -m "Add TX1 crisis status - all 6 game servers down due to IP allocation mismatch"
|
|
git push
|
|
|
|
echo ""
|
|
echo "✅ TX1 crisis status added to session-handoff.md"
|
|
echo "✅ Complete context for future Claude session"
|
|
echo "✅ Committed to Git"
|
|
echo "✅ Pushed to Gitea (will auto-sync to GitHub)"
|
|
echo ""
|
|
echo "DOCUMENTATION IS NOW 100% CURRENT!"
|
|
echo "Ready for new session with full context."
|
|
|