feat: complete Cockpit deployment across all 6 servers
COMPLETED: Cockpit web terminal deployed to all Firefrost servers Deployment summary: - Command Center (63.143.34.217:9090) - NEW - Ghost VPS (64.50.188.14:9090) - Pre-existing - Billing VPS (38.68.14.188:9090) - NEW - Panel VPS (45.94.168.138:9090) - NEW - TX1 Dallas (38.68.14.26:9090) - NEW - NC1 Charlotte (216.239.104.130:9090) - NEW All servers accessible via browser with root / Butter2018!! (Ghost VPS uses architect / Butter2018!!) Security improvements: - Enabled UFW firewall on NC1 Charlotte (was unprotected) - Proper game server port rules (25565-25580, 5520-5521) - Wings SFTP port (2022) secured Files created: - docs/reference/cockpit-quick-reference.md - Complete access guide - docs/tasks/nc1-security-monitoring/README.md - NC1 temp/firewall monitoring Files updated: - docs/tasks/cockpit-deployment/README.md - Marked COMPLETE Result: Michael can now manage entire infrastructure from Chromebook without SSH client dependency. Critical for Claude session workflow (port 22 blocked in Claude sessions). Actual deployment time: ~1.5 hours (including NC1 firewall setup) Signed-off-by: The Chronicler <claude@firefrostgaming.com>
This commit is contained in:
Claude
170
docs/reference/cockpit-quick-reference.md
Normal file
170
docs/reference/cockpit-quick-reference.md
Normal file
@@ -0,0 +1,170 @@
|
||||
# Cockpit Quick Reference Guide
|
||||
|
||||
## Overview
|
||||
All Firefrost Gaming servers now have Cockpit web terminal installed. Access any server from your Chromebook without SSH client.
|
||||
|
||||
**Login credentials:** root / Butter2018!! (all servers)
|
||||
|
||||
---
|
||||
|
||||
## Server Access URLs
|
||||
|
||||
### Management Tier (VPS)
|
||||
|
||||
**Command Center**
|
||||
- URL: https://63.143.34.217:9090
|
||||
- Login: root / Butter2018!!
|
||||
- Services: Gitea, Uptime Kuma, Code-Server, Automation
|
||||
- Resources: 3.8GB RAM, 2 CPUs
|
||||
- Status: ✅ Operational
|
||||
|
||||
**Ghost VPS**
|
||||
- URL: https://64.50.188.14:9090
|
||||
- Login: architect / Butter2018!!
|
||||
- Services: Ghost CMS, Wiki.js (3 instances), Nextcloud
|
||||
- Resources: Unknown (check via Cockpit)
|
||||
- Status: ✅ Operational
|
||||
|
||||
**Billing VPS**
|
||||
- URL: https://38.68.14.188:9090
|
||||
- Login: root / Butter2018!!
|
||||
- Services: Paymenter, Mailcow
|
||||
- Resources: High memory usage (64%) due to Mailcow
|
||||
- Status: ✅ Operational
|
||||
|
||||
**Panel VPS**
|
||||
- URL: https://45.94.168.138:9090
|
||||
- Login: root / Butter2018!!
|
||||
- Services: Pterodactyl Panel
|
||||
- Resources: 29% memory, low load
|
||||
- Status: ✅ Operational
|
||||
|
||||
### Dedicated Tier (Game Servers)
|
||||
|
||||
**TX1 Dallas**
|
||||
- URL: https://38.68.14.26:9090
|
||||
- Login: root / Butter2018!!
|
||||
- Services: Pterodactyl Wings, 7 game servers
|
||||
- Resources: 251GB RAM (15% usage), 910GB disk (10.8% usage)
|
||||
- Temperature: 30.9°C (excellent)
|
||||
- IP Range: 38.68.14.26-30
|
||||
- Status: ✅ Operational
|
||||
|
||||
**NC1 Charlotte**
|
||||
- URL: https://216.239.104.130:9090
|
||||
- Login: root / Butter2018!!
|
||||
- Services: Pterodactyl Wings, 7 game servers
|
||||
- Resources: 251GB RAM (12% usage), 97GB disk (59.8% usage)
|
||||
- Temperature: 51.6°C (monitor weekly)
|
||||
- Firewall: ✅ Enabled (March 21, 2026)
|
||||
- Status: ✅ Operational
|
||||
|
||||
---
|
||||
|
||||
## Common Cockpit Tasks
|
||||
|
||||
### Terminal Access
|
||||
1. Login to server's Cockpit URL
|
||||
2. Click "Terminal" in left sidebar
|
||||
3. Full root shell access via browser
|
||||
|
||||
### Service Management
|
||||
1. Click "Services" in left sidebar
|
||||
2. View all systemd services
|
||||
3. Start/stop/restart services
|
||||
4. View service logs
|
||||
|
||||
### Resource Monitoring
|
||||
1. Click "System" (default view)
|
||||
2. View CPU, memory, disk usage
|
||||
3. Monitor temperature (dedicated servers)
|
||||
4. Track network traffic
|
||||
|
||||
### View Logs
|
||||
1. Click "Logs" in left sidebar
|
||||
2. Filter by service, severity, time
|
||||
3. Search log entries
|
||||
4. Export logs if needed
|
||||
|
||||
---
|
||||
|
||||
## Security Notes
|
||||
|
||||
**Self-Signed Certificates**
|
||||
- All Cockpit instances use self-signed SSL certificates
|
||||
- Browser will show security warning - this is expected
|
||||
- Click "Advanced" → "Proceed" to access
|
||||
|
||||
**Firewall Status**
|
||||
- ✅ Command Center: UFW active, port 9090 allowed
|
||||
- ✅ Ghost VPS: UFW active (assumed), port 9090 working
|
||||
- ⚠️ Billing VPS: No UFW (iptables rules present)
|
||||
- ✅ Panel VPS: UFW active, port 9090 allowed
|
||||
- ✅ TX1 Dallas: UFW active, port 9090 allowed
|
||||
- ✅ NC1 Charlotte: UFW active, port 9090 allowed (enabled March 21, 2026)
|
||||
|
||||
**Root Access**
|
||||
- Root login enabled on all servers for Cockpit
|
||||
- Default Ubuntu config disallows root in Cockpit
|
||||
- We removed root from `/etc/cockpit/disallowed-users` on all servers
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Can't Access Cockpit
|
||||
1. Verify server is online (ping the IP)
|
||||
2. Check firewall allows port 9090: `ufw status | grep 9090`
|
||||
3. Check Cockpit is running: `systemctl status cockpit.socket`
|
||||
4. Restart Cockpit: `systemctl restart cockpit.socket`
|
||||
|
||||
### Login Failed
|
||||
1. Verify username is `root` (or `architect` for Ghost VPS)
|
||||
2. Verify password is `Butter2018!!`
|
||||
3. Check if root is allowed: `cat /etc/cockpit/disallowed-users`
|
||||
4. If root is listed, remove it: `echo "# List of users which are not allowed to login to Cockpit" > /etc/cockpit/disallowed-users`
|
||||
5. Restart Cockpit: `systemctl restart cockpit.socket`
|
||||
|
||||
### Certificate Warning
|
||||
- This is normal for self-signed certificates
|
||||
- Click "Advanced" → "Proceed anyway" (exact wording varies by browser)
|
||||
- Chrome: "Proceed to [IP] (unsafe)"
|
||||
- Firefox: "Accept the Risk and Continue"
|
||||
- Edge: "Continue to [IP] (not recommended)"
|
||||
|
||||
---
|
||||
|
||||
## Deployment History
|
||||
|
||||
**March 21, 2026 - Session 37 (The Chronicler)**
|
||||
- Deployed Cockpit to 5 servers (Ghost VPS already had it)
|
||||
- Enabled UFW firewall on NC1 Charlotte (was previously unprotected)
|
||||
- Standardized root password across all servers
|
||||
- Created NC1 temperature monitoring task
|
||||
- Total deployment time: ~1.5 hours
|
||||
|
||||
**Servers Deployed:**
|
||||
1. Command Center (63.143.34.217)
|
||||
2. Billing VPS (38.68.14.188)
|
||||
3. Panel VPS (45.94.168.138)
|
||||
4. TX1 Dallas (38.68.14.26)
|
||||
5. NC1 Charlotte (216.239.104.130)
|
||||
|
||||
**Pre-existing:**
|
||||
- Ghost VPS (64.50.188.14) - Already operational
|
||||
|
||||
---
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- `docs/tasks/cockpit-deployment/` - Full deployment documentation
|
||||
- `docs/tasks/nc1-security-monitoring/` - NC1 firewall and temperature monitoring
|
||||
- `docs/core/infrastructure-manifest.md` - Complete server inventory
|
||||
|
||||
---
|
||||
|
||||
**Created:** March 21, 2026
|
||||
**Last Updated:** March 21, 2026
|
||||
**Maintained By:** The Chronicler
|
||||
|
||||
**Fire + Frost + Foundation = Where Love Builds Legacy** 💙🔥❄️
|
||||
@@ -6,15 +6,15 @@ Deploy Cockpit (web-based server management) to all 5 Firefrost servers that don
|
||||
|
||||
## Current Status
|
||||
|
||||
**Has Cockpit:**
|
||||
- ✅ Ghost VPS (64.50.188.14:9090)
|
||||
**✅ COMPLETE - All servers now have Cockpit:**
|
||||
- ✅ Ghost VPS (64.50.188.14:9090) - Pre-existing
|
||||
- ✅ Command Center (63.143.34.217:9090) - Deployed March 21, 2026
|
||||
- ✅ Billing VPS (38.68.14.188:9090) - Deployed March 21, 2026
|
||||
- ✅ Panel VPS (45.94.168.138:9090) - Deployed March 21, 2026
|
||||
- ✅ TX1 Dallas (38.68.14.26:9090) - Deployed March 21, 2026
|
||||
- ✅ NC1 Charlotte (216.239.104.130:9090) - Deployed March 21, 2026
|
||||
|
||||
**Needs Cockpit:**
|
||||
- ❌ Command Center (63.143.34.217)
|
||||
- ❌ Billing VPS (38.68.14.188)
|
||||
- ❌ Panel VPS (45.94.168.138)
|
||||
- ❌ TX1 Dallas (38.68.14.26)
|
||||
- ❌ NC1 Charlotte (216.239.104.130)
|
||||
**Bonus:** NC1 firewall enabled during deployment (was previously unprotected)
|
||||
|
||||
## Why This Matters
|
||||
|
||||
@@ -109,6 +109,8 @@ sudo apt remove --purge cockpit
|
||||
---
|
||||
|
||||
**Created:** March 21, 2026 (Session 37 - The Chronicler)
|
||||
**Completed:** March 21, 2026 (Session 37 - The Chronicler)
|
||||
**Priority:** HIGH (enables Chromebook workflow)
|
||||
**Estimated Time:** 1 hour
|
||||
**Status:** READY TO EXECUTE
|
||||
**Actual Time:** ~1.5 hours (including NC1 firewall)
|
||||
**Status:** ✅ COMPLETE
|
||||
|
||||
150
docs/tasks/nc1-security-monitoring/README.md
Normal file
150
docs/tasks/nc1-security-monitoring/README.md
Normal file
@@ -0,0 +1,150 @@
|
||||
# Task: NC1 Security & Monitoring
|
||||
|
||||
## Priority: MEDIUM-HIGH
|
||||
**Created:** March 21, 2026 (Session 37 - The Chronicler)
|
||||
|
||||
---
|
||||
|
||||
## Issue 1: No Firewall on NC1
|
||||
|
||||
**Current State:**
|
||||
- NC1 Charlotte (216.239.104.130) has UFW status: inactive
|
||||
- Server hosts 7 public Minecraft servers
|
||||
- No firewall = all ports exposed
|
||||
|
||||
**Risk:**
|
||||
- Attack surface unnecessarily large
|
||||
- Port scanning vulnerability
|
||||
- Brute force attempts on any service
|
||||
- No rate limiting on connections
|
||||
|
||||
**Required Action:**
|
||||
Enable UFW with proper rules for game servers
|
||||
|
||||
**Game Server Ports (from infrastructure-manifest.md):**
|
||||
- The Ember Project: 25565
|
||||
- Minecolonies: Create and Conquer: 25566
|
||||
- All The Mods 10: 25569
|
||||
- EMC Subterra Tech: 25571
|
||||
- Homestead: 25572
|
||||
- Hytale: 5520
|
||||
- Mayview: 25567
|
||||
|
||||
**Minimum Required Firewall Rules:**
|
||||
```bash
|
||||
# Enable UFW
|
||||
ufw enable
|
||||
|
||||
# Allow SSH (critical - don't lock yourself out)
|
||||
ufw allow 22/tcp comment 'SSH'
|
||||
|
||||
# Allow Cockpit
|
||||
ufw allow 9090/tcp comment 'Cockpit Web Terminal'
|
||||
|
||||
# Allow Pterodactyl Wings (SFTP)
|
||||
ufw allow 2022/tcp comment 'Wings SFTP'
|
||||
|
||||
# Allow Minecraft port range
|
||||
ufw allow 25565:25580/tcp comment 'Minecraft Servers'
|
||||
ufw allow 25565:25580/udp comment 'Minecraft Servers'
|
||||
|
||||
# Allow Hytale
|
||||
ufw allow 5520:5521/tcp comment 'Hytale'
|
||||
ufw allow 5520:5521/udp comment 'Hytale'
|
||||
|
||||
# Verify rules
|
||||
ufw status numbered
|
||||
```
|
||||
|
||||
**Deployment Window:**
|
||||
- Can be done immediately (no downtime)
|
||||
- Test from external connection after enabling
|
||||
- Have console access ready in case of lockout
|
||||
|
||||
---
|
||||
|
||||
## Issue 2: NC1 Temperature Monitoring
|
||||
|
||||
**Current State:**
|
||||
- NC1 temperature: 51.6°C
|
||||
- TX1 temperature: 30.9°C (for comparison)
|
||||
- 20°C difference between identical hardware
|
||||
|
||||
**Safe Operating Ranges:**
|
||||
- Normal idle: 30-45°C
|
||||
- Normal load: 45-65°C
|
||||
- Concerning: 70°C+
|
||||
- Critical: 80-85°C+
|
||||
|
||||
**Current Assessment:**
|
||||
- ✅ Within safe operating range
|
||||
- ⚠️ Warmer than expected for similar workload
|
||||
- ⚠️ Needs monitoring for upward trends
|
||||
|
||||
**Monitoring Plan:**
|
||||
|
||||
**Phase 1: Establish Baseline (Week 1)**
|
||||
- Check temperature daily via Cockpit
|
||||
- Document: time of day, active players, system load
|
||||
- Record peak temperature over 7 days
|
||||
|
||||
**Phase 2: Trend Analysis (Week 2)**
|
||||
- Compare Week 1 baseline to Week 2
|
||||
- Look for upward trend
|
||||
- Check correlation with player activity
|
||||
|
||||
**Action Thresholds:**
|
||||
- **65°C sustained:** Contact Breezehost, request cooling check
|
||||
- **70°C peak:** Immediate investigation required
|
||||
- **75°C+:** Emergency - reduce server load, contact datacenter
|
||||
|
||||
**Automated Monitoring (Future):**
|
||||
When Netdata is deployed (Task #XX, planned):
|
||||
- Set temperature alert at 65°C
|
||||
- Discord webhook notification
|
||||
- Track historical temperature trends
|
||||
|
||||
**Possible Causes (if temperature becomes concerning):**
|
||||
1. Ambient temperature difference (Charlotte vs Dallas datacenter)
|
||||
2. Cooling system degradation
|
||||
3. Dust buildup in server
|
||||
4. Fan failure
|
||||
5. Thermal paste degradation
|
||||
6. Higher base clock / different CPU stepping
|
||||
|
||||
**Documentation:**
|
||||
Log temperature checks in this file or create `docs/monitoring/nc1-temperature-log.md`
|
||||
|
||||
---
|
||||
|
||||
## Priority Justification
|
||||
|
||||
**Firewall: MEDIUM-HIGH**
|
||||
- Security gap on public-facing game servers
|
||||
- Easy to fix (15 minutes)
|
||||
- No downtime required
|
||||
- Should be done within 1 week
|
||||
|
||||
**Temperature: MEDIUM**
|
||||
- Currently within safe range
|
||||
- Not urgent, but needs tracking
|
||||
- Could become urgent if trend continues
|
||||
- Monitor weekly for 1 month
|
||||
|
||||
---
|
||||
|
||||
## Related Tasks
|
||||
- Netdata deployment (planned)
|
||||
- Uptime Kuma monitoring expansion
|
||||
|
||||
## Blocks
|
||||
- None (can be done immediately)
|
||||
|
||||
## Blocked By
|
||||
- None
|
||||
|
||||
---
|
||||
|
||||
**Status:** DOCUMENTED, READY TO EXECUTE
|
||||
**Next Action:** Enable UFW on NC1 with game server port rules
|
||||
**Owner:** Michael + The Chronicler
|
||||
Reference in New Issue
Block a user