firefrost-operations-manual/docs/quick-reference/common-operations.md

# 🚀 QUICK REFERENCE - Common Operations

**One-page quick reference for daily operations**  
**Print and keep handy!**

---

## 🔐 EMERGENCY CREDENTIALS ACCESS

**Vaultwarden:** vault.firefrostgaming.com  
**If Vaultwarden down:** Check emergency credential sheet

---

## 🖥️ SERVER ACCESS

```bash
# Command Center (Dallas hub)
ssh root@63.143.34.217

# TX1 (Dallas game servers)
ssh root@38.68.14.26

# NC1 (Charlotte game servers)  
ssh root@216.239.104.130

# Panel (Control plane)
ssh root@45.94.168.138

# Billing VPS
ssh root@38.68.14.188

# Ghost VPS (Docs/Wiki)
ssh root@64.50.188.14
```

---

## 🎮 RESTART SINGLE SERVER

**Via Pterodactyl Panel:**
1. Go to panel.firefrostgaming.com
2. Select server
3. Click "Restart" button
4. Wait 2-3 minutes
5. Verify server online

**Via API:**
```bash
curl -X POST "https://panel.firefrostgaming.com/api/client/servers/{uuid}/power" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"signal":"restart"}'
```

---

## 🔄 RESTART ALL SERVERS (Staggered)

**Manual (when automation down):**
```bash
# On Command Center
python3 /opt/automation/staggered-restart/staggered-restart.py
```

**Scheduled (cron):**
- Runs automatically at 4:00 AM daily
- Check logs: `tail -f /var/log/staggered-restart.log`

---

## 💾 MANUAL BACKUP

**Single server world:**
```bash
# On Command Center
python3 /opt/automation/world-backup/world-backup.py --server "ATM10"
```

**All servers:**
```bash
python3 /opt/automation/world-backup/world-backup.py
```

**Check backup status:**
- NextCloud: downloads.firefrostgaming.com/backups/worlds/

---

## 📊 CHECK SERVER HEALTH

**TPS (in-game):**
```
/tps
/forge tps
```

**Resource usage (SSH):**
```bash
# Quick overview
htop

# Memory
free -h

# Disk space
df -h

# Network
iftop
```

**Via Pterodactyl:**
- View server → Graphs tab

---

## 🔥 PERFORMANCE ISSUES

**High CPU:**
```bash
# Find process
top
# Kill if needed
kill [PID]
```

**High Memory:**
```bash
# Check usage
free -h
# Restart server if critical
```

**Low TPS:**
```
# In-game
/kill @e[type=!player]  # Clear entities
# Then restart server
```

**High Disk I/O:**
```bash
iostat -x 1
# Check what's writing
iotop
```

---

## 🌐 FROSTWALL TUNNEL CHECK

**Command Center:**
```bash
# Check tunnel status
ip link show | grep gre

# Test connectivity
ping 10.0.1.2  # TX1
ping 10.0.2.2  # NC1

# Restart if needed
systemctl restart networking
```

---

## 🚨 CHECK SERVICE STATUS

```bash
# Any systemd service
systemctl status [service-name]

# Common services
systemctl status nginx
systemctl status gitea
systemctl status vaultwarden
systemctl status netdata
```

---

## 📝 VIEW LOGS

```bash
# Service logs (last 50 lines)
journalctl -u [service] -n 50

# Follow logs live
journalctl -u [service] -f

# All system logs
journalctl -xe

# Specific log files
tail -f /var/log/[logfile]
```

---

## 🔧 RESTART SERVICES

```bash
# Restart service
systemctl restart [service]

# Restart web server
systemctl restart nginx

# Restart all Pterodactyl
systemctl restart pteroq wings

# Restart automation
systemctl restart staggered-restart
```

---

## 🎯 WHITELIST PLAYER

**Via Web Dashboard:**
1. Go to whitelist.firefrostgaming.com
2. Enter Minecraft username
3. Select server
4. Click "Add to Whitelist"

**Manual (in-game console):**
```
/whitelist add [username]
/whitelist reload
```

---

## 👥 ADD STAFF PERMISSIONS

**LuckPerms (in-game):**
```
/lp user [username] parent set admin
/lp user [username] permission set [perm] true
```

**Pterodactyl Panel:**
1. Users → Create User
2. Assign to servers
3. Set permissions

---

## 📈 CHECK UPTIME

**Uptime Kuma:**
- Go to status.firefrostgaming.com
- View all service status

**Manual check:**
```bash
uptime
systemctl status [service]
```

---

## 💬 DISCORD NOTIFICATIONS

**Server Status:**
- Posted automatically to #server-status
- Configured via webhooks

**Manual notification:**
```bash
curl -X POST [DISCORD_WEBHOOK_URL] \
  -H "Content-Type: application/json" \
  -d '{"content":"[Your message]"}'
```

---

## 🗄️ DATABASE ACCESS

**MySQL (if needed):**
```bash
mysql -u root -p
SHOW DATABASES;
USE [database];
SHOW TABLES;
```

**Pterodactyl database:**
```bash
mysql -u pterodactyl -p pterodactyl
```

---

## 🔐 SECURITY QUICK CHECKS

**Check for attacks:**
```bash
# Failed SSH attempts
grep "Failed password" /var/log/auth.log | tail -20

# Fail2Ban status
fail2ban-client status sshd

# UFW status
ufw status
```

---

## 📦 UPDATE SYSTEM

```bash
# Update packages
apt update && apt upgrade -y

# Check what's outdated
apt list --upgradable

# Security updates only
unattended-upgrades
```

---

## 🆘 EMERGENCY STOP

**Stop specific server:**
- Pterodactyl panel → Stop button

**Stop all game servers:**
```bash
# Via Pterodactyl API (script)
for uuid in [server-uuids]; do
  curl -X POST ".../power" -d '{"signal":"stop"}'
done
```

**Stop critical service:**
```bash
systemctl stop [service]
```

---

## 📞 WHEN TO ESCALATE

**Yellow Alert (⚠️):**
- Single server down >15 min
- Performance degraded >30 min
- Any revenue system affected

**Red Alert (🚨):**
- Multiple services down
- All game servers unreachable  
- Provider outage
- Security breach

**See:** `docs/emergency-protocols/`

---

## 🔗 QUICK LINKS

- **Panel:** panel.firefrostgaming.com
- **Status:** status.firefrostgaming.com
- **Vault:** vault.firefrostgaming.com
- **Docs:** docs.firefrostgaming.com
- **Git:** git.firefrostgaming.com

---

**Fire + Frost + Foundation** 💙🔥❄️

**Print Date:** 2026-02-17  
**Version:** 1.0