firefrost-operations-manual/docs/troubleshooting/nc1-cleanup-guidelines.md

# NC1 Server Cleanup Guidelines

## Overview
This document provides safe cleanup procedures for the NC1 Charlotte server to prevent accidental deletion of critical game files.

---

## Critical Warning: Know What You're Deleting

**THE GOLDEN RULE:** Always list files before deleting them with `ls -lah` to verify what will be removed.

---

## Safe Cleanup Procedures

### 1. Journal/Log Cleanup (SAFE)
```bash
# Clean systemd journals
journalctl --vacuum-time=7d
journalctl --vacuum-size=1G

# Clean old logs
find /var/log -type f -name "*.log.*" -mtime +30 -delete
find /var/log -type f -name "*.gz" -mtime +30 -delete
```

### 2. Docker Cleanup (SAFE)
```bash
# Remove unused containers, networks, images
docker system prune -af --volumes

# Remove old images only
docker image prune -a
```

### 3. APT Cache Cleanup (SAFE)
```bash
apt-get clean
apt-get autoclean
apt-get autoremove -y
```

---

## Game Server Specific Cleanup

### ❌ DANGEROUS - DO NOT USE
```bash
# THESE WILL DELETE CRITICAL FILES:
rm -f /var/lib/pterodactyl/volumes/<uuid>/*.zip
rm -rf /var/lib/pterodactyl/volumes/<uuid>/backups
```

### ✅ SAFE - Server-Specific Cleanup

#### Hytale Server
**UUID:** `13c80cb8-f6f8-4bfe-9cdb-823d7e951584`

**Critical Files - NEVER DELETE:**
- `Assets.zip` (3.3GB) - Core game assets
- `Server/` directory - Server binaries
- `hytale-downloader*` - Update tool
- `config.json` - Server configuration
- `.hytale-downloader-credentials.json` - Auth tokens
- `universe/` - World data (unless intentionally wiping)

**Safe to Delete:**
```bash
cd /var/lib/pterodactyl/volumes/13c80cb8-f6f8-4bfe-9cdb-823d7e951584

# Old manual backups (verify first!)
ls -lah universe-backup-*.tar.gz
rm -f universe-backup-*.tar.gz  # Only after verifying dates

# Server package after extraction
rm -f Server-Package.zip

# Old backup directory (ONLY if you have external backups)
ls -lah backups/  # Verify first!
rm -rf backups/   # ONLY if confirmed backed up elsewhere

# Old logs
rm -f logs/*.log.*.gz
```

#### Minecraft Servers (ATM 10, etc.)

**Critical Files - NEVER DELETE:**
- `world/` - Main world
- `world_nether/` - Nether dimension
- `world_the_end/` - End dimension
- Server JAR files
- `server.properties`
- `ops.json`, `whitelist.json`

**Safe to Delete:**
```bash
cd /var/lib/pterodactyl/volumes/<minecraft-uuid>

# Old crash reports
rm -rf crash-reports/*.txt

# Old logs
rm -f logs/*.log.gz

# Old backups (verify externally backed up first!)
ls -lah *backup*.tar.gz
rm -f *backup*.tar.gz  # Only after external verification
```

---

## Pre-Cleanup Checklist

Before ANY cleanup operation:

1. **Stop affected servers** (if deleting from server directories)
2. **List files to be deleted:**
   ```bash
   ls -lah <target-directory>
   ```
3. **Verify file sizes and dates** - ensure you're not deleting recent/large critical files
4. **Have external backups** - if deleting backups, ensure they exist elsewhere
5. **Document what you're deleting** - take notes for recovery if needed

---

## Emergency Recovery

### If You Accidentally Deleted Critical Files

#### Hytale Assets.zip Recovery:
```bash
cd /var/lib/pterodactyl/volumes/13c80cb8-f6f8-4bfe-9cdb-823d7e951584

# Re-download using hytale-downloader
./hytale-downloader-linux-amd64 -download-path Server-Package.zip

# Extract nested Assets.zip
unzip -j Server-Package.zip Assets.zip

# Verify size (should be 3.3GB)
ls -lh Assets.zip

# Clean up package
rm Server-Package.zip
```

#### Hytale World Data Recovery:
If you deleted `universe/` or its contents:
```bash
# Check for backup tarballs
ls -lah universe-backup-*.tar.gz

# Restore from most recent backup
tar -xzf universe-backup-YYYYMMDD-HHMMSS.tar.gz

# If no backup exists, server will generate fresh world on restart
```

#### Minecraft World Recovery:
```bash
# Check for world backups
ls -lah world-backup-*.tar.gz

# Restore
tar -xzf world-backup-YYYYMMDD-HHMMSS.tar.gz

# If no backup, you may have lost the world permanently
```

---

## Disk Space Analysis Before Cleanup

**Always analyze first to target the actual space hogs:**

```bash
# Check overall disk usage
df -h

# Find largest directories
du -h --max-depth=1 / | sort -hr | head -20

# Find largest files
find / -type f -size +100M -exec ls -lh {} \; 2>/dev/null | sort -k5 -hr | head -20

# Check Pterodactyl volumes specifically
du -h --max-depth=2 /var/lib/pterodactyl/volumes/ | sort -hr | head -20

# Check Docker space usage
docker system df
```

---

## Recommended Cleanup Schedule

### Weekly:
- Docker system prune
- Journal vacuum (keep 7 days)
- Old log rotation

### Monthly:
- Review game server backups
- Clean old crash reports
- APT cache cleanup

### Quarterly:
- Full disk analysis
- Archive old game server backups to external storage
- Review Pterodactyl volume usage

---

## Space Reclamation Examples

**From our 2026-02-16 cleanup session:**

| Action | Space Reclaimed |
|--------|----------------|
| Delete Hytale backups | ~10GB |
| Journal vacuum (7d) | 3.7GB |
| Docker system prune | 3.5GB |
| **Total** | **17.42GB** |

**Lessons learned:**
- ✅ Cleaned journals safely
- ✅ Removed Docker cruft
- ❌ Accidentally deleted `Assets.zip` with wildcard `*.zip`
- ❌ Accidentally wiped Hytale world (was in `/backups/`)

---

## Command Safety Tips

### Use Specific Patterns, Not Wildcards
```bash
# ❌ DANGEROUS - Deletes ALL zip files:
rm -f *.zip

# ✅ SAFE - Only deletes backup zips:
rm -f backup-*.zip
rm -f *-backup.zip

# ✅ SAFER - List first, then delete:
ls -lah *.zip
# Review output, then delete specific files
rm -f specific-file.zip
```

### Always Use `-i` for Interactive Deletion
```bash
# Prompts before each deletion:
rm -i *.zip

# Prompts before recursive deletion:
rm -ri backups/
```

### Use `find` with Confirmation
```bash
# Find and list candidates:
find /var/log -name "*.gz" -mtime +30 -ls

# Review, then delete:
find /var/log -name "*.gz" -mtime +30 -delete
```

---

## Pterodactyl Panel Considerations

**Prefer Panel-based operations when possible:**
- Backups: Use Pterodactyl's built-in backup system
- File deletion: Use file manager (provides trash/restore)
- Server reinstall: Use panel reinstall feature

**Only use SSH/direct access when:**
- Bulk operations needed
- Panel is unavailable
- Advanced troubleshooting required

---

## Quick Reference: What NOT to Delete

### Hytale (`13c80cb8-f6f8-4bfe-9cdb-823d7e951584`)
```
❌ Assets.zip
❌ Server/
❌ hytale-downloader*
❌ config.json
❌ .hytale-downloader-credentials.json
❌ universe/ (unless wiping intentionally)
```

### Minecraft (All servers)
```
❌ world*/
❌ *.jar (server files)
❌ server.properties
❌ ops.json, whitelist.json
❌ Recent backups (check dates!)
```

### System-Wide
```
❌ /var/lib/pterodactyl/volumes/ (entire directory)
❌ Critical Docker images (in use)
❌ Recent journals (< 7 days)
```

---

**Last Updated:** 2026-02-16  
**NC1 Server:** 216.239.104.130 (Charlotte, NC datacenter)  
**Maintained by:** Firefrost Gaming