# 📊 Service Metrics & SLA Definitions

**Status:** Operational Standards  
**Owner:** Michael "The Wizard" Krause  
**Last Updated:** 2026-02-17

---

## 🎯 SERVICE LEVEL AGREEMENTS (SLAs)

### Overall Infrastructure SLA

**Target Uptime:** 99.5% monthly  
**Allowed Downtime:** ~3.6 hours per month  
**Measurement:** Uptime Kuma historical data  

---

## 📈 PERFORMANCE TARGETS

### Game Servers

**TPS (Ticks Per Second):**
- **Target:** 19.5-20.0 TPS
- **Acceptable:** 18.0-19.5 TPS  
- **Degraded:** 15.0-18.0 TPS
- **Critical:** <15.0 TPS (Yellow Alert)

**Player Connection:**
- **Target:** <100ms latency
- **Acceptable:** 100-200ms latency
- **Degraded:** 200-300ms latency
- **Critical:** >300ms latency

**Server Uptime:**
- **Target:** 99.5% per server monthly
- **Scheduled Maintenance:** 30 minutes daily (4:00 AM restart)
- **Unplanned Downtime:** <2 hours monthly per server

---

### Management Services

**Pterodactyl Panel:**
- **Uptime Target:** 99.9% monthly
- **Response Time:** <2 seconds page load
- **API Response:** <500ms per request

**Billing (Paymenter):**
- **Uptime Target:** 99.9% monthly (revenue-critical)
- **Payment Processing:** <30 seconds
- **Page Load:** <3 seconds

**Wiki/Documentation:**
- **Uptime Target:** 99.0% monthly
- **Search Response:** <1 second
- **Page Load:** <2 seconds

---

## 💾 BACKUP METRICS

**World Backups:**
- **Frequency:** Daily at 3:30 AM
- **Retention:** 7 daily, 4 weekly, 12 monthly
- **Success Rate Target:** 100% (all 11 servers)
- **Recovery Time Objective (RTO):** 30 minutes
- **Recovery Point Objective (RPO):** 24 hours (daily backups)

**Configuration Backups:**
- **Frequency:** On every change + daily
- **Retention:** 30 days
- **Storage:** Git repository + off-server

---

## 🌐 NETWORK METRICS

**Frostwall Tunnels:**
- **Uptime Target:** 99.9% per tunnel
- **Latency:** <10ms additional overhead
- **Packet Loss:** <0.1%
- **Health Check:** Every 5 minutes

**Bandwidth Usage:**
- **TX1 Node:** ~500GB/month baseline
- **NC1 Node:** ~800GB/month baseline
- **Alert Threshold:** >80% of allocated bandwidth

---

## 🔒 SECURITY METRICS

**Fail2Ban:**
- **SSH Ban Threshold:** 3 failed attempts
- **Ban Duration:** 1 hour (first offense)
- **Monitoring:** Check banned IPs daily

**Firewall:**
- **Blocked Attempts:** Monitor daily
- **Rule Changes:** Logged and reviewed
- **Audit Frequency:** Weekly

**Vulnerability Scans:**
- **Frequency:** Monthly
- **Critical Patches:** Within 48 hours
- **Security Updates:** Within 7 days

---

## 💰 COST METRICS

### Infrastructure Costs (Monthly)

**Dedicated Servers:**
- TX1 Dallas: ~$150/month
- NC1 Charlotte: ~$150/month
- **Total Dedicated:** ~$300/month

**VPS Services:**
- Command Center: ~$20/month
- Panel: ~$15/month
- Billing VPS: ~$10/month
- Ghost VPS: ~$15/month
- **Total VPS:** ~$60/month

**Additional Services:**
- Domain registration: ~$15/year
- Cloudflare: $0 (free tier)
- Backups/Storage: ~$10/month

**Total Monthly Infrastructure:** ~$370/month

---

### Revenue Metrics

**Subscription Tiers:**
- Sovereign: $99/month
- Consular: $49/month
- Community: Free

**Targets:**
- **Break-even:** 4 Sovereign OR 8 Consular subscribers
- **Profit Target:** 10+ paying subscribers
- **Growth Rate:** +2 subscribers per month

---

## 📊 CAPACITY PLANNING

### Current Capacity (Feb 2026)

**TX1 Dallas:**
- CPU: 32 vCPUs (avg 40% usage)
- RAM: 256GB (avg 60% usage - 150GB)
- Disk: 2TB (40% usage - 800GB)
- **Headroom:** 5 more servers possible

**NC1 Charlotte:**
- CPU: 32 vCPUs (avg 50% usage)
- RAM: 256GB (avg 70% usage - 180GB)
- Disk: 2TB (45% usage - 900GB)
- **Headroom:** 3-4 more servers possible

**Scaling Triggers:**
- RAM usage sustained >80%: Add more RAM or migrate servers
- CPU usage sustained >70%: Optimize or add node
- Disk usage >80%: Add storage or implement cleanup

---

### Growth Projections

**Q1 2026 (Current):**
- 11 game servers
- ~50 active players
- ~5 paying subscribers (projected)

**Q2 2026 (Target):**
- 13-15 game servers
- ~100 active players  
- ~12 paying subscribers

**Q3 2026 (Growth):**
- 15-18 game servers
- ~150 active players
- ~20 paying subscribers

**Capacity Limit (Current Infrastructure):**
- Maximum: ~20 servers across both nodes
- Need 3rd node if exceeding 20 servers

---

## ⏱️ RESPONSE TIME TARGETS

**Incident Response:**
- **Critical (Red Alert):** Acknowledge in 5 min, resolve in 1 hour
- **High (Yellow Alert):** Acknowledge in 15 min, resolve in 30 min
- **Medium:** Respond in 1 hour, resolve in 4 hours
- **Low:** Respond in 24 hours, resolve in 1 week

**Support Tickets:**
- **Urgent:** Response in 2 hours
- **Normal:** Response in 12 hours
- **Low Priority:** Response in 48 hours

---

## 🎮 PLAYER EXPERIENCE METRICS

**Connection Success Rate:**
- **Target:** >99% of connection attempts succeed
- **Measurement:** Player reports + server logs

**Server Stability:**
- **Target:** <1 crash per server per month
- **Measurement:** Pterodactyl crash reports

**Player Retention:**
- **Target:** >60% monthly active players return
- **Measurement:** Login tracking

**Support Satisfaction:**
- **Target:** >90% positive feedback
- **Measurement:** Player surveys

---

## 📉 FAILURE METRICS

**Mean Time Between Failures (MTBF):**
- **Target:** >720 hours (30 days) per service
- **Current:** Track and improve monthly

**Mean Time To Repair (MTTR):**
- **Critical Services:** <30 minutes
- **Game Servers:** <15 minutes
- **Non-critical:** <2 hours

**Change Success Rate:**
- **Target:** >95% of changes deploy without incident
- **Measurement:** Track deployments vs rollbacks

---

## 📋 MONITORING DASHBOARDS

**Uptime Kuma:**
- All services monitored
- Status page: status.firefrostgaming.com
- Alert thresholds configured

**Netdata (Planned):**
- Real-time performance metrics
- Historical data retention: 7 days
- Alert integration with Discord

**Pterodactyl:**
- Server resource usage graphs
- Player connection logs
- Crash reports

---

## 🔔 ALERT THRESHOLDS

**Uptime Kuma Alerts:**
- Service down >5 minutes → Discord notification
- Service down >15 minutes → Email alert
- Service down >30 minutes → SMS/Call escalation

**Resource Alerts:**
- CPU >80% for 10 min → Warning
- RAM >90% for 5 min → Critical
- Disk >90% → Critical
- Network down → Critical immediate

**Performance Alerts:**
- TPS <15 for 15 min → Warning
- TPS <10 for 5 min → Critical
- Latency >300ms for 10 min → Warning

---

## 📊 REPORTING SCHEDULE

**Daily:**
- Automated backup success/failure report
- Critical alerts summary

**Weekly:**
- Uptime summary (per service)
- Performance trends
- Failed login attempts
- Bandwidth usage

**Monthly:**
- SLA compliance report
- Cost analysis
- Capacity utilization
- Growth metrics
- Incident post-mortems

**Quarterly:**
- Infrastructure review
- Capacity planning update
- Security audit summary
- Financial performance

---

## 🎯 SUCCESS METRICS

**Infrastructure:**
- ✅ 99.5% uptime achieved
- ✅ All backups successful
- ✅ Zero data loss incidents
- ✅ Response times within SLA

**Business:**
- ✅ Revenue > costs (profitability)
- ✅ Subscriber growth on track
- ✅ Player retention >60%
- ✅ Positive community sentiment

**Operations:**
- ✅ Incidents resolved within targets
- ✅ Change success rate >95%
- ✅ Security posture maintained
- ✅ Documentation complete and current

---

**Fire + Frost + Foundation = Where Love Builds Legacy** 💙🔥❄️

---

**Document Status:** ACTIVE  
**Review Schedule:** Monthly  
**Next Review:** 2026-03-17  
**Version:** 1.0
