94 lines
2.5 KiB
Markdown
94 lines
2.5 KiB
Markdown
# NC1 Inter-Datacenter Routing Resolution
|
|
|
|
**Date:** February 9, 2026
|
|
**Ticket:** #5ae82fd3
|
|
**Issue:** TX1 Dallas ↔ NC1 Charlotte could not communicate
|
|
**Resolution Time:** ~4.5 hours
|
|
**Status:** ✅ RESOLVED
|
|
|
|
---
|
|
|
|
## Problem Statement
|
|
|
|
TX1 (Dallas) and NC1 (Charlotte) could not communicate directly with each other, preventing:
|
|
- Uptime Kuma monitoring of NC1 from Command Center
|
|
- Cross-datacenter database architecture
|
|
- BungeeCord/Velocity proxy networks
|
|
- Direct server-to-server backups
|
|
|
|
---
|
|
|
|
## Diagnostic Work
|
|
|
|
**Test 1: TX1 → NC1 (Dallas to Charlotte)**
|
|
```
|
|
root@TX1# ping -c 4 216.239.104.130
|
|
From 38.68.14.25 icmp_seq=1 Destination Net Unreachable
|
|
--- 100% packet loss ---
|
|
```
|
|
Result: TX1's gateway immediately returned "Network Unreachable"
|
|
|
|
**Test 2: NC1 → TX1 (Charlotte to Dallas)**
|
|
```
|
|
root@NC1# traceroute -n 38.68.14.26
|
|
1 216.239.104.129 0.092 ms
|
|
2 * * *
|
|
[times out after 9 hops]
|
|
```
|
|
Result: Asymmetric routing - NC1 could route toward Dallas but packets timed out
|
|
|
|
**Test 3: Firewall Check**
|
|
```
|
|
root@TX1# ufw status numbered | grep 216.239
|
|
[no results]
|
|
```
|
|
Result: Not a firewall issue - pure routing problem
|
|
|
|
---
|
|
|
|
## Root Cause
|
|
|
|
Missing routing table entry on Breezehost's infrastructure to allow inter-datacenter communication between Dallas and Charlotte facilities.
|
|
|
|
---
|
|
|
|
## Resolution
|
|
|
|
**Breezehost Action:** "Just needed a route added on our end" - Brandon E, Feb 9, 2026
|
|
|
|
**Verification:**
|
|
```
|
|
root@command-center# ping -c 3 216.239.104.130
|
|
3 packets transmitted, 3 received, 0% packet loss
|
|
```
|
|
|
|
**Impact:**
|
|
- ✅ NC1 now fully monitored by Uptime Kuma (100% uptime)
|
|
- ✅ Cross-datacenter architecture options enabled
|
|
- ✅ Complete infrastructure visibility
|
|
|
|
---
|
|
|
|
## Lessons Learned
|
|
|
|
1. **Diagnostic work pays off:** Providing comprehensive test results (ping, traceroute, firewall checks) enabled fast resolution
|
|
2. **Ask before assuming:** We assumed inter-DC isolation was intentional - it wasn't
|
|
3. **Breezehost is responsive:** 4.5 hour turnaround on infrastructure routing changes
|
|
4. **Document everything:** Having ticket history enables future reference
|
|
|
|
---
|
|
|
|
## Future Opportunities Unlocked
|
|
|
|
Now that TX1 ↔ NC1 communication works, we can explore:
|
|
- Centralized MySQL/PostgreSQL database (single instance, both datacenters)
|
|
- BungeeCord/Velocity proxy networks (seamless player transfers)
|
|
- Direct backup replication (NC1 → TX1 or vice versa)
|
|
- Load balancing across datacenters
|
|
- Failover architectures
|
|
|
|
---
|
|
|
|
**Breezehost Contact:** Brandon E (Staff)
|
|
**Appreciation:** "Like always, you guys are legends, I appreciate you" - Michael
|