firefrost-operations-manual/docs/services/trinity-console.md

Trinity Console - Admin Control Center

Service: Trinity Console (Arbiter 3.0 Admin Panel)
Purpose: Unified admin interface for Firefrost Gaming operations
Server: Command Center (63.143.34.217)
Status: ✅ 100% Complete - All 7 modules operational
Deployed: April 1, 2026, 5:00 AM CDT
Completed: April 1, 2026, 11:30 AM CDT
Deployed by: The Rigger (Chronicler #51)
Completed by: Chronicler #52
Built by: Zephyr (Chronicler #50) + Gemini AI

---

Overview

Trinity Console is the administrative control center for Firefrost Gaming. It provides The Trinity (Michael, Meg, Holly) with a unified web interface to manage players, monitor servers, track revenue, handle grace periods, audit actions, and diagnose role issues.

URL: https://discord-bot.firefrostgaming.com/admin

Built for RV Life: Designed to run remotely with cellular connection support, enabling full operation from anywhere in the United States.

---

Architecture

Technology Stack:

• Backend: Node.js + Express.js
• Frontend: EJS templates + htmx + Tailwind CSS
• Database: PostgreSQL (arbiter_db)
• Authentication: Discord OAuth2 + Passport.js
• Security: CSRF protection (csurf middleware)
• Hosting: Command Center VPS, Nginx reverse proxy

Integration Points:

Trinity Console
    ├─ PostgreSQL Database (subscriptions, users, audit logs)
    ├─ Pterodactyl Panel API (server management, whitelists)
    ├─ Discord API (role diagnostics, member data)
    └─ Paymenter (future: billing integration)

---

Access Control

Authorized Users (The Trinity):

• Holly (unicorn20089): Discord ID 269225344572063754
• Michael (Frostystyle): Discord ID 219309716021444609
• Meg (Gingerfury66): Discord ID 669981568059703316

Authentication Flow:

1. Visit /admin → Redirects to Discord OAuth
2. Discord authorization → Callback to Trinity Console
3. Middleware checks Discord ID against ADMIN_USERS whitelist
4. Session created → Access granted

Security:

• Discord OAuth2 for identity verification
• Session-based authentication
• CSRF token protection on all POST requests
• Trinity-only access (403 for unauthorized users)

---

Modules

Trinity Console consists of 7 integrated modules:

1. Dashboard

Purpose: High-level overview of all operations
Features:

• Active subscriber count
• Total MRR (Monthly Recurring Revenue)
• Server online status (12 game servers)
• Last sync timestamp
• Quick navigation to other modules

2. Servers (Server Matrix)

Purpose: Real-time game server monitoring and control
Features:

• List all 12 Minecraft servers (TX1 Dallas, NC1 Charlotte)
• Server online/offline status
• Whitelist enabled/disabled status
• Last whitelist sync timestamp
• Force sync whitelist (per server or all servers)
• Toggle whitelist mode
• Automatic polling for status updates

Key Actions:

• Sync Now: Forces immediate whitelist sync with Pterodactyl Panel
• Sync All: Syncs all 12 servers simultaneously
• Toggle Whitelist: Enables/disables whitelist mode (requires server restart warning)

3. Players (Player Management)

Purpose: Subscriber and player database management
Features:

• Search players by Discord ID or Minecraft username
• View player Minecraft skins (rendered live)
• Display subscription tier (Fire/Frost/Admin/Sovereign)
• Show status (active, lifetime, grace_period, cancelled)
• Change subscription tiers via dropdown (all tiers including Admin)
• Toggle staff status independently of subscription tier
• Pagination for large player lists
• Mobile-responsive design
• All changes logged in Audit Log

Tier Display:

• Fire Path: Orange badges (Fire Elemental, Fire Knight, Fire Master, Fire Legend)
• Frost Path: Cyan badges (Frost Elemental, Frost Knight, Frost Master, Frost Legend)
• Universal: Purple badges (The Awakened, Admin, Sovereign)

Status Indicators:

• 🟢 Green dot: Active or lifetime subscriptions
• 🟡 Yellow dot: Grace period (payment failed, 3 days to recover)
• 🔴 Red dot: Cancelled or expired

Actions Column:

• Tier Dropdown: Change any player's subscription tier (including Admin assignment)
• Staff Checkbox: Mark players as staff members (independent of subscription)
• Both actions create audit log entries
• Table auto-refreshes after changes
• Example use case: Moderator on Elemental tier = Tier: Elemental, Staff: ✓

Staff Tracking:

• Staff status stored in users.is_staff column
• Completely separate from subscription tier
• Allows hiring team members who are also subscribers
• Clear separation between employment and subscription

4. Financials & Analytics

Purpose: Revenue tracking and business intelligence
Status: ✅ Complete (Deployed April 1, 2026)

Global Health Metrics:

• Active Subscribers: Total count excluding grace period
• Monthly Recurring Revenue (MRR): Sum of active subscription values
• Annual Run Rate (ARR): MRR × 12
• At Risk: Subscribers in grace period + associated MRR
• Lifetime Revenue: Total from Sovereign tier ($499 × count)

Fire vs Frost Path Intelligence:

• Side-by-side comparison cards with gradient backgrounds
• Subscriber count per path
• Monthly revenue per path
• Marketing intelligence for path performance decisions

Tier Performance Table:

• All tiers with subscriber counts (Active + At Risk)
• MRR breakdown by tier
• Color-coded by path (Fire/Frost/Universal)
• Sortable by tier level (highest to lowest)

Use Cases:

• Real-time revenue tracking from RV
• Path popularity analysis for marketing
• At-risk subscriber identification
• Tier upgrade/downgrade trend analysis
• Financial health monitoring

5. Grace Period Dashboard

Purpose: Recovery mission control for failed payments
Features:

• "We Don't Kick People Out" philosophy
• List all subscribers in grace period
• Total at-risk MRR calculation
• Countdown timers (hours remaining until expiry)
• Payment failure reasons
• Manual override actions:
  • Extend 24h: Adds 24 hours to grace period
  • Manual Payment: Marks payment as received, restores active status
• Email all at-risk button (Phase 2)

Grace Period Policy:

• Payment failure → Automatic 3-day grace period
• After 3 days → Downgrade to permanent Awakened tier ($1)
• Never remove access completely
• Chargeback → Immediate permanent ban (no grace period)

6. Audit Log (Accountability Audit)

Purpose: Permanent record of all Trinity actions
Features:

• Logs every administrative action
• Filter by admin (Michael, Meg, Holly)
• Filter by action type (force_sync, tier_change, manual_override, etc.)
• Timestamp for each action
• Target identifier (which server or player was affected)
• JSON details for complex actions
• Chronological feed (newest first)

Logged Actions:

• Server whitelist syncs
• Grace period extensions
• Manual payment overrides
• Role assignments
• Tier changes
• Server configuration changes

Purpose: Accountability, troubleshooting, and compliance tracking

7. Role Audit (Discord Role Diagnostics)

Purpose: Detect and fix Discord role mismatches
Features:

• Run Diagnostic Scan: Compares Discord roles vs database subscriptions
• Finds subscribers without Discord roles
• Finds Discord members with roles but no subscription
• One-click resync for mismatched users
• Perfect sync confirmation message

Use Cases:

• New subscriber not getting role → Detected and fixed
• Cancelled subscriber still has role → Detected and removed
• Manual role grants (for testing) → Detected as mismatch
• Post-migration role cleanup

---

Database Schema

Trinity Console uses 3 new tables plus enhanced subscriptions:

New Tables

1. player_history

CREATE TABLE player_history (
    id SERIAL PRIMARY KEY,
    discord_id VARCHAR(255) REFERENCES users(discord_id),
    previous_tier INT,
    new_tier INT,
    change_reason VARCHAR(255), -- 'upgrade', 'downgrade', 'payment_failed', 'manual'
    changed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

2. admin_audit_log

CREATE TABLE admin_audit_log (
    id SERIAL PRIMARY KEY,
    admin_discord_id VARCHAR(255),
    admin_username VARCHAR(255),
    action_type VARCHAR(50), -- 'force_sync', 'manual_role_assign', 'tier_change'
    target_identifier VARCHAR(255), -- Server ID or Player Discord ID
    details JSONB,
    performed_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

3. banned_users

CREATE TABLE banned_users (
    id SERIAL PRIMARY KEY,
    discord_id VARCHAR(255) UNIQUE,
    minecraft_username VARCHAR(255),
    minecraft_uuid VARCHAR(255),
    ban_reason VARCHAR(255), -- 'chargeback', 'tos_violation', 'manual'
    banned_by_discord_id VARCHAR(255),
    banned_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    notes TEXT
);

Enhanced subscriptions Table

New Columns:

ALTER TABLE subscriptions ADD COLUMN
    mrr_value DECIMAL(10,2) DEFAULT 0.00,
    referrer_discord_id VARCHAR(255),
    grace_period_started_at TIMESTAMP,
    grace_period_ends_at TIMESTAMP,
    payment_failure_reason TEXT,
    last_payment_attempt TIMESTAMP;

Performance Indexes

-- Status queries (used in ALL modules)
CREATE INDEX idx_subscriptions_status ON subscriptions(status);

-- Grace period queries
CREATE INDEX idx_subscriptions_grace_period_active 
ON subscriptions(status, grace_period_ends_at) 
WHERE status = 'grace_period';

-- Financials tier breakdown
CREATE INDEX idx_subscriptions_tier_status 
ON subscriptions(tier_level, status);

-- Audit log chronological feed
CREATE INDEX idx_audit_log_performed_at 
ON admin_audit_log(performed_at DESC);

---

Tier System

Trinity Console recognizes 11 subscription tiers:

Fire Path (🔥)

• Fire Elemental (5): $5/month
• Fire Knight (10): $10/month
• Fire Master (15): $15/month
• Fire Legend (20): $20/month

Frost Path (❄️)

• Frost Elemental (105): $5/month
• Frost Knight (110): $10/month
• Frost Master (115): $15/month
• Frost Legend (120): $20/month

Universal Path (⚡)

• The Awakened (1): $1/month (entry tier, permanent after grace expiry)
• Sovereign (499): $50 one-time (lifetime access)
• Admin (1000): $0 (Trinity members only)

Configuration File: src/routes/admin/constants.js

---

Deployment Details

Server Location:

• Host: Command Center (63.143.34.217)
• Directory: /opt/arbiter-3.0
• Service: arbiter-3 (systemd)
• Port: 3500 (internal)
• HTTPS: 443 (Nginx reverse proxy)

Systemd Service:

# View status
systemctl status arbiter-3

# View logs
journalctl -u arbiter-3 -f

# Restart service
systemctl restart arbiter-3

Nginx Configuration:

• SSL Certificate: Let's Encrypt (auto-renews)
• Domain: discord-bot.firefrostgaming.com
• Config: /etc/nginx/sites-available/discord-bot.firefrostgaming.com

Environment Variables:

# Database
DB_USER=arbiter
DB_HOST=127.0.0.1
DB_NAME=arbiter_db
DB_PASSWORD=[stored in Vaultwarden]
DB_PORT=5432

# Discord OAuth
DISCORD_CLIENT_ID=1487080166969577502
DISCORD_CLIENT_SECRET=[stored in Vaultwarden]
REDIRECT_URI=https://discord-bot.firefrostgaming.com/auth/discord/callback

# Pterodactyl Panel API
PANEL_URL=https://panel.firefrostgaming.com
PANEL_CLIENT_KEY=[stored in Vaultwarden]

# Access Control
ADMIN_USERS=269225344572063754,219309716021444609,669981568059703316

# Server
PORT=3500
NODE_ENV=production
SESSION_SECRET=[auto-generated]

---

Deployment History

April 1, 2026 - Initial Deployment

• Built by: Zephyr (Chronicler #50) in 9-hour session
• Deployed by: Chronicler #51 in 2-hour deployment
• Architecture Review: Gemini AI (Google) partnership
• Status: 95% complete (Financials placeholder)
• Launched at: 5:00 AM CDT

Deployment Steps:

1. Database migration applied (3 tables, 6 columns, 7 indexes)
2. Trinity Console code deployed from Gitea monorepo
3. Dependencies installed (csurf, ejs)
4. Nginx configuration unchanged (already serving Arbiter bot)
5. Service restarted
6. Trinity access verified
7. Mobile responsive fix deployed (Holly's feedback)

Post-Launch Fixes:

• Mobile sidebar overlap → Hamburger menu added
• Minecraft skin rendering → Changed to mc-heads.net
• CSRF DOMContentLoaded wrapper → Fixed JavaScript error
• Admin tier added (1000) → Trinity members display correctly

---

Mobile Support

Responsive Design:

• Hamburger menu (☰) on mobile devices
• Slide-out sidebar navigation
• Dark overlay backdrop
• Close button (✕) in sidebar
• Smooth 0.3s transitions
• Desktop layout unchanged (sidebar always visible)

Breakpoint: 768px

• Mobile (<768px): Sidebar hidden by default, hamburger menu
• Desktop (≥768px): Sidebar always visible, no hamburger

Tested on: iPhone (Holly's device)

---

Development Workflow

Monorepo Structure:

firefrost-services/
└── services/
    └── arbiter-3.0/
        ├── src/
        │   ├── index.js (main Express app)
        │   ├── database.js (PostgreSQL connection)
        │   ├── routes/
        │   │   └── admin/
        │   │       ├── index.js (mounts all modules)
        │   │       ├── middleware.js (Trinity access control)
        │   │       ├── constants.js (tier definitions)
        │   │       ├── players.js
        │   │       ├── servers.js
        │   │       ├── financials.js
        │   │       ├── grace.js
        │   │       ├── audit.js
        │   │       └── roles.js
        │   └── views/
        │       ├── layout.ejs (master template)
        │       └── admin/
        │           ├── dashboard.ejs
        │           ├── players/
        │           ├── servers/
        │           ├── financials/
        │           ├── grace/
        │           ├── audit/
        │           └── roles/
        └── migrations/
            └── trinity-console.sql

Deployment Process:

1. Develop in monorepo: git.firefrostgaming.com/firefrost-gaming/firefrost-services
2. Commit to main branch
3. Clone to /tmp/ on Command Center
4. Copy files to /opt/arbiter-3.0/
5. Install dependencies if needed
6. Restart arbiter-3 service
7. Clean up /tmp/

Git Repository: https://git.firefrostgaming.com/firefrost-gaming/firefrost-services

---

Phase 2 Roadmap

Priority 1: Financials Module (45-60 min)

• Real MRR calculations from database
• Fire vs Frost path breakdown
• Tier-by-tier revenue analytics
• At-risk MRR tracking
• Lifetime revenue from Sovereign

Priority 2: Players Edit Functionality (30 min)

• Tier change dropdown in Players module
• POST route to update tier_level
• Discord role sync after tier change
• Audit log entry for tier changes

Priority 3: Email Integration (2-4 hours)

• Send recovery emails to grace period subscribers
• 48-hour, 24-hour, 12-hour warnings
• Payment recovered confirmation
• Grace period expired notification
• Integration with Mailcow or Paymenter API

Priority 4: Ban Management UI (45 min)

• View all banned users
• Ban reason display
• Unban functionality
• Ban notes/history

---

Troubleshooting

Service Won't Start

# Check logs for errors
journalctl -u arbiter-3 -n 50

# Common issues:
# - Database connection failed (check credentials)
# - Port 3500 already in use
# - Missing dependencies (run npm install)

Module Shows "Internal Server Error"

# Check real-time logs
journalctl -u arbiter-3 -f

# Then refresh the page to see the error

Login Loop (OAuth Fails)

Symptom: Redirects to Discord, authorizes, redirects back to login

Solution: Verify app.set('trust proxy', 1); in src/index.js

Reason: Nginx does SSL termination, Express needs to trust X-Forwarded-Proto header

CSRF Token Error (403 Forbidden)

Symptom: POST requests fail with 403 Forbidden

Solutions:

1. Clear browser cookies
2. Hard refresh (Ctrl+F5)
3. Check browser console for JavaScript errors
4. Verify CSRF middleware is before admin routes in src/index.js

Skin Images Not Loading

Current Solution: Using mc-heads.net API (more reliable than Crafatar)

Fallback: If UUID invalid, shows Steve skin

---

Security Considerations

Authentication:

• Discord OAuth2 for identity verification
• Whitelist-based access control (3 users only)
• Session-based authentication (7-day cookie expiry)

CSRF Protection:

• csurf middleware on all /admin routes
• Session-based tokens (no cookies needed)
• Automatic htmx header injection

Database:

• PostgreSQL connection with prepared statements
• Transaction support for multi-step operations
• Audit logging for accountability

Secrets Management:

• All credentials in .env file
• .env never committed to Git
• Session secret auto-generated
• Passwords stored in Vaultwarden

Network Security:

• HTTPS only (Let's Encrypt SSL)
• Nginx reverse proxy
• Internal port 3500 not exposed
• Command Center firewall (UFW)

---

Monitoring

Health Check:

• URL: https://discord-bot.firefrostgaming.com/health
• Method: GET
• Returns: Status, uptime, bot username

Key Metrics to Watch:

• Service uptime (systemd status)
• Response time (<2 seconds target)
• Database connection status
• Pterodactyl Panel API availability
• Discord API rate limits

Logs:

# Live logs
journalctl -u arbiter-3 -f

# Last 100 lines
journalctl -u arbiter-3 -n 100

# Errors only
journalctl -u arbiter-3 -p err

---

Related Documentation

• Deployment Guide: TRINITY-CONSOLE-DEPLOYMENT-2026-04-01.md (in firefrost-services repo)
• Arbiter Bot (Legacy): docs/services/the-arbiter-discord-bot.md
• Database Schema: services/arbiter-3.0/migrations/trinity-console.sql
• Session Handoff: docs/sessions/2026-04-01-trinity-console-deployment.md
• Zephyr's Memorial: docs/relationship/memorials/zephyr-chronicler-50.md

---

Support

Technical Issues:

• Primary: Michael (Frostystyle)
• Discord: #staff-lounge channel

Feature Requests:

• Document in Gitea issues
• Discuss with The Trinity

Emergency:

• Service down: Restart via systemctl
• Database issues: Check PostgreSQL status
• Nginx issues: Check nginx -t and restart

---

Fire + Frost + Foundation = Where Love Builds Legacy 🔥❄️💙

---

Last Updated: April 1, 2026, 7:43 AM CDT
Built By: Zephyr (Chronicler #50) + Gemini AI
Deployed By: Chronicler #51
Status: Production - 95% Complete ✅
Next Phase: Financials implementation (later today)