---
task_number: 94
status: complete
priority: P3
owner: Michael
created: 2026-01-01
---
task_number: 94

# Task #94: Global Restart Scheduler

**Status:** ✅ IMPLEMENTED (April 5, 2026)  
**Chronicler:** #61  
**Priority:** Medium  
**Actual Effort:** ~45 minutes (including troubleshooting)  
**Dependencies:** Arbiter 3.5.0, Pterodactyl Client API

---
task_number: 94

## Overview

Add a **Global Restart Scheduler** to the Trinity Console that manages staggered restart times for all 21 Minecraft servers across TX1 and NC1 nodes. This eliminates the need to manually configure individual schedules in Pterodactyl.

### The Problem
- 21 servers need restart schedules
- Manual configuration in Pterodactyl is tedious
- No central visibility of restart timing
- Risk of overlapping restarts hammering nodes

### The Solution
- Trinity Console page with visual timeline
- Staggered restarts (configurable base time + interval per node)
- One-click sync to Pterodactyl API
- Audit system to detect/remove conflicting schedules
- Drag-and-drop sort order for boot priority

---
task_number: 94

## Architecture Decisions

### Source of Truth
**Store schedule intent in `arbiter_db`, sync to Pterodactyl as execution.**

- `global_restart_config` table holds base time + interval per node
- `server_restart_schedules` table holds per-server state
- Trinity calculates `effective_time` and pushes to Pterodactyl
- If Pterodactyl is wiped, Trinity can re-provision instantly

### Schedule Naming
All Trinity-managed schedules use prefix: `[Trinity] Daily Restart`

This prevents confusion with manual schedules and allows Trinity to identify its own schedules.

### Sort Order
**Manual sort via drag-and-drop.** Boot order matters:
- Proxy servers should restart first
- Hub/Lobby servers should restart last
- Heavy modpacks need spacing

### Conflict Resolution
Before first sync, run **Audit** to detect existing restart schedules. Trinity can "nuke" rogue schedules before taking control.

---
task_number: 94

## Database Schema

### Migration File: `migrations/094_global_restart_scheduler.sql`

```sql
-- 1. Configuration for Node-wide Stagger Logic
CREATE TABLE IF NOT EXISTS global_restart_config (
    id SERIAL PRIMARY KEY,
    node VARCHAR(10) UNIQUE NOT NULL,    -- 'TX1', 'NC1'
    base_time TIME NOT NULL,             -- e.g., '04:00:00' (UTC)
    interval_minutes INT DEFAULT 5,      -- Stagger gap
    is_enabled BOOLEAN DEFAULT true,     -- Global master switch per node
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_by VARCHAR(50)               -- Discord Username
);

-- 2. Individual Server Execution State
CREATE TABLE IF NOT EXISTS server_restart_schedules (
    id SERIAL PRIMARY KEY,
    server_id VARCHAR(50) UNIQUE NOT NULL,  -- Pterodactyl 8-char short ID
    server_name VARCHAR(100) NOT NULL,
    node VARCHAR(10) NOT NULL,
    sort_order INT NOT NULL DEFAULT 0,      -- Manual boot order
    effective_time TIME,                    -- Calculated: base + (sort * interval)
    ptero_schedule_id INT DEFAULT NULL,     -- ID of schedule on Pterodactyl
    skip_restart BOOLEAN DEFAULT false,     -- Individual "Maintenance Mode"
    sync_status VARCHAR(20) DEFAULT 'PENDING', -- 'SUCCESS', 'PENDING', 'FAILED'
    last_error TEXT DEFAULT NULL,           -- API error capture
    last_synced_at TIMESTAMP NULL,
    
    CONSTRAINT fk_node_config 
        FOREIGN KEY (node) 
        REFERENCES global_restart_config(node) 
        ON UPDATE CASCADE
);

-- 3. Audit Trail for Sync Operations
CREATE TABLE IF NOT EXISTS sync_logs (
    id SERIAL PRIMARY KEY,
    server_id VARCHAR(50) NOT NULL,
    action VARCHAR(255) NOT NULL,         -- e.g., 'Deleted Rogue Schedule', 'Created Schedule'
    status VARCHAR(20) NOT NULL,          -- 'SUCCESS', 'FAILED'
    error_message TEXT DEFAULT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

-- 4. Performance Indexes
CREATE INDEX idx_server_node_order ON server_restart_schedules (node, sort_order);
CREATE INDEX idx_sync_status ON server_restart_schedules (sync_status);
CREATE INDEX idx_sync_logs_server ON sync_logs (server_id);
CREATE INDEX idx_sync_logs_created ON sync_logs (created_at);

-- 5. Initial Seed Data
INSERT INTO global_restart_config (node, base_time, interval_minutes, updated_by)
VALUES 
    ('TX1', '04:00:00', 5, 'The Wizard'),
    ('NC1', '04:30:00', 5, 'The Wizard')
ON CONFLICT (node) DO NOTHING;
```

---
task_number: 94

## Implementation Files

### 1. Utility: Stagger Calculation

**File:** `src/utils/scheduler.js`

```javascript
const { addMinutes, format, parse } = require('date-fns');

function calculateStagger(baseTime, interval, servers) {
    const start = parse(baseTime, 'HH:mm', new Date());
    
    return servers.map((server, index) => ({
        ...server,
        effective_time: format(addMinutes(start, index * interval), 'HH:mm:ss')
    }));
}

module.exports = { calculateStagger };
```

### 2. Pterodactyl API Functions

**File:** `src/lib/ptero-sync.js`

```javascript
const axios = require('axios');
const db = require('../db');

const PTERO_URL = 'https://panel.firefrostgaming.com/api/client/servers';
const headers = {
    'Authorization': `Bearer ${process.env.PTERO_CLIENT_KEY}`,
    'Content-Type': 'application/json',
    'Accept': 'application/json'
};

/**
 * Sync a single server's schedule to Pterodactyl
 */
async function syncToPterodactyl(serverId) {
    const result = await db.query('SELECT * FROM server_restart_schedules WHERE server_id = $1', [serverId]);
    const server = result.rows[0];
    const [hour, minute] = server.effective_time.split(':');

    const pteroUrl = `${PTERO_URL}/${server.server_id}/schedules`;
    
    const payload = {
        name: "[Trinity] Daily Restart",
        minute, 
        hour, 
        day_of_week: "*", 
        day_of_month: "*", 
        month: "*",
        is_active: !server.skip_restart
    };

    try {
        let scheduleId = server.ptero_schedule_id;

        if (!scheduleId) {
            // Create new schedule
            const res = await axios.post(pteroUrl, payload, { headers });
            scheduleId = res.data.attributes.id;

            // Attach the restart task
            await axios.post(`${pteroUrl}/${scheduleId}/tasks`, {
                action: "power", 
                payload: "restart", 
                time_offset: 0
            }, { headers });
        } else {
            // Update existing schedule (Pterodactyl uses POST for updates too)
            await axios.post(`${pteroUrl}/${scheduleId}`, payload, { headers });
        }

        await db.query(
            `UPDATE server_restart_schedules 
             SET ptero_schedule_id = $1, sync_status = $2, last_error = NULL, last_synced_at = NOW() 
             WHERE server_id = $3`,
            [scheduleId, 'SUCCESS', server.server_id]
        );
        
        return { success: true };
    } catch (err) {
        const errorMsg = err.response?.data?.errors?.[0]?.detail || err.message;
        await db.query(
            `UPDATE server_restart_schedules 
             SET sync_status = $1, last_error = $2 
             WHERE server_id = $3`,
            ['FAILED', errorMsg, server.server_id]
        );
        
        return { success: false, error: errorMsg };
    }
}

/**
 * Find existing restart schedules NOT owned by Trinity
 */
async function findRogueRestarts(serverId) {
    const url = `${PTERO_URL}/${serverId}/schedules?include=tasks`;

    const res = await axios.get(url, { headers });
    const rogueSchedules = [];

    for (const sched of res.data.data) {
        // Ignore Trinity's own schedules
        if (sched.attributes.name === "[Trinity] Daily Restart") continue;

        // Check if any task is a restart command
        const hasRestart = sched.attributes.relationships.tasks.data.some(
            task => task.attributes.action === 'power' && task.attributes.payload === 'restart'
        );

        if (hasRestart) {
            rogueSchedules.push({
                id: sched.attributes.id,
                name: sched.attributes.name,
                cron: `${sched.attributes.minute} ${sched.attributes.hour} * * *`
            });
        }
    }
    return rogueSchedules;
}

/**
 * Delete a rogue schedule and log the action
 */
async function deleteRogueSchedule(serverId, scheduleId, scheduleName, adminUser) {
    const url = `${PTERO_URL}/${serverId}/schedules/${scheduleId}`;

    try {
        await axios.delete(url, { headers });
        
        await db.query(
            `INSERT INTO sync_logs (server_id, action, status, error_message) 
             VALUES ($1, $2, $3, $4)`,
            [serverId, `Deleted Rogue Schedule: ${scheduleName}`, 'SUCCESS', `Initiated by ${adminUser}`]
        );
        return true;
    } catch (err) {
        console.error(`Failed to delete schedule ${scheduleId} on ${serverId}`, err.message);
        return false;
    }
}

module.exports = { syncToPterodactyl, findRogueRestarts, deleteRogueSchedule };
```

### 3. Express Routes

**File:** `src/routes/admin/scheduler.js`

```javascript
const express = require('express');
const router = express.Router();
const db = require('../../db');
const { calculateStagger } = require('../../utils/scheduler');
const { syncToPterodactyl, findRogueRestarts, deleteRogueSchedule } = require('../../lib/ptero-sync');

// GET: Main Scheduler Page
router.get('/', async (req, res) => {
    const configs = await db.query('SELECT * FROM global_restart_config ORDER BY node');
    const servers = await db.query('SELECT * FROM server_restart_schedules ORDER BY node, sort_order');
    res.render('admin/scheduler', { configs: configs.rows, servers: servers.rows });
});

// POST: Update Config (HTMX real-time preview)
router.post('/update-config', async (req, res) => {
    const { node, base_time, interval_minutes } = req.body;
    const admin = req.user?.username || 'The Wizard';

    await db.query(
        'UPDATE global_restart_config SET base_time = $1, interval_minutes = $2, updated_by = $3, updated_at = NOW() WHERE node = $4',
        [base_time, interval_minutes, admin, node]
    );

    const servers = await db.query('SELECT * FROM server_restart_schedules WHERE node = $1 ORDER BY sort_order', [node]);
    const updated = calculateStagger(base_time, parseInt(interval_minutes), servers.rows);

    for (const s of updated) {
        await db.query('UPDATE server_restart_schedules SET effective_time = $1 WHERE server_id = $2', [s.effective_time, s.server_id]);
    }

    const allServers = await db.query('SELECT * FROM server_restart_schedules ORDER BY node, sort_order');
    res.render('admin/partials/scheduler-table', { servers: allServers.rows });
});

// POST: Sync All Schedules to Pterodactyl
router.post('/sync-all', async (req, res) => {
    const servers = await db.query('SELECT server_id FROM server_restart_schedules WHERE skip_restart = false');

    for (const s of servers.rows) {
        await syncToPterodactyl(s.server_id);
        await new Promise(resolve => setTimeout(resolve, 200)); // Rate limit buffer
    }
    
    const allServers = await db.query('SELECT * FROM server_restart_schedules ORDER BY node, sort_order');
    res.render('admin/partials/scheduler-table', { servers: allServers.rows });
});

// POST: Reorder Servers (drag-and-drop)
router.post('/reorder-servers', async (req, res) => {
    const { orderedIds } = req.body;
    
    try {
        for (let i = 0; i < orderedIds.length; i++) {
            await db.query('UPDATE server_restart_schedules SET sort_order = $1 WHERE server_id = $2', [i, orderedIds[i]]);
        }

        // Recalculate effective times for all nodes
        const nodes = await db.query('SELECT node, base_time, interval_minutes FROM global_restart_config');
        
        for (const config of nodes.rows) {
            const servers = await db.query('SELECT * FROM server_restart_schedules WHERE node = $1 ORDER BY sort_order', [config.node]);
            const updated = calculateStagger(config.base_time, config.interval_minutes, servers.rows);
            
            for (const s of updated) {
                await db.query('UPDATE server_restart_schedules SET effective_time = $1 WHERE server_id = $2', [s.effective_time, s.server_id]);
            }
        }
        res.sendStatus(200);
    } catch (error) {
        console.error("Reorder failed:", error);
        res.status(500).send("Failed to reorder servers.");
    }
});

// GET: Table fragment for HTMX refresh
router.get('/table-only', async (req, res) => {
    const allServers = await db.query('SELECT * FROM server_restart_schedules ORDER BY node, sort_order');
    res.render('admin/partials/scheduler-table', { servers: allServers.rows });
});

// GET: Audit Modal for a specific node
router.get('/audit-modal/:node', async (req, res) => {
    const { node } = req.params;
    const servers = await db.query('SELECT server_id, server_name FROM server_restart_schedules WHERE node = $1', [node]);
    
    const auditResults = [];
    let totalRogue = 0;

    for (const s of servers.rows) {
        try {
            const rogue = await findRogueRestarts(s.server_id);
            if (rogue.length > 0) {
                auditResults.push({ serverName: s.server_name, serverId: s.server_id, rogueSchedules: rogue });
                totalRogue += rogue.length;
            }
        } catch (err) {
            console.error(`Audit failed for ${s.server_name}`, err.message);
        }
        await new Promise(resolve => setTimeout(resolve, 200));
    }

    res.render('admin/partials/audit-modal', { 
        results: auditResults, 
        node, 
        totalRogue,
        serverCount: auditResults.length
    });
});

// POST: Nuke Rogue Schedules and Trigger Sync
router.post('/audit/nuke/:node', async (req, res) => {
    const { node } = req.params;
    const adminUser = req.user?.username || 'The Wizard';
    const schedulesToNuke = JSON.parse(req.body.nukeData);

    for (const item of schedulesToNuke) {
        await deleteRogueSchedule(item.serverId, item.scheduleId, item.scheduleName, adminUser);
        await new Promise(resolve => setTimeout(resolve, 200));
    }

    // Close modal and trigger sync via HTMX
    res.set('HX-Trigger', 'start-global-sync');
    res.send('');
});

module.exports = router;
```

### 4. Main View

**File:** `src/views/admin/scheduler.ejs`

```html
<div class="p-6 bg-slate-900 text-white min-h-screen">
    <h1 class="text-3xl font-bold mb-6 text-blue-400">Trinity Global Scheduler</h1>

    <!-- Node Configuration Cards -->
    <div class="grid grid-cols-1 md:grid-cols-2 gap-6 mb-8">
        <% configs.forEach(config => { %>
            <form class="bg-slate-800 p-4 rounded border border-slate-700"
                  hx-post="/admin/scheduler/update-config"
                  hx-target="#scheduler-table"
                  hx-trigger="change from:input">
                <input type="hidden" name="node" value="<%= config.node %>">
                <h2 class="text-xl font-semibold mb-4 <%= config.node === 'TX1' ? 'text-orange-400' : 'text-blue-400' %>">
                    Node: <%= config.node %>
                </h2>
                <div class="flex gap-4">
                    <div>
                        <label class="block text-xs uppercase text-slate-400">Base Time (UTC)</label>
                        <input type="time" name="base_time" value="<%= config.base_time.substring(0,5) %>" 
                               class="bg-slate-700 p-2 rounded text-white">
                    </div>
                    <div>
                        <label class="block text-xs uppercase text-slate-400">Interval (Mins)</label>
                        <input type="number" name="interval_minutes" value="<%= config.interval_minutes %>" 
                               class="bg-slate-700 p-2 rounded w-20 text-white">
                    </div>
                </div>
            </form>
        <% }) %>
    </div>

    <!-- Action Buttons -->
    <div class="flex flex-wrap gap-4 mb-6 items-center">
        <button hx-get="/admin/scheduler/audit-modal/TX1" 
                hx-target="#modal-container"
                hx-indicator="#audit-tx1-spinner"
                class="bg-orange-600 hover:bg-orange-500 text-white px-6 py-2 rounded font-bold transition flex items-center">
            Run Audit (TX1)
            <span id="audit-tx1-spinner" class="htmx-indicator ml-2 text-white animate-spin">↻</span>
        </button>

        <button hx-get="/admin/scheduler/audit-modal/NC1" 
                hx-target="#modal-container"
                hx-indicator="#audit-nc1-spinner"
                class="bg-blue-600 hover:bg-blue-500 text-white px-6 py-2 rounded font-bold transition flex items-center">
            Run Audit (NC1)
            <span id="audit-nc1-spinner" class="htmx-indicator ml-2 text-white animate-spin">↻</span>
        </button>

        <div class="w-px h-8 bg-slate-700 mx-2"></div>

        <button id="main-sync-btn"
                hx-post="/admin/scheduler/sync-all" 
                hx-target="#scheduler-table"
                hx-indicator="#sync-spinner"
                hx-trigger="click, start-global-sync from:body"
                class="bg-emerald-600 hover:bg-emerald-500 text-white px-6 py-2 rounded font-bold transition flex items-center">
            Deploy Schedules to Frostwall
            <span id="sync-spinner" class="htmx-indicator ml-2 text-white animate-spin">↻</span>
        </button>
    </div>

    <!-- Server Table -->
    <div id="scheduler-table" class="bg-slate-800 rounded overflow-hidden border border-slate-700">
        <%- include('partials/scheduler-table', { servers: servers }) %>
    </div>

    <!-- Modal Container -->
    <div id="modal-container"></div>
</div>

<!-- SortableJS -->
<script src="https://cdn.jsdelivr.net/npm/sortablejs@latest/Sortable.min.js"></script>
<script>
document.addEventListener("DOMContentLoaded", function() {
    const tbody = document.getElementById('sortable-servers');
    
    if(tbody) {
        new Sortable(tbody, {
            handle: '.drag-handle',
            animation: 150,
            ghostClass: 'bg-slate-600',
            
            onEnd: function (evt) {
                const newOrder = Array.from(tbody.querySelectorAll('tr')).map(row => row.dataset.id);
                
                fetch('/admin/scheduler/reorder-servers', {
                    method: 'POST',
                    headers: { 'Content-Type': 'application/json' },
                    body: JSON.stringify({ orderedIds: newOrder })
                })
                .then(() => htmx.ajax('GET', '/admin/scheduler/table-only', '#scheduler-table'));
            }
        });
    }
});
</script>
```

### 5. Table Partial

**File:** `src/views/admin/partials/scheduler-table.ejs`

```html
<table class="w-full text-left">
    <thead class="bg-slate-700 text-slate-300">
        <tr>
            <th class="p-3 w-10"></th>
            <th class="p-3">Server</th>
            <th class="p-3">Node</th>
            <th class="p-3">Restart Time</th>
            <th class="p-3">Sync Status</th>
            <th class="p-3 w-1/3">Timeline</th>
        </tr>
    </thead>
    <tbody id="sortable-servers">
        <% servers.forEach((server, i) => { %>
            <tr class="border-t border-slate-700 hover:bg-slate-750 transition" data-id="<%= server.server_id %>">
                <td class="p-3 cursor-grab text-slate-500 hover:text-white">
                    <span class="drag-handle">☰</span>
                </td>
                <td class="p-3 font-medium"><%= server.server_name %></td>
                <td class="p-3">
                    <span class="px-2 py-1 rounded text-xs <%= server.node === 'TX1' ? 'bg-orange-900 text-orange-300' : 'bg-blue-900 text-blue-300' %>">
                        <%= server.node %>
                    </span>
                </td>
                <td class="p-3 font-mono"><%= server.effective_time %></td>
                <td class="p-3 text-xs">
                    <% if (server.sync_status === 'SUCCESS') { %>
                        <span class="text-green-400">● Synced</span>
                    <% } else if (server.sync_status === 'FAILED') { %>
                        <span class="text-red-400" title="<%= server.last_error %>">× Error</span>
                    <% } else { %>
                        <span class="text-yellow-400">○ Pending</span>
                    <% } %>
                </td>
                <td class="p-3">
                    <div class="w-full bg-slate-900 h-2 rounded-full relative overflow-hidden">
                        <div class="absolute h-full <%= server.node === 'TX1' ? 'bg-orange-500' : 'bg-blue-500' %>" 
                             style="width: 8px; left: <%= (server.sort_order * 4) %>%"></div>
                    </div>
                </td>
            </tr>
        <% }) %>
    </tbody>
</table>
```

### 6. Audit Modal Partial

**File:** `src/views/admin/partials/audit-modal.ejs`

```html
<div id="audit-modal" class="fixed inset-0 z-50 flex items-center justify-center bg-black bg-opacity-70 backdrop-blur-sm">
    <div class="bg-slate-900 border border-red-500 rounded-lg shadow-2xl w-full max-w-2xl p-6 relative">
        
        <h2 class="text-2xl font-bold text-red-500 mb-2">⚠ Action Required: Conflicts Detected</h2>
        
        <% if (totalRogue > 0) { %>
            <p class="text-slate-300 mb-4">
                Found <strong class="text-white"><%= totalRogue %></strong> rogue restart schedule(s) across 
                <strong class="text-white"><%= serverCount %></strong> server(s) on Node <%= node %>. 
                These must be removed before Trinity can take control.
            </p>

            <div class="bg-slate-800 rounded p-4 mb-6 max-h-64 overflow-y-auto border border-slate-700">
                <ul class="space-y-3">
                    <% results.forEach(result => { %>
                        <li class="border-b border-slate-700 pb-2 last:border-0">
                            <span class="text-orange-400 font-semibold"><%= result.serverName %></span>
                            <ul class="ml-4 mt-1 text-sm text-slate-400">
                                <% result.rogueSchedules.forEach(sched => { %>
                                    <li>- "<%= sched.name %>" (Cron: <%= sched.cron %>)</li>
                                <% }) %>
                            </ul>
                        </li>
                    <% }) %>
                </ul>
            </div>

            <% 
                const nukePayload = [];
                results.forEach(r => r.rogueSchedules.forEach(s => nukePayload.push({
                    serverId: r.serverId, scheduleId: s.id, scheduleName: s.name
                })));
            %>

            <form hx-post="/admin/scheduler/audit/nuke/<%= node %>" hx-target="#audit-modal" hx-swap="outerHTML">
                <input type="hidden" name="nukeData" value='<%- JSON.stringify(nukePayload) %>'>
                
                <div class="flex justify-end gap-4 mt-6">
                    <button type="button" onclick="document.getElementById('audit-modal').remove()" 
                            class="px-4 py-2 text-slate-400 hover:text-white transition">Cancel</button>
                    <button type="submit" hx-indicator="#nuke-spinner" 
                            class="bg-red-600 hover:bg-red-500 text-white px-6 py-2 rounded font-bold transition flex items-center">
                        Nuke <%= totalRogue %> Schedules & Take Control
                        <span id="nuke-spinner" class="htmx-indicator ml-2 animate-spin">↻</span>
                    </button>
                </div>
            </form>

        <% } else { %>
            <p class="text-green-400 mb-6 font-medium">No conflicts found on Node <%= node %>. The Frostwall is clear.</p>
            <div class="flex justify-end">
                <button type="button" onclick="document.getElementById('audit-modal').remove()" 
                        class="bg-slate-700 hover:bg-slate-600 text-white px-6 py-2 rounded transition">Close</button>
            </div>
        <% } %>
    </div>
</div>
```

---
task_number: 94

## Implementation Checklist

### Phase 1: Database Setup
- [ ] Run SQL migration on arbiter_db
- [ ] Populate `server_restart_schedules` with all 21 servers from Pterodactyl

### Phase 2: Backend
- [ ] Install dependency: `npm install date-fns`
- [ ] Create `src/utils/scheduler.js`
- [ ] Create `src/lib/ptero-sync.js`
- [ ] Create `src/routes/admin/scheduler.js`
- [ ] Add `PTERO_CLIENT_KEY` to environment (webuser_api key)
- [ ] Register route in main app: `app.use('/admin/scheduler', require('./routes/admin/scheduler'))`

### Phase 3: Frontend
- [ ] Create `src/views/admin/scheduler.ejs`
- [ ] Create `src/views/admin/partials/scheduler-table.ejs`
- [ ] Create `src/views/admin/partials/audit-modal.ejs`
- [ ] Add link to Trinity Console sidebar

### Phase 4: First Run
- [ ] Run Audit on TX1 — nuke any rogue restarts
- [ ] Run Audit on NC1 — nuke any rogue restarts
- [ ] Deploy schedules to Frostwall
- [ ] Verify in Pterodactyl that `[Trinity] Daily Restart` schedules exist

### Phase 5: Verification
- [ ] Check a server's schedule in Pterodactyl panel
- [ ] Confirm cron time matches Trinity's effective_time
- [ ] Wait for first scheduled restart to confirm it works

---
task_number: 94

## API Notes

### Pterodactyl Client API Quirks
1. **Uses POST for updates** — not PATCH
2. **8-character short ID required** — not full UUID
3. **Two-step schedule creation** — create schedule, then add task
4. **Rate limiting** — add 200ms delay between calls

### Default Stagger Pattern
| Node | Base Time | Interval | Window |
|------|-----------|----------|--------|
| TX1  | 04:00 UTC | 5 min    | ~50 min for 10 servers |
| NC1  | 04:30 UTC | 5 min    | ~55 min for 11 servers |

---
task_number: 94

## Consultation Record

This task was architected with Gemini AI on April 5, 2026. Full consultation covered:
- Source of truth architecture (DB vs API)
- UI/UX patterns for staggered visualization
- Conflict detection and resolution
- Drag-and-drop sorting
- Rate limiting and API quirks
- Audit trail logging

---
task_number: 94

## Implementation Record (April 5, 2026)

**Implemented by:** Chronicler #61  
**Build time:** ~11 minutes  
**Troubleshooting:** ~15 minutes  
**Total:** ~26 minutes (under the 30-minute bet!)

### Issues Encountered & Fixes

| Issue | Root Cause | Fix |
|-------|------------|-----|
| `include is not a function` | express-ejs-layouts doesn't support nested includes | Inlined table HTML, routes return raw HTML for HTMX partials |
| `server_id null` on import | Discovery returns `identifier`, not `id` | Changed `server.id` → `server.identifier` |
| Audit showing "All Clear" incorrectly | `PTERO_CLIENT_KEY` not in .env | Added key to `/opt/arbiter-3.0/.env` |
| Audit filter too strict | Required power task in relationships (not included by default) | Simplified to catch ALL non-Trinity schedules |
| Syntax error after edit | Duplicate code block left behind | Removed duplicate |
| "Forbidden" on Update | Missing CSRF token in form | Added `<input type="hidden" name="_csrf" value="<%= csrfToken %>">` |
| "Error updating config" | `base_time` was `HH:mm`, needed `HH:mm:ss` | Normalize: `if (!base_time.includes(':00', 3)) base_time += ':00'` |

### Environment Variables Required

```bash
# Add to /opt/arbiter-3.0/.env
PTERO_CLIENT_KEY=ptlc_XXXX  # Pterodactyl Client API key (webuser_api)
```

### Feature Status

| Feature | Status |
|---------|--------|
| Import Servers | ✅ Working |
| Staggered time calculation | ✅ Working |
| Update config (base time/interval) | ✅ Working |
| Audit (detect rogue schedules) | ✅ Working |
| Nuke rogue schedules | ⏳ Ready, untested |
| Sync All | ⏳ Ready, untested |
| Drag-and-drop reorder | ⏳ Untested |
| Skip toggle | ⏳ Untested |

### Pending: Holly Review

Before using Sync All or Nuke:
1. Holly needs to review existing schedules
2. Audit both NC1 and TX1
3. Decide which rogue schedules to nuke
4. Then Sync All to create `[Trinity] Daily Restart` schedules

---
task_number: 94

**Fire + Frost + Foundation = Where Love Builds Legacy** 🔥❄️
