firefrost-operations-manual/docs/reference/architecture-decisions/firefrost-services-monorepo-decision.md

# Gemini Consultation: firefrost-services Monorepo Architecture
## Architectural Decision Record

**Date:** March 31, 2026  
**Participants:** Michael (The Wizard), Claude (The Golden Chronicler #50), Gemini (Architectural Consultant)  
**Topic:** Monorepo structure and version management strategy for Firefrost Gaming backend services  
**Decision Status:** ✅ APPROVED — Ready for implementation  
**Related Task:** Task #87 (Arbiter 2.1 deployment will be first service in new structure)

---

## Context

Firefrost Gaming has developed multiple custom Node.js backend services:
- **Arbiter** — Discord bot for subscription automation and role management
- **Whitelist Manager** — Pterodactyl Panel integration for Minecraft whitelist automation
- **Modpack Version Checker** — Monitors modpack updates across game servers
- **Future services** — Additional tools and integrations planned

These services currently exist as separate, unversioned deployments. We need a sustainable monorepo structure that:
- Supports remote management from RV (September 2027 goal)
- Works with simple `git pull` workflow (Chromebook accessibility requirement)
- Handles service-specific versioning
- Enables code sharing between services
- Aligns with industry best practices

---

## The Question

**Should we use directory-based version archives or Git tags for version management?**

### Approach A: Directory-Based Versions
```
firefrost-services/
├── arbiter/
│   ├── current/
│   ├── v2.1/
│   ├── v2.0/
│   └── archive/
```

**Pros:** Visual history, easy comparison  
**Cons:** Code duplication, breaks Node.js module resolution, anti-pattern

### Approach B: Git Tags for Versions
```
firefrost-services/
├── arbiter/
│   ├── src/
│   └── package.json
```

With service-prefixed tags: `arbiter-v2.1.0`, `whitelist-v1.0.0`

**Pros:** Native Git features, clean tree, industry standard  
**Cons:** Less visible (but proper use of Git)

---

## Gemini's Recommendation

**✅ Approach B (Git Tags) is the definitive winner.**

### Key Reasoning

**Directory-based versioning is an anti-pattern:**
- Bloats repository size
- Makes line-by-line change tracking impossible
- Breaks standard Node.js module resolution
- Not how Git is designed to work

**Git tags are purpose-built for versioning:**
- Clean codebase structure
- Full history via `git log`
- Easy rollbacks via `git checkout <tag>`
- Industry standard approach

---

## Approved Architecture

### Repository Structure

```
firefrost-services/
├── package.json              # Root workspace configuration
├── package-lock.json
├── .gitignore
├── README.md                 # Monorepo overview
├── services/
│   ├── arbiter/
│   │   ├── src/
│   │   │   └── index.js
│   │   ├── deploy/
│   │   │   └── arbiter.service
│   │   ├── .env.example
│   │   ├── package.json
│   │   └── README.md
│   ├── whitelist-manager/
│   └── modpack-version-checker/
├── shared/
│   ├── src/
│   │   ├── utils/
│   │   └── logger/
│   ├── package.json          # Named "@firefrost/shared"
│   └── README.md
└── future/                   # Experimental ideas
    └── README.md
```

### Version Management Strategy

**Service-Prefixed Git Tags:**
- Arbiter releases: `arbiter-v2.1.0`, `arbiter-v2.2.0`
- Whitelist Manager: `whitelist-v1.0.0`, `whitelist-v1.1.0`
- Modpack Checker: `modpack-v1.0.0`

This solves the "Service A is v2.1, Service B is v1.0" problem perfectly.

### npm Workspaces Configuration

**Root package.json:**
```json
{
  "name": "firefrost-services",
  "private": true,
  "workspaces": [
    "services/*",
    "shared"
  ]
}
```

**Shared package (shared/package.json):**
```json
{
  "name": "@firefrost/shared",
  "version": "1.0.0",
  "main": "src/index.js"
}
```

**Service dependency (services/arbiter/package.json):**
```json
{
  "name": "@firefrost/arbiter",
  "dependencies": {
    "discord.js": "^14.0.0",
    "@firefrost/shared": "*"
  }
}
```

**Key benefit:** Single `npm install` at root handles all services and shared code automatically.

---

## Deployment Workflow

### Standard Deployment (Update to Newer Version)

```bash
# On the server running Arbiter
cd /var/www/firefrost-services

# Step 1: Fetch latest tags
git fetch --all --tags

# Step 2: Check out specific version
git checkout arbiter-v2.1.0

# Step 3: Install dependencies
npm install

# Step 4: Restart service
sudo systemctl restart arbiter
```

### Rollback Workflow

```bash
# Step 1: Return to main branch
git checkout main

# Step 2: Pull latest
git pull origin main

# Step 3: Check out rollback version
git checkout arbiter-v2.0.0

# Step 4: Restart service
sudo systemctl restart arbiter
```

### Critical: Detached HEAD State

When you `git checkout <tag>`, Git enters "detached HEAD" state (static snapshot).

**Important:** You cannot run `git pull` while in detached HEAD state.

To update, you must return to main branch first, pull, then checkout the new tag.

---

## systemd Configuration

**Correct configuration for monorepo services:**

```ini
[Unit]
Description=Arbiter Discord Bot
After=network.target

[Service]
Type=simple
User=arbiter
WorkingDirectory=/var/www/firefrost-services/services/arbiter
ExecStart=/usr/bin/node src/index.js
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
```

**Key change from standalone deployment:**
- `WorkingDirectory` set to service folder (not repo root)
- `ExecStart` uses relative path from WorkingDirectory
- This ensures proper file resolution and environment variable loading

---

## Environment Variables Strategy

**Approved approach:** `.env.example` committed + `.gitignore` for actual `.env`

### Implementation

1. **Create `.env.example` in each service directory** (committed to Git):
   ```
   DISCORD_TOKEN=your_token_here
   PAYMENTER_WEBHOOK_SECRET=your_secret_here
   DATABASE_URL=postgresql://user:pass@host:port/db
   ```

2. **Add `.env` to root `.gitignore`** (prevents accidental commits):
   ```
   .env
   node_modules/
   ```

3. **On server, manually create actual `.env`** in service directory with real credentials

**Why this works:**
- ✅ Secrets never committed to Git
- ✅ Configuration template visible to developers
- ✅ Industry standard approach
- ✅ Easy to document what each service needs

---

## Multi-Server Deployment

**Scenario:** Arbiter runs on Server A, Whitelist Manager runs on Server B

### Process

1. **Clone entire repo to both servers:**
   ```bash
   # Server A (Command Center)
   git clone https://token@git.firefrostgaming.com/firefrost-gaming/firefrost-services.git
   
   # Server B (Panel VPS)
   git clone https://token@git.firefrostgaming.com/firefrost-gaming/firefrost-services.git
   ```

2. **Server A deployment (Arbiter only):**
   ```bash
   cd firefrost-services
   git checkout arbiter-v2.1.0
   npm install
   # Create services/arbiter/.env with actual tokens
   sudo systemctl enable arbiter
   sudo systemctl start arbiter
   ```

3. **Server B deployment (Whitelist Manager only):**
   ```bash
   cd firefrost-services
   git checkout whitelist-v1.0.0
   npm install
   # Create services/whitelist-manager/.env with actual tokens
   sudo systemctl enable whitelist-manager
   sudo systemctl start whitelist-manager
   ```

**Result:** Entire codebase exists on both servers, but only the specific service configured in systemd runs on each.

---

## Key Principles Established

### 1. Simplicity Over Complexity
- Native npm workspaces (no Lerna, Nx, or Turborepo)
- Standard Git features (no custom versioning tools)
- Direct systemd management (no PM2 or other process managers)

### 2. Security First
- Secrets never committed to Git
- `.env.example` templates for documentation
- Manual `.env` creation on servers

### 3. Service Independence
- Services never import from other services directly
- Shared code lives in `@firefrost/shared` package
- Each service can version independently

### 4. Sustainability
- Structure supports decades-long maintenance
- Simple enough to manage remotely from RV
- Clear rollback procedures

### 5. Accessibility
- Works with `git pull` workflow (Chromebook compatible)
- Minimal typing required for deployments
- Clear, documented procedures

---

## Anti-Patterns to Avoid

**Gemini explicitly warned against:**

1. **Service interdependency** — Never import code directly from other services. Use `@firefrost/shared` instead.

2. **Over-complicated tooling** — Don't reach for Lerna/Nx/Turborepo for a small team. Standard npm workspaces are sufficient.

3. **Stale systemd paths** — Always update `WorkingDirectory` and `ExecStart` to match new monorepo structure.

4. **Secrets in Git** — Never put actual `.env` files in repository, even temporarily.

---

## Implementation Plan

### Phase 1: Repository Setup
1. Create `firefrost-services` repo in Gitea
2. Set up directory structure
3. Create root `package.json` with workspaces
4. Add `.gitignore` and README

### Phase 2: Arbiter 2.1 Migration (First Service)
1. Move Arbiter 2.1 code into `services/arbiter/`
2. Create `@firefrost/shared` package
3. Update systemd configuration
4. Test deployment workflow
5. Tag as `arbiter-v2.1.0`

### Phase 3: Additional Services
1. Migrate Whitelist Manager
2. Migrate Modpack Version Checker
3. Tag each with appropriate versions

### Phase 4: Future Development
1. Add experimental services to `future/`
2. Document service creation process
3. Establish CI/CD if needed

---

## Success Metrics

**This architecture is successful if:**
- ✅ Single `npm install` deploys all services
- ✅ Rollbacks take <5 minutes
- ✅ New services can be added without restructuring
- ✅ Documentation is clear enough for future staff
- ✅ Michael can manage from RV in 2027

---

## Related Documentation

- **Task #87:** `docs/tasks/arbiter-2-1-cancellation-flow/README.md` (Arbiter 2.1 architecture)
- **Gemini Session Transcript:** This document
- **Operations Manual:** `docs/core/infrastructure-manifest.md` (will be updated post-deployment)

---

## Acknowledgments

**Gemini's contributions:**
- Validated Git tags as industry standard
- Explained npm workspaces mechanics
- Caught systemd `WorkingDirectory` requirement
- Recommended `.env.example` + `.gitignore` security pattern
- Documented detached HEAD workflow clearly

**Chronicler #49's contributions:**
- Documented Arbiter 2.1 architecture
- Identified need for monorepo structure
- Established "We Don't Kick People Out" philosophy

---

## Final Decision

**APPROVED: Proceed with Approach B (Git Tags + npm Workspaces)**

Repository location: `git.firefrostgaming.com/firefrost-gaming/firefrost-services`

First deployment: Arbiter 2.1 (validates workflow, tests monorepo structure)

**For children not yet born.** 💙🔥❄️

---

**Document Status:** ✅ APPROVED  
**Implementation Status:** Ready to begin  
**Next Action:** Create `firefrost-services` repository in Gitea  
**Responsible:** The Golden Chronicler (#50) + Michael (The Wizard)

**Documented by:** The Golden Chronicler (Chronicler #50)  
**Date:** March 31, 2026  
**Session:** Gemini architectural consultation on monorepo strategy