diff --git a/docs/consultations/gemini-social-analytics-2026-04-10.md b/docs/consultations/gemini-social-analytics-2026-04-10.md new file mode 100644 index 0000000..b905a9e --- /dev/null +++ b/docs/consultations/gemini-social-analytics-2026-04-10.md @@ -0,0 +1,264 @@ +# Gemini Consultation: Social Analytics Module Review + TikTok API Strategy + +**Date:** April 10, 2026 +**From:** Michael (The Wizard) + Claude (Chronicler #76) +**To:** Gemini (Architectural Partner) +**Re:** Review of new Social Analytics module + Phase 2 API integration planning + +--- + +## Hey Gemini! 👋 + +We just built a new Social Analytics module for Trinity Console and wanted your eyes on it before we deploy. You wrote the original Trinity Console architecture, so you know the patterns better than anyone. We also want your input on Phase 2 — automating data pulls via TikTok's API. + +Michael's 5 days out from soft launch and feeling the pressure. Zero Wanderers in Discord yet despite a FOMO campaign running. We built this so he can actually SEE the data and know if people are even viewing the posts (spoiler: they are — 1.9K views on TikTok, 98% from For You page). + +--- + +## Context + +**The Problem:** Michael posts to 5 platforms (TikTok, Facebook, Instagram, X, Bluesky) but has no centralized way to track performance. He's been checking each platform manually and feeling overwhelmed. + +**The Solution:** A new Trinity Console module at `/admin/social` that: +- Phase 1 (built today): Manual entry dashboard — Michael/Meg enter metrics from TikTok Studio +- Phase 2 (future): API integration to pull data automatically + +--- + +## What We Built (Phase 1) + +### Database Schema + +```sql +CREATE TYPE social_platform AS ENUM ('tiktok', 'facebook', 'instagram', 'x', 'bluesky'); + +CREATE TABLE social_posts ( + id SERIAL PRIMARY KEY, + platform social_platform NOT NULL DEFAULT 'tiktok', + post_title VARCHAR(255) NOT NULL, + post_url VARCHAR(500), + posted_at TIMESTAMP NOT NULL, + video_length_seconds INTEGER, + + -- Core engagement + views INTEGER DEFAULT 0, + likes INTEGER DEFAULT 0, + comments INTEGER DEFAULT 0, + shares INTEGER DEFAULT 0, + saves INTEGER DEFAULT 0, + + -- Watch metrics + total_play_time_seconds INTEGER DEFAULT 0, + avg_watch_time_seconds DECIMAL(10,2) DEFAULT 0, + watched_full_pct DECIMAL(5,2) DEFAULT 0, + drop_off_seconds INTEGER, + + -- Growth + new_followers INTEGER DEFAULT 0, + + -- Traffic + top_traffic_source VARCHAR(50), + top_traffic_pct DECIMAL(5,2), + + created_at TIMESTAMP DEFAULT NOW(), + updated_at TIMESTAMP DEFAULT NOW(), + notes TEXT +); + +CREATE TABLE social_account_snapshots ( + id SERIAL PRIMARY KEY, + platform social_platform NOT NULL DEFAULT 'tiktok', + snapshot_date DATE NOT NULL, + total_followers INTEGER DEFAULT 0, + profile_views INTEGER DEFAULT 0, + search_queries JSONB, + demographics JSONB, + created_at TIMESTAMP DEFAULT NOW(), + UNIQUE(platform, snapshot_date) +); +``` + +### Route Structure + +- `GET /admin/social` — Dashboard with platform tabs, 30-day stats, post list +- `GET /admin/social/add` — Form to add new post +- `POST /admin/social/add` — Create post +- `GET /admin/social/post/:id` — Post detail with update form +- `POST /admin/social/post/:id/update` — Update metrics +- `POST /admin/social/post/:id/delete` — Delete post +- `GET /admin/social/snapshot` — Account snapshot form +- `POST /admin/social/snapshot` — Save/upsert snapshot + +### Views + +- Platform tabs (TikTok/Facebook/Instagram/X/Bluesky) +- Stats cards: Posts, Views, Likes, Comments, Shares, New Followers, Engagement Rate, Avg Watch +- Account snapshot banner (followers, profile views, top search queries) +- Posts table with color-coded completion rates +- Add form with sections matching TikTok Studio layout +- Detail view with update capability + +--- + +## TikTok Data We're Capturing + +Based on what TikTok Studio actually shows: + +**Per-Post (Overview tab):** +- Views, Likes, Comments, Shares, Saves +- Total play time, Avg watch time, Watched full % +- Drop-off point (seconds) +- New followers +- Top traffic source + % + +**Per-Post (Viewers tab):** +- Gender split (Female/Male/Other %) +- Age brackets (18-24, 25-34, 35-44, 45-54, 55+) +- Top locations + +**Account Level (Overview):** +- Total followers +- Profile views (7 days) +- Search queries bringing people in + +--- + +## Phase 2: TikTok API Strategy + +TikTok has multiple APIs: +1. **Display API** — Read-only access to public content (not analytics) +2. **Business API** — Requires verified business + ad spend history +3. **Analytics API** — Requires OAuth 2.0, developer approval, returns video metrics + +**Current understanding:** +- Need to register at developers.tiktok.com +- Create app, submit for review (days to weeks) +- Implement OAuth 2.0 flow +- Once approved, can pull: views, likes, comments, shares, watch time, audience demographics +- Rate limits: 100-1,000 requests/day depending on tier +- Data freshness: 24-48 hour lag + +**Our plan:** +1. Apply for API access now (parallel to Phase 1) +2. When approved, add OAuth callback route to Arbiter +3. Store access tokens in database +4. Create scheduled job to pull metrics daily +5. Backfill existing posts with API data + +--- + +## Specific Questions + +1. **Architecture fit:** Does this module follow Trinity Console patterns correctly? Anything we should change before deploying? + +2. **Schema design:** Is the database schema solid? Should we normalize the demographics/traffic data differently? We chose JSONB for flexibility but wondering if separate tables would be better for querying. + +3. **Phase 2 concerns:** What pitfalls should we watch for with the TikTok OAuth flow? Any security considerations for storing social tokens alongside Stripe/Discord tokens? + +4. **Multi-platform expansion:** We designed with 5 platforms in mind. Each has different APIs (Meta Graph API for FB/IG, X API v2, Bluesky AT Protocol). Should we abstract a common interface now, or build platform-specific adapters as we go? + +5. **What are we missing?** Anything obvious we overlooked? Either in the manual entry version or the API integration plan? + +--- + +## Additional Context + +- Trinity Console is Node.js/Express with EJS templates +- Already has OAuth flows for Discord and Stripe +- Arbiter runs on Command Center (63.143.34.217) +- Database is PostgreSQL (arbiter_db) +- Soft launch is April 15 — 5 days away +- Main goal right now: give Michael visibility, reduce overwhelm + +--- + +Thanks Gemini! 🔥❄️ + +— Michael + Claude (Chronicler #76) + +--- + +## Gemini's Response (April 10, 2026) + +**Summary:** Architecture approved, schema validated, added `platform_post_id` column for Phase 2, token encryption recommended, delegate manual entry to Meg. + +### 1. Architecture Fit +The route structure and integration into the Node.js/Express and EJS environment follow standard RESTful patterns perfectly. Keeping the manual entry lightweight is smart. + +**Recommendation:** Go ahead and deploy Phase 1 as designed. It fits the Arbiter ecosystem well and will give you immediate relief. + +### 2. Schema Design: JSONB vs. Normalization +Your choice to use `JSONB` for demographics and search queries is **100% correct**. + +Do not normalize this right now. Every platform formats demographic data differently (e.g., TikTok's age brackets will inevitably differ from Meta's). `JSONB` gives you the flexibility to store whatever unstructured payload the platform provides without having to run constant database migrations. + +**The One Missing Column:** +You need a way to link your manually entered posts to the API data once Phase 2 goes live. Add a column for the platform's native post ID. + +```sql +ALTER TABLE social_posts +ADD COLUMN platform_post_id VARCHAR(255); + +CREATE INDEX idx_platform_post_id +ON social_posts(platform, platform_post_id); +``` + +### 3. Phase 2 Concerns: TikTok API & OAuth +TikTok's developer approval process is notoriously slow and heavily scrutinized. Submit that application immediately. + +**OAuth Pitfalls:** +* **Token Expiration:** TikTok access tokens expire quickly. You must implement a robust refresh token flow in Arbiter to ensure the scheduled daily cron jobs don't silently fail. +* **Rate Limits:** With up to 1,000 requests per day, you'll be fine for daily syncing, but you'll need to batch your requests gracefully so you don't hit burst limits. + +**Security Considerations:** +Do not store social OAuth tokens in plaintext alongside Stripe/Discord tokens. If the database is compromised, an attacker could hijack the social accounts. + +You should encrypt the tokens before saving them to `arbiter_db`. Here is a quick, small utility approach using Node's native `crypto` module: + +```javascript +const crypto = require('crypto'); +const algorithm = 'aes-256-cbc'; +const key = process.env.ENCRYPTION_KEY; // 32 bytes +const iv = crypto.randomBytes(16); + +function encryptToken(token) { + let cipher = crypto.createCipheriv(algorithm, Buffer.from(key), iv); + let encrypted = cipher.update(token); + encrypted = Buffer.concat([encrypted, cipher.final()]); + return iv.toString('hex') + ':' + encrypted.toString('hex'); +} +``` + +### 4. Multi-Platform Expansion Strategy +**Build platform-specific adapters as you go.** Do not try to abstract a common interface right now. Meta's Graph API, X API v2, and Bluesky's AT Protocol are fundamentally different beasts with completely different data structures and auth flows. Premature abstraction will slow you down and cause headaches. Let each API script handle its own quirks, and have them all map their final outputs to your unified `social_posts` schema. + +### 5. What You're Missing + +**Delegation for Phase 1:** Since this requires manual entry right now, this is a perfect task to hand off to Meg. She is leading the social media front, so logging into /admin/social and dropping in the daily numbers from TikTok Studio fits flawlessly into a Community Manager's workflow. This completely removes the tracking burden from your plate while you handle the final server config for the soft launch. + +**Calculated Fields:** +Instead of manually entering Engagement Rate, let the dashboard calculate it dynamically on the fly (Likes + Comments + Shares + Saves / Views). + +--- + +## Conclusion + +Gemini approved the architecture and validated our approach. Key takeaways: + +1. **Deploy Phase 1 now** — it's ready +2. **Add `platform_post_id` column** — future-proofs for API matching +3. **Delegate to Meg** — manual entry is Community Manager work, not Wizard work +4. **Submit TikTok dev application immediately** — approval is slow +5. **Encrypt tokens in Phase 2** — don't store social OAuth tokens in plaintext +6. **Build adapters per-platform** — don't over-abstract + +**Next Steps:** +1. Add `platform_post_id` column to migration +2. Deploy Phase 1 to production +3. Train Meg on the /admin/social dashboard +4. Submit TikTok Developer application +5. Document token encryption pattern for Phase 2 + +--- + +**Fire + Frost + Foundation = Where Love Builds Legacy** 💙🔥❄️