ALPHA_PROJECT — System Context for GitHub Copilot

Who I am

I am a Cloud Architect with 6 years of Big4 consulting experience (currently at manager level at EY), working daily on Azure, Dynamics 365 / Dataverse, .NET, and enterprise integration patterns. I run a production-grade homelab on Kubernetes at home, and I am building ALPHA_PROJECT as a personal initiative in my spare time (evenings and weekends).

---

What ALPHA_PROJECT is

ALPHA_PROJECT is a proactive, multi-agent personal AI assistant built entirely on self-hosted infrastructure. It is not a chatbot. It is an autonomous system that:

• Monitors my digital life (email, calendar, home automation, finances, infrastructure)
• Maintains a persistent, structured memory of facts, habits, and preferences
• Takes initiative to notify me of relevant events, correlations, and pending actions
• Interacts with me via voice (Amazon Echo / Alexa custom skill named "Pompeo") and Telegram
• Runs local LLMs on dedicated hardware — no cloud AI inference (except GitHub Copilot completions, available via EY license at zero cost)

The assistant is named Pompeo (the Alexa skill wake word).

---

Infrastructure

LLM Server (new, dedicated node — outside the Kubernetes cluster)

• CPU: AMD Ryzen 5 4500
• RAM: 16 GB DDR4
• GPU: NVIDIA GeForce RTX 3060 (16 GB VRAM)
• Runtime: Ollama (API-compatible with OpenAI)
• Primary model: Qwen2.5-14B-Instruct Q4_K_M (fits entirely in VRAM, no offload)
• Secondary model: Qwen2.5-Coder-14B-Instruct Q4_K_M (for code-related tasks)
• Embedding model: TBD — to be served via Ollama (e.g. nomic-embed-text)
• Constraint: zero RAM offload — all models must fit entirely in 16 GB VRAM

Kubernetes Homelab Cluster

Production-grade self-hosted stack. Key components relevant to ALPHA_PROJECT:

Component	Role
n8n	Primary orchestrator and workflow engine for all agents
Node-RED	Event-driven automation, Home Assistant bridge
Patroni / PostgreSQL	Persistent structured memory store
Qdrant	Vector store for semantic/episodic memory — qdrant.persistence.svc.cluster.local:6333
NATS / Redis Streams	Message broker between agents (to be chosen and deployed)
Authentik	SSO / IAM (OIDC)
Home Assistant	IoT hub — device tracking, automations, sensors
MikroTik	Network — VLANs, firewall rules, device presence detection
Paperless-ngx	Document archive (docs.mt-home.uk)
Actual Budget	Personal finance
Mealie	Meal planning / recipes
Immich	Photo library
Outline	Internal wiki / knowledge base
Radarr / Sonarr	Media management
Jenkins	CI/CD
AdGuard	DNS filtering
WireGuard	VPN
Minio	S3-compatible object storage
Longhorn	Distributed block storage
Velero	Disaster recovery / backup

External Services (in use)

• Gmail — primary email
• Google Calendar — calendar (multiple calendars: Work, Family, Formula 1, WEC, Inter, Birthdays, Tasks, Pulizie, Spazzatura, Festività Italia, Varie)
• Amazon Echo — voice interface for Pompeo
• AWS Lambda — bridge between Alexa skill and n8n webhook
• Telegram — notifications, logging, manual document upload
• GitHub Copilot (GPT-4.1 via api.githubcopilot.com) — LLM completions at zero cost (EY license)

Internal Services / Custom

• orchestrator.mt-home.uk — n8n instance
• docs.mt-home.uk — Paperless-ngx
• filewizard.home.svc.cluster.local:8000 — custom OCR microservice (async, job-based API)

---

Architecture Overview

Multi-Agent Design

ALPHA_PROJECT uses specialized agents, each responsible for a specific data domain. All agents are implemented as n8n workflows.

Agent	Trigger	Responsibility
Mail Agent	Cron every 15-30 min	Read Gmail, classify emails, extract facts, detect invoices/bills
Finance Agent	Triggered by Mail Agent or Telegram	Process PDF invoices/bills, archive to Paperless, persist to memory
Calendar Agent	Cron + on-demand	Read Google Calendar, detect upcoming events, cross-reference with other agents
Infrastructure Agent	Cron + alert webhooks	Monitor Kubernetes cluster health, disk usage, failed jobs
IoT Agent	Event-driven (Home Assistant webhooks)	Monitor device presence, home state, learn behavioral patterns
Newsletter Agent	Cron morning	Digest newsletters, extract relevant articles
Proactive Arbiter	Cron (adaptive frequency) + high-priority queue messages	Consume agent outputs, correlate, decide what to notify

Message Broker (Blackboard Pattern)

Agents do not call each other directly. They publish observations to a central message queue (NATS JetStream or Redis Streams — TBD). The Proactive Arbiter consumes the queue, batches low-priority messages, and immediately processes high-priority ones.

Message schema (all agents must conform):

{
  "agent": "mail",
  "priority": "low|high",
  "event_type": "new_fact|reminder|alert|behavioral_observation",
  "subject": "brief description",
  "detail": {},
  "source_ref": "optional reference to postgres record or external ID",
  "timestamp": "ISO8601",
  "expires_at": "ISO8601 or null"
}

Memory Architecture

Three layers of persistence:

1. Structured memory — PostgreSQL (Patroni)

Episodic facts, finance records, reminders, behavioral observations. Fast, queryable, expirable.

-- Generic episodic facts
CREATE TABLE memory_facts (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  source TEXT NOT NULL,         -- 'email', 'calendar', 'iot', 'paperless', ...
  category TEXT,                -- 'finance', 'personal', 'work', 'health', ...
  subject TEXT,
  detail JSONB,                 -- flexible per-source payload
  action_required BOOLEAN DEFAULT false,
  action_text TEXT,
  created_at TIMESTAMP DEFAULT now(),
  expires_at TIMESTAMP,         -- facts have a TTL
  qdrant_id UUID                -- FK to vector store
);

-- Finance documents (frequent structured queries)
CREATE TABLE finance_documents (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  paperless_doc_id INT,
  correspondent TEXT,
  amount NUMERIC(10,2),
  currency TEXT DEFAULT 'EUR',
  doc_date DATE,
  doc_type TEXT,
  tags TEXT[],
  created_at TIMESTAMP DEFAULT now()
);

-- Behavioral context (used by IoT agent and Arbiter)
CREATE TABLE behavioral_context (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  event_type TEXT,              -- 'sport_event', 'dog_walk', 'work_session', ...
  start_at TIMESTAMP,
  end_at TIMESTAMP,
  do_not_disturb BOOLEAN DEFAULT false,
  home_presence_expected BOOLEAN,
  notes TEXT
);

2. Semantic memory — Qdrant — qdrant.persistence.svc.cluster.local:6333

Vector embeddings for similarity search. Three collections with multi-tenant design: isolation via user_id payload field ("martin", "shared", future users).

Collection	Content
episodes	Conversations, episodic facts with timestamp
knowledge	Documents, Outline notes, newsletters, knowledge base
preferences	Preferences, habits, behavioral patterns

Each Qdrant point includes a metadata payload for pre-filtering (user_id, source, date, category, action_required) to avoid full-scan similarity searches.

3. Profile memory — PostgreSQL (static table)

User preferences, fixed facts, communication style. Updated manually or via explicit agent action.

Embedding Strategy

• Embeddings are generated via Ollama (nomic-embed-text or equivalent) once the LLM server is online
• During bootstrap phase: embeddings generated via GitHub Copilot (text-embedding-3-small at api.githubcopilot.com/embeddings) — same token acquisition pattern already in use
• Never embed raw content — always embed LLM-generated summaries + extracted entities

Proactive Notification Logic

The Arbiter runs on an adaptive schedule:

Time slot	Frequency	Behavior
23:00–07:00	Never	Silence
07:00–09:00	Once	Morning briefing (calendar, reminders, pending actions)
09:00–19:00	Every 2-3h	Only high-priority or correlated events
19:00–22:00	Once	Evening recap + next day preview

High-priority queue messages bypass the schedule and trigger immediate notification.

Notification is sent via Amazon Echo / Pompeo (TTS) for voice, and Telegram for logging. Every Arbiter decision (notify / discard / defer) is logged to a dedicated Telegram audit channel.

Voice Interface (Pompeo)

• Amazon Echo → Alexa Custom Skill → AWS Lambda (bridge) → n8n webhook → Ollama (Qwen2.5-14B) → TTS response back to Echo
• Wake phrase: "Pompeo"
• Lambda is intentionally thin — it only translates the Alexa request format to the n8n webhook payload and returns the TTS response

---

Existing n8n Workflows (already in production)

📬 Gmail — Daily Digest [Schedule] (1lIKvVJQIcva30YM)

• Runs every 3 hours (+ test webhook)
• Fetches unread emails from the last 3 hours
• Calls GPT-4.1 (via Copilot) to classify each email: category, sentiment, labels, action_required, whether it has a Paperless-relevant PDF attachment
• Applies Gmail labels, marks as read, trashes spam
• If a bill/invoice PDF is detected → triggers the Upload Bolletta webhook
• Sends a digest report to Telegram

📄 Paperless — Upload Bolletta [Email] (vbzQ3fgUalOPdcOq)

• Triggered by webhook from Daily Digest (payload includes email_id)
• Downloads the PDF attachment from Gmail API
• Fetches Paperless metadata (correspondents, document types, tags, storage paths, similar existing documents)
• Calls GPT-4.1 to infer Paperless metadata (correspondent, doc type, tags, storage path, filename, date)
• Uploads PDF to Paperless, polls task status, patches metadata on the created document
• Sends Telegram confirmation

📄 Paperless — Upload Documento [Telegram] (ZX5rLSETg6Xcymps)

• Triggered by Telegram bot (user sends a PDF with caption starting with "Documento")
• Downloads file from Telegram
• Sends to FileWizard OCR microservice (async job), polls for result
• Same GPT-4.1 metadata inference pipeline as above
• Uploads to Paperless (filename = original filename without extension), patches metadata
• Sends Telegram confirmation with link to document
• Cleans up FileWizard: deletes processed files, then clears job history

Common pattern across all three: GitHub Copilot token is obtained fresh at each run (GET https://api.github.com/copilot_internal/v2/token), then used for POST https://api.githubcopilot.com/chat/completions with model gpt-4.1.

n8n Credentials (IDs)

ID	Name	Type
qvOikS6IF0H5khr8	Gmail OAuth2	OAuth2
uTXHLqcCJxbOvqN3	Telegram account	Telegram API
vBwUxlzKrX3oDHyN	GitHub Copilot OAuth Token	HTTP Header Auth
uvGjLbrN5yQTQIzv	Paperless-NGX API	HTTP Header Auth

---

Coding Conventions

• n8n workflows: nodes named in Italian, descriptive emoji prefixes on trigger nodes
• Workflow naming: {icon} {App} — {Azione} {Tipo} [{Sorgente}] (e.g. 📄 Paperless — Upload Documento [Telegram])
• HTTP nodes: always use predefinedCredentialType for authenticated services already configured in n8n credentials
• GPT body: use contentType: "raw" + rawContentType: "application/json" + JSON.stringify({...}) inline expression — never specifyBody: string
• LLM output parsing: always defensive — handle missing choices, malformed JSON, empty responses gracefully
• Copilot token: always fetched fresh per workflow run, never cached across executions
• Binary fields: Telegram node file.get with download: true stores binary in field named data (not attachment)
• Postgres: use UUID primary keys with gen_random_uuid(), JSONB for flexible payloads, always include created_at
• Qdrant upsert: always include full metadata payload for filtering; use message_id / thread_id / doc_id as logical dedup keys

---

TO-DO

Phase 0 — Infrastructure Bootstrap (prerequisite for everything)

• Deploy Qdrant on the Kubernetes cluster ✅ 2026-03-21
  • Collections: episodes, knowledge, preferences (multi-tenant via user_id payload field)
  • Payload indexes: user_id, source, category, date, action_required
  • Endpoint: qdrant.persistence.svc.cluster.local:6333
• Run PostgreSQL migrations on Patroni ✅ 2026-03-21
  • Database pompeo creato (Zalando Operator)
  • Tabelle: user_profile, memory_facts, finance_documents, behavioral_context, agent_messages
  • Multi-tenancy: campo user_id su tutte le tabelle, seed martin + shared
  • Script DDL: alpha/db/postgres.sql
• Verify embedding endpoint via Copilot (text-embedding-3-small) as bootstrap fallback
• Plan migration to local Ollama embedding model once LLM server is online

---

Phase 1 — Memory Integration into Existing Workflows

• Daily Digest: after Parse risposta GPT-4.1, add:

  • Postgres INSERT into memory_facts (source=email, category, subject, detail JSONB, action_required, expires_at)
  • Embedding generation (Copilot endpoint) → Qdrant upsert into episodes (user_id=martin)
  • Thread dedup: use thread_id as logical key, update existing Qdrant point if thread already exists

• Upload Bolletta + Upload Documento (Telegram): after Paperless - Patch Metadati, add:

  • Postgres INSERT into finance_documents (correspondent, amount, doc_date, doc_type, tags, paperless_doc_id)
  • Postgres INSERT into memory_facts (source=paperless, category=finance, cross-reference)
  • Embedding of OCR text chunks → Qdrant upsert into knowledge (user_id=martin)

---

Phase 2 — New Agents

• Calendar Agent

  • Poll Google Calendar (all relevant calendars)
  • Persist upcoming events to Postgres (memory_facts + behavioral_context for leisure events)
  • Weekly cluster embedding (chunk per week, not per event)
  • Dedup recurring events: embed only first occurrence, store rest in Postgres only

• Finance Agent (extend beyond Paperless)

  • Read Actual Budget export or API
  • Persist transactions, monthly summaries to finance_documents
  • Trend analysis prompt for periodic financial summary

• Infrastructure Agent

  • Webhook receiver for Kubernetes/Longhorn/Minio alerts
  • Cron-based cluster health check (disk, pod status, backup freshness)
  • Publishes to message broker with priority: high for critical alerts

• IoT Agent

  • Home Assistant webhook → Node-RED → n8n
  • Device presence tracking → behavioral_context
  • Pattern recognition via Qdrant similarity on historical episodes (e.g. "Tuesday evening, outside, laptop on")

• Newsletter Agent

  • Separate Gmail label for newsletters (excluded from Daily Digest main flow)
  • Morning cron: summarize + extract relevant articles → knowledge

---

Phase 3 — Message Broker + Proactive Arbiter

• Choose and deploy broker: NATS JetStream (preferred — lightweight, native Kubernetes) or Redis Streams
• Define final message schema (draft above, to be validated)
• Implement Proactive Arbiter n8n workflow:
  • Adaptive schedule (morning briefing, midday, evening recap)
  • Consume queue batch → LLM correlation prompt → structured notify/defer/discard output
  • High-priority bypass path
  • All decisions logged to Telegram audit channel
• Implement correlation logic: detect when 2+ agents report related events (e.g. IoT presence + calendar event + open reminder)

---

Phase 4 — Voice Interface (Pompeo)

• Create Alexa Custom Skill ("Pompeo")
• AWS Lambda bridge (thin translator: Alexa request → n8n webhook → TTS response)
• n8n webhook handler: receive transcribed text → prepend memory context → Ollama inference → return TTS string
• TTS response pipeline back to Echo
• Proactive push: Arbiter → Lambda → Echo notification (Alexa proactive events API)

---

Phase 5 — Generalization and Backlog

• OCR on email attachments in Daily Digest: generalize the ingest pipeline to extract text from any PDF attachment (not just bills), using FileWizard OCR — produce richer embeddings and enable full-text retrieval on any emailed document
• Flusso Cedolino (payslip pipeline):
  • Trigger: Gmail label Lavoro/Cedolino or Telegram upload
  • PDF → FileWizard OCR → GPT-4.1 metadata extraction (month, gross, net, deductions)
  • Paperless upload with tag Cedolino
  • Persist structured data to finance_documents (custom fields for payslip)
  • Trend embedding in knowledge for finance agent queries
• Behavioral habit modeling: aggregate behavioral_context records over time, generate periodic "habit summary" embeddings in preferences
• Outline → Qdrant pipeline: sync selected Outline documents into knowledge on edit/publish event
• Chrome browsing history ingestion (privacy-filtered): evaluate browser extension or local export → embedding pipeline for interest/preference modeling
• "Posti e persone" graph: structured contact/location model in Postgres, populated from email senders, calendar attendees, Home Assistant presence data
• Local embedding model: migrate from Copilot text-embedding-3-small to Ollama-served model (e.g. nomic-embed-text) once LLM server is stable