commit e257a6795fd714b8417bc07e09129974667c9f90 Author: Martin Tahiraj Date: Sat Mar 21 00:10:52 2026 +0100 docs: add ALPHA_PROJECT system context diff --git a/README.md b/README.md new file mode 100644 index 0000000..ce14267 --- /dev/null +++ b/README.md @@ -0,0 +1,359 @@ +# ALPHA_PROJECT — System Context for GitHub Copilot + +## Who I am + +I am a Cloud Architect with 6 years of Big4 consulting experience (currently at manager level at EY), working daily on Azure, Dynamics 365 / Dataverse, .NET, and enterprise integration patterns. I run a production-grade homelab on Kubernetes at home, and I am building ALPHA_PROJECT as a personal initiative in my spare time (evenings and weekends). + +--- + +## What ALPHA_PROJECT is + +ALPHA_PROJECT is a **proactive, multi-agent personal AI assistant** built entirely on self-hosted infrastructure. It is not a chatbot. It is an autonomous system that: + +- Monitors my digital life (email, calendar, home automation, finances, infrastructure) +- Maintains a persistent, structured memory of facts, habits, and preferences +- Takes initiative to notify me of relevant events, correlations, and pending actions +- Interacts with me via voice (Amazon Echo / Alexa custom skill named **"Pompeo"**) and Telegram +- Runs local LLMs on dedicated hardware — no cloud AI inference (except GitHub Copilot completions, available via EY license at zero cost) + +The assistant is named **Pompeo** (the Alexa skill wake word). + +--- + +## Infrastructure + +### LLM Server (new, dedicated node — outside the Kubernetes cluster) + +- **CPU**: AMD Ryzen 5 4500 +- **RAM**: 16 GB DDR4 +- **GPU**: NVIDIA GeForce RTX 3060 (16 GB VRAM) +- **Runtime**: Ollama (API-compatible with OpenAI) +- **Primary model**: Qwen2.5-14B-Instruct Q4_K_M (fits entirely in VRAM, no offload) +- **Secondary model**: Qwen2.5-Coder-14B-Instruct Q4_K_M (for code-related tasks) +- **Embedding model**: TBD — to be served via Ollama (e.g. `nomic-embed-text`) +- **Constraint**: zero RAM offload — all models must fit entirely in 16 GB VRAM + +### Kubernetes Homelab Cluster + +Production-grade self-hosted stack. Key components relevant to ALPHA_PROJECT: + +| Component | Role | +|---|---| +| **n8n** | Primary orchestrator and workflow engine for all agents | +| **Node-RED** | Event-driven automation, Home Assistant bridge | +| **Patroni / PostgreSQL** | Persistent structured memory store | +| **Qdrant** | Vector store for semantic/episodic memory *(to be deployed)* | +| **NATS / Redis Streams** | Message broker between agents *(to be chosen and deployed)* | +| **Authentik** | SSO / IAM (OIDC) | +| **Home Assistant** | IoT hub — device tracking, automations, sensors | +| **MikroTik** | Network — VLANs, firewall rules, device presence detection | +| **Paperless-ngx** | Document archive (`docs.mt-home.uk`) | +| **Actual Budget** | Personal finance | +| **Mealie** | Meal planning / recipes | +| **Immich** | Photo library | +| **Outline** | Internal wiki / knowledge base | +| **Radarr / Sonarr** | Media management | +| **Jenkins** | CI/CD | +| **AdGuard** | DNS filtering | +| **WireGuard** | VPN | +| **Minio** | S3-compatible object storage | +| **Longhorn** | Distributed block storage | +| **Velero** | Disaster recovery / backup | + +### External Services (in use) + +- **Gmail** — primary email +- **Google Calendar** — calendar (multiple calendars: Work, Family, Formula 1, WEC, Inter, Birthdays, Tasks, Pulizie, Spazzatura, Festività Italia, Varie) +- **Amazon Echo** — voice interface for Pompeo +- **AWS Lambda** — bridge between Alexa skill and n8n webhook +- **Telegram** — notifications, logging, manual document upload +- **GitHub Copilot** (GPT-4.1 via `api.githubcopilot.com`) — LLM completions at zero cost (EY license) + +### Internal Services / Custom + +- `orchestrator.mt-home.uk` — n8n instance +- `docs.mt-home.uk` — Paperless-ngx +- `filewizard.home.svc.cluster.local:8000` — custom OCR microservice (async, job-based API) + +--- + +## Architecture Overview + +### Multi-Agent Design + +ALPHA_PROJECT uses specialized agents, each responsible for a specific data domain. All agents are implemented as **n8n workflows**. + +| Agent | Trigger | Responsibility | +|---|---|---| +| **Mail Agent** | Cron every 15-30 min | Read Gmail, classify emails, extract facts, detect invoices/bills | +| **Finance Agent** | Triggered by Mail Agent or Telegram | Process PDF invoices/bills, archive to Paperless, persist to memory | +| **Calendar Agent** | Cron + on-demand | Read Google Calendar, detect upcoming events, cross-reference with other agents | +| **Infrastructure Agent** | Cron + alert webhooks | Monitor Kubernetes cluster health, disk usage, failed jobs | +| **IoT Agent** | Event-driven (Home Assistant webhooks) | Monitor device presence, home state, learn behavioral patterns | +| **Newsletter Agent** | Cron morning | Digest newsletters, extract relevant articles | +| **Proactive Arbiter** | Cron (adaptive frequency) + high-priority queue messages | Consume agent outputs, correlate, decide what to notify | + +### Message Broker (Blackboard Pattern) + +Agents do not call each other directly. They publish observations to a **central message queue** (NATS JetStream or Redis Streams — TBD). The **Proactive Arbiter** consumes the queue, batches low-priority messages, and immediately processes high-priority ones. + +Message schema (all agents must conform): + +```json +{ + "agent": "mail", + "priority": "low|high", + "event_type": "new_fact|reminder|alert|behavioral_observation", + "subject": "brief description", + "detail": {}, + "source_ref": "optional reference to postgres record or external ID", + "timestamp": "ISO8601", + "expires_at": "ISO8601 or null" +} +``` + +### Memory Architecture + +Three layers of persistence: + +**1. Structured memory — PostgreSQL (Patroni)** + +Episodic facts, finance records, reminders, behavioral observations. Fast, queryable, expirable. + +```sql +-- Generic episodic facts +CREATE TABLE memory_facts ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + source TEXT NOT NULL, -- 'email', 'calendar', 'iot', 'paperless', ... + category TEXT, -- 'finance', 'personal', 'work', 'health', ... + subject TEXT, + detail JSONB, -- flexible per-source payload + action_required BOOLEAN DEFAULT false, + action_text TEXT, + created_at TIMESTAMP DEFAULT now(), + expires_at TIMESTAMP, -- facts have a TTL + qdrant_id UUID -- FK to vector store +); + +-- Finance documents (frequent structured queries) +CREATE TABLE finance_documents ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + paperless_doc_id INT, + correspondent TEXT, + amount NUMERIC(10,2), + currency TEXT DEFAULT 'EUR', + doc_date DATE, + doc_type TEXT, + tags TEXT[], + created_at TIMESTAMP DEFAULT now() +); + +-- Behavioral context (used by IoT agent and Arbiter) +CREATE TABLE behavioral_context ( + id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + event_type TEXT, -- 'sport_event', 'dog_walk', 'work_session', ... + start_at TIMESTAMP, + end_at TIMESTAMP, + do_not_disturb BOOLEAN DEFAULT false, + home_presence_expected BOOLEAN, + notes TEXT +); +``` + +**2. Semantic memory — Qdrant** + +Vector embeddings for similarity search. Three collections: + +| Collection | Content | +|---|---| +| `martin_episodes` | Conversations, episodic facts with timestamp | +| `martin_knowledge` | Documents, Outline notes, newsletters, knowledge base | +| `martin_preferences` | Preferences, habits, behavioral patterns | + +Each Qdrant point includes a metadata payload for pre-filtering (source, date, category, action_required) to avoid full-scan similarity searches. + +**3. Profile memory — PostgreSQL (static table)** + +User preferences, fixed facts, communication style. Updated manually or via explicit agent action. + +### Embedding Strategy + +- Embeddings are generated via Ollama (`nomic-embed-text` or equivalent) once the LLM server is online +- During bootstrap phase: embeddings generated via GitHub Copilot (`text-embedding-3-small` at `api.githubcopilot.com/embeddings`) — same token acquisition pattern already in use +- Never embed raw content — always embed **LLM-generated summaries + extracted entities** + +### Proactive Notification Logic + +The Arbiter runs on an **adaptive schedule**: + +| Time slot | Frequency | Behavior | +|---|---|---| +| 23:00–07:00 | Never | Silence | +| 07:00–09:00 | Once | Morning briefing (calendar, reminders, pending actions) | +| 09:00–19:00 | Every 2-3h | Only high-priority or correlated events | +| 19:00–22:00 | Once | Evening recap + next day preview | + +High-priority queue messages bypass the schedule and trigger immediate notification. + +Notification is sent via **Amazon Echo / Pompeo** (TTS) for voice, and **Telegram** for logging. Every Arbiter decision (notify / discard / defer) is logged to a dedicated Telegram audit channel. + +### Voice Interface (Pompeo) + +- Amazon Echo → **Alexa Custom Skill** → **AWS Lambda** (bridge) → **n8n webhook** → Ollama (Qwen2.5-14B) → TTS response back to Echo +- Wake phrase: "Pompeo" +- Lambda is intentionally thin — it only translates the Alexa request format to the n8n webhook payload and returns the TTS response + +--- + +## Existing n8n Workflows (already in production) + +### 📬 Gmail — Daily Digest [Schedule] (`1lIKvVJQIcva30YM`) + +- Runs every 3 hours (+ test webhook) +- Fetches unread emails from the last 3 hours +- Calls GPT-4.1 (via Copilot) to classify each email: category, sentiment, labels, action_required, whether it has a Paperless-relevant PDF attachment +- Applies Gmail labels, marks as read, trashes spam +- If a bill/invoice PDF is detected → triggers the **Upload Bolletta** webhook +- Sends a digest report to Telegram + +### 📄 Paperless — Upload Bolletta [Email] (`vbzQ3fgUalOPdcOq`) + +- Triggered by webhook from Daily Digest (payload includes `email_id`) +- Downloads the PDF attachment from Gmail API +- Fetches Paperless metadata (correspondents, document types, tags, storage paths, similar existing documents) +- Calls GPT-4.1 to infer Paperless metadata (correspondent, doc type, tags, storage path, filename, date) +- Uploads PDF to Paperless, polls task status, patches metadata on the created document +- Sends Telegram confirmation + +### 📄 Paperless — Upload Documento [Telegram] (`ZX5rLSETg6Xcymps`) + +- Triggered by Telegram bot (user sends a PDF with caption starting with "Documento") +- Downloads file from Telegram +- Sends to FileWizard OCR microservice (async job), polls for result +- Same GPT-4.1 metadata inference pipeline as above +- Uploads to Paperless (filename = original filename without extension), patches metadata +- Sends Telegram confirmation with link to document +- Cleans up FileWizard: deletes processed files, then clears job history + +**Common pattern across all three**: GitHub Copilot token is obtained fresh at each run (`GET https://api.github.com/copilot_internal/v2/token`), then used for `POST https://api.githubcopilot.com/chat/completions` with model `gpt-4.1`. + +### n8n Credentials (IDs) + +| ID | Name | Type | +|---|---|---| +| `qvOikS6IF0H5khr8` | Gmail OAuth2 | OAuth2 | +| `uTXHLqcCJxbOvqN3` | Telegram account | Telegram API | +| `vBwUxlzKrX3oDHyN` | GitHub Copilot OAuth Token | HTTP Header Auth | +| `uvGjLbrN5yQTQIzv` | Paperless-NGX API | HTTP Header Auth | + +--- + +## Coding Conventions + +- **n8n workflows**: nodes named in Italian, descriptive emoji prefixes on trigger nodes +- **Workflow naming**: `{icon} {App} — {Azione} {Tipo} [{Sorgente}]` (e.g. `📄 Paperless — Upload Documento [Telegram]`) +- **HTTP nodes**: always use `predefinedCredentialType` for authenticated services already configured in n8n credentials +- **GPT body**: use `contentType: "raw"` + `rawContentType: "application/json"` + `JSON.stringify({...})` inline expression — never `specifyBody: string` +- **LLM output parsing**: always defensive — handle missing `choices`, malformed JSON, empty responses gracefully +- **Copilot token**: always fetched fresh per workflow run, never cached across executions +- **Binary fields**: Telegram node `file.get` with `download: true` stores binary in field named `data` (not `attachment`) +- **Postgres**: use UUID primary keys with `gen_random_uuid()`, JSONB for flexible payloads, always include `created_at` +- **Qdrant upsert**: always include full metadata payload for filtering; use `message_id` / `thread_id` / `doc_id` as logical dedup keys + +--- + +## TO-DO + +### Phase 0 — Infrastructure Bootstrap *(prerequisite for everything)* + +- [ ] Deploy **Qdrant** on the Kubernetes cluster + - Create collections: `martin_episodes`, `martin_knowledge`, `martin_preferences` + - Configure payload indexes on: `source`, `category`, `date`, `action_required` +- [ ] Run **PostgreSQL migrations** on Patroni + - Create tables: `memory_facts`, `finance_documents`, `behavioral_context` + - Add index on `memory_facts(source, category, expires_at)` +- [ ] Verify embedding endpoint via Copilot (`text-embedding-3-small`) as bootstrap fallback +- [ ] Plan migration to local Ollama embedding model once LLM server is online + +--- + +### Phase 1 — Memory Integration into Existing Workflows + +- [ ] **Daily Digest**: after `Parse risposta GPT-4.1`, add: + - Postgres INSERT into `memory_facts` (source=email, category, subject, detail JSONB, action_required, expires_at) + - Embedding generation (Copilot endpoint) → Qdrant upsert into `martin_episodes` + - Thread dedup: use `thread_id` as logical key, update existing Qdrant point if thread already exists + +- [ ] **Upload Bolletta** + **Upload Documento (Telegram)**: after `Paperless - Patch Metadati`, add: + - Postgres INSERT into `finance_documents` (correspondent, amount, doc_date, doc_type, tags, paperless_doc_id) + - Postgres INSERT into `memory_facts` (source=paperless, category=finance, cross-reference) + - Embedding of OCR text chunks → Qdrant upsert into `martin_knowledge` + +--- + +### Phase 2 — New Agents + +- [ ] **Calendar Agent** + - Poll Google Calendar (all relevant calendars) + - Persist upcoming events to Postgres (`memory_facts` + `behavioral_context` for leisure events) + - Weekly cluster embedding (chunk per week, not per event) + - Dedup recurring events: embed only first occurrence, store rest in Postgres only + +- [ ] **Finance Agent** (extend beyond Paperless) + - Read Actual Budget export or API + - Persist transactions, monthly summaries to `finance_documents` + - Trend analysis prompt for periodic financial summary + +- [ ] **Infrastructure Agent** + - Webhook receiver for Kubernetes/Longhorn/Minio alerts + - Cron-based cluster health check (disk, pod status, backup freshness) + - Publishes to message broker with `priority: high` for critical alerts + +- [ ] **IoT Agent** + - Home Assistant webhook → Node-RED → n8n + - Device presence tracking → `behavioral_context` + - Pattern recognition via Qdrant similarity on historical episodes (e.g. "Tuesday evening, outside, laptop on") + +- [ ] **Newsletter Agent** + - Separate Gmail label for newsletters (excluded from Daily Digest main flow) + - Morning cron: summarize + extract relevant articles → `martin_knowledge` + +--- + +### Phase 3 — Message Broker + Proactive Arbiter + +- [ ] Choose and deploy broker: **NATS JetStream** (preferred — lightweight, native Kubernetes) or Redis Streams +- [ ] Define final message schema (draft above, to be validated) +- [ ] Implement **Proactive Arbiter** n8n workflow: + - Adaptive schedule (morning briefing, midday, evening recap) + - Consume queue batch → LLM correlation prompt → structured `notify/defer/discard` output + - High-priority bypass path + - All decisions logged to Telegram audit channel +- [ ] Implement **correlation logic**: detect when 2+ agents report related events (e.g. IoT presence + calendar event + open reminder) + +--- + +### Phase 4 — Voice Interface (Pompeo) + +- [ ] Create Alexa Custom Skill ("Pompeo") +- [ ] AWS Lambda bridge (thin translator: Alexa request → n8n webhook → TTS response) +- [ ] n8n webhook handler: receive transcribed text → prepend memory context → Ollama inference → return TTS string +- [ ] TTS response pipeline back to Echo +- [ ] Proactive push: Arbiter → Lambda → Echo notification (Alexa proactive events API) + +--- + +### Phase 5 — Generalization and Backlog + +- [ ] **OCR on email attachments in Daily Digest**: generalize the ingest pipeline to extract text from any PDF attachment (not just bills), using FileWizard OCR — produce richer embeddings and enable full-text retrieval on any emailed document +- [ ] **Flusso Cedolino** (payslip pipeline): + - Trigger: Gmail label `Lavoro/Cedolino` or Telegram upload + - PDF → FileWizard OCR → GPT-4.1 metadata extraction (month, gross, net, deductions) + - Paperless upload with tag `Cedolino` + - Persist structured data to `finance_documents` (custom fields for payslip) + - Trend embedding in `martin_knowledge` for finance agent queries +- [ ] Behavioral habit modeling: aggregate `behavioral_context` records over time, generate periodic "habit summary" embeddings in `martin_preferences` +- [ ] Outline → Qdrant pipeline: sync selected Outline documents into `martin_knowledge` on edit/publish event +- [ ] Chrome browsing history ingestion (privacy-filtered): evaluate browser extension or local export → embedding pipeline for interest/preference modeling +- [ ] "Posti e persone" graph: structured contact/location model in Postgres, populated from email senders, calendar attendees, Home Assistant presence data +- [ ] Local embedding model: migrate from Copilot `text-embedding-3-small` to Ollama-served model (e.g. `nomic-embed-text`) once LLM server is stable