# truf: Build Spec for the Coding Agent You are the implementing engineer for **truf**, an RSS/Atom/JSON Feed aggregator web app. The human owner is learning web development and will read, run, and modify your code. Optimize for **correct, simple, readable, well-verified** code, not cleverness or feature volume. --- ## 0. How to work (read first) 1. **Work milestone by milestone** (Section 6). Do not start milestone N+1 until milestone N's acceptance checks pass. 2. **Plan briefly, then act.** Before each milestone, write a 5 to 10 line plan in `docs/PROGRESS.md`. Then implement. Do not over-deliberate. 3. **Verify, don't assume.** After every meaningful change, run the checks in Section 7. Fix failures before moving on. Never claim something works without running it. 4. **Read before you write.** Inspect existing files before editing. Keep changes small and focused. Prefer editing over rewriting. 5. **Commit per logical step** with a clear message (`feat(fetcher): add conditional GET`). Never commit secrets or `.env`. 6. **Stop and ask** (write the question in `docs/QUESTIONS.md`, then halt) when: a requirement is ambiguous, two reasonable designs conflict with this spec, you need a dependency not listed in Section 3, or a check fails twice for reasons you cannot explain. 7. **Do not over-engineer.** No features, abstractions, config options, or dependencies that the current milestone does not need. Three similar lines beat a premature abstraction. 8. **Explain for a learner.** Add a short comment only where the _why_ is non-obvious. Keep `docs/PROGRESS.md` updated with what you did, what you verified, and what is next. --- ## 1. Product goal A user subscribes to feed URLs. truf polls those feeds **centrally**, stores items once in a shared time-ordered stream, and shows each user a personal view (their subscriptions, read/unread state) that updates live. Core flow: `server polls feeds -> normalizes + deduplicates items into a central store -> per-user views query by subscription -> new items pushed to connected clients` ### In scope (v1) - Add/remove a subscription by URL (with feed auto-discovery from a normal site URL) - Central polling with conditional requests - Item list per user (newest first, paginated), mark read/unread, unread counts - Live updates to the open page - Basic accounts and sessions ### Out of scope (do NOT build) - OPML import/export, search, folders/tags, themes, mobile apps, email digests, payments - Browser-side direct fetching from feed origins (blocked by CORS in most cases). If you believe it is needed, put the question in `docs/QUESTIONS.md`. - Redis, unless a milestone below explicitly introduces it --- ## 2. Design-first rule (important) The owner wants to discuss the data design **before** any SQL exists. **Milestone 0 produces `docs/DESIGN.md` only: no schema SQL, no migrations.** It must describe in prose and simple diagrams: the entities and how they relate, how items are identified and deduplicated, how read state is represented, how polling is scheduled, and how live updates reach clients. List trade-offs and your recommendation for each decision. Then **stop and wait for owner approval** (record "AWAITING APPROVAL" in `docs/PROGRESS.md`). --- ## 3. Tech constraints | Area | Requirement | | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Runtime | **Node.js 24 LTS** | | Modules | **ES modules only** (`"type": "module"`, `import`/`export`, no `require`, no CommonJS) | | Server | Express (current major) | | Frontend | React with Vite, plain JavaScript or JSDoc types (no TypeScript unless asked) | | Database | Default: **SQLite** for v1, accessed only through a small repository module so **Postgres** can replace it later. Do not leak SQLite-specific SQL outside `server/db/`. | | HTTP client | Node's built-in `fetch` (undici). No axios. | | Tests | Built-in `node:test` + `node:assert/strict` for the server. Vitest + Testing Library for React only. | | Lint/format | ESLint + Prettier, configured in Milestone 0 | | Config | Environment variables loaded from `.env` (via `node --env-file`), validated at startup, documented in `.env.example` | **Allowed dependencies (propose others in `docs/QUESTIONS.md` first):** `express`, a SQLite driver (`better-sqlite3` or built-in `node:sqlite`; justify the pick in DESIGN.md), a feed parser (`rss-parser` or `fast-xml-parser` + own normalizer), `sanitize-html`, `argon2` or `bcrypt`, `express-session` (or equivalent), `helmet`, `express-rate-limit`, `zod`, `pino`, `react`, `react-dom`, `react-router`, `@tanstack/react-query`, `vite`, `vitest`, `eslint`, `prettier`. Idiomatic style: `async/await`, no callbacks, named exports preferred, small modules, errors thrown as typed `Error` subclasses, no global mutable state outside the app entry. --- ## 4. Target repository layout ``` truf/ package.json # workspaces: server, client .env.example docs/ DESIGN.md PROGRESS.md QUESTIONS.md server/ src/ app.js # express app factory (no listen) index.js # entry: load config, start server + poller config.js # env validation db/ # connection, migrations, repositories feeds/ # fetch, parse, normalize, discover poller/ # scheduler, per-host limiter routes/ # thin route handlers services/ # business logic security/ # ssrf guard, sanitizer test/ fixtures/ # saved real-world feed XML/JSON samples client/ src/ ... ``` Routes stay thin (parse, call service, respond). Business logic lives in `services/`. Data access lives only in `db/`. --- ## 5. Non-negotiable requirements ### Feed fetching - Timeout (10s), max response size (e.g. 5 MB), max 5 redirects, and an identifying `User-Agent` (`truf/`). - Send `If-None-Match` / `If-Modified-Since` using stored `ETag` / `Last-Modified`; handle `304`. - Honor `Retry-After` and apply exponential backoff on repeated failures; mark feeds as failing after a threshold and surface the last error. - Per-host concurrency limit so one slow host never blocks the rest; global concurrency cap. - Never run two polls of the same feed at once. ### Parsing and items - Support RSS 2.0, Atom, and JSON Feed. Malformed feeds must produce a recorded error, never crash the process. - Normalize to one internal item shape. **Deduplication:** prefer the feed's GUID/id; if absent or unstable, fall back to a hash of (canonical link, title, published date). Document the rule in DESIGN.md and test it. - Missing or future dates: fall back to fetch time, clamp future dates. ### Security (treat feeds as hostile) - **SSRF guard:** before any outbound request, resolve DNS and reject loopback, private, link-local, and metadata ranges (IPv4 and IPv6); re-check on every redirect hop; only allow `http`/`https`. Unit-test with hostile URLs. - **XSS:** sanitize all feed HTML with an allow-list before storing or rendering. Never use `dangerouslySetInnerHTML` on unsanitized data. Force external links to `rel="noopener noreferrer nofollow"`. - Passwords hashed with argon2/bcrypt; session cookies `HttpOnly`, `Secure` in production, `SameSite=Lax`; CSRF protection for state-changing routes. - Validate every request body/query with `zod`. Use `helmet`, rate-limit auth and subscribe endpoints. - Parameterized queries only. No string-built SQL. ### Reliability - Graceful shutdown (stop poller, drain requests, close DB). - Structured logging with `pino`; log feed URL + error class on failures, never secrets. --- ## 6. Milestones and acceptance criteria **M0: Design + scaffold.** Produce `docs/DESIGN.md` (Section 2), repo scaffold, lint/format/test scripts, `.env.example`, a `/healthz` route with a passing test. _Accept:_ `npm run check` passes on a clean clone. **Then STOP for owner approval of the design.** **M1: Fetch, parse, normalize (no web, no DB).** `server/src/feeds/` with fetch (Section 5 rules), SSRF guard, parser, normalizer, feed auto-discovery. A CLI `node server/src/feeds/cli.js ` prints normalized items. _Accept:_ tests against fixtures for RSS, Atom, JSON Feed, malformed XML, 304 handling, redirect limits, oversized body, and SSRF-blocked URLs all pass. **M2: Storage, dedupe, poller.** Migrations and repositories per the approved design; ingest pipeline with deduplication; scheduler with backoff and per-host limits. _Accept:_ integration tests using a local fake feed server show that re-polling the same feed inserts no duplicates, 304 skips work, and a failing feed backs off without affecting others. **M3: API.** Auth/sessions; endpoints to add/remove subscriptions, list items (cursor pagination, newest first), mark read/unread (single and bulk), unread counts. _Accept:_ API tests cover auth required, validation errors, pagination stability when new items arrive, and per-user isolation of read state. **M4: React UI.** Login, subscription manager, item list with infinite scroll, read/unread toggle with optimistic update (rollback on error), unread badges, loading/empty/error states, keyboard-friendly. _Accept:_ component tests for list, toggle, and error rollback; manual smoke checklist in PROGRESS.md. **M5: Live updates.** Server-Sent Events stream per user; clients receive "new items" notifications and merge them without losing scroll position. _Accept:_ test that two clients with different subscriptions receive only their own events; reconnect works. **M6: Hardening + ship.** Rate limits, security headers, CSRF review, retention policy for old items, Dockerfile + `docker-compose.yml`, README with setup/run/deploy steps. _Accept:_ `docker compose up` runs the app; README steps work from scratch; all checks green. --- ## 7. Verification commands (must exist and pass) ``` npm run lint # eslint npm run format:check npm test # server (node:test) + client (vitest) npm run check # runs all of the above ``` Run `npm run check` before every commit and before declaring a milestone done. If a command is missing, creating it is part of the milestone. --- ## 8. Definition of done (per milestone) - Acceptance criteria met, with the command output summarized in `docs/PROGRESS.md` - No lint errors, no skipped or commented-out tests, no `TODO` without a matching entry in `docs/QUESTIONS.md` - No new dependency outside Section 3 - Docs updated; commits are small and descriptive When a milestone is done, write a short summary (what changed, what you verified, known limitations, what you need from the owner), then **wait for the owner's go-ahead** before continuing.