rfd-fyi

This repository provides a simple, less-distracting frontend for Hot Deals posted on https://forums.redflagdeals.com.

The frontend is a Vite/Vue 3 app. Cloudflare Pages serves the static output, Pages Functions serve /topics.json, /enrichment.json, /html, /health.json, /admin/refresh, and /admin/enrich from Cloudflare KV, and a scheduled Cloudflare Worker refreshes cached topics to avoid excessive requests to RedFlagDeals itself.

Architecture

flowchart TD
  Browser[Browser] -->|GET /| Pages[Cloudflare Pages static assets]
  Browser -->|GET /topics.json| TopicsFn[Pages Function: /topics.json]
  Browser -->|GET /enrichment.json| EnrichFn[Pages Function: /enrichment.json]
  Browser -->|GET /html| HtmlFn[Pages Function: /html]
  Browser -->|GET /health.json| HealthFn[Pages Function: /health.json]

  TopicsFn -->|read topics.json| KV[(Cloudflare KV: TOPICS_KV)]
  EnrichFn -->|read enrichment.json| KV
  HtmlFn -->|read topics.json + enrichment.json| KV
  HealthFn -->|read refresh-status.json| KV

  Enricher[Enricher<br/>tools/enricher + LiteLLM] -->|GET topics + enrichment| TopicsFn
  Enricher -->|POST /admin/enrich| EnrichAdmin[Pages Function: /admin/enrich]
  EnrichAdmin -->|write enrichment.json| KV

  Cron[Cloudflare Cron Trigger<br/>every 10 minutes] --> RefreshWorker[Scheduled Worker: rfd-fyi-refresh]
  RefreshWorker -->|fetch topic pages| RFD[RedFlagDeals API]
  RefreshWorker -->|fetch redirect rules| Redirects[Redirect rules JSON]
  RefreshWorker -->|write topics.json| KV

  AdminPages[Manual refresh<br/>POST /admin/refresh] --> TopicsRefreshFn[Pages Function: /admin/refresh]
  AdminWorker[Manual refresh<br/>GET /refresh] --> RefreshWorker
  TopicsRefreshFn -->|fetch topic pages| RFD
  TopicsRefreshFn -->|fetch redirect rules| Redirects
  TopicsRefreshFn -->|write topics.json| KV

Cloudflare deployment

Install dependencies and log in to Cloudflare:

npm ci
npx wrangler login

Create a KV namespace and a preview namespace:

npx wrangler kv namespace create TOPICS_KV
npx wrangler kv namespace create TOPICS_KV --preview

Copy the returned namespace IDs into both:

• wrangler.toml
• worker/wrangler.toml

Build and deploy the Pages app:

npm run build
npm run pages:deploy
# or
npm run build && just deploy-pages

Deploy the scheduled refresh Worker:

npm run worker:deploy
# or
just deploy-worker

Deploy both:

just deploy

The Worker runs every 10 minutes and writes the latest topics to KV. Pages reads that cached JSON at /topics.json and renders a no-JavaScript view at /html. /health.json reports the status of the last refresh.

Deal tags

Deals can be tagged with a closed vocabulary (computing, grocery, dining, telecom, …) by an LLM — minimax-m3 through a LiteLLM proxy by default, or a local Ollama. Because Cloudflare Workers cannot reach a LAN, the tagger pushes results in rather than being called: see tools/enricher/.

Tags live in their own KV key and are served from /enrichment.json, which the frontend joins onto topics by topic_id. /topics.json is unchanged, and the app renders normally when no tags exist. Click a tag chip — or type #gaming into the filter box — to filter by tag.

Self-hosted Docker Compose

Docker Compose runs the Pages app and a small refresh service locally, using a persistent local KV volume instead of Cloudflare. The first refresh runs after the app is ready, and subsequent refreshes run every 10 minutes by default.

From the repository root:

docker compose up

Open http://localhost:8788. The cache survives container restarts in the rfd-state volume. Use docker compose down -v only when you also want to remove the cached topics.

The defaults can be overridden with environment variables or a .env file (by default, the listener binds to 127.0.0.1):

RFD_HOST=127.0.0.1
RFD_PORT=8788
RFD_REFRESH_SECRET=local
RFD_REFRESH_INTERVAL_SECONDS=600
RFD_BASE_URL=https://forums.redflagdeals.com
REDIRECTS_URL=https://raw.githubusercontent.com/davegallant/rfd-redirect-stripper/main/redirects.json

Optional manual refresh endpoints:

# For the Pages /admin/refresh endpoint
npx wrangler pages secret put REFRESH_SECRET --project-name rfd-fyi
curl -X POST -H "Authorization: Bearer $REFRESH_SECRET" https://<your-pages-domain>/admin/refresh

# For the Worker /refresh endpoint
npx wrangler secret put REFRESH_SECRET --config worker/wrangler.toml
curl -H "Authorization: Bearer $REFRESH_SECRET" https://rfd-fyi-refresh.<your-subdomain>.workers.dev/refresh

Tests and checks

Run the unit test suite:

npm test -- --run

Run coverage:

npm run test:coverage -- --run

Run linting and a production build:

npm run lint
npm run build

GitHub Actions runs npm ci, tests, linting, and build on every push and pull request.

Local Development

Full Cloudflare Pages local dev

Run the Cloudflare Pages build locally, including Pages Functions:

npm run pages:dev

On Nix systems, just dev also supplies /etc/ssl/certs/ca-certificates.crt to Workerd when SSL_CERT_FILE is unset, so local Functions can make outbound HTTPS requests. An existing SSL_CERT_FILE value takes precedence.

Wrangler serves the app at:

http://localhost:8788

Local Pages KV starts empty, so the app may initially show no deals. Seed local KV by calling the manual refresh endpoint in another shell:

curl -X POST -H "Authorization: Bearer dev" http://localhost:8788/admin/refresh

A successful refresh returns something like:

{"refreshed":120}

After that, these local endpoints should return populated data:

http://localhost:8788/topics.json
http://localhost:8788/html
http://localhost:8788/health.json

Refresh Worker local dev

To run the scheduled refresh Worker locally:

npm run worker:dev

Frontend-only Vite dev

For frontend-only Vite development:

npm run serve

The Vite dev server proxies /topics.json, /enrichment.json, and /html to the configured production Pages origin by default, so it should show live deals without seeding local KV. Override with VITE_API_ORIGIN if needed:

VITE_API_ORIGIN=http://localhost:8788 npm run serve