diff --git a/README.md b/README.md index ba386a3..b95a710 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,7 @@ Nail salon booking platform — Go 1.25 backend + SvelteKit 5 frontend + Docker. - **loginInProgress rate limiting**: switched from `map[string]bool` to `map[string]time.Time` with 30-second staleness check, 20-entry cap (returns 429 when full), ticker goroutine cleans up stuck entries - **Profile picture upload limit**: 15MB client-side check before crop dialog in account page - **Portfolio image upload backend limit**: separate `portfolioBodyLimit` (20MB) applied to `/images` route, distinct from `uploadBodyLimit` (15MB) for profile pictures +- **GDPR compliance system**: Full Article 15 Subject Access Request via `/gdpr` — async Go endpoint (`GET /api/user/gdpr-export`) with 12h in-memory cache and background generation, 16-section SQL export (`export_all_user_data()`) covering profile, bookings (with override pricing), payments, refunds, saved cards, social logins, loyalty redemptions, booking discounts, edit requests, affiliate payouts, verification codes, forgiven no-shows, patch tests, referrals, notification preferences. Frontend: skeleton loading, 2s polling, styled report cards/tables, PDF export (print CSS hides navbar + verification banner), raw JSON download. Conditional rendering for empty sections, VAT breakdowns (hidden until non-zero), deposits required (hidden when 0). Account deletion (`DELETE /api/user/account`) extended with external system scrubbing — S3 profile picture deletion, Square saved card deletion — before SQL-level anonymization. `anonymize_user()` SQL function extended with child table PII scrubbing (social logins, saved cards soft-delete with PCI clearance, verification code expiry, time blocker reservation scrubbing, edit request notes nulling, notification preference deletion). `AnonymizeStaleGuestAccounts()` extended with additional field scrubbing (profile_pic_url, referral_code, notes, data_retention_consent). 25 tests covering all backend additions. ## Project Structure @@ -130,7 +131,7 @@ cd backend && go build -o bin/backend ./main.go # Frontend cd frontend && npm ci && npm run build -# Tests (446/449 passing, 3 skipped) +# Tests (576/579 passing, 3 skipped) cd backend && go test -tags "test,dev" ./... ``` diff --git a/obsidian/Crussell/Future Work - Gap Backlog.md b/obsidian/Crussell/Future Work - Gap Backlog.md index be75c44..908b4c2 100644 --- a/obsidian/Crussell/Future Work - Gap Backlog.md +++ b/obsidian/Crussell/Future Work - Gap Backlog.md @@ -1,4 +1,4 @@ -**Last Updated:** June 2026 — JWT revocation with JTI, portfolio image upload limits (20MB frontend/backend), notes validation (max=1000000), CharCounter component, loginInProgress rate limiting cap, profile picture 15MB limit, formatDateISO utility, NavBar/TodayCalendar/lunch protection refinements +**Last Updated:** June 2026 — GDPR compliance system (async export endpoint with 12h cache, 16-section SQL export, /gdpr frontend with PDF export + JSON download, DeleteAccountHandler external system scrubbing, anonymize_user child table PII scrubbing, AnonymizeStaleGuestAccounts extended field scrubbing), portfolio image upload limits (20MB frontend/backend), notes validation (max=1000000), CharCounter component, loginInProgress rate limiting cap, profile picture 15MB limit, formatDateISO utility, NavBar/TodayCalendar/lunch protection refinements **Status:** Living backlog — add to this as gaps are discovered --- @@ -29,7 +29,7 @@ No external dependencies. No paid services. No API keys needed. | ----- | -------------------------------------------------------- | -------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ~~5~~ | ~~**Admin notification panel**~~ ✅ | ~~M (1-2d)~~ | ~~Frontend~~ | ~~Backend fully wired (GET/acknowledge). No frontend UI to display notifications. Admin has no visibility into pending bookings, cancellations, no-shows.~~ Two-tier system live: `new_booking` (all) + `pending_booking` (notes/today). Priority ordering, bell icon, `/admin/notifications` page, acknowledge flow, enriched responses. | 6 | **Reservation/anonymization cron** | S (2-3h) | Backend | `CleanupOldReservations()` and `AnonymizeStaleGuestAccounts()` only fire on availability fetch. If no one fetches availability, expired reservations persist and stale guests aren't anonymized. Should be a background ticker in `main.go`. | -| 7 | **GDPR data export endpoint** | M (1d) | Backend | `export_all_user_data()` SQL function exists (JSON export). No Go handler wired. Required for GDPR Article 15 SAR requests. | +| 7 | ~~**GDPR data export endpoint**~~ ✅ | ~~M (1d)~~ | ~~Backend~~ | ~~`export_all_user_data()` SQL function exists (JSON export). No Go handler wired. Required for GDPR Article 15 SAR requests.~~ **Complete June 2026.** Async `GET /api/user/gdpr-export` with 12h in-memory cache, background goroutine generation (navigation away doesn't cancel), cache states (MISS/HIT/GENERATING). `/gdpr` frontend page with skeleton loading, 2s polling, styled report cards/tables, PDF export (print CSS hides navbar + verification banner), raw JSON download. 16-section SQL export: profile, bookings (with override pricing), payments, patch tests, referrals, notification preferences, saved cards, refunds, social logins, loyalty redemptions (with booking date/services), booking discounts (with campaign names), edit requests, affiliate payouts, verification codes, forgiven no-shows. Conditional rendering for empty sections, VAT breakdowns (hidden until non-zero), deposits required (hidden when 0). `DeleteAccountHandler` extended with external system scrubbing (S3 profile pic, Square saved cards). `anonymize_user()` extended with child table PII scrubbing. `AnonymizeStaleGuestAccounts()` extended with additional field scrubbing. 25 tests. | | 8 | **VAT/Tax export endpoints** | M (1-2d) | Backend | `get_vat_return_data()`, `export_sales_transactions()` SQL functions exist. No admin API to trigger them. Needed for HMRC compliance. | | ~~9~~ | ~~**Walk-in guest reservation → booking transition**~~ ✅ | S (1h) | Frontend | Done — guest accounts created at submit time, reservation system uses explicit reservation_type field, both walk-in and call-in use 15min TTL. | | 10 | **Password reset flow not wired to frontend** | S (2-3h) | Frontend | Backend has `/api/verify/generate` and `/api/verify/check` endpoints. Login page has no "forgot password" link or form. | @@ -146,7 +146,7 @@ Require paid accounts, API approval, or external service credentials. **Do not a ┌─────────────────────────────────────────────────────────┐ │ LOCAL (UNBLOCKED) │ │ │ -│ #1 Delete account ──→ #7 GDPR export ──→ #14 SQL func │ +│ #1 Delete account ✅──→ #7 GDPR export ✅──→ #14 SQL func ✅ │ │ │ │ #3 Approval decline ✅──→ #13 Booking reschedule │ │ │ @@ -198,9 +198,9 @@ Require paid accounts, API approval, or external service credentials. **Do not a ### Phase 3 — Compliance + Reliability (Week 3) 15. **#6** Reservation/anonymization background cron (2-3h) -16. **#7** GDPR data export endpoint (1d) +16. ~~**#7**~~ ~~GDPR data export endpoint~~ ✅ — complete (async endpoint + /gdpr frontend + 25 tests) 17. **#8** VAT/Tax export endpoints (1-2d) -18. **#14** Create `delete_guest_user()` SQL function (1h) ✅ +18. ~~**#14**~~ ~~Create `delete_guest_user()` SQL function~~ ✅ — complete 19. **#27** Graceful shutdown (1h) ✅ 20. ~~**#36**~~ ~~Idempotency keys for bookings~~ ✅ — implemented 21. **#30** XSS input sanitization (2-3h) diff --git a/obsidian/Crussell/Overview.md b/obsidian/Crussell/Overview.md index 1e3f5ab..845d5b5 100644 --- a/obsidian/Crussell/Overview.md +++ b/obsidian/Crussell/Overview.md @@ -171,9 +171,10 @@ flowchart TD - **Amount precision**: API responses use pence (int64) for all monetary values to avoid floating-point issues; internal DB storage uses pounds (float) ### Data & Compliance -- GDPR anonymization: guest PII scrubbed 6 months after booking start_time -- Registered user account deletion via `anonymize_user()` SQL function -- GDPR data export function (`export_all_user_data()`) — not yet wired to endpoint +- **GDPR Article 15 Subject Access Request**: Full data export via `/gdpr` frontend page and `GET /api/user/gdpr-export` async endpoint. 12h in-memory cache with background generation (navigation away doesn't cancel). 16-section JSON export covering: user profile, bookings (with override pricing and total_price), payments, patch tests, referrals, notification preferences, saved cards, refunds, social logins, loyalty redemptions (with booking date/services), booking discounts (with campaign names), edit requests, affiliate payouts, verification codes, forgiven no-shows. Frontend: skeleton loading, 2s polling, styled report cards/tables, PDF export (print CSS hides navbar + verification banner), raw JSON download. Conditional rendering for empty sections, VAT breakdowns (hidden until any payment has non-zero VAT), deposits required (hidden when 0). +- GDPR anonymization: guest PII scrubbed 6 months after booking start_time. `AnonymizeStaleGuestAccounts()` extended with additional field scrubbing (profile_pic_url, referral_code, notes, data_retention_consent). +- Registered user account deletion via `anonymize_user()` SQL function — extended with child table PII scrubbing (social logins deleted, saved cards soft-deleted with PCI data cleared, verification codes expired, time blocker reservations scrubbed, edit request notes nulled, notification preferences deleted). `DeleteAccountHandler` extended with external system scrubbing — S3 profile picture deletion, Square saved card deletion — before SQL-level anonymization. +- GDPR data export function (`export_all_user_data()`) — **wired to async Go endpoint with 12h cache** - VAT/tax export functions (`get_vat_return_data()`, `export_sales_transactions()`) — not yet wired - Image metadata stripping (EXIF/GPS removed on upload) - Partial unique email index: guests can share emails, registered users cannot @@ -211,7 +212,7 @@ All flows integrate with holiday/exceptional hours and time blockers. All bookin ## Test Coverage -**446/449 tests passing** (3 skipped) across 12+ test packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, cash/gift card payments, enriched edit request workflows, GetBookingsByCreatedRange endpoint, scheduling exceptional hours, referral code validation, and JWT revocation via JTI logout. +**576/579 tests passing** (3 skipped) across 12+ test packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, cash/gift card payments, enriched edit request workflows, GetBookingsByCreatedRange endpoint, scheduling exceptional hours, referral code validation, JWT revocation via JTI logout, and GDPR compliance (export handler cache states, anonymize_user child table scrubbing, export_all_user_data 16-section export, AnonymizeStaleGuestAccounts field scrubbing). | Package | Coverage Area | |---------|--------------| @@ -222,7 +223,7 @@ All flows integrate with holiday/exceptional hours and time blockers. All bookin | `handlers/admin` | Admin bookings, today view, users, services, GetBookingsByCreatedRange | | `handlers/scheduling` | Working hours, exceptional groups, available hours, time blockers, exceptional hours validation | | `handlers/services` | Service eligibility (age + patch test filtering) | -| `handlers/user` | User profile, guest creation, loyalty | +| `handlers/user` | User profile, guest creation, loyalty, GDPR export handler, anonymize_user child table scrubbing, export_all_user_data comprehensive export, AnonymizeStaleGuestAccounts field scrubbing | | `handlers/portfolio` | Image upload, listing, tags, filters | | `handlers/notifications` | Admin notifications (GET, acknowledge) | | `handlers/handlers_test.go` | Common handler tests | diff --git a/obsidian/Crussell/Technical Manual.md b/obsidian/Crussell/Technical Manual.md index 32f0f76..2e57de4 100644 --- a/obsidian/Crussell/Technical Manual.md +++ b/obsidian/Crussell/Technical Manual.md @@ -53,7 +53,7 @@ Backend (:8080) | `handlers/webhooks` | square.go | Square webhook handler for payment status updates | | `handlers/admin` | users.go, analytics.go, discount_campaigns.go | Admin user management, discount campaigns, analytics (stub) | | `handlers/today` | today.go | Current/next appointment, today's grid, pending approvals | -| `handlers/user` | profile.go, account.go, guest.go, loyalty.go, customer_relationship.go | User profile, guest creation, loyalty, contact info | +| `handlers/user` | profile.go, account.go, guest.go, loyalty.go, customer_relationship.go, gdpr_export.go | User profile, guest creation, loyalty, contact info, GDPR export (async with 12h cache) | | `handlers/services` | services.go | Service catalog, eligibility filtering, patch_test_duration_hours auto-creates patch test records | | `handlers/scheduling` | default-hours.go, exceptional-hours.go, time-blockers.go | Working hours, exceptional groups, time blockers | | `handlers/portfolio` | images.go | Image upload, listing, tags, filters | @@ -109,6 +109,7 @@ Backend (:8080) | `/demo` | demo/+page.svelte | Demo mode | | `/booking-confirmed/[id]` | booking-confirmed/[id]/+page.svelte | Booking confirmation page with payment summary, deposit info | | `/pay-tip/[id]` | pay-tip/[id]/+page.svelte | Tip payment page — percentage-based tips (10%, 15%, 20%) or custom amount | +| `/gdpr` | gdpr/+page.svelte | GDPR data export — skeleton loading, 2s polling, styled report cards/tables, PDF export, raw JSON download | | `/admin/notifications` | admin/notifications/+page.svelte | Admin notifications page with priority sorting, acknowledge flow | | `/api/[...path]` | api/[...path]/+server.ts | API catch-all proxy (dev) | @@ -228,7 +229,8 @@ src/lib/components/ | PUT | `/api/user/change-password` | Change password | | GET | `/api/user/notification-preferences` | Get notification preferences | | PUT | `/api/user/notification-preferences` | Update notification preferences | -| DELETE | `/api/user/account` | Delete account (GDPR anonymization) | +| DELETE | `/api/user/account` | Delete account (GDPR anonymization + external system scrubbing: S3, Square) | +| GET | `/api/user/gdpr-export` | Async GDPR data export (12h cache, background generation, returns "generating" on cache miss) | | GET | `/api/user/loyalty` | Get loyalty stamp count | | GET | `/api/bookings` | List user's bookings | | GET | `/api/bookings/{id}` | Get specific booking | @@ -368,10 +370,9 @@ src/lib/components/ | `generate_payment_id()` | Wrapper for payments table | | `generate_verification_code()` | 12-char verification code | | `generate_referral_code()` | 12-char referral code with collision detection | -| `anonymize_user(target_id)` | GDPR right-to-be-erased for registered users | +| `anonymize_user(target_id)` | GDPR right-to-be-erased for registered users — extended with child table PII scrubbing (social logins, saved cards, verification codes, time blockers, edit notes, notification prefs) | | `delete_guest_user(target_id)` | Full removal of guest account | -| `update_data_consent(target_id, consent)` | Update GDPR consent flags | -| `export_all_user_data(target_user_id)` | GDPR Article 15 SAR (JSON export) | +| `export_all_user_data(target_user_id)` | GDPR Article 15 SAR — 16-section JSON export (profile, bookings with overrides, payments, refunds, saved cards, social logins, loyalty redemptions, booking discounts, edit requests, affiliate payouts, verification codes, forgiven no-shows, patch tests, referrals, notification preferences, export_metadata) | | `get_vat_return_data(start, end)` | VAT return summary for MTD | | `export_sales_transactions(start, end, include_vat)` | Tax-compatible transaction export | | `get_monthly_business_summary(start, end)` | Monthly revenue breakdown | @@ -449,7 +450,7 @@ src/lib/components/ **GDPR Anonymization:** `AnonymizeStaleGuestAccounts()` runs on every availability fetch: - Scrubs PII 6 months after booking's `start_time` - Preserved: `account_role`, `account_type`, `deposits_required`, `id`, `created_at` -- Scrubbed: name → "Guest Anonymized", email → "anon-{id}@anon.invalid", phone → "000000000000", DOB → "1900-01-01" +- Scrubbed: name → "Guest Anonymized", email → "anon-{id}@anon.invalid", phone → "000000000000", DOB → "1900-01-01", profile_pic_url → NULL, referral_code → NULL, notes → NULL, data_retention_consent → FALSE - Excludes users with active/pending bookings --- @@ -820,7 +821,7 @@ go test -tags "test,dev" -v -p 1 -count=2 ./... # Run twice for flaky detection ### Test Coverage -**446/449 tests passing** (3 skipped) across 12+ packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, and cash/gift card payments, GetBookingsByCreatedRange endpoint, scheduling exceptional hours validation, referral code registration, and JWT revocation via JTI logout. +**576/579 tests passing** (3 skipped) across 12+ packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, and cash/gift card payments, GetBookingsByCreatedRange endpoint, scheduling exceptional hours validation, referral code registration, JWT revocation via JTI logout, and GDPR compliance (export handler cache states, anonymize_user child table scrubbing, export_all_user_data 16-section export, AnonymizeStaleGuestAccounts field scrubbing). - `handlers/auth` — Authentication (login, register, referral code validation, refresh, verification) - `handlers/bookings` — User booking flow, guest bookings, reservations, edit requests, discounts, closing hours validation, active booking limits, GetBookingsByCreatedRange - `handlers/payments` — Square payments (terminal, online, refunds, tips, saved cards)