popertots/Crussell
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update README and Obsidian docs with GDPR compliance system

Browse Source 
Update README feature list with GDPR compliance system details. Update Overview.md Data & Compliance section with full Article 15 SAR details. Update Technical Manual: add /gdpr route, gdpr_export.go handler, GET /api/user/gdpr-export endpoint, update Key Functions table, update Guest User System section. Update Future Work backlog: mark #7 GDPR export as complete, update dependency map and Phase 3 execution order. Update test count: 446/449 -> 576/579.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This commit is contained in:
popertots
2026-06-05 12:13:34 +01:00
co-authored by Sisyphus
parent 36ad38f670
commit 6cc65341a0
4 changed files with 21 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2 -1
View File
@@ -55,6 +55,7 @@ Nail salon booking platform — Go 1.25 backend + SvelteKit 5 frontend + Docker.
- **loginInProgress rate limiting**: switched from `map[string]bool` to `map[string]time.Time` with 30-second staleness check, 20-entry cap (returns 429 when full), ticker goroutine cleans up stuck entries
- **Profile picture upload limit**: 15MB client-side check before crop dialog in account page
- **Portfolio image upload backend limit**: separate `portfolioBodyLimit` (20MB) applied to `/images` route, distinct from `uploadBodyLimit` (15MB) for profile pictures
- **GDPR compliance system**: Full Article 15 Subject Access Request via `/gdpr` — async Go endpoint (`GET /api/user/gdpr-export`) with 12h in-memory cache and background generation, 16-section SQL export (`export_all_user_data()`) covering profile, bookings (with override pricing), payments, refunds, saved cards, social logins, loyalty redemptions, booking discounts, edit requests, affiliate payouts, verification codes, forgiven no-shows, patch tests, referrals, notification preferences. Frontend: skeleton loading, 2s polling, styled report cards/tables, PDF export (print CSS hides navbar + verification banner), raw JSON download. Conditional rendering for empty sections, VAT breakdowns (hidden until non-zero), deposits required (hidden when 0). Account deletion (`DELETE /api/user/account`) extended with external system scrubbing — S3 profile picture deletion, Square saved card deletion — before SQL-level anonymization. `anonymize_user()` SQL function extended with child table PII scrubbing (social logins, saved cards soft-delete with PCI clearance, verification code expiry, time blocker reservation scrubbing, edit request notes nulling, notification preference deletion). `AnonymizeStaleGuestAccounts()` extended with additional field scrubbing (profile_pic_url, referral_code, notes, data_retention_consent). 25 tests covering all backend additions.
## Project Structure
@@ -130,7 +131,7 @@ cd backend && go build -o bin/backend ./main.go
# Frontend
cd frontend && npm ci && npm run build
# Tests (446/449 passing, 3 skipped)
# Tests (576/579 passing, 3 skipped)
cd backend && go test -tags "test,dev" ./...
```
obsidian/Crussell/Future Work - Gap Backlog.md
+5 -5
View File
@@ -1,4 +1,4 @@
**Last Updated:** June 2026 — JWT revocation with JTI, portfolio image upload limits (20MB frontend/backend), notes validation (max=1000000), CharCounter component, loginInProgress rate limiting cap, profile picture 15MB limit, formatDateISO utility, NavBar/TodayCalendar/lunch protection refinements
**Last Updated:** June 2026 — GDPR compliance system (async export endpoint with 12h cache, 16-section SQL export, /gdpr frontend with PDF export + JSON download, DeleteAccountHandler external system scrubbing, anonymize_user child table PII scrubbing, AnonymizeStaleGuestAccounts extended field scrubbing), portfolio image upload limits (20MB frontend/backend), notes validation (max=1000000), CharCounter component, loginInProgress rate limiting cap, profile picture 15MB limit, formatDateISO utility, NavBar/TodayCalendar/lunch protection refinements
**Status:** Living backlog — add to this as gaps are discovered
---
@@ -29,7 +29,7 @@ No external dependencies. No paid services. No API keys needed.
| ----- | -------------------------------------------------------- | -------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ~~5~~ | ~~**Admin notification panel**~~ ✅ | ~~M (1-2d)~~ | ~~Frontend~~ | ~~Backend fully wired (GET/acknowledge). No frontend UI to display notifications. Admin has no visibility into pending bookings, cancellations, no-shows.~~ Two-tier system live: `new_booking` (all) + `pending_booking` (notes/today). Priority ordering, bell icon, `/admin/notifications` page, acknowledge flow, enriched responses.
| 6 | **Reservation/anonymization cron** | S (2-3h) | Backend | `CleanupOldReservations()` and `AnonymizeStaleGuestAccounts()` only fire on availability fetch. If no one fetches availability, expired reservations persist and stale guests aren't anonymized. Should be a background ticker in `main.go`. |
| 7 | **GDPR data export endpoint** | M (1d) | Backend | `export_all_user_data()` SQL function exists (JSON export). No Go handler wired. Required for GDPR Article 15 SAR requests. |
| 7 | ~~**GDPR data export endpoint**~~ ✅ | ~~M (1d)~~ | ~~Backend~~ | ~~`export_all_user_data()` SQL function exists (JSON export). No Go handler wired. Required for GDPR Article 15 SAR requests.~~ **Complete June 2026.** Async `GET /api/user/gdpr-export` with 12h in-memory cache, background goroutine generation (navigation away doesn't cancel), cache states (MISS/HIT/GENERATING). `/gdpr` frontend page with skeleton loading, 2s polling, styled report cards/tables, PDF export (print CSS hides navbar + verification banner), raw JSON download. 16-section SQL export: profile, bookings (with override pricing), payments, patch tests, referrals, notification preferences, saved cards, refunds, social logins, loyalty redemptions (with booking date/services), booking discounts (with campaign names), edit requests, affiliate payouts, verification codes, forgiven no-shows. Conditional rendering for empty sections, VAT breakdowns (hidden until non-zero), deposits required (hidden when 0). `DeleteAccountHandler` extended with external system scrubbing (S3 profile pic, Square saved cards). `anonymize_user()` extended with child table PII scrubbing. `AnonymizeStaleGuestAccounts()` extended with additional field scrubbing. 25 tests. |
| 8 | **VAT/Tax export endpoints** | M (1-2d) | Backend | `get_vat_return_data()`, `export_sales_transactions()` SQL functions exist. No admin API to trigger them. Needed for HMRC compliance. |
| ~~9~~ | ~~**Walk-in guest reservation → booking transition**~~ ✅ | S (1h) | Frontend | Done — guest accounts created at submit time, reservation system uses explicit reservation_type field, both walk-in and call-in use 15min TTL. |
| 10 | **Password reset flow not wired to frontend** | S (2-3h) | Frontend | Backend has `/api/verify/generate` and `/api/verify/check` endpoints. Login page has no "forgot password" link or form. |
@@ -146,7 +146,7 @@ Require paid accounts, API approval, or external service credentials. **Do not a
┌─────────────────────────────────────────────────────────┐
│ LOCAL (UNBLOCKED) │
│ │
│ #1 Delete account ──→ #7 GDPR export ──→ #14 SQL func │
│ #1 Delete account ✅──→ #7 GDPR export ✅──→ #14 SQL func ✅ │
│ │
│ #3 Approval decline ✅──→ #13 Booking reschedule │
│ │
@@ -198,9 +198,9 @@ Require paid accounts, API approval, or external service credentials. **Do not a
### Phase 3 — Compliance + Reliability (Week 3)
15. **#6** Reservation/anonymization background cron (2-3h)
16. **#7** GDPR data export endpoint (1d)
16. ~~**#7**~~ ~~GDPR data export endpoint~~ ✅ — complete (async endpoint + /gdpr frontend + 25 tests)
17. **#8** VAT/Tax export endpoints (1-2d)
18. **#14** Create `delete_guest_user()` SQL function (1h) ✅
18. ~~**#14**~~ ~~Create `delete_guest_user()` SQL function~~ ✅ — complete
19. **#27** Graceful shutdown (1h) ✅
20. ~~**#36**~~ ~~Idempotency keys for bookings~~ ✅ — implemented
21. **#30** XSS input sanitization (2-3h)
obsidian/Crussell/Overview.md
+6 -5
View File
@@ -171,9 +171,10 @@ flowchart TD
- **Amount precision**: API responses use pence (int64) for all monetary values to avoid floating-point issues; internal DB storage uses pounds (float)
### Data & Compliance
- GDPR anonymization: guest PII scrubbed 6 months after booking start_time
- Registered user account deletion via `anonymize_user()` SQL function
- GDPR data export function (`export_all_user_data()`) — not yet wired to endpoint
- **GDPR Article 15 Subject Access Request**: Full data export via `/gdpr` frontend page and `GET /api/user/gdpr-export` async endpoint. 12h in-memory cache with background generation (navigation away doesn't cancel). 16-section JSON export covering: user profile, bookings (with override pricing and total_price), payments, patch tests, referrals, notification preferences, saved cards, refunds, social logins, loyalty redemptions (with booking date/services), booking discounts (with campaign names), edit requests, affiliate payouts, verification codes, forgiven no-shows. Frontend: skeleton loading, 2s polling, styled report cards/tables, PDF export (print CSS hides navbar + verification banner), raw JSON download. Conditional rendering for empty sections, VAT breakdowns (hidden until any payment has non-zero VAT), deposits required (hidden when 0).
- GDPR anonymization: guest PII scrubbed 6 months after booking start_time. `AnonymizeStaleGuestAccounts()` extended with additional field scrubbing (profile_pic_url, referral_code, notes, data_retention_consent).
- Registered user account deletion via `anonymize_user()` SQL function — extended with child table PII scrubbing (social logins deleted, saved cards soft-deleted with PCI data cleared, verification codes expired, time blocker reservations scrubbed, edit request notes nulled, notification preferences deleted). `DeleteAccountHandler` extended with external system scrubbing — S3 profile picture deletion, Square saved card deletion — before SQL-level anonymization.
- GDPR data export function (`export_all_user_data()`) — **wired to async Go endpoint with 12h cache**
- VAT/tax export functions (`get_vat_return_data()`, `export_sales_transactions()`) — not yet wired
- Image metadata stripping (EXIF/GPS removed on upload)
- Partial unique email index: guests can share emails, registered users cannot
@@ -211,7 +212,7 @@ All flows integrate with holiday/exceptional hours and time blockers. All bookin
## Test Coverage
**446/449 tests passing** (3 skipped) across 12+ test packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, cash/gift card payments, enriched edit request workflows, GetBookingsByCreatedRange endpoint, scheduling exceptional hours, referral code validation, and JWT revocation via JTI logout.
**576/579 tests passing** (3 skipped) across 12+ test packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, cash/gift card payments, enriched edit request workflows, GetBookingsByCreatedRange endpoint, scheduling exceptional hours, referral code validation, JWT revocation via JTI logout, and GDPR compliance (export handler cache states, anonymize_user child table scrubbing, export_all_user_data 16-section export, AnonymizeStaleGuestAccounts field scrubbing).
| Package | Coverage Area |
|---------|--------------|
@@ -222,7 +223,7 @@ All flows integrate with holiday/exceptional hours and time blockers. All bookin
| `handlers/admin` | Admin bookings, today view, users, services, GetBookingsByCreatedRange |
| `handlers/scheduling` | Working hours, exceptional groups, available hours, time blockers, exceptional hours validation |
| `handlers/services` | Service eligibility (age + patch test filtering) |
| `handlers/user` | User profile, guest creation, loyalty |
| `handlers/user` | User profile, guest creation, loyalty, GDPR export handler, anonymize_user child table scrubbing, export_all_user_data comprehensive export, AnonymizeStaleGuestAccounts field scrubbing |
| `handlers/portfolio` | Image upload, listing, tags, filters |
| `handlers/notifications` | Admin notifications (GET, acknowledge) |
| `handlers/handlers_test.go` | Common handler tests |
obsidian/Crussell/Technical Manual.md
+8 -7
View File
@@ -53,7 +53,7 @@ Backend (:8080)
| `handlers/webhooks` | square.go | Square webhook handler for payment status updates |
| `handlers/admin` | users.go, analytics.go, discount_campaigns.go | Admin user management, discount campaigns, analytics (stub) |
| `handlers/today` | today.go | Current/next appointment, today's grid, pending approvals |
| `handlers/user` | profile.go, account.go, guest.go, loyalty.go, customer_relationship.go | User profile, guest creation, loyalty, contact info |
| `handlers/user` | profile.go, account.go, guest.go, loyalty.go, customer_relationship.go, gdpr_export.go | User profile, guest creation, loyalty, contact info, GDPR export (async with 12h cache) |
| `handlers/services` | services.go | Service catalog, eligibility filtering, patch_test_duration_hours auto-creates patch test records |
| `handlers/scheduling` | default-hours.go, exceptional-hours.go, time-blockers.go | Working hours, exceptional groups, time blockers |
| `handlers/portfolio` | images.go | Image upload, listing, tags, filters |
@@ -109,6 +109,7 @@ Backend (:8080)
| `/demo` | demo/+page.svelte | Demo mode |
| `/booking-confirmed/[id]` | booking-confirmed/[id]/+page.svelte | Booking confirmation page with payment summary, deposit info |
| `/pay-tip/[id]` | pay-tip/[id]/+page.svelte | Tip payment page — percentage-based tips (10%, 15%, 20%) or custom amount |
| `/gdpr` | gdpr/+page.svelte | GDPR data export — skeleton loading, 2s polling, styled report cards/tables, PDF export, raw JSON download |
| `/admin/notifications` | admin/notifications/+page.svelte | Admin notifications page with priority sorting, acknowledge flow |
| `/api/[...path]` | api/[...path]/+server.ts | API catch-all proxy (dev) |
@@ -228,7 +229,8 @@ src/lib/components/
| PUT | `/api/user/change-password` | Change password |
| GET | `/api/user/notification-preferences` | Get notification preferences |
| PUT | `/api/user/notification-preferences` | Update notification preferences |
| DELETE | `/api/user/account` | Delete account (GDPR anonymization) |
| DELETE | `/api/user/account` | Delete account (GDPR anonymization + external system scrubbing: S3, Square) |
| GET | `/api/user/gdpr-export` | Async GDPR data export (12h cache, background generation, returns "generating" on cache miss) |
| GET | `/api/user/loyalty` | Get loyalty stamp count |
| GET | `/api/bookings` | List user's bookings |
| GET | `/api/bookings/{id}` | Get specific booking |
@@ -368,10 +370,9 @@ src/lib/components/
| `generate_payment_id()` | Wrapper for payments table |
| `generate_verification_code()` | 12-char verification code |
| `generate_referral_code()` | 12-char referral code with collision detection |
| `anonymize_user(target_id)` | GDPR right-to-be-erased for registered users |
| `anonymize_user(target_id)` | GDPR right-to-be-erased for registered users — extended with child table PII scrubbing (social logins, saved cards, verification codes, time blockers, edit notes, notification prefs) |
| `delete_guest_user(target_id)` | Full removal of guest account |
| `update_data_consent(target_id, consent)` | Update GDPR consent flags |
| `export_all_user_data(target_user_id)` | GDPR Article 15 SAR (JSON export) |
| `export_all_user_data(target_user_id)` | GDPR Article 15 SAR — 16-section JSON export (profile, bookings with overrides, payments, refunds, saved cards, social logins, loyalty redemptions, booking discounts, edit requests, affiliate payouts, verification codes, forgiven no-shows, patch tests, referrals, notification preferences, export_metadata) |
| `get_vat_return_data(start, end)` | VAT return summary for MTD |
| `export_sales_transactions(start, end, include_vat)` | Tax-compatible transaction export |
| `get_monthly_business_summary(start, end)` | Monthly revenue breakdown |
@@ -449,7 +450,7 @@ src/lib/components/
**GDPR Anonymization:** `AnonymizeStaleGuestAccounts()` runs on every availability fetch:
- Scrubs PII 6 months after booking's `start_time`
- Preserved: `account_role`, `account_type`, `deposits_required`, `id`, `created_at`
- Scrubbed: name → "Guest Anonymized", email → "anon-{id}@anon.invalid", phone → "000000000000", DOB → "1900-01-01"
- Scrubbed: name → "Guest Anonymized", email → "anon-{id}@anon.invalid", phone → "000000000000", DOB → "1900-01-01", profile_pic_url → NULL, referral_code → NULL, notes → NULL, data_retention_consent → FALSE
- Excludes users with active/pending bookings
---
@@ -820,7 +821,7 @@ go test -tags "test,dev" -v -p 1 -count=2 ./... # Run twice for flaky detection
### Test Coverage
**446/449 tests passing** (3 skipped) across 12+ packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, and cash/gift card payments, GetBookingsByCreatedRange endpoint, scheduling exceptional hours validation, referral code registration, and JWT revocation via JTI logout.
**576/579 tests passing** (3 skipped) across 12+ packages. Comprehensive coverage of online payments (deposit, full, partial, balance), saved card operations, tip payments, terminal payments, refunds, idempotency, webhook handling, and cash/gift card payments, GetBookingsByCreatedRange endpoint, scheduling exceptional hours validation, referral code registration, JWT revocation via JTI logout, and GDPR compliance (export handler cache states, anonymize_user child table scrubbing, export_all_user_data 16-section export, AnonymizeStaleGuestAccounts field scrubbing).
- `handlers/auth` — Authentication (login, register, referral code validation, refresh, verification)
- `handlers/bookings` — User booking flow, guest bookings, reservations, edit requests, discounts, closing hours validation, active booking limits, GetBookingsByCreatedRange
- `handlers/payments` — Square payments (terminal, online, refunds, tips, saved cards)