feat: ship Tier 1 — auth, authz, rate limits, real notifications, CSV import, billing, backups/DR, privacy

Closes every block in docs/TIER1_PLAN.md from the Claude-scope side. The
homelab / cloud setup steps (SES verification, restore drill, lawyer-
drafted ToS) remain operator-owned but are unblocked.

Block A — Authentication
- Migration 0003: password_hash, email_verified, email_verification_tokens,
  password_reset_tokens, refresh_tokens (with replaced_by family chain).
- Bcrypt hasher, HS256 JWT signer, single-use refresh tokens with rotation
  + replay-detection (revokes the family on reuse).
- /auth/signup, /login, /refresh, /logout, /verify-email,
  /forgot-password, /reset-password — enumeration-safe.
- requireAuth middleware + GET /me.
- Frontend useAuth/useApi with auto-refresh-on-401, login/signup/verify/
  forgot/reset pages, route-guard middleware.

Block B — Authorisation
- EventRepo.GetForHost; Update/Delete scoped by host_id.
- All host routes behind requireAuth + ownership; cross-tenant returns
  404 (no enumeration). ?host_id removed.
- WS auth via short-lived single-use tickets (POST /auth/ws-ticket).
- Tests: TestCrossTenantIsolation — 9 probes.

Block C — Rate limiting
- Redis sliding-window via Lua (atomic ZADD+ZCARD+PEXPIRE).
- Per-route limits matching the plan (signup IP, login IP+email, RSVP/
  access by token, events/guests/tokens by user_id).
- 429 with Retry-After header and JSON body.
- Auth lockout: 5 failed logins → account locked, only password reset
  clears it.
- Frontend: useErrMessage normalises 429 + locked messaging.

Block D — Real notifications
- Migration 0004: provider_message_id, bounce_type, complained columns
  + unsubscribes (CITEXT) suppression table.
- Branded HTML + plaintext templates for verification, reset, invitation,
  confirmation, reminder. Per-page templates avoid html/template's
  contextual-escape collisions.
- Senders: SESv2, Twilio (SMS), SMTP (Mailpit-friendly), Resend HTTP.
- PickEmailSender priority Resend > SMTP > SES > Log — system boots
  cleanly in dev with Mailpit; production flips one env var.
- Webhook endpoints (Twilio status + SES SNS) — bounces add to suppression;
  signature verification stubbed pending creds.
- Auto-send: POST /tokens publishes invitation.send; notifier renders +
  delivers via the configured backend; suppression list honoured.
- Bulk + per-row invitation flow: POST /events/{id}/guests/invitations/bulk
  returns per-guest tokens so phone-only guests can be SMS'd manually.
- Unsubscribe: signed HMAC token (no TTL) + /unsubscribe/[token] page.
- WhatsApp Option A+: wa.me click-to-chat wizard with per-guest progress
  tracking, isLikelyE164 validation, edit-from-wizard.
- Token rotate (POST /tokens/rotate) invalidates the old URL — used by
  the regenerate-link flow.
- Mailpit added to docker-compose for dev inbox.

Block E — CSV import
- Streaming parser: tolerant header detection, UTF-8 BOM + UTF-16 LE/BE
  decoding, row-level validation, 5,000-row cap.
- Strict E.164 phone validation with helpful error message.
- POST /preview + /import + GET /template; preview UI on event page;
  atomic per-batch with dedup on existing emails.

Phone capture across UI
- PhoneInput component: country picker (~50 ISO codes) + national input +
  live E.164 preview + inline length validation.
- Used in Add Guest and Edit Guest modals. Smart paste-handling extracts
  country code from full E.164 strings.

Block F — Billing (Stripe)
- Migration 0005: subscriptions table (user_id → tier/status/period_end +
  Stripe customer/sub ids). Partial unique index keeps one granting sub
  per user.
- internal/billing: Tier + Limits model (Free 1/50, Pro 10/1000, Business
  ∞/5000), Stripe SDK wrapper with IgnoreAPIVersionMismatch for newer
  account API versions.
- /billing/checkout-session, /billing/portal, /billing/status,
  /webhooks/stripe (signature-verified, lifecycle events).
- Tier enforcement: 402 on POST /events, /guests, /import with
  {error, reason, tier, used, limit, upgrade_url} body.
- Frontend: useBilling composable, /dashboard/billing page (current plan,
  usage bars, tier cards), global UpgradeModal triggered by useApi's
  402 interceptor.
- Customer portal kept for self-service cancel/payment-method changes.

Block G — Backups & DR (application side)
- Every migration has a tested .down.sql.
- TestMigrationRoundtrip applies all ups → all downs → all ups against a
  fresh container; catches asymmetric down migrations.
- cmd/restore-verify: 28-check post-restore invariant tool (schema
  presence, no orphans across 10 FK relationships, email uniqueness,
  single-active subscription, row-count snapshot).
- docs/RUNBOOK_RESTORE.md: 9-step restore procedure with RTO/RPO
  targets, drill instructions, rollback path.

Block H — Privacy compliance (application side)
- Migration 0006: deleted_at + terms_accepted_at + privacy_policy_accepted_at
  on users. Partial index on email for live-only uniqueness.
- GET /me/data-export — synchronous JSON dump (user, events, guests,
  tokens, rsvps, access_logs, notifications).
- DELETE /me — soft-delete with PII scrub + refresh-token revocation;
  re-signup with same email works.
- POST /me/accept-terms — idempotent consent recording.
- Frontend /privacy + /terms placeholder pages with substantive (pending
  legal review) copy; footer links; signup terms checkbox; TermsGateModal
  for accounts created before the rollout; export + delete buttons on
  /dashboard/billing.

Tests
- All migrations verified up/down/up.
- Integration suite: TestE2EHappyPath, TestAuthFlow, TestCrossTenantIsolation,
  TestRateLimitSignup, TestLoginLockout, TestUnsubscribeFlow,
  TestSESBounceWebhook, TestTwilioStatusWebhook, TestCsvImportFlow,
  TestCsvImportAtomicRollback, TestBulkIssueInvitations, TestBulkIssueExplicitSubset,
  TestTokenIssuePublishesInvitation, TestTokenIssueWithoutGuestEmailSkipsInvitation,
  TestGuestUpdate, TestGuestDelete, TestTokenRotate, TestSMTPSenderAgainstMailpit,
  TestFreeTierEventLimit, TestFreeTierGuestLimit, TestBusinessTierBypassesLimits,
  TestDataExport, TestDeleteMe, TestAcceptTerms, TestMigrationRoundtrip.
  Full suite runs in ~120s against real Postgres + NATS + Redis + Mailpit.
- Unit suite green across internal/auth, internal/csvimport,
  internal/notification, internal/ratelimit, internal/domain.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Kwaku Danso
2026-05-16 23:54:22 +01:00
parent a0ed34f860
commit 59b8781659
124 changed files with 13702 additions and 445 deletions
+250
View File
@@ -0,0 +1,250 @@
// Package csvimport parses a guest-list CSV into structured rows, with
// tolerant header detection (Excel, Numbers, Google Sheets variants) and
// per-row validation. Streaming-friendly so a 5,000-row import doesn't
// load the entire file into a slice before we know if column 1 is junk.
package csvimport
import (
"bufio"
"encoding/csv"
"errors"
"fmt"
"io"
"net/mail"
"regexp"
"strconv"
"strings"
"golang.org/x/text/encoding/unicode"
"golang.org/x/text/transform"
)
// Row is a single validated guest. Empty Email / Phone are allowed (a
// phone-only or name-only guest is valid per the plan).
type Row struct {
Name string
Email string
Phone string
PlusOnes int
}
// RowError flags one row with the human-readable reason it can't be
// imported. The line number is 1-based and matches the source CSV
// (header counts as line 1, first data row is line 2) so the frontend
// can highlight the offending row.
type RowError struct {
Row int `json:"row"`
Reason string `json:"reason"`
}
// Result is the outcome of one parse pass.
type Result struct {
Rows []Row `json:"rows,omitempty"`
Errors []RowError `json:"errors,omitempty"`
TotalCount int `json:"total_count"` // total data rows seen (excluding header)
}
// Options tune limits + behaviour.
type Options struct {
MaxRows int // hard cap; rows beyond MaxRows return an error instead of being silently dropped
}
const DefaultMaxRows = 5000
// Strict E.164: optional leading +, then a non-zero leading digit (country
// codes never start with 0), followed by 614 more digits — total 715
// significant digits. Spaces / dashes / parens are tolerated by stripping
// before validation, but local-format numbers like "0244…" or "07700…"
// are rejected here so the host fixes them at upload time rather than at
// WhatsApp-send time.
var phoneRe = regexp.MustCompile(`^\+?[1-9][0-9]{6,14}$`)
// Parse reads a CSV from r and returns the parsed result. Encoding is
// auto-detected: UTF-8 with or without BOM, plus UTF-16 LE/BE BOMs
// (commonly produced by Mac Numbers exports).
func Parse(r io.Reader, opt Options) (*Result, error) {
max := opt.MaxRows
if max <= 0 {
max = DefaultMaxRows
}
rd, err := decodingReader(r)
if err != nil {
return nil, err
}
csvr := csv.NewReader(rd)
csvr.FieldsPerRecord = -1 // tolerate ragged rows; we re-validate column count ourselves
csvr.TrimLeadingSpace = true
header, err := csvr.Read()
if err != nil {
if errors.Is(err, io.EOF) {
return nil, errors.New("csv is empty")
}
return nil, fmt.Errorf("read header: %w", err)
}
cols, err := detectColumns(header)
if err != nil {
return nil, err
}
out := &Result{Rows: make([]Row, 0, 64)}
lineNo := 1 // header was line 1
for {
rec, err := csvr.Read()
if err == io.EOF {
break
}
lineNo++
if err != nil {
out.Errors = append(out.Errors, RowError{Row: lineNo, Reason: fmt.Sprintf("malformed csv: %v", err)})
continue
}
out.TotalCount++
if out.TotalCount > max {
return nil, fmt.Errorf("import exceeds maximum of %d rows", max)
}
// Skip fully-empty rows silently — these appear at the end of
// Excel exports a lot.
if rowEmpty(rec) {
out.TotalCount-- // don't count it
continue
}
row, rerr := buildRow(rec, cols)
if rerr != "" {
out.Errors = append(out.Errors, RowError{Row: lineNo, Reason: rerr})
continue
}
out.Rows = append(out.Rows, row)
}
return out, nil
}
func rowEmpty(rec []string) bool {
for _, v := range rec {
if strings.TrimSpace(v) != "" {
return false
}
}
return true
}
// decodingReader strips a UTF-8 BOM and decodes UTF-16 LE/BE when their
// BOM is present, returning a UTF-8 reader. Other byte orders fall through
// as raw UTF-8.
func decodingReader(r io.Reader) (*bufio.Reader, error) {
br := bufio.NewReader(r)
bom, err := br.Peek(3)
if err != nil && !errors.Is(err, io.EOF) {
return nil, err
}
switch {
case len(bom) >= 3 && bom[0] == 0xEF && bom[1] == 0xBB && bom[2] == 0xBF:
_, _ = br.Discard(3)
return br, nil
case len(bom) >= 2 && bom[0] == 0xFF && bom[1] == 0xFE:
_, _ = br.Discard(2)
dec := unicode.UTF16(unicode.LittleEndian, unicode.IgnoreBOM).NewDecoder()
return bufio.NewReader(transform.NewReader(br, dec)), nil
case len(bom) >= 2 && bom[0] == 0xFE && bom[1] == 0xFF:
_, _ = br.Discard(2)
dec := unicode.UTF16(unicode.BigEndian, unicode.IgnoreBOM).NewDecoder()
return bufio.NewReader(transform.NewReader(br, dec)), nil
}
return br, nil
}
// columnSet records which column index each known field lives in. -1 means
// the column was not supplied; only Name is mandatory.
type columnSet struct {
name, email, phone, plusOnes int
}
func detectColumns(header []string) (columnSet, error) {
cs := columnSet{name: -1, email: -1, phone: -1, plusOnes: -1}
for i, raw := range header {
key := normaliseHeader(raw)
switch key {
case "name", "guestname", "fullname":
cs.name = i
case "email", "emailaddress", "e-mail":
cs.email = i
case "phone", "telephone", "mobile", "phonenumber":
cs.phone = i
case "plusones", "plus1", "plus-one", "plus-ones", "+1", "guests", "additionalguests":
cs.plusOnes = i
}
}
if cs.name < 0 {
return cs, fmt.Errorf("required column 'name' not found in header: %v", header)
}
return cs, nil
}
func normaliseHeader(s string) string {
s = strings.ToLower(strings.TrimSpace(s))
// Drop spaces + underscores. Keep `+`, `-` so "+1" / "plus-one" still
// match exactly.
return strings.NewReplacer(" ", "", "_", "").Replace(s)
}
func buildRow(rec []string, cs columnSet) (Row, string) {
get := func(i int) string {
if i < 0 || i >= len(rec) {
return ""
}
return strings.TrimSpace(rec[i])
}
row := Row{
Name: get(cs.name),
Email: strings.ToLower(get(cs.email)),
Phone: get(cs.phone),
}
if row.Name == "" {
return row, "name is required"
}
if row.Email != "" {
if _, err := mail.ParseAddress(row.Email); err != nil {
return row, "invalid email"
}
}
if row.Phone != "" {
stripped := stripPhone(row.Phone)
if !phoneRe.MatchString(stripped) {
return row, "phone must be in international format with country code (e.g. +447700900123) — local numbers starting with 0 won't work for SMS or WhatsApp"
}
// Normalise: ensure stored form always starts with "+".
if !strings.HasPrefix(stripped, "+") {
stripped = "+" + stripped
}
row.Phone = stripped
}
if raw := get(cs.plusOnes); raw != "" {
n, err := strconv.Atoi(raw)
if err != nil || n < 0 {
return row, "plus_ones must be a non-negative integer"
}
row.PlusOnes = n
}
return row, ""
}
var phoneStripper = strings.NewReplacer(" ", "", "-", "", "(", "", ")", "", " ", "")
func stripPhone(s string) string {
return phoneStripper.Replace(s)
}
// TemplateCSV is the sample file served at /events/{id}/guests/import/template.
// Phone numbers MUST include the country code (e.g. +44 for UK, +233 for
// Ghana). Local-format numbers like "0244..." or "07700..." will be
// rejected at upload — the sample below shows the expected shape.
const TemplateCSV = "name,email,phone,plus_ones\n" +
"Alex Doe,alex@example.com,+447700900123,1\n" +
"Sam Patel,sam@example.com,,0\n" +
"Jordan Lee,,+15551234567,2\n" +
"Mira Patel,mira@example.com,+233244123456,0\n"
+157
View File
@@ -0,0 +1,157 @@
package csvimport
import (
"strings"
"testing"
)
func TestParseHappyPath(t *testing.T) {
in := `name,email,phone,plus_ones
Alex Doe,alex@example.com,+447700900123,1
Sam Patel,SAM@example.com,,0
Jordan Lee,,+1 (555) 123-4567,2
`
r, err := Parse(strings.NewReader(in), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if got, want := len(r.Rows), 3; got != want {
t.Fatalf("rows: got %d want %d (errors=%+v)", got, want, r.Errors)
}
if r.Rows[1].Email != "sam@example.com" {
t.Errorf("email not lowercased: %q", r.Rows[1].Email)
}
if r.Rows[2].Phone != "+15551234567" {
t.Errorf("phone not stripped: %q", r.Rows[2].Phone)
}
if r.Rows[0].Phone != "+447700900123" {
t.Errorf("phone should keep leading +: %q", r.Rows[0].Phone)
}
}
func TestParsePhoneNormalisedToPlus(t *testing.T) {
// E.164 without explicit "+" is accepted and normalised to include one.
in := "name,phone\nAlex,447700900123\n"
r, err := Parse(strings.NewReader(in), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(r.Rows) != 1 || r.Rows[0].Phone != "+447700900123" {
t.Fatalf("expected normalised phone, got: rows=%+v errors=%+v", r.Rows, r.Errors)
}
}
func TestParsePhoneRejectsLocalFormat(t *testing.T) {
// Local UK / GH style numbers (leading 0, no country code) must be
// rejected — they break SMS routing and WhatsApp click-to-chat.
in := `name,phone
UK Local,07700900123
GH Local,0244123456
With Plus And Zero,+0244123456
`
r, err := Parse(strings.NewReader(in), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(r.Errors) != 3 {
t.Fatalf("expected 3 errors for local-format phones, got %d: %+v", len(r.Errors), r.Errors)
}
}
func TestParseHeaderVariants(t *testing.T) {
cases := []string{
"Name,Email,Phone,Plus Ones\nMira,m@x.com,,1\n",
"Guest Name,E-Mail,Telephone,+1\nMira,m@x.com,,1\n",
"full_name,email_address,mobile,plusones\nMira,m@x.com,,1\n",
}
for i, in := range cases {
t.Run(string(rune('a'+i)), func(t *testing.T) {
r, err := Parse(strings.NewReader(in), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(r.Rows) != 1 {
t.Fatalf("rows=%d errors=%+v", len(r.Rows), r.Errors)
}
if r.Rows[0].PlusOnes != 1 {
t.Errorf("plusones not detected: %+v", r.Rows[0])
}
})
}
}
func TestParseRowValidation(t *testing.T) {
in := `name,email,phone,plus_ones
,a@x.com,,1
Valid Guest,not-an-email,,0
Phone Person,,abc,0
Negative,,,-1
Email Only,e@x.com,,
`
r, err := Parse(strings.NewReader(in), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(r.Errors) != 4 {
t.Fatalf("expected 4 errors, got %d: %+v", len(r.Errors), r.Errors)
}
// "Email Only" row is valid: name present, email parses, phone+plus blank.
if len(r.Rows) != 1 || r.Rows[0].Name != "Email Only" {
t.Fatalf("expected 1 valid row, got %+v", r.Rows)
}
}
func TestParseUTF8BOM(t *testing.T) {
in := "\xEF\xBB\xBFname,email\nMira,m@x.com\n"
r, err := Parse(strings.NewReader(in), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(r.Rows) != 1 || r.Rows[0].Name != "Mira" {
t.Fatalf("BOM not stripped: %+v", r.Rows)
}
}
func TestParseUTF16LE(t *testing.T) {
// "name,email\nMira,m@x.com\n" in UTF-16 LE with BOM.
in := []byte{0xFF, 0xFE}
for _, r := range "name,email\nMira,m@x.com\n" {
in = append(in, byte(r), byte(r>>8))
}
r, err := Parse(strings.NewReader(string(in)), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(r.Rows) != 1 || r.Rows[0].Name != "Mira" {
t.Fatalf("UTF-16 LE not decoded: %+v", r.Rows)
}
}
func TestParseEmptyTrailingRows(t *testing.T) {
in := "name,email\nMira,m@x.com\n,,\n\n,\n"
r, err := Parse(strings.NewReader(in), Options{})
if err != nil {
t.Fatalf("parse: %v", err)
}
if r.TotalCount != 1 {
t.Fatalf("trailing blanks counted: TotalCount=%d", r.TotalCount)
}
}
func TestParseMissingNameHeader(t *testing.T) {
in := "email,phone\na@x.com,\n"
if _, err := Parse(strings.NewReader(in), Options{}); err == nil {
t.Fatal("expected error for missing name column")
}
}
func TestParseMaxRows(t *testing.T) {
var b strings.Builder
b.WriteString("name\n")
for i := 0; i < 11; i++ {
b.WriteString("X\n")
}
if _, err := Parse(strings.NewReader(b.String()), Options{MaxRows: 10}); err == nil {
t.Fatal("expected error when exceeding MaxRows")
}
}