Operations, Local Development, and Test Automation - Seeding, local keys, and reproducible sandbox data

Overview

This part of the repo provides the local operational primitives needed to stand up the API with stable credentials, seed repeatable tenant and applicant records, and export those records for comparison across runs. It is the layer that makes local Fastify and Prisma development predictable without depending on production material.

The scripts are built around two distinct workflows: scripts/seed-dev-auth.sh creates a minimal tenant plus API client for local authentication, while scripts/seed-sandbox-dummy-data.sh populates deterministic US and JP sandbox scenarios, consent records, evidence snapshots, and decision rows. Together with scripts/run-api.sh and services/us-api/.env.example, they define how local JWT material, partner bearer keys, and reproducible sandbox fixtures are inferred and reused.

Architecture Overview

flowchart TB
    Dev[Developer]

    subgraph LocalAuth[Local Authentication Material]
        RunApi[scripts run-api.sh]
        SeedDevAuth[scripts seed-dev-auth.sh]
        EnvExample[services us-api env example]
    end

    subgraph SandboxData[Deterministic Sandbox Data]
        SeedSandbox[scripts seed-sandbox-dummy-data.sh]
        PrismaGen[Prisma generate]
        PrismaClient[Prisma client]
    end

    subgraph Persistence[Persistence and Artifacts]
        Postgres[(PostgreSQL)]
        Artifacts[workspace artifacts JSON CSV]
    end

    Dev --> RunApi
    RunApi --> EnvExample
    RunApi -->|exports US and JP JWT keys| EnvExample

    Dev --> SeedDevAuth
    SeedDevAuth --> PrismaClient
    PrismaClient --> Postgres

    Dev --> SeedSandbox
    SeedSandbox --> PrismaGen
    SeedSandbox --> PrismaClient
    PrismaClient --> Postgres
    SeedSandbox --> Artifacts

Component Structure

1. Local Authentication Material

scripts/run-api.sh

This script is the local key bootstrapper for the regional API surfaces. It reuses any already-exported JWT variables, and if they are missing it generates one ephemeral RSA keypair, exports it to both US and JP token flows, and removes the temporary key files.

Observed behavior

  • Reuses US_JWT_PRIVATE_KEY, US_JWT_PUBLIC_KEY, JP_JWT_PRIVATE_KEY, and JP_JWT_PUBLIC_KEY when all are already present.
  • Generates a 2048-bit RSA keypair when those variables are absent.
  • Writes the keys to temporary files, reads them back into the environment, and deletes the temp files.
  • Prints a local bootstrap message indicating the JWT signing keys were generated.

Local key inference

  • The script does not introduce separate key pairs per region.
  • The same ephemeral private/public pair is assigned to both US and JP local token flows.
  • This keeps local token verification aligned across regional surfaces while avoiding persistent key material.

scripts/seed-dev-auth.sh

This script creates a minimal local tenant and partner API client so local requests can authenticate against the API. It uses the provided API_KEY value as the clientId, stores a fixed synthetic clientSecretHash, and grants wildcard scopes.

Inputs and defaults

Variable Type Default Purpose
API_KEY string dev-key Bearer credential stored as clientId
TENANT_NAME string dev-tenant Tenant name created for local auth
CLIENT_NAME string dev-client Human-readable API client name

Persistence behavior

  • Creates a new tenant with prisma.tenant.create.
  • Upserts apiClient by clientId.
  • Uses clientSecretHash: "dev" on create.
  • Sets scopes: ["*"] on create and update.
  • Prints tenant_id and api_key to stdout.
  • Disconnects Prisma at the end.

[!IMPORTANT] scripts/seed-dev-auth.sh always creates a new tenant row. It reuses the API client by clientId, but it does not look up an existing tenant by name before creating one.

Typical local invocation

API_KEY=dev-key TENANT_NAME=dev-tenant CLIENT_NAME=dev-client ./scripts/seed-dev-auth.sh

services/us-api/.env.example

This file defines the US API runtime configuration contract for local development. It shows the expected shape of the JWT keys, partner API keys, and supporting runtime values that the service consumes.

Variable Type Example Purpose
PORT number 8080 Service port
DATABASE_URL string postgresql://user:pass@localhost:5432/us_api PostgreSQL connection string
JWT_PRIVATE_KEY PEM string -----BEGIN PRIVATE KEY----- JWT signing key
JWT_PUBLIC_KEY PEM string -----BEGIN PUBLIC KEY----- JWT verification key
PARTNER_API_KEYS comma-separated string partner_key_1,partner_key_2,sotheby_partner_token Local partner bearer keys
WEBHOOK_SIGNING_SECRET string webhook_secret Webhook signing material
TOKEN_TTL_SECONDS number 3600 Local token lifetime
LINK_BASE_URL string https://joinlita.com/us Link-generation base URL
SCORING_CALLBACK_URL string https://example.com Callback URL used by the US surface

2. Deterministic Sandbox Data

scripts/seed-sandbox-dummy-data.sh

This script seeds a stable sandbox tenant, a sandbox API client, deterministic applicant scenarios, consent records, and evidence snapshots. It also exports the resulting payload as JSON, CSV, or both so reviewers can compare runs.

Inputs and defaults

Variable Type Default Purpose
TENANT_NAME string sandbox-tenant Tenant name reused across runs
API_KEY string sandbox-key Bearer credential stored as clientId
CLIENT_NAME string sandbox-client Human-readable API client name
OUTPUT_FORMAT string both json, csv, or both
OUTPUT_DIR string ./workspace/artifacts Artifact destination
DATABASE_URL string postgresql://postgres:postgres@localhost:5432/lita-api Prisma connection string when not supplied
.env file optional Sourced when present before defaults are applied

Execution behavior

  • Runs pnpm -C packages/db prisma generate before using Prisma.
  • Reuses or creates the sandbox tenant by name.
  • Upserts the sandbox apiClient by clientId.
  • Seeds three fixed applicant profiles.
  • Recomputes the decision table from deterministic scoring rules.
  • Creates or updates US and JP consent and evidence records.
  • Writes JSON and CSV artifacts into OUTPUT_DIR.
  • Prints stable identifiers and output file paths.
  • Disconnects Prisma at the end.

Named helper methods

Method Description Returns
computeScore Calculates a bounded score from income, debt, NSF count, and liquidity using a currency-aware formula. A rounded integer score between 520 and 830
scoreToOutcome Maps the score to approved, manual_review, or declined. Outcome string
reasonCodesFor Returns fixed reason code arrays for each outcome. String array
escapeCsvValue Escapes CSV values containing commas, quotes, or newlines. CSV-safe string
toCsvRows Serializes the export payload into sectioned CSV rows. CSV text
upsertUsFixtures Ensures the US consent and US evidence snapshot exist for the seeded tenant. Consent record
upsertJpFixtures Ensures the JP consent and JP evidence snapshot exist for the seeded tenant. Consent record

Deterministic applicant scenarios

Field Type Notes
externalUserId string Stable external user identifier
region string US or JP
currency string USD or JPY
employmentStatus string full_time or contract
monthlyIncome number Used in score calculation
debtToIncome number Used in score calculation
avgBalance90d number Used in score calculation
nsfCount180d number Used in score calculation

Seeded scenario set

  • sandbox-us-prime-001
  • sandbox-us-thin-002
  • sandbox-jp-stable-003

Scoring rules

  • Income is normalized differently for JPY and non-JPY inputs.
  • Debt and NSF activity reduce the score.
  • Average balance increases the score through a capped liquidity bonus.
  • The final score is clamped to the 520 to 830 range and rounded.
Region Consent applicant Consent status Purpose Data categories
US sandbox-applicant-us active GENERAL_PERSONAL_FINANCE ACCOUNTS, BALANCES, TRANSACTIONS, INCOME, CASHFLOW_FEATURES
JP sandbox-applicant-jp active GENERAL_PERSONAL_FINANCE ACCOUNTS, BALANCES, TRANSACTIONS, INCOME, CASHFLOW_FEATURES

US evidence snapshot

  • evidenceRef: us-sandbox-stable-v2
  • featureSet: cashflow_v2
  • featuresJson keys:
    • avg_balance_90d
    • nsf_count_180d
    • monthly_income_estimate
    • debt_to_income

JP evidence snapshot

  • evidenceRef: jp-sandbox-stable-v2
  • featureSet: cashflow_v2
  • featuresJson keys:
    • avg_balance_90d
    • nsf_count_180d
    • monthly_income_estimate
    • debt_to_income

Exported payload shape

Property Type Description
generatedAt string ISO timestamp for the run
tenant object Tenant summary
apiClient object API client summary
profiles array Seeded decision rows
consents object US and JP consent summaries
evidenceSnapshots object US and JP evidence summaries

tenant object

Property Type Description
id string Tenant identifier
name string Tenant name

apiClient object

Property Type Description
clientId string Local bearer credential
name string Client display name

profiles item

Property Type Description
externalUserId string Stable external user identifier
region string US or JP
decisionId string Created decision record identifier
outcome string Decision result
score number Deterministic score used for the decision

consents.us and consents.jp

Property Type Description
consentId string Consent identifier
applicantId string Applicant identifier
status string Consent status
expiresAt string ISO timestamp

evidenceSnapshots.us and evidenceSnapshots.jp

Property Type Description
evidenceRef string Stable evidence reference
featureSet string Feature set name
features object Raw feature JSON

CSV export layout

Section Columns
Profile rows external_user_id, region, outcome, score, decision_id
Consent rows consent_region, consent_id, applicant_id, status, expires_at
Evidence rows evidence_region, evidence_ref, feature_set, features_json

[!NOTE] generatedAt is regenerated on every run, and both consent records use plusDays(30) for expiresAt, so those timestamps will differ across executions even when the seed inputs remain the same.

Artifact output behavior

  • OUTPUT_FORMAT=json writes sandbox-dummy-data.json.
  • OUTPUT_FORMAT=csv writes sandbox-dummy-data.csv.
  • OUTPUT_FORMAT=both writes both files.
  • OUTPUT_DIR is created recursively before write.
  • JSON output is pretty-printed with two-space indentation and ends with a newline.
  • CSV rows are escaped so embedded commas, quotes, and newlines stay valid.
  • Console output includes sandbox_tenant_id, sandbox_api_key, credit_profiles_seeded, us_consent_id, jp_consent_id, and artifact file paths.

Typical local invocation

OUTPUT_FORMAT=both OUTPUT_DIR=./workspace/artifacts ./scripts/seed-sandbox-dummy-data.sh

workspace/README.md

workspace/README.md

This README defines the repository’s long-lived memory folder. It instructs contributors to keep durable operational notes in SOUL.md, MEMORIES.md, DECISIONS.md, and TASKS.md, and says meaningful docs or code changes should update at least one of those files in the same commit.

Feature Flows

1. Local Key Bootstrap and Dev Auth Seeding

sequenceDiagram
    participant Dev as Developer
    participant RunApi as run-api.sh
    participant Node as Node RSA generator
    participant SeedAuth as seed-dev-auth.sh
    participant Prisma as Prisma client
    participant DB as PostgreSQL

    Dev->>RunApi: Start local API bootstrap
    alt JWT env vars already exist
        RunApi-->>Dev: Reuse existing keys
    else Missing JWT env vars
        RunApi->>Node: generateKeyPairSync
        Node-->>RunApi: Private and public PEM keys
        RunApi->>RunApi: Export US and JP key vars
        RunApi->>RunApi: Remove temp key files
    end

    Dev->>SeedAuth: Seed local API client
    SeedAuth->>Prisma: create tenant and upsert apiClient
    Prisma->>DB: Write tenant and apiClient rows
    DB-->>Prisma: tenant_id and api_key
    Prisma-->>SeedAuth: Seed complete
    SeedAuth-->>Dev: Print tenant_id and api_key

Flow notes

  • The local JWT material can be supplied externally or generated on demand.
  • The bearer credential used by local scripts is the API client clientId, not the stored secret hash.
  • The seeded API client uses wildcard scopes so local flows can authenticate without extra per-scope setup.

2. Sandbox Scenario Seeding and Artifact Export

sequenceDiagram
    participant Dev as Developer
    participant Seed as seed-sandbox-dummy-data.sh
    participant Prisma as Prisma client
    participant DB as PostgreSQL
    participant FS as File system
    participant Console as Console

    Dev->>Seed: Seed sandbox data
    Seed->>Seed: Source .env when present
    Seed->>Seed: Resolve DATABASE_URL and output paths
    Seed->>Seed: Run prisma generate
    Seed->>Prisma: Open client and reuse or create tenant
    Prisma->>DB: Write tenant and apiClient rows
    Seed->>Prisma: Upsert creditProfile rows
    Prisma->>DB: Write seeded profile data
    Seed->>Prisma: Delete old decisions and create new ones
    Prisma->>DB: Refresh decision rows
    Seed->>Prisma: Upsert US and JP consent and evidence records
    Prisma->>DB: Write regional sandbox records
    Seed->>FS: Write JSON and CSV artifacts
    Seed->>Console: Print identifiers and artifact paths
    Seed->>Prisma: Disconnect client
    Seed-->>Dev: Deterministic sandbox ready

Flow notes

  • The tenant is reused by name, so repeated runs keep the sandbox identity stable.
  • Decisions are rebuilt for the seeded profiles on each run, which keeps the decision set aligned with the current scoring rules.
  • Artifact files are written under workspace/artifacts unless OUTPUT_DIR overrides the location.

State Management

Entity Pattern Behavior
Tenant in seed-dev-auth.sh Create only A new tenant row is created every run
API client in both scripts Upsert by clientId The local bearer credential is stable across reruns
Tenant in seed-sandbox-dummy-data.sh Find first by name, then create The sandbox tenant is reused when present
Credit profile Upsert by compound key tenantId and externalUserId keep the row stable
Decision Delete many, then create Old decisions for the seeded profile are replaced
US consent Find first active record, then create Existing active consent is reused when present
JP consent Find first active record, then create Existing active consent is reused when present
US evidence snapshot Upsert by evidenceRef Stable evidence reference keeps the snapshot aligned
JP evidence snapshot Upsert by evidenceRef Stable evidence reference keeps the snapshot aligned

Error Handling

  • Both seed scripts use set -euo pipefail, so command failures and unset variables stop execution.
  • scripts/seed-sandbox-dummy-data.sh throws when OUTPUT_FORMAT is not one of json, csv, or both.
  • scripts/seed-sandbox-dummy-data.sh still validates the output format before printing success markers, so an invalid format never produces a successful completion message.
  • scripts/run-api.sh removes temporary key files after copying the generated PEM values into environment variables.

Dependencies

  • bash
  • pnpm
  • node
  • packages/db Prisma Client generation
  • PostgreSQL through DATABASE_URL
  • Local filesystem output under workspace/artifacts
  • Regional US and JP runtime environment variables defined in services/us-api/.env.example

Testing Considerations

  • Re-run scripts/seed-sandbox-dummy-data.sh to confirm the same three external users still produce the same seeded profile identities.
  • Compare sandbox-dummy-data.json across runs when you want a human-readable snapshot of the seeded tenant, API client, decisions, consents, and evidence references.
  • Compare sandbox-dummy-data.csv when you want a compact diffable artifact for review tooling.
  • Normalize or ignore generatedAt and the expiresAt fields when comparing runs, because both are timestamp-driven.
  • Use the printed credit_profiles_seeded, us_consent_id, and jp_consent_id markers to confirm the expected record count and regional fixture creation.
  • Use OUTPUT_FORMAT=json or OUTPUT_FORMAT=csv when only one artifact type is needed for review.

Key Classes Reference

Class Responsibility
run-api.sh Generates or reuses local JWT signing material for US and JP token flows
seed-dev-auth.sh Seeds a local tenant and partner API client for bearer-authenticated development
seed-sandbox-dummy-data.sh Seeds deterministic sandbox profiles, consents, evidence, decisions, and export artifacts
services/us-api/.env.example Defines the US API local runtime configuration contract
workspace/README.md Defines the repo memory update rule for durable local operational notes