Deployment, Operations, and Observability - Local Development, Build, and Container Workflows

Overview

This part of the repository defines how internal engineers bring the platform up locally, verify it, and package it for container-based execution. The workflow centers on a Next.js frontend, a FastAPI backend, and supporting database and storage prerequisites that must be in place before the application behaves like the deployed stack.

The local developer path is intentionally direct: load environment variables, start the compose stack, verify the application aggregation endpoint, and inspect logs when something is off. The build path is equally explicit in the checked-in Dockerfiles: the frontend image is built on Node 20, the production container is slimmed down to runtime-only dependencies, and the local image is set up for iterative development with npm run dev.

Architecture Overview

flowchart TB
    Dev[Developer workstation]
    Env[Env file and shell variables]
    Compose[Local compose stack]
    FE[Frontend container Next.js]
    BE[Backend container FastAPI]
    DB[MySQL database]
    AppFile[Applications json file]
    Telemetry[Logging and telemetry]
    USBuild[US API TypeScript build]
    USTest[US API test build]

    Dev -->|source .env| Env
    Dev -->|docker compose up| Compose
    Env --> Compose
    Compose --> FE
    Compose --> BE
    Compose --> DB
    FE -->|browser pages and API routes| BE
    FE -->|persist local submissions| AppFile
    BE -->|SQLAlchemy and schema checks| DB
    FE -->|registerOTel and Sentry| Telemetry
    BE -->|validation and auth logs| Telemetry
    USBuild -->|rootDir src outDir dist| USBuild
    USTest -->|CommonJS test compile| USTest

Local Development Environment

Required tools and inputs

[!IMPORTANT] documents/deployment.md still describes the frontend as “React with Vite”, but the checked-in frontend build files use Next.js 16 and Node 20-alpine. The local and container workflows in this repository follow the Next.js-based files, not the Vite description.

documents/development.md

Tool or Input Requirement Why it matters in this repo
Visual Studio Code Recommended Used with the formatter setup referenced in the doc
git Required For branch sync and pull request flow
Docker and Docker Compose Required Used to start the local stack
Google Cloud SDK Required Used in deployment-related workflows
Python 3.10 Required Matches the backend development baseline
Node.js 20.9+ / npm Required Matches the frontend build/runtime baseline
.env file Required Loaded before starting the stack
ADMIN_EMAILS / REALTOR_EMAILS Optional Promotes or demotes reviewer/admin accounts
DEFAULT_REALTOR_PASSWORD Optional Overrides the shared realtor password

Environment file setup

The documented setup starts by renaming env_template in the repository root to .env and filling in the required API keys. The same document also notes that keys should be provided by other engineers rather than generated ad hoc.

For reviewer-only flows, the repo ships with preloaded accounts and role assignment rules:

The last two addresses are automatically treated as admins in every environment, while ADMIN_EMAILS and REALTOR_EMAILS can be used to change role assignments before the stack starts.

Local Startup and Verification

Start the local stack

documents/development.md

set -a && source .env && set +a
docker compose -f platform/deployment/local/docker-compose.yml up -d --build

The local stack is expected to expose the frontend at http://localhost:3000. The documented sanity check is to curl the applications endpoint:

curl http://localhost:3000/api/v1/applications

A healthy environment returns either:

  • an empty array when there are no submissions, or
  • the merged list of enabled applications.

How the applications data is assembled

The frontend applications route merges two data sources:

Source Behavior
Remote backend applications Pulled from the backend container when ENABLE_REMOTE_APPLICATIONS=true or NEXT_PUBLIC_ENABLE_REMOTE_APPLICATIONS=true
Local submissions Persisted to data/applications.json inside the frontend container, or to the path named by APPLICATIONS_DATA_FILE

This means the local frontend can show dashboard data even when the remote integration is disabled, while still opting into backend data when the environment permits it.

Troubleshooting the applications route

documents/development.md

If the dashboard looks empty or the applicant detail page fails, the repo documents a short sequence of checks:

  1. Inspect the IDs returned by the aggregated list.
  2. Confirm the detail endpoint responds for the selected identifier.
  3. Check the local data/applications.json file.
  4. Verify BACKEND_API_URL is present in the frontend container environment.
  5. Tail the frontend logs while reproducing the click.

Those checks are designed to isolate whether the problem is data absence, identifier mismatch, or environment drift.

Build and runtime mismatch guard

The local frontend instructions call out a specific runtime constraint: Next.js 16 needs Node.js >=20.9.0. If the compose stack repeatedly fails while waiting on http://localhost:3000/api/v1/applications, the documented fix is to rebuild after pulling the latest frontend image changes so the container runs on Node 20 rather than Node 18.

Container Workflows

Frontend production image

frontend/Dockerfile

Stage Base image Responsibilities
Builder node:20-alpine Accepts NEXT_PUBLIC_BACKEND_API_URL and NEXT_PUBLIC_AMPLITUDE_API_KEY, installs dependencies, copies the app, and runs npm run build
Runner node:20-alpine Sets NODE_ENV=production, copies .next, public, and package*.json, installs production dependencies, exposes port 3000, and starts the app

Build-time inputs

Input Type Purpose
NEXT_PUBLIC_BACKEND_API_URL build arg / env Frontend build-time API base URL
NEXT_PUBLIC_AMPLITUDE_API_KEY build arg / env Frontend analytics key

The production image keeps the runtime package set minimal by reinstalling only production dependencies in the final stage.

Frontend local image

frontend/Dockerfile.local

Stage Base image Responsibilities
Single stage node:20-alpine Installs dependencies, copies the project, exposes port 3000, and starts npm run dev

This image is the local development container entrypoint for the frontend and is the one that matches the “hot edit and refresh” style workflow described in the local docs.

US API TypeScript build contract

services/us-api/tsconfig.json

services/us-api/tsconfig.test.json

File Key settings Effect
tsconfig.json target: ES2022, module: ES2022, moduleResolution: node, outDir: dist, rootDir: src, strict: true Defines the production build contract for the service
tsconfig.test.json Extends tsconfig.json, module: CommonJS, moduleResolution: Node, isolatedModules: true, types: ["node", "jest"] Defines the Jest/test compilation contract

This TypeScript configuration is the verified build surface for the US API codebase: source lives in src, compiled output lands in dist, and tests run under a Jest-friendly CommonJS configuration.

Backend Runtime Composition and Error Handling

backend/src/main.py

backend/src/main.py

Module-level properties

Property Type Description
app FastAPI Main FastAPI application instance
ALLOWED_ORIGINS list[str] Comma-separated origin list loaded from ALLOWED_ORIGINS, defaulting to http://localhost:3000

The module wires the application together by importing the routers for auth, Salt Edge, Equifax, TransUnion batch, Japan credit, open banking, Lita, applicants, billing, and credit upload. It also imports MySQL database and schema helpers plus country and locale resolvers, which shows that the app is the central composition point for backend runtime concerns.

Public functions

Method Description
health_check Returns a simple health payload used to confirm the app is running
validation_exception_handler Logs FastAPI request validation failures and returns a 422 response with detail errors
auth_exception_handler Logs AuthError failures and returns a 401 response with WWW-Authenticate: Bearer

Error handling behavior

  • RequestValidationError is logged at error level with the HTTP method, request path, and validation details.
  • AuthError is logged at warning level and returned with both detail and error_code.
  • The auth handler also sets the WWW-Authenticate header to Bearer, which matters for clients and browser debugging.

Backend request-handling trace

sequenceDiagram
    participant Client as Client
    participant App as FastAPI app
    participant Val as validation_exception_handler
    participant Auth as auth_exception_handler
    participant Log as Logging

    Client->>App: Request with invalid body
    App->>Val: Raise RequestValidationError
    Val->>Log: error method path details
    Val-->>Client: 422 JSONResponse

    Client->>App: Request with invalid auth
    App->>Auth: Raise AuthError
    Auth->>Log: warning method path message
    Auth-->>Client: 401 JSONResponse with Bearer header

Observability

Frontend telemetry registration

frontend/instrumentation.ts

Module-level properties

Property Type Description
serviceName string OpenTelemetry service name from OTEL_SERVICE_NAME, defaulting to lita-frontend

Public functions

Method Description
register Registers OpenTelemetry and conditionally loads the correct Sentry config for the runtime

Runtime behavior

Condition Behavior
Always Calls registerOTel({ serviceName })
NEXT_RUNTIME === "nodejs" Imports ./sentry.server.config
NEXT_RUNTIME === "edge" Imports ./sentry.edge.config

The checked-in frontend environment example also exposes the observability settings that go with this bootstrap path:

  • NEXT_PUBLIC_SENTRY_DSN
  • SENTRY_DSN
  • NEXT_PUBLIC_SENTRY_ENVIRONMENT
  • SENTRY_ENVIRONMENT
  • NEXT_PUBLIC_SENTRY_TRACES_SAMPLE_RATE
  • SENTRY_TRACES_SAMPLE_RATE
  • NEXT_PUBLIC_SENTRY_REPLAYS_SESSION_SAMPLE_RATE
  • NEXT_PUBLIC_SENTRY_REPLAYS_ON_ERROR_SAMPLE_RATE
  • SENTRY_AUTH_TOKEN
  • SENTRY_ORG
  • SENTRY_PROJECT
  • OTEL_SERVICE_NAME
  • OTEL_EXPORTER_OTLP_ENDPOINT
  • OTEL_EXPORTER_OTLP_HEADERS

Logging path in local operations

The repository uses standard Python logging in backend/src/main.py for request validation and auth errors, and the frontend bootstrap routes operational telemetry through OpenTelemetry and Sentry. That gives engineers two clear places to inspect when a local stack behaves differently from the expected UI flow: backend exception logs and frontend telemetry initialization.

Database Prerequisites and Schema Updates

MySQL schema expectations

documents/deployment.md

The deployment notes state that when new API endpoints rely on additional MySQL tables, the schema must be applied to long-lived databases because the initialization script only runs the first time the volume is created. The same document calls out that without this step, form submissions can fail because the target table does not exist.

The documented helper is:

cd ~/MVP_fastapi
MYSQL_USER="myuser" \
MYSQL_PASSWORD="mypassword" \
platform/database/apply_schema.sh

After running it, the doc suggests verifying that tables exist with:

docker-compose -f platform/deployment/prod/docker-compose.yml exec mysql \
  bash -c "MYSQL_PWD=\"$MYSQL_PASSWORD\" mysql -u\"$MYSQL_USER\" -e 'USE platform; SHOW TABLES LIKE \"%applications\";'"

Data persistence for application submissions

The local frontend API stores onboarding submissions in a file-backed JSON store by default:

  • data/applications.json
  • override with APPLICATIONS_DATA_FILE

The production notes also describe a GCS-backed variant using:

  • APPLICATIONS_DATA_GCS_BUCKET
  • APPLICATIONS_DATA_GCS_OBJECT

That split is important operationally: local development uses file-backed persistence for speed, while deployed container workflows can switch to durable storage.

Repository Validation and Build Expectations

Verified validation commands

documents/deployment.md

documents/lita-testing-guide.md

Command Purpose
pytest tests/ -v Backend unit test run documented for deployment validation
pytest backend/tests/interface/test_lita_endpoints.py -q Targeted Lita API regression coverage
./scripts/lita_smoke_test.sh One-shot local smoke check for the consent, token, snapshot, and scoring flow
docker compose -f platform/deployment/local/docker-compose.yml up -d --build Start the local frontend/backend stack
curl http://localhost:3000/api/v1/applications Verify the frontend aggregated applications endpoint
docker compose -f platform/deployment/local/docker-compose.yml logs frontend -f Follow frontend logs while reproducing issues

Build expectations for internal engineers

  • Frontend production builds are performed with Node 20-alpine and npm run build.
  • Local frontend iteration uses npm run dev.
  • The US API codebase is compiled from src to dist using the TypeScript settings in tsconfig.json.
  • Test compilation for the US API switches to CommonJS and Jest-compatible types via tsconfig.test.json.
  • Long-lived MySQL databases must have the current schema applied before endpoint flows that write application records can succeed.

Feature Flows

1. Start the local stack and verify applications data

sequenceDiagram
    participant Eng as Engineer
    participant Shell as Shell
    participant Compose as Docker Compose
    participant FE as Frontend container
    participant BE as Backend container
    participant DB as MySQL database
    participant FS as applications json file

    Eng->>Shell: export env and start compose
    Shell->>Compose: docker compose up -d --build
    Compose->>FE: build and start Next.js frontend
    Compose->>BE: build and start FastAPI backend
    Compose->>DB: start database service
    Eng->>FE: GET /api/v1/applications
    FE->>BE: fetch remote applications when enabled
    FE->>FS: read stored applications
    FE-->>Eng: merged JSON response

2. Submit application data through the frontend

The repository’s local workflow stores application submissions in the frontend container’s JSON store, then merges those submissions into later list responses. That is the operational path that makes local dashboard refreshes immediate even before remote applicant data is enabled.

3. Surface validation and auth failures cleanly

The backend converts FastAPI validation errors and auth failures into explicit JSON responses, which makes local debugging straightforward when a request shape or token is wrong. The logging context includes the HTTP method and request path, so the error message stays attached to the failing call.

Integration Points

  • platform/deployment/local/docker-compose.yml for local stack startup
  • platform/deployment/prod/docker-compose.yml for database and production container operations
  • platform/database/apply_schema.sh for schema synchronization
  • frontend/Dockerfile for production image builds
  • frontend/Dockerfile.local for local frontend iteration
  • backend/src/main.py for backend runtime composition and error handling
  • frontend/instrumentation.ts for telemetry bootstrap
  • services/us-api/tsconfig.json and services/us-api/tsconfig.test.json for the TypeScript build contract

Dependencies

  • FastAPI
  • SQLAlchemy
  • MySQL
  • Next.js 16
  • Node.js 20.9+
  • Python 3.10
  • Docker and Docker Compose
  • OpenTelemetry
  • Sentry
  • Google Cloud SDK for deployment-related workflows

Testing Considerations

  • Confirm the frontend container starts on Node 20, not Node 18.
  • Verify curl http://localhost:3000/api/v1/applications returns either [] or a merged list of applications.
  • Check that BACKEND_API_URL is present in the frontend container when remote applications are expected.
  • Re-run the schema helper before testing write paths against a persistent database volume.
  • Watch the backend logs for 422 validation errors and 401 auth errors when request payloads or credentials are wrong.
  • Confirm the frontend telemetry bootstrap loads the correct Sentry config for the runtime.

Key Classes Reference

Class Responsibility
development.md Documents local setup, startup, troubleshooting, and contributor workflow
deployment.md Documents deployment validation, schema application, and container rollout expectations
Dockerfile Builds the production frontend image
Dockerfile.local Runs the frontend in local development mode
main.py Composes the FastAPI app and handles validation and auth errors
instrumentation.ts Registers OpenTelemetry and Sentry for the frontend
tsconfig.json Defines the US API production TypeScript build contract
tsconfig.test.json Defines the US API Jest-friendly test compile contract