Deployment, Operations, and Observability
Overview
This deployment slice covers how the application is built into production images, pushed to Artifact Registry, and rolled out onto the Compute Engine host with Docker Compose. It also covers the schema-apply step that must run before production form submissions depend on new tables, and the edge routing split that sends browser traffic through Fastly and Google Cloud while separating frontend and API paths.
Operationally, the production stack is configured so static frontend assets can be cached at the edge, API paths can stay fresh, and failed backend writes can fall back to local persistence when the Next.js application route is used. Observability is bootstrapped in the frontend through OpenTelemetry and conditional Sentry imports, with runtime telemetry controlled by environment variables in frontend/.env.local.example.
Architecture Overview
flowchart LR
subgraph CICD[CI CD Delivery]
GitHubActions[GitHub Actions]
ArtifactRegistry[Artifact Registry]
end
subgraph Edge[Edge Routing]
Fastly[Fastly]
LoadBalancer[Google Cloud HTTPS Load Balancer]
end
subgraph Runtime[Production Runtime]
Frontend[Frontend Container]
Backend[Backend Container]
MySQL[MySQL Container]
Compose[Docker Compose Prod Stack]
SchemaApply[platform database apply schema]
end
subgraph Observability[Observability]
Instrumentation[frontend instrumentation ts]
OTel[OpenTelemetry]
Sentry[Sentry]
end
Browser[Browser] --> Fastly
Fastly --> LoadBalancer
LoadBalancer --> Frontend
LoadBalancer --> Backend
GitHubActions --> ArtifactRegistry
ArtifactRegistry --> Compose
Compose --> Frontend
Compose --> Backend
Compose --> MySQL
SchemaApply --> MySQL
Frontend --> Instrumentation
Instrumentation --> OTel
Instrumentation --> Sentry
Production Image Builds
documents/deployment.md, frontend/Dockerfile, frontend/Dockerfile.local
The production deployment uses GitHub Actions to build and push container images to Artifact Registry. The backend and frontend images are tagged with latest under us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/mvp-repo/..., then pulled on the Compute Engine host during rollout.
| File | Purpose | Build or runtime behavior |
|---|---|---|
frontend/Dockerfile |
Production frontend image | Builds with node:20-alpine, accepts NEXT_PUBLIC_BACKEND_API_URL and NEXT_PUBLIC_AMPLITUDE_API_KEY as build arguments, runs npm run build, then starts with npm run start on port 3000. |
frontend/Dockerfile.local |
Local frontend container | Uses node:20-alpine, installs dependencies, and starts the dev server with npm run dev on port 3000. |
documents/deployment.md |
Deployment runbook | Defines the GitHub Actions build-and-push workflow and the Compute Engine rollout sequence. |
Artifact Registry image tags
[!NOTE]
documents/deployment.mddescribes a.env.productionfile withVITE_BACKEND_API_URLandALLOWED_ORIGINS, whilefrontend/DockerfileconsumesNEXT_PUBLIC_BACKEND_API_URLandNEXT_PUBLIC_AMPLITUDE_API_KEYat build time. The production image therefore depends on theNEXT_PUBLIC_...build arguments, not theVITE_...names in the deployment note.
The documented image targets are:
us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/mvp-repo/backend:latestus-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/mvp-repo/frontend:latest
Those tags are pulled on the Compute Engine host after gcloud auth configure-docker us-central1-docker.pkg.dev has been run on that machine.
Production Rollout on Compute Engine
documents/deployment.md, platform/deployment/prod/ docker compose files
The production rollout is SSH-driven and runs on the Compute Engine instance. The documented sequence is:
- SSH into the instance.
- Pull the latest Git branch state with
git fetch origin main && git rebase origin/main. - Stop the current stack with
docker-compose down. - Authenticate Docker to Artifact Registry with
gcloud auth configure-docker us-central1-docker.pkg.dev. - Pull the fresh images with
docker-compose pull. - Restart the stack with
docker-compose up -d --build.
Rollout sequence
sequenceDiagram
participant GA as GitHub Actions
participant AR as Artifact Registry
participant CE as Compute Engine
participant DC as Docker Compose
participant DB as MySQL
GA->>AR: Push backend and frontend images
CE->>CE: git fetch origin main and git rebase origin main
CE->>DC: docker compose down
CE->>CE: gcloud auth configure docker us central1 docker pkg dev
CE->>DC: docker compose pull
CE->>DC: docker compose up -d --build
DC->>DB: Start or restart database container
DC-->>CE: Services ready
Deployment secrets
The runbook lists these repository secrets for CI/CD:
| Secret | Purpose |
|---|---|
GCP_SA_KEY |
Service account key used by GitHub Actions |
GCP_PROJECT_ID |
Artifact Registry and GCP project selection |
COMPUTE_ENGINE_IP |
SSH target for rollout |
COMPUTE_ENGINE_USER |
SSH username for the Compute Engine host |
SSH_PRIVATE_KEY |
Private key used by appleboy/ssh-action |
Schema Rollout
documents/deployment.md, platform/database/apply_schema.sh
Production database initialization only runs when the persistent MySQL volume is created for the first time. For long-lived databases, schema changes must be applied explicitly after the code has been pulled onto the Compute Engine instance.
The documented helper is:
cd ~/MVP_fastapi
MYSQL_USER="myuser" \
MYSQL_PASSWORD="mypassword" \
platform/database/apply_schema.sh
The helper reuses the Docker Compose setup to execute platform/database/1_create_database.sql inside the running MySQL container. It is explicitly described as safe to run multiple times, and it creates missing tables such as phone_applications and applications.
Schema rollout verification
The runbook verifies the schema with a direct MySQL check inside the container:
docker-compose -f platform/deployment/prod/docker-compose.yml exec mysql \
bash -c "MYSQL_PWD=\"$MYSQL_PASSWORD\" mysql -u\"$MYSQL_USER\" -e 'USE platform; SHOW TABLES LIKE \"%applications\";'"
Schema rollout order
- Pull the latest code on the Compute Engine host.
- Run
platform/database/apply_schema.sh. - Verify the expected tables exist.
- Restart or continue the production stack.
- Only then rely on application pages that submit to
/api/v1/applications.
Schema rollout flow
sequenceDiagram
participant Ops as Operator
participant Host as Compute Engine Host
participant Compose as Docker Compose
participant MySQL as MySQL Container
participant App as Application Submission
Ops->>Host: Pull latest code
Ops->>Host: Run platform database apply schema
Host->>Compose: Reuse compose config
Compose->>MySQL: Execute 1_create_database sql
MySQL-->>Compose: Create missing tables
Ops->>MySQL: Verify applications tables
App->>Compose: Submit form data
Compose-->>App: Persist succeeds after schema exists
Traffic Routing and Request Flow
[!IMPORTANT] The deployment notes tie this step directly to the
/applicationsubmission flow. If the table does not exist, backend inserts fail and the production form cannot persist records correctly.
documents/deployment.md, frontend/app/api/applications/route.ts, frontend/app/api/v1/applications/route.ts
The production traffic model is split across Fastly and Google Cloud:
- Fastly fronts
app.joinlita.com. - The Google Cloud HTTPS Load Balancer is the origin.
- TLS certificates are managed by Fastly.
- A DNS CNAME, ALIAS, or ANAME maps
app.joinlita.comto the Fastly service domain. - Static frontend assets are cached at the edge.
- API paths such as
/api/*bypass cache.
The load balancer then splits traffic by host and path rule:
| Rule | Target |
|---|---|
/api/* |
mvp-backend-api |
| all other requests | mvp-backend |
The documented backend services use named ports:
| Backend service | Purpose | Named port | Port |
|---|---|---|---|
mvp-backend |
Frontend UI | http-frontend |
80 |
mvp-backend-api |
API endpoints | http-backend |
8000 |
Request routing sequence
sequenceDiagram
participant Browser as Browser
participant Fastly as Fastly
participant LB as Google Cloud HTTPS Load Balancer
participant Frontend as Frontend Service
participant API as API Service
Browser->>Fastly: Request app joinlita dot com
Fastly->>LB: Forward origin request
alt Static or app page
LB->>Frontend: Route non api path
Frontend-->>LB: HTML or static asset
else API request
LB->>API: Route api path
API-->>LB: JSON response
end
LB-->>Fastly: Response
Fastly-->>Browser: Cached or uncached response
Fastly caching model
| Path type | Cache behavior |
|---|---|
| Static frontend assets | Cached |
/api/* |
Bypassed |
Runtime Configuration and Observability
[!NOTE]
documents/deployment.mdsends/api/*to the Google Cloud API backend service, while the Next.js frontend also definesfrontend/app/api/v1/applications/route.ts. The external routing split therefore places public/apitraffic on the Google Cloud API origin, while the frontend route remains part of the Next.js application container and its local or alternate deployment paths.
frontend/.env.local.example, frontend/instrumentation.ts, documents/deployment.md
The frontend runtime is configured through environment variables for authentication, telemetry, and build-time behavior. The repository includes a detailed example file for local and environment setup, while the deployment notes define additional production-specific values for the application storage fallback path.
Frontend runtime variables
| Variable | Purpose |
|---|---|
AUTH0_DOMAIN |
Auth0 tenant domain without protocol |
AUTH0_CLIENT_ID |
Public Auth0 client identifier |
AUTH0_CLIENT_SECRET |
Auth0 client secret |
AUTH0_ORGANIZATION_ID |
Optional Auth0 organization identifier |
NEXT_PUBLIC_AUTH0_ORGANIZATION_ID |
Client-side fallback for organization-aware login redirects |
AUTH0_SECRET |
Cookie encryption secret |
AUTH0_BASE_URL |
Local or deployed base URL |
AUTH0_ISSUER_BASE_URL |
Auth0 issuer base URL |
AUTH0_MANAGEMENT_CLIENT_ID |
Auth0 Management API client ID |
AUTH0_MANAGEMENT_CLIENT_SECRET |
Auth0 Management API client secret |
AUTH0_MANAGEMENT_AUDIENCE |
Auth0 Management API audience |
NEXT_PUBLIC_SENTRY_DSN |
Client-side Sentry DSN |
SENTRY_DSN |
Server-side Sentry DSN |
NEXT_PUBLIC_SENTRY_ENVIRONMENT |
Client-side Sentry environment |
SENTRY_ENVIRONMENT |
Server-side Sentry environment |
NEXT_PUBLIC_SENTRY_TRACES_SAMPLE_RATE |
Client-side tracing sample rate |
SENTRY_TRACES_SAMPLE_RATE |
Server-side tracing sample rate |
NEXT_PUBLIC_SENTRY_REPLAYS_SESSION_SAMPLE_RATE |
Session replay sample rate |
NEXT_PUBLIC_SENTRY_REPLAYS_ON_ERROR_SAMPLE_RATE |
Replay-on-error sample rate |
SENTRY_AUTH_TOKEN |
Sentry auth token |
SENTRY_ORG |
Sentry organization slug |
SENTRY_PROJECT |
Sentry project slug |
NEXT_PUBLIC_AMPLITUDE_API_KEY |
Frontend analytics key |
OTEL_SERVICE_NAME |
OpenTelemetry service name |
OTEL_EXPORTER_OTLP_ENDPOINT |
OpenTelemetry exporter endpoint |
OTEL_EXPORTER_OTLP_HEADERS |
OpenTelemetry exporter headers |
Production storage variables
The deployment notes also define submission storage overrides for the application route:
| Variable | Purpose |
|---|---|
APPLICATIONS_DATA_GCS_BUCKET |
Google Cloud Storage bucket for persisted application submissions |
APPLICATIONS_DATA_GCS_OBJECT |
Optional object name, defaulting to applications.json |
When those variables are omitted, the route falls back to the local data/applications.json file for development convenience.
OpenTelemetry and Sentry bootstrap
frontend/instrumentation.ts initializes telemetry during frontend startup.
| Property | Type | Description |
|---|---|---|
serviceName |
string |
Set from OTEL_SERVICE_NAME or defaults to lita-frontend |
| Method | Description |
|---|---|
register |
Registers OpenTelemetry with registerOTel, then conditionally imports ./sentry.server.config for Node.js runtime or ./sentry.edge.config for Edge runtime. |
Telemetry startup sequence
sequenceDiagram
participant Next as Next js runtime
participant Inst as instrumentation ts
participant OTel as OpenTelemetry
participant Sentry as Sentry config
Next->>Inst: register
Inst->>OTel: registerOTel with serviceName
alt nodejs runtime
Inst->>Sentry: import sentry server config
else edge runtime
Inst->>Sentry: import sentry edge config
end
Deployment API Surface
GET /api/v1/applications
frontend/app/api/v1/applications/route.ts, frontend/app/api/applications/route.ts
This endpoint serves application records as JSON by default and can also render an HTML table when format=html is provided. It reads backend data when available, falls back to stored application records, and returns an empty array if nothing exists.
List Applications
{
"title": "List Applications",
"description": "Returns application records as JSON for deployment checks and dashboard data loading. The same handler can also render HTML when format=html is requested.",
"method": "GET",
"baseUrl": "<AppBaseUrl>",
"endpoint": "/api/v1/applications",
"headers": [],
"queryParams": [
{
"key": "format",
"value": "json",
"required": false
}
],
"pathParams": [],
"bodyType": "none",
"requestBody": "",
"formData": [],
"rawBody": "",
"responses": {
"200": {
"description": "Success",
"body": "[\n {\n \"id\": 101,\n \"identifier\": \"app-101\",\n \"email\": \"[email protected]\",\n \"phone\": \"+1 555 0100\",\n \"country\": \"US\",\n \"employment_status\": \"Employed\",\n \"financing_consent\": \"granted\",\n \"full_name\": \"Demo Applicant\",\n \"date_of_birth\": \"1990-04-12\",\n \"profession\": \"Product Manager\",\n \"current_employer\": \"Lita Housing\",\n \"job_title\": \"Operations Lead\",\n \"industry\": \"Real Estate\",\n \"website\": \"https://example.com\",\n \"linkedin\": \"https://linkedin.com/in/demo-applicant\",\n \"agree_policy\": true,\n \"receive_updates\": false,\n \"skipped_connect_accounts\": false,\n \"connected_accounts\": {\n \"personal\": [],\n \"business\": [],\n \"investments\": []\n },\n \"metadata\": {\n \"application_reference\": \"AP_1A2B3C4\",\n \"source\": \"production\"\n },\n \"created_at\": \"2026-03-31T12:00:00.000Z\"\n }\n]"
}
}
}
POST /api/v1/applications
frontend/app/api/v1/applications/route.ts, frontend/app/api/applications/route.ts
This endpoint validates the incoming payload, proxies it to the backend when the backend is reachable, and falls back to local JSON persistence when the backend write fails. In the fallback path it assigns a numeric id, stamps created_at, writes the record to the local storage file, and returns a synthesized 201 response.
Create Application
{
"title": "Create Application",
"description": "Submits an application payload to the backend and falls back to local persistence when backend submission fails.",
"method": "POST",
"baseUrl": "<AppBaseUrl>",
"endpoint": "/api/v1/applications",
"headers": [
{
"key": "Content-Type",
"value": "application/json",
"required": true
}
],
"queryParams": [],
"pathParams": [],
"bodyType": "json",
"requestBody": "{\n \"id\": 101,\n \"email\": \"[email protected]\",\n \"phone\": \"+1 555 0100\",\n \"country\": \"US\",\n \"employment_status\": \"Employed\",\n \"financing_consent\": \"granted\",\n \"full_name\": \"Demo Applicant\",\n \"date_of_birth\": \"1990-04-12\",\n \"profession\": \"Product Manager\",\n \"current_employer\": \"Lita Housing\",\n \"job_title\": \"Operations Lead\",\n \"industry\": \"Real Estate\",\n \"website\": \"https://example.com\",\n \"linkedin\": \"https://linkedin.com/in/demo-applicant\",\n \"agree_policy\": true,\n \"receive_updates\": false,\n \"skipped_connect_accounts\": false,\n \"connected_accounts\": {\n \"personal\": [\n {\n \"id\": \"acc_001\",\n \"name\": \"Primary Checking\",\n \"type\": \"checking\"\n }\n ],\n \"business\": [],\n \"investments\": []\n },\n \"metadata\": {\n \"application_reference\": \"AP_1A2B3C4\",\n \"source\": \"production\"\n },\n \"created_at\": \"2026-03-31T12:05:00.000Z\"\n}",
"formData": [],
"rawBody": "",
"responses": {
"201": {
"description": "Local fallback success",
"body": "{\n \"id\": 102,\n \"created_at\": \"2026-03-31T12:05:00.000Z\",\n \"persisted\": \"local_fallback\",\n \"detail\": \"Backend persistence is currently unavailable. Saved locally so submission can continue.\"\n}"
}
}
}
Application submission flow
sequenceDiagram
participant User as User
participant UI as Review Page
participant Route as api applications route
participant Backend as Backend API
participant Store as Local submissions file
User->>UI: Submit application
UI->>Route: POST application payload
Route->>Route: Validate payload and resolve backend URLs
Route->>Backend: Forward JSON to backend
alt Backend accepts write
Backend-->>Route: Created record
Route-->>UI: 201 response from backend
else Backend rejects or is unavailable
Route->>Store: Read and append local submission
Store-->>Route: Persisted locally
Route-->>UI: 201 local fallback response
end
Error Handling
The deployment and application-route code uses explicit fallback paths instead of failing hard when the backend is unavailable.
| Location | Behavior |
|---|---|
frontend/app/api/applications/route.ts |
Logs a warning when backend submission fails, then stores the payload locally and returns a 201 fallback response. |
frontend/app/api/applications/route.ts |
Returns 400 for invalid JSON payloads or validation failures. |
frontend/instrumentation.ts |
Chooses runtime-specific telemetry imports based on NEXT_RUNTIME. |
platform/database/apply_schema.sh |
Can be rerun safely to apply missing tables without special casing first-run behavior. |
Caching Strategy
| Layer | Cache behavior |
|---|---|
| Fastly | Caches static frontend assets and bypasses /api/*. |
| Next.js backend fetches | Use cache: "no-store" for application submission and read paths that must stay fresh. |
| Schema rollout | No cache; changes are applied directly into the running MySQL container. |
The deployment notes also emphasize that the application data route should remain fresh on production deployments so dashboard reads reflect the latest submissions. The edge cache is therefore limited to static frontend content.
Operational Verification
The documented post-deployment checks are:
- Open
https://app.joinlita.com. - Confirm client-side routing such as
/loginand/register. - Verify
/api/...requests route correctly through the split origin model. - Check Load Balancer health checks in the GCP Console.
- Inspect backend and frontend container logs on the Compute Engine host.
Log inspection
The runbook uses gcloud compute ssh to reach the host and then checks container logs through Docker Compose. The documented command style is:
gcloud compute ssh instance-20250206-052936
cd MVP_fastapi
docker-compose -f platform/deployment/prod/docker-compose.yaml logs backend
docker-compose -f platform/deployment/prod/docker-compose.yml logs frontend
Dependencies
| Dependency | Role in deployment or observability |
|---|---|
| GitHub Actions | Builds and pushes container images |
| Artifact Registry | Stores versioned backend and frontend images |
| Compute Engine | Hosts the production Docker Compose stack |
| Docker Compose | Starts, stops, and updates containers |
| Fastly | Edge delivery, TLS, and static asset caching |
| Google Cloud HTTPS Load Balancer | Origin routing for frontend and API paths |
| MySQL container | Production relational data store |
| Google Cloud Storage | Optional storage backend for application submissions |
| OpenTelemetry | Frontend instrumentation bootstrap |
| Sentry | Node and Edge runtime error reporting |
| Auth0 | Frontend authentication runtime configuration |
| Amplitude | Frontend analytics key configuration |
Testing Considerations
| Scenario | What to verify |
|---|---|
| Fresh deployment | Images pull from Artifact Registry and containers start cleanly. |
| Schema update | platform/database/apply_schema.sh creates missing tables before submissions are posted. |
| Application submission | POST /api/v1/applications succeeds, or falls back to local persistence when backend writes fail. |
| Edge routing | /api/* reaches the API backend service, while non-API traffic reaches the frontend service. |
| Static caching | Frontend assets are cached by Fastly, but API traffic bypasses cache. |
| Observability startup | frontend/instrumentation.ts registers OpenTelemetry and loads the correct Sentry config for the runtime. |
Key Classes Reference
| Class | Responsibility |
|---|---|
documents/deployment.md |
Production deployment runbook, routing model, and verification steps |
platform/database/apply_schema.sh |
Applies the current schema to a long-lived production MySQL volume |
frontend/Dockerfile |
Builds the production frontend image for Artifact Registry |
frontend/Dockerfile.local |
Builds the local development frontend container |
frontend/.env.local.example |
Documents runtime environment variables for Auth0, Sentry, Amplitude, and OpenTelemetry |
frontend/instrumentation.ts |
Registers OpenTelemetry and loads runtime-specific Sentry configuration |
frontend/app/api/applications/route.ts |
Implements application submission proxying and local fallback persistence |
frontend/app/api/v1/applications/route.ts |
Exposes the production-facing v1 alias for the application route |