Deployment, Operations, and Observability
Overview
This section covers the release path for Lita’s web application from verified code to a running production stack. The documented workflow uses GitHub Actions to run the backend test gate, publish Docker images to Artifact Registry, and then trigger an SSH-based rollout onto the Compute Engine host that runs docker-compose in platform/deployment/prod.
Operationally, the deployment notes also define how to verify the rollout, where to inspect container logs, and how to run pre-launch stability checks before launch. The observability surface is centered on the frontend’s instrumentation.ts, which wires OpenTelemetry and Sentry at runtime, plus the deployment-time environment variables that control telemetry behavior.
Architecture Overview
flowchart TB
subgraph GitHubActions[GitHub Actions CI CD]
Tests[Backend pytest gate]
BuildBackend[Build and push backend image]
BuildFrontend[Build and push frontend image]
DeploySSH[Deploy via SSH]
end
subgraph ArtifactRegistry[Artifact Registry]
BackendImage[backend latest image]
FrontendImage[frontend latest image]
end
subgraph ComputeEngine[Compute Engine rollout host]
DeployDir[platform deployment prod]
PullImages[docker compose pull]
Recreate[docker compose up -d --build]
end
subgraph FrontendRuntime[Frontend runtime container]
ProdDockerfile[frontend Dockerfile]
LocalDockerfile[frontend Dockerfile local]
Telemetry[instrumentation ts]
end
subgraph Observability[Observability sinks]
OpenTelemetry[OpenTelemetry]
Sentry[Sentry]
Logs[Container logs]
end
Tests --> BuildBackend
Tests --> BuildFrontend
BuildBackend --> BackendImage
BuildFrontend --> FrontendImage
DeploySSH --> DeployDir
DeployDir --> PullImages
PullImages --> Recreate
Recreate --> FrontendRuntime
FrontendRuntime --> Telemetry
Telemetry --> OpenTelemetry
Telemetry --> Sentry
ComputeEngine --> Logs
CI/CD Image Build and Publishing
The deployment document defines a two-image publish step: one image for the backend and one for the frontend. Both are built with docker/build-push-action@v4 and pushed to Artifact Registry under us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/mvp-repo/....
GitHub Actions release gates
| Stage | Tooling | Behavior |
|---|---|---|
| Test gate | pytest tests/ -v |
Runs backend unit tests before any image publish step. |
| Backend image publish | docker/build-push-action@v4 |
Builds from ./backend, pushes backend:latest. |
| Frontend image publish | docker/build-push-action@v4 |
Builds from ./frontend, pushes frontend:latest. |
| Rollout | appleboy/[email protected] |
Connects to the Compute Engine host and runs the deployment script. |
Artifact Registry authentication surface
The documented GitHub Actions setup depends on these repository secrets:
| Secret | Purpose |
|---|---|
GCP_SA_KEY |
Service account credential used by the workflow. |
GCP_PROJECT_ID |
Project ID used in Artifact Registry image tags. |
COMPUTE_ENGINE_IP |
SSH target for the rollout step. |
COMPUTE_ENGINE_USER |
SSH username for the Compute Engine host. |
SSH_PRIVATE_KEY |
Private key passed to appleboy/ssh-action. |
The rollout script also authenticates Docker on the VM with:
gcloud auth configure-docker us-central1-docker.pkg.dev
That step is required before docker-compose pull can fetch the published images from Artifact Registry.
Publish commands documented in the deployment guide
- name: Build and Push Backend Image
uses: docker/build-push-action@v4
with:
context: ./backend
push: true
tags: us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/mvp-repo/backend:latest
- name: Build and Push Frontend Image
uses: docker/build-push-action@v4
with:
context: ./frontend
push: true
tags: us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/mvp-repo/frontend:latest
Rollout Workflow Over SSH
The production deployment is executed on the Compute Engine instance after the images are pushed. The SSH action runs a small release script inside ~/MVP_fastapi/platform/deployment/prod.
Rollout sequence
sequenceDiagram
participant Dev as Developer
participant GH as GitHub Actions
participant AR as Artifact Registry
participant VM as Compute Engine host
participant DC as docker compose
Dev->>GH: Push to protected branch
GH->>GH: Run pytest gate
GH->>AR: Build and push backend image
GH->>AR: Build and push frontend image
GH->>VM: appleboy ssh action
VM->>VM: git fetch origin main and git rebase origin main
VM->>VM: docker compose down
VM->>VM: gcloud auth configure docker us-central1 docker pkg dev
VM->>DC: docker compose pull
VM->>DC: docker compose up -d --build
DC-->>VM: Containers restarted with published images
VM-->>GH: Rollout completes
Remote commands
cd ~/MVP_fastapi/platform/deployment/prod
git fetch origin main && git rebase origin/main
docker-compose down
gcloud auth configure-docker us-central1-docker.pkg.dev
docker-compose pull
docker-compose up -d --build
Rollout notes
git fetchplusgit rebasekeeps the VM aligned withorigin/main.docker-compose downclears the previous stack before the new images are pulled.docker-compose pullconsumes the Artifact Registry images published by GitHub Actions.docker-compose up -d --buildbrings the services back up in detached mode.
Production and Local Container Composition
The repository shows two hardened container patterns for the frontend: a production multi-stage build and a dedicated local hot-reload build. These are the clearest documented examples of how the application distinguishes release images from development images.
Frontend Production Dockerfile
frontend/Dockerfile
The production build uses a two-stage image to separate build-time dependencies from the runtime container.
| Stage | Base Image | Behavior |
|---|---|---|
builder |
node:20-alpine |
Installs dependencies, copies source, and runs npm run build. |
runner |
node:20-alpine |
Copies the built output, installs production dependencies, and starts the app. |
Build-time inputs
| Name | Type | Purpose |
|---|---|---|
NEXT_PUBLIC_BACKEND_API_URL |
ARG / ENV |
Injected into the build so the client bundle knows the backend base URL. |
NEXT_PUBLIC_AMPLITUDE_API_KEY |
ARG / ENV |
Injected into the build for client-side analytics config. |
NODE_ENV |
ENV |
Set to production in the runtime stage. |
Runtime composition
| File element | Behavior |
|---|---|
COPY --from=builder /app/.next ./.next |
Ships the compiled Next.js build output. |
COPY --from=builder /app/public ./public |
Ships static assets. |
RUN npm install --production --legacy-peer-deps |
Installs runtime-only dependencies. |
EXPOSE 3000 |
Declares the application port. |
CMD ["npm", "run", "start"] |
Starts the production server. |
Public methods
None. This file defines the container image build and runtime entrypoint only.
Frontend Local Dockerfile
frontend/Dockerfile.local
This file is the local hot-reload variant for development. It keeps the source tree in the container and starts the app with npm run dev.
| File element | Behavior |
|---|---|
FROM node:20-alpine |
Uses the same base image family as production. |
RUN npm install --legacy-peer-deps |
Installs the full dependency set needed for local development. |
COPY . . |
Copies the entire frontend source tree. |
EXPOSE 3000 |
Declares the dev server port. |
CMD ["npm", "run", "dev"] |
Starts the hot-reload development server. |
Production vs local difference
| Aspect | frontend/Dockerfile |
frontend/Dockerfile.local |
|---|---|---|
| Build style | Multi-stage | Single-stage |
| Runtime dependencies | Production only | Full dev dependency set |
| Startup command | npm run start |
npm run dev |
| Intended use | Release image | Local development and hot reload |
Frontend build-time environment contract
frontend/.env.local.example
This file defines the frontend’s local telemetry and runtime configuration surface.
| Variable | Purpose |
|---|---|
AUTH0_DOMAIN |
Auth0 tenant domain. |
AUTH0_CLIENT_ID |
Auth0 client identifier. |
AUTH0_CLIENT_SECRET |
Auth0 client secret. |
AUTH0_ORGANIZATION_ID |
Optional organization-aware login setting. |
NEXT_PUBLIC_AUTH0_ORGANIZATION_ID |
Client-side organization fallback. |
AUTH0_SECRET |
Cookie encryption secret. |
AUTH0_BASE_URL |
Local application base URL. |
AUTH0_ISSUER_BASE_URL |
Auth0 issuer base URL. |
AUTH0_MANAGEMENT_CLIENT_ID |
Auth0 management client ID. |
AUTH0_MANAGEMENT_CLIENT_SECRET |
Auth0 management client secret. |
AUTH0_MANAGEMENT_AUDIENCE |
Auth0 management API audience. |
NEXT_PUBLIC_SENTRY_DSN |
Browser-side Sentry DSN. |
SENTRY_DSN |
Server-side Sentry DSN. |
NEXT_PUBLIC_SENTRY_ENVIRONMENT |
Browser-side Sentry environment label. |
SENTRY_ENVIRONMENT |
Server-side Sentry environment label. |
NEXT_PUBLIC_SENTRY_TRACES_SAMPLE_RATE |
Browser-side trace sampling. |
SENTRY_TRACES_SAMPLE_RATE |
Server-side trace sampling. |
NEXT_PUBLIC_SENTRY_REPLAYS_SESSION_SAMPLE_RATE |
Browser replay sampling. |
NEXT_PUBLIC_SENTRY_REPLAYS_ON_ERROR_SAMPLE_RATE |
Browser replay-on-error sampling. |
SENTRY_AUTH_TOKEN |
Sentry upload/auth token. |
SENTRY_ORG |
Sentry organization slug. |
SENTRY_PROJECT |
Sentry project slug. |
NEXT_PUBLIC_AMPLITUDE_API_KEY |
Browser analytics key. |
OTEL_SERVICE_NAME |
OpenTelemetry service name. |
OTEL_EXPORTER_OTLP_ENDPOINT |
OTLP endpoint for telemetry export. |
OTEL_EXPORTER_OTLP_HEADERS |
Extra OTLP export headers. |
Pre-Launch Stability Checks
The deployment document defines a separate scheduled workflow named DWILAR | Pre-Launch Stability. It is designed to catch release regressions before launch by running smoke tests, a short load burst, and a CPU guard against a local FastAPI instance.
Workflow behavior
| Check | Script | Purpose |
|---|---|---|
| Backend smoke | scripts/lita_smoke_test.sh |
Exercises the backend using the SQLite fallback. |
| Frontend smoke | scripts/prelaunch_frontend_smoke_test.sh |
Starts the Next.js app and verifies the frontend health route can reach the backend. |
| Load and CPU guard | Part of the scheduled workflow | Surfaces runaway CPU spikes and stability regressions before launch. |
Local execution examples from the deployment guide
python -m pip install -r backend/requirements.txt
NEXT_PUBLIC_BACKEND_API_URL="https://app.joinlita.com" \
LITA_CPU_LIMIT=90 LITA_LOAD_REQUESTS=40 bash scripts/lita_smoke_test.sh
NEXT_PUBLIC_BACKEND_API_URL="https://app.joinlita.com" \
BACKEND_API_URL="https://app.joinlita.com" \
bash scripts/prelaunch_frontend_smoke_test.sh
Operational inputs used by the smoke checks
| Variable | Purpose |
|---|---|
NEXT_PUBLIC_BACKEND_API_URL |
Points the frontend smoke run at the backend URL under test. |
BACKEND_API_URL |
Backend URL used by the frontend smoke script. |
LITA_CPU_LIMIT |
CPU threshold used by the load guard. |
LITA_LOAD_REQUESTS |
Number of requests used in the short load burst. |
Pre-launch sequence
sequenceDiagram
participant Op as Operator
participant WF as GitHub Actions workflow
participant BE as FastAPI backend
participant FE as Next.js frontend
Op->>WF: Run DWILAR Pre Launch Stability
WF->>BE: Execute lita_smoke_test.sh
WF->>FE: Execute prelaunch_frontend_smoke_test.sh
WF->>WF: Run short load burst and CPU guard
WF-->>Op: Pass or fail result
Observability and Runtime Telemetry
The frontend exposes a small but explicit observability bootstrap through frontend/instrumentation.ts. That file selects a service name, registers OpenTelemetry, and loads Sentry configuration based on the runtime environment.
Telemetry bootstrap
frontend/instrumentation.ts
| Element | Type | Purpose |
|---|---|---|
serviceName |
constant | Uses OTEL_SERVICE_NAME or falls back to lita-frontend. |
register |
exported function | Registers telemetry and loads runtime-specific Sentry config. |
Public methods
| Method | Description |
|---|---|
register |
Registers OpenTelemetry and conditionally loads Sentry configuration for the current runtime. |
Runtime behavior
| Condition | Behavior |
|---|---|
process.env.NEXT_RUNTIME === "nodejs" |
Imports ./sentry.server.config. |
process.env.NEXT_RUNTIME === "edge" |
Imports ./sentry.edge.config. |
| Any runtime | Calls registerOTel({ serviceName }). |
Observability flow
sequenceDiagram
participant Next as Next.js runtime
participant Inst as instrumentation ts
participant OTel as OpenTelemetry
participant Sentry as Sentry config
Next->>Inst: register
Inst->>OTel: registerOTel serviceName
alt nodejs runtime
Inst->>Sentry: import sentry server config
else edge runtime
Inst->>Sentry: import sentry edge config
end
Operational logging surface
The deployment guide uses two log inspection paths after rollout:
gcloud compute ssh instance-20250206-052936
cd MVP_fastapi
docker-compose -f platform/deployment/prod/docker-compose.yaml logs backend
docker-compose -f platform/deployment/prod/docker-compose.yml logs frontend
These commands are the documented path for checking container logs after deployment and for verifying that the backend and frontend started cleanly on the production host.
Deployment Process
The documented release process is intentionally linear: run tests, publish images, update the database schema when required, and roll the stack on the Compute Engine host. The schema step is part of the deployment guide because new API endpoints may depend on fresh tables before they can serve traffic.
Release phases
| Phase | Outcome |
|---|---|
| Testing | Backend unit tests must pass before image publishing. |
| Publishing | Backend and frontend images are pushed to Artifact Registry. |
| Schema update | Long-lived databases are updated when the release introduces new tables. |
| Rollout | The production host fetches the new images and restarts the stack. |
| Verification | Operators confirm the application responds and logs are clean. |
Operational verification checklist
- Confirm the app is reachable at
https://app.joinlita.com. - Confirm client-side routes continue to work after deployment.
- Confirm
/api/...requests are routed to the backend through the production edge path. - Inspect GCP health checks and container logs if the rollout behaves unexpectedly.
Error Handling
The documented workflow fails fast at each stage instead of attempting a partial release. Test failures stop publishing, registry authentication failures prevent image pushes, and SSH rollout failures stop the remote script before the new stack can be considered live.
| Failure point | Result |
|---|---|
pytest tests/ -v fails |
Images are not published. |
| Artifact Registry auth fails | Build artifacts cannot be pushed or pulled. |
appleboy/ssh-action fails |
The Compute Engine rollout does not start. |
docker-compose pull fails |
The host cannot fetch the new release images. |
docker-compose up -d --build fails |
The production stack does not restart successfully. |
Dependencies
The deployment stack depends on these concrete tools and services from the repository documentation:
- GitHub Actions
docker/build-push-action@v4appleboy/[email protected]- Google Cloud Artifact Registry
- Google Compute Engine
gcloud auth configure-dockerdocker-composepytest- Node.js 20 on the frontend image
- OpenTelemetry and Sentry for frontend observability
Testing Considerations
The deployment and operations workflow is validated through the following concrete checks:
- Backend unit tests:
pytest tests/ -v - Scheduled pre-launch smoke workflow:
DWILAR | Pre-Launch Stability - Backend smoke script:
scripts/lita_smoke_test.sh - Frontend smoke script:
scripts/prelaunch_frontend_smoke_test.sh - Production rollout verification via
docker-compose logs - Runtime telemetry initialization via
frontend/instrumentation.ts
Key Classes Reference
| Class | Responsibility |
|---|---|
documents/deployment.md |
Canonical deployment, publishing, rollout, and verification workflow. |
frontend/Dockerfile |
Production multi-stage frontend image build. |
frontend/Dockerfile.local |
Local hot-reload frontend container. |
frontend/instrumentation.ts |
Frontend telemetry bootstrap for OpenTelemetry and Sentry. |
frontend/.env.local.example |
Frontend telemetry and runtime configuration surface. |