Engineering Reference, Workspace Memory, and Supporting Documents - Utility Scripts, Validation Helpers, and Local Data Generation Assets
Overview
This part of the repository contains the small command-line and test-only helpers used to create sample housing applications, clear application data from a database, and generate deterministic fixture payloads for local validation. The utilities are intentionally narrow: they either exercise the housing-application API with realistic values or build in-memory objects that drive scoring and endpoint tests.
The operational distinction is important. backend/src/scripts/create_test_application.py is a data-seeding helper that talks to the running backend and creates real application records. backend/src/scripts/clear_housing_applications.py is a destructive database maintenance script that removes all housing applications from the configured database. The remaining helpers are test fixtures and generators that stay inside the local test harness.
Script and Helper Map
| File | Role | Primary Effect | Safe for Local Use | Safe for Shared or Deployment Environments |
|---|---|---|---|---|
backend/src/scripts/create_test_application.py |
API seeding helper | Creates a test housing application, then reads it back and prints a compact JSON summary | Yes | Only with an intentionally targeted backend |
backend/src/scripts/clear_housing_applications.py |
Database cleanup helper | Deletes every housing application in the configured database | Yes, on disposable databases | No |
backend/tests/usecase/test_credit_scoring.py |
Test data builder | Builds a baseline HousingApplication for credit score tests |
Yes | N/A |
backend/tests/usecase/test_credit_score_runtime.py |
Test data builder | Builds a richer HousingApplication with Plaid-like connected accounts and transactions |
Yes | N/A |
backend/tests/usecase/test_dummy_account_detection.py |
Test data builder | Builds a HousingApplication for dummy-account heuristic tests |
Yes | N/A |
backend/tests/ml/test_pipeline_validation.py |
Validation fixture builders | Builds canonical bureau and banking objects for gating tests | Yes | N/A |
backend/tests/interface/test_application_endpoint.py |
Payload builder | Builds a realistic application payload for endpoint tests | Yes | N/A |
backend/tests/test_applicants.py |
Test harness helpers | Creates isolated SQLAlchemy engines, sessions, and FastAPI app instances for endpoint tests | Yes | N/A |
Execution Overview
flowchart LR
subgraph LocalScripts[Local Utility Scripts]
CreateTestApp[create_test_application py]
ClearApplications[clear_housing_applications py]
end
subgraph TestData[Local Test Data Builders]
CreditScoringBuilder[test_credit_scoring build_application]
RuntimeBuilder[test_credit_score_runtime build_application]
DummyAccountBuilder[test_dummy_account_detection build_application]
PipelineBuilders[test_pipeline_validation bureau and banking]
EndpointPayloadBuilder[test_application_endpoint build_payload]
end
subgraph BackendServices[Backend Services]
ApplicationsAPI[Housing applications API]
HousingRepo[SQLAlchemy housing application repository]
TestHarness[FastAPI and SQLite test harness]
end
CreateTestApp --> ApplicationsAPI
ApplicationsAPI --> CreateTestApp
ClearApplications --> HousingRepo
HousingRepo --> ClearApplications
CreditScoringBuilder --> TestHarness
RuntimeBuilder --> TestHarness
DummyAccountBuilder --> TestHarness
PipelineBuilders --> TestHarness
EndpointPayloadBuilder --> TestHarness
Utility Scripts
backend/src/scripts/create_test_application.py
[!WARNING]
backend/src/scripts/clear_housing_applications.pyperforms a full delete against the active database session. It is only appropriate for local, throwaway, or otherwise fully disposable databases.
File path: backend/src/scripts/create_test_application.py
This script creates a test housing application through the backend HTTP API and then fetches the created record by ID. It is designed to generate a realistic applicant payload, exercise backend validation and provisioning paths, and print a concise result that includes the application ID, resolved credit score, login credentials, and the detail URL.
Functions
| Method | Description |
|---|---|
_default_payload |
Builds the JSON body used for the initial application POST, including applicant identity, residency, housing preferences, income fields, policy consent, and a sample connected account. |
_build_headers |
Builds request headers for JSON submission and attaches Authorization: Bearer <token> when APPLICATIONS_API_TOKEN is present. |
_request_json |
Sends an HTTP request, decodes the JSON response, and raises a RuntimeError for transport failures, HTTP errors, or invalid JSON responses. |
main |
Parses CLI arguments, posts a test application, reads the created record back, and prints a JSON summary to stdout. |
CLI Arguments
| Argument | Description | Default |
|---|---|---|
--base-url |
Backend API base URL | APPLICATIONS_API_BASE_URL or http://localhost:8000/api |
--email |
Applicant email address | [email protected] |
--full-name |
Applicant full name | Test Applicant |
Environment Assumptions
| Variable | Purpose |
|---|---|
APPLICATIONS_API_BASE_URL |
Overrides the backend base URL used for the POST and GET calls |
APPLICATIONS_API_TOKEN |
Optional bearer token sent as an Authorization header |
What It Generates
- A fully populated application payload with:
- email, phone, country, property type, purchase intent, budget range, savings, income, currency, employment status
financing_consent,agree_policy,receive_updates- a sample
connected_accountsobject
- A printed JSON result containing:
application_idcredit_scorelogin_credentialsdetail_url
Runtime Behavior
The script first posts to the applications endpoint, then immediately fetches the created application by ID. The returned credit_score prefers the detail response and falls back to the creation response when needed. If the backend returns malformed JSON or the request fails, the script stops with a RuntimeError instead of continuing with partial data.
Safety Profile
- Safe locally: yes
- Safe in staging: only when the target backend is meant to receive seeded test applicants
- Safe in production: only if intentionally creating records in the live system
Sequence of Work
sequenceDiagram
participant O as Operator
participant S as main
participant A as Backend API
participant R as Request helper
O->>S: Run create_test_application py
S->>S: Build default payload and headers
S->>R: POST application request
R->>A: Submit applicant record
A-->>R: Created application JSON
R-->>S: Created application data
S->>R: GET application by id
R->>A: Fetch application detail
A-->>R: Application detail JSON
R-->>S: Detail response
S-->>O: Print summary JSON
[!NOTE] The backend create flow triggered by this script also performs downstream work visible in
src/interface/http_endpoints.py: it validates required fields, merges metadata into connected accounts, submits the housing application, attempts open-banking consent sync, attempts credential provisioning, and publishes anapplication_createdevent when the application has both an ID and timestamp.
backend/src/scripts/clear_housing_applications.py
File path: backend/src/scripts/clear_housing_applications.py
This script removes every housing application from the database configured for the backend session. It is a maintenance helper for resetting local or test data, not a selective cleanup tool.
Functions
| Method | Description |
|---|---|
_build_argument_parser |
Defines the --yes flag and the command description. |
_confirm_deletion |
Prompts for an interactive confirmation token and only proceeds when the response is exactly delete. |
main |
Opens a database session, constructs SQLAlchemyHousingApplicationRepository, deletes all housing applications, and prints the deletion count. |
CLI Arguments
| Argument | Description | Default |
|---|---|---|
--yes |
Skips the interactive confirmation prompt | False |
Environment Assumptions
| Dependency | Purpose |
|---|---|
SessionLocal |
Provides the SQLAlchemy session bound to the configured backend database |
SQLAlchemyHousingApplicationRepository |
Performs the actual full-table delete |
| Database configuration | Determines which persistent store is cleared |
What It Generates or Removes
- Removes all housing application records
- Prints
Deleted N housing application(s). - Emits
Aborted. No records were deleted.when confirmation is not provided
Runtime Behavior
The script prompts before deletion unless --yes is passed. If standard input is unavailable, _confirm_deletion returns False and the script exits without deleting anything. The database session is always closed in a finally block.
Safety Profile
- Safe locally: yes, only against disposable databases
- Safe in staging: no, unless the environment is intentionally resettable
- Safe in production: no
Sequence of Work
sequenceDiagram
participant O as Operator
participant S as main
participant P as confirm prompt
participant DB as Database session
participant R as SQLAlchemy repository
O->>S: Run clear_housing_applications py
S->>P: Ask for delete confirmation
P-->>S: delete or abort
alt confirmed
S->>DB: Open SessionLocal
S->>R: delete_all
R-->>S: Deleted count
S-->>O: Print deletion summary
else aborted
S-->>O: Print abort message
end
Validation Helpers and Local Data Generation Assets
backend/tests/usecase/test_credit_scoring.py
File path: backend/tests/usecase/test_credit_scoring.py
This helper builds a baseline HousingApplication object for heuristic credit scoring tests. It provides a predictable applicant profile with empty connected accounts by default, while still allowing overrides for targeted assertions.
| Method | Description |
|---|---|
_build_application |
Creates a HousingApplication with deterministic identity, financial, and review-state fields, then applies caller overrides. |
Generated Test Shape
purchase_intentdefaults tobuyconnected_accountsdefaults to an empty mappingreview_statusdefaults topendingcreated_atis initialized todatetime.now()- All primary application fields are populated so scoring logic can run without fixture assembly in each test
backend/tests/usecase/test_credit_score_runtime.py
File path: backend/tests/usecase/test_credit_score_runtime.py
This helper builds a richer HousingApplication for runtime scoring tests. It includes Plaid-like connected accounts and a small transaction history so the runtime scorer can exercise affordability, liquidity, salary detection, and breakdown generation paths.
| Method | Description |
|---|---|
_build_application |
Creates a HousingApplication with sample connected account data, balances, and transactions, then applies overrides. |
Generated Test Shape
- Includes one connected account under the
plaidkey - Includes sample checking balances
- Includes salary and rent-style transactions
- Sets
archived=Falseandreview_status="pending" - Supports contract-validation tests for Vertex AI responses
backend/tests/usecase/test_dummy_account_detection.py
File path: backend/tests/usecase/test_dummy_account_detection.py
This helper builds a HousingApplication used by dummy-account detection tests. It is tuned for heuristics that inspect repeated counterparties, balances, and account-holder name matching.
| Method | Description |
|---|---|
_build_application |
Creates a HousingApplication with optional full_name and connected_accounts overrides for dummy-account detection scenarios. |
Generated Test Shape
- Defaults
full_nametoTobias Andersen Yoshi - Uses
created_at=datetime.now(timezone.utc) - Accepts caller-provided connected account structures
- Keeps the rest of the housing application fields realistic for heuristic evaluation
backend/tests/ml/test_pipeline_validation.py
File path: backend/tests/ml/test_pipeline_validation.py
These helper constructors produce canonical ML inputs for pipeline validation. They are used to verify the gating rules around bureau availability, banking availability, coverage, and recency.
| Method | Description |
|---|---|
_banking |
Creates a CanonicalOpenBankingData object with a provider name, account count, coverage days, and optional transaction set. |
_bureau |
Creates a CanonicalBureauData object with score, recency, and optional fraud or AML flags. |
Generated Validation Shapes
_bankingdefaults to a Plaid-like provider and one synthetic salary transaction_bureaudefaults to a TransUnion-like bureau score with a recent report date- Both helpers are suitable for testing stale-report and missing-data gates
backend/tests/interface/test_application_endpoint.py
File path: backend/tests/interface/test_application_endpoint.py
This builder creates a realistic application payload for endpoint testing. It mirrors the payload structure used by the application create flow, including connected accounts.
| Method | Description |
|---|---|
build_payload |
Returns a dictionary that matches the application-create request shape used in endpoint tests. |
Generated Payload Shape
- email, phone, country, property type, purchase intent, budget range, savings, income, and currency
financing_consent,agree_policy,receive_updates- a
connected_accountsobject with a samplepersonalaccount entry
backend/tests/test_applicants.py
File path: backend/tests/test_applicants.py
These test harness helpers create isolated FastAPI and SQLAlchemy state for applicant endpoint tests. They are used to keep the consent-validation and applicant-management tests independent from the real backend database.
| Method | Description |
|---|---|
_get_test_engine |
Builds a single in-memory SQLite engine and enables foreign keys on connect. |
_create_session |
Drops and recreates all tables, then opens a fresh session for test isolation. |
_create_customer |
Inserts an active customer with a generated API key and returns both the model and plaintext key. |
_create_app |
Builds a FastAPI app, mounts the applicants router, registers the AuthError handler, and overrides get_db. |
Test Harness Behavior
- Uses SQLite in memory
- Recreates schema per test to avoid cross-test contamination
- Converts auth failures into
401responses withWWW-Authenticate: Bearer - Supports tests that verify consent-grant validation behavior
Local Use vs Deployment Use
| Asset | Local Use | Deployment Use |
|---|---|---|
create_test_application.py |
Good for seeding and smoke testing against local backend instances | Only when the target environment is intentionally being seeded |
clear_housing_applications.py |
Good for resetting disposable databases | Not appropriate unless data loss is intended |
| Test fixture builders | Good for deterministic unit and integration tests | Not used directly in deployment |
| FastAPI and SQLite test harness helpers | Good for isolated test runs | Not used directly in deployment |
Error Handling
The scripts and helpers use simple, explicit failure handling.
create_test_application.pyraisesRuntimeErrorwhen the request fails, the backend returns an HTTP error, or the response body is not valid JSON.clear_housing_applications.pyaborts cleanly when confirmation is not received and always closes its session in afinallyblock.- The test fixture helpers are intentionally narrow and rely on Python object construction rather than runtime recovery.
Dependencies
Runtime and Library Dependencies
| File | Dependency Surface |
|---|---|
backend/src/scripts/create_test_application.py |
argparse, json, os, urllib.request, urllib.error |
backend/src/scripts/clear_housing_applications.py |
argparse, SessionLocal, SQLAlchemyHousingApplicationRepository |
backend/tests/test_applicants.py |
FastAPI test client, SQLAlchemy engine/session helpers, AuthError handler |
backend/tests/ml/test_pipeline_validation.py |
Canonical ML schema objects |
Backend and Database Dependencies
- Application-create and read helpers depend on the running backend service.
- Database cleanup depends on the backend SQLAlchemy session configuration.
- Test harness helpers depend on SQLite or the configured test database.
Testing Considerations
| Scenario | Why It Matters |
|---|---|
Create helper with no APPLICATIONS_API_TOKEN |
Verifies the script can run against unauthenticated local backends when allowed |
| Create helper with invalid base URL | Verifies transport failure handling |
| Create helper with non-JSON response | Verifies the JSON decoding guard |
| Clear helper without confirmation | Verifies the destructive path is blocked |
Clear helper with --yes |
Verifies the non-interactive cleanup path |
| Fixture builders with override inputs | Verifies that test payloads remain adjustable without rewriting the builders |
| Pipeline validation builders | Verifies completeness, stale-report, and coverage gates with deterministic canonical data |
Key Classes Reference
| Class | Responsibility |
|---|---|
create_test_application.py |
Creates a test housing application through the backend API and prints a summary |
clear_housing_applications.py |
Deletes all housing applications from the configured database |
test_credit_scoring.py |
Builds a baseline HousingApplication test fixture |
test_credit_score_runtime.py |
Builds a runtime scoring fixture with connected accounts and transactions |
test_dummy_account_detection.py |
Builds a dummy-account detection fixture |
test_pipeline_validation.py |
Builds canonical banking and bureau validation fixtures |
test_application_endpoint.py |
Builds an application endpoint payload fixture |
test_applicants.py |
Creates isolated FastAPI and SQLAlchemy test harnesses |