Open Banking and Financial Data Domain - US Data Gateway Service and Consented Data Retrieval Boundary

Overview

The us-data-gateway Go service is the consented retrieval boundary for US financial data. It exposes the /api/v1/us/data/* read paths that the broader US API layer can route to, and it enforces token verification, consent status checks, and data-category gating before any provider data is returned.

The service is intentionally small and boundary-focused: main.go wires the HTTP server, auth.go verifies signed access claims and session revocation state, handlers.go enforces consent and category access, and provider.go abstracts the data source behind a narrow read-only interface. The current bootstrap always wires provider.MockProvider{}, so the runtime behavior is deterministic and test-friendly while still preserving the provider boundary.

Architecture Overview

flowchart TB
    subgraph USApiLayer[US API Service Map]
        USDataRoutes[api v1 us data routes]
    end

    subgraph Gateway[US Data Gateway Service]
        Main[cmd us data gateway main.go]
        Config[internal config Load]
        Auth[internal auth TokenVerifier Verify]
        Handlers[internal handlers Handler]
        Provider[internal provider Provider]
        MockProvider[internal provider MockProvider]
    end

    subgraph Infra[Infrastructure]
        Postgres[PostgreSQL consents table]
        Redis[Redis session hashes]
    end

    USDataRoutes --> Main
    Main --> Config
    Main --> Auth
    Main --> Handlers
    Main --> Provider
    Provider --> MockProvider
    Auth --> Redis
    Handlers --> Postgres
    Handlers --> Auth
    Handlers --> Provider

Module and Runtime Dependencies

go.mod

[!NOTE] cfg.Provider is loaded from environment, but main.go instantiates provider.MockProvider{} in both the mock case and the default branch. The provider selection setting does not change runtime behavior in this build.

services/us-data-gateway/go.mod

The module is a standalone Go service module named us-data-gateway.

Package Purpose
github.com/go-redis/redis/v8 Redis client used by auth.TokenVerifier
github.com/golang-jwt/jwt/v5 RS256 JWT parsing and public-key verification
github.com/jackc/pgx/v5 PostgreSQL connection pool used by the handlers

Service Bootstrap and Wiring

main.go

services/us-data-gateway/cmd/us-data-gateway/main.go

main() loads configuration, creates the PostgreSQL pool, creates the Redis client, parses the JWT public key, selects a provider implementation, and registers HTTP handlers on a http.ServeMux. It binds the consented data endpoints directly under /api/v1/us/data/* and exposes /health for liveness.

Public Functions

Function Description
main Boots the HTTP server and registers all routes
parsePublicKey Parses the JWT public key PEM into *rsa.PublicKey

Helper Functions

Function Description
init Sets REDIS_URL=localhost:6379 when the environment variable is unset

Wiring and Lifecycle

Step Behavior
1 config.Load() reads runtime settings
2 pgxpool.New(ctx, cfg.DatabaseURL) opens the consent database pool
3 redis.NewClient(&redis.Options{Addr: cfg.RedisURL}) creates the Redis client
4 parsePublicKey(cfg.JWTPublicKey) loads the RS256 verification key
5 handlers.Handler{...} is assembled with DB, Provider, and Auth
6 http.ListenAndServe(":"+cfg.Port, mux) starts the HTTP listener

config.go

[!NOTE] init() mutates the process environment before config.Load() runs by setting REDIS_URL to localhost:6379 when absent. Load() then reads the resulting environment value directly.

services/us-data-gateway/internal/config/config.go

Load() resolves the service settings from environment variables and explicit defaults.

Config

Property Type Description
Port string HTTP listen port, default 8084
DatabaseURL string PostgreSQL connection string
RedisURL string Redis address used by auth.TokenVerifier
JWTPublicKey string PEM-encoded RSA public key used for JWT verification
Provider string Provider selector read by main.go

Public Methods

Method Description
Load Builds a Config from the process environment

Helper Functions

Function Description
getEnv Reads an environment variable or falls back to the supplied default

auth.go

services/us-data-gateway/internal/auth/auth.go

TokenVerifier performs the first authorization gate. It verifies the bearer token with RS256, extracts claims, and checks Redis for revocation state under session:<jti> before the request is allowed to continue.

Claims

Property Type Description
ConsentID string JWT claim mapped from cid
Purpose string JWT claim mapped from pur
Categories []string JWT claim mapped from cat
jwt.RegisteredClaims jwt.RegisteredClaims Embedded standard JWT registered claims

TokenVerifier

Property Type Description
PublicKey *rsa.PublicKey RS256 verification key supplied by main.go
Redis *redis.Client Redis session store used to check revocation

Public Methods

Method Description
Verify Parses and validates the JWT, then checks the Redis session record for revocation

Verification Flow

Stage Behavior
JWT parse Uses jwt.ParseWithClaims and rejects non-RS256 tokens
Claim extraction Requires a valid Claims payload and a valid token
Redis lookup Reads HGetAll on session:<claims.ID>
Revocation check Rejects the token when session["revoked"] == "true"

Consented Data Retrieval Handlers

handlers.go

[!NOTE] TokenVerifier.Verify returns plain errors for invalid signatures, invalid payloads, revoked sessions, and Redis failures. handlers.writeError preserves status codes only for *apiError, so these verification failures are surfaced as 500 INTERNAL_ERROR rather than a dedicated 403 or 409 response.

services/us-data-gateway/internal/handlers/handlers.go

Handler is the consented read boundary. Every data endpoint first calls authorize, then applies category checks from the JWT claims, then validates the consent row in PostgreSQL, and only then calls the provider.

Handler

Property Type Description
DB *pgxpool.Pool PostgreSQL pool used for consent lookup
Provider provider.Provider Data-source abstraction used for account, balance, transaction, and income retrieval
Auth auth.TokenVerifier JWT and Redis-based authorization gate

consentRecord

Property Type Description
ID string Consent identifier loaded from PostgreSQL
Status string Current consent status
ExpiresAt time.Time Consent expiration timestamp
Categories []string Consent categories loaded from PostgreSQL

apiError

Property Type Description
Status int HTTP status code emitted by writeError
Code string Error code serialized to the response body

Public Methods

Method Description
Accounts Returns consent-scoped accounts
Balances Returns consent-scoped balances
Transactions Returns consent-scoped transactions filtered by optional date range
TransactionsSync Returns consent-scoped transaction sync deltas using an optional cursor
Income Returns consent-scoped income observations with an optional lookback window

Internal Methods and Helpers

Method Description
ensureConsentActive Loads the consent row and rejects inactive or expired consent
authorize Extracts the bearer token and delegates verification to Auth.Verify
writeError Serializes apiError values or falls back to 500 INTERNAL_ERROR
writeJSON Sets Content-Type: application/json, writes the status code, and encodes the payload
contains Performs exact string membership checks against the JWT category list

Endpoint Enforcement Matrix

Endpoint Required JWT Category Consent Check Provider Call
Accounts ACCOUNTS ensureConsentActive FetchAccounts
Balances BALANCES ensureConsentActive FetchBalances
Transactions TRANSACTIONS ensureConsentActive FetchTransactions
TransactionsSync TRANSACTIONS ensureConsentActive FetchTransactionsSync
Income INCOME ensureConsentActive FetchIncome

Runtime Flow

Step Behavior
1 authorize requires an Authorization: Bearer ... header
2 TokenVerifier.Verify validates the JWT and Redis session state
3 The handler logs request_id and consent_id with log.Printf
4 Category membership is checked against claims.Categories
5 ensureConsentActive queries PostgreSQL for consent state and expiration
6 The provider method is called with the consent ID and request-specific parameters
7 writeJSON returns the provider result as JSON

The handlers check category membership before consent validity. A request with the wrong category is rejected even if the consent is also expired or inactive.

handlers_test.go

services/us-data-gateway/internal/handlers/handlers_test.go

These tests validate the error serialization boundary for the handler helpers.

Test Behavior Covered
TestCategoryNotAllowedReturns403 Verifies ErrCategoryNotAllowed serializes as HTTP 403 with CATEGORY_NOT_ALLOWED
TestConsentRevokedReturns409 Verifies ErrConsentRevoked serializes as HTTP 409 with CONSENT_REVOKED

Provider Abstraction

provider.go

services/us-data-gateway/internal/provider/provider.go

provider.Provider defines the service’s read-only data access contract. The interface is shaped around consent-scoped retrieval and sync support.

Account

Property Type Description
AccountToken string Stable account token
Name string Account display name
Type string Account type
Mask string Account mask

Balance

Property Type Description
AccountToken string Associated account token
Available float64 Available balance
Current float64 Current balance
Currency string ISO currency code

Transaction

Property Type Description
TransactionID string Transaction identifier
AccountToken string Associated account token
Amount float64 Transaction amount
Currency string ISO currency code
PostedAt string Posting timestamp
MerchantName string Merchant name
MerchantCategory string Merchant category

TransactionRemoval

Property Type Description
TransactionID string Identifier of a removed transaction

TransactionSync

Property Type Description
Added []Transaction Newly added transactions
Modified []Transaction Modified transactions
Removed []TransactionRemoval Removed transaction identifiers
NextCursor string Cursor for the next sync page

Income

Property Type Description
IncomeID string Income observation identifier
Amount float64 Income amount
Currency string ISO currency code
PayDate string Payment date

Public Methods

Method Description
FetchAccounts Returns consent-scoped accounts
FetchBalances Returns consent-scoped balances
FetchTransactions Returns transactions within an optional date range
FetchTransactionsSync Returns a sync delta with added, modified, and removed items
FetchIncome Returns income observations for the requested lookback window

mock.go

services/us-data-gateway/internal/provider/mock.go

MockProvider is the current runtime provider implementation. It returns deterministic data for every method and does not vary results by consentID, fromDate, toDate, cursor, or lookbackDays.

MockProvider

Property Type Description
struct{} struct{} Stateless provider implementation

Public Methods

Method Description
FetchAccounts Returns two deterministic deposit accounts
FetchBalances Returns balances for the deterministic accounts
FetchTransactions Returns two deterministic transactions
FetchTransactionsSync Returns a deterministic sync delta with one added, one modified, and one removed transaction
FetchIncome Returns two deterministic income observations

Infrastructure Services

Handler.ensureConsentActive reads from PostgreSQL through *pgxpool.Pool. The query targets the consents table and reads id, status, expires_at, and categories.

Data Flow

Step Behavior
1 A handler receives a JWT-authenticated request
2 ensureConsentActive queries consents by consent ID
3 The handler rejects any row where status != "ACTIVE"
4 The handler rejects any row where time.Now() is after expires_at

Query Shape

SELECT id, status, expires_at, categories
FROM consents
WHERE id=$1

Redis Session Store

TokenVerifier.Verify uses Redis as the session revocation boundary.

Data Flow

Step Behavior
1 JWT verification extracts claims.ID
2 Redis is queried with HGetAll("session:<jti>")
3 The token is rejected if revoked == "true"

Session Key Pattern

Key Pattern Purpose
session:<claims.ID> Stores token-session fields used for revocation checks

Session Fields Used

Field Meaning
revoked When set to true, the token is rejected

Request Logging

The service writes operational logs directly with the standard library log package.

Logged Events

Location Log Event
Handler.Accounts request_id, consent_id, action=accounts
Handler.Balances request_id, consent_id, action=balances
Handler.Transactions request_id, consent_id, action=transactions
Handler.TransactionsSync request_id, consent_id, action=transactions_sync
Handler.Income request_id, consent_id, action=income
main.go startup message and listen-failure logging

API Integration

GET /api/v1/us/data/accounts

Get Accounts

{
    "title": "Get Accounts",
    "description": "Returns consent-scoped accounts after Bearer JWT verification, Redis revocation checking, active-consent validation, and ACCOUNTS category enforcement.",
    "method": "GET",
    "baseUrl": "<USDataGatewayBaseUrl>",
    "endpoint": "/api/v1/us/data/accounts",
    "headers": [
        {
            "key": "Authorization",
            "value": "Bearer <access_token>",
            "required": true
        }
    ],
    "queryParams": [],
    "pathParams": [],
    "bodyType": "none",
    "requestBody": "",
    "formData": [],
    "rawBody": "",
    "responses": {
        "200": {
            "description": "Success",
            "body": "{\n    \"accounts\": [\n        {\n            \"account_token\": \"acct_tok_1\",\n            \"name\": \"Checking\",\n            \"type\": \"depository\",\n            \"mask\": \"1234\"\n        }\n    ]\n}"
        },
        "403": {
            "description": "Missing bearer token or disallowed category",
            "body": "{\n    \"error\": \"TOKEN_REQUIRED\"\n}"
        },
        "409": {
            "description": "Consent revoked or expired",
            "body": "{\n    \"error\": \"CONSENT_REVOKED\"\n}"
        },
        "500": {
            "description": "Verification, consent lookup, or provider failure",
            "body": "{\n    \"error\": \"INTERNAL_ERROR\"\n}"
        }
    }
}

GET /api/v1/us/data/balances

Get Balances

{
    "title": "Get Balances",
    "description": "Returns consent-scoped balances after Bearer JWT verification, Redis revocation checking, active-consent validation, and BALANCES category enforcement.",
    "method": "GET",
    "baseUrl": "<USDataGatewayBaseUrl>",
    "endpoint": "/api/v1/us/data/balances",
    "headers": [
        {
            "key": "Authorization",
            "value": "Bearer <access_token>",
            "required": true
        }
    ],
    "queryParams": [],
    "pathParams": [],
    "bodyType": "none",
    "requestBody": "",
    "formData": [],
    "rawBody": "",
    "responses": {
        "200": {
            "description": "Success",
            "body": "{\n    \"balances\": [\n        {\n            \"account_token\": \"acct_tok_1\",\n            \"available\": 1200,\n            \"current\": 1500,\n            \"currency\": \"USD\"\n        }\n    ]\n}"
        },
        "403": {
            "description": "Missing bearer token or disallowed category",
            "body": "{\n    \"error\": \"TOKEN_REQUIRED\"\n}"
        },
        "409": {
            "description": "Consent revoked or expired",
            "body": "{\n    \"error\": \"CONSENT_REVOKED\"\n}"
        },
        "500": {
            "description": "Verification, consent lookup, or provider failure",
            "body": "{\n    \"error\": \"INTERNAL_ERROR\"\n}"
        }
    }
}

GET /api/v1/us/data/transactions

Get Transactions

{
    "title": "Get Transactions",
    "description": "Returns consent-scoped transactions after Bearer JWT verification, Redis revocation checking, active-consent validation, and TRANSACTIONS category enforcement. The optional from and to query parameters are passed through to the provider.",
    "method": "GET",
    "baseUrl": "<USDataGatewayBaseUrl>",
    "endpoint": "/api/v1/us/data/transactions",
    "headers": [
        {
            "key": "Authorization",
            "value": "Bearer <access_token>",
            "required": true
        }
    ],
    "queryParams": [
        {
            "key": "from",
            "value": "2024-10-01",
            "required": false
        },
        {
            "key": "to",
            "value": "2024-10-31",
            "required": false
        }
    ],
    "pathParams": [],
    "bodyType": "none",
    "requestBody": "",
    "formData": [],
    "rawBody": "",
    "responses": {
        "200": {
            "description": "Success",
            "body": "{\n    \"transactions\": [\n        {\n            \"transaction_id\": \"txn_1\",\n            \"account_token\": \"acct_tok_1\",\n            \"amount\": 1200,\n            \"currency\": \"USD\",\n            \"posted_at\": \"2024-10-01T00:00:00Z\",\n            \"merchant_name\": \"Rent Co\",\n            \"merchant_category\": \"RENT\"\n        }\n    ]\n}"
        },
        "403": {
            "description": "Missing bearer token or disallowed category",
            "body": "{\n    \"error\": \"TOKEN_REQUIRED\"\n}"
        },
        "409": {
            "description": "Consent revoked or expired",
            "body": "{\n    \"error\": \"CONSENT_REVOKED\"\n}"
        },
        "500": {
            "description": "Verification, consent lookup, or provider failure",
            "body": "{\n    \"error\": \"INTERNAL_ERROR\"\n}"
        }
    }
}

GET /api/v1/us/data/transactions/sync

Get Transaction Sync

{
    "title": "Get Transaction Sync",
    "description": "Returns consent-scoped transaction sync data after Bearer JWT verification, Redis revocation checking, active-consent validation, and TRANSACTIONS category enforcement. The optional cursor query parameter is passed through to the provider.",
    "method": "GET",
    "baseUrl": "<USDataGatewayBaseUrl>",
    "endpoint": "/api/v1/us/data/transactions/sync",
    "headers": [
        {
            "key": "Authorization",
            "value": "Bearer <access_token>",
            "required": true
        }
    ],
    "queryParams": [
        {
            "key": "cursor",
            "value": "cursor_next_1",
            "required": false
        }
    ],
    "pathParams": [],
    "bodyType": "none",
    "requestBody": "",
    "formData": [],
    "rawBody": "",
    "responses": {
        "200": {
            "description": "Success",
            "body": "{\n    \"added\": [\n        {\n            \"transaction_id\": \"txn_3\",\n            \"account_token\": \"acct_tok_2\",\n            \"amount\": 45,\n            \"currency\": \"USD\",\n            \"posted_at\": \"2024-10-03T00:00:00Z\",\n            \"merchant_name\": \"Coffee\",\n            \"merchant_category\": \"FOOD\"\n        }\n    ],\n    \"modified\": [\n        {\n            \"transaction_id\": \"txn_2\",\n            \"account_token\": \"acct_tok_1\",\n            \"amount\": 90,\n            \"currency\": \"USD\",\n            \"posted_at\": \"2024-10-02T00:00:00Z\",\n            \"merchant_name\": \"Card\",\n            \"merchant_category\": \"DEBT\"\n        }\n    ],\n    \"removed\": [\n        {\n            \"transaction_id\": \"txn_4\"\n        }\n    ],\n    \"next_cursor\": \"cursor_next_1\"\n}"
        },
        "403": {
            "description": "Missing bearer token or disallowed category",
            "body": "{\n    \"error\": \"TOKEN_REQUIRED\"\n}"
        },
        "409": {
            "description": "Consent revoked or expired",
            "body": "{\n    \"error\": \"CONSENT_REVOKED\"\n}"
        },
        "500": {
            "description": "Verification, consent lookup, or provider failure",
            "body": "{\n    \"error\": \"INTERNAL_ERROR\"\n}"
        }
    }
}

GET /api/v1/us/data/income

Get Income

{
    "title": "Get Income",
    "description": "Returns consent-scoped income observations after Bearer JWT verification, Redis revocation checking, active-consent validation, and INCOME category enforcement. The optional lookback_days query parameter is parsed as an integer and defaults to 180 when omitted or invalid.",
    "method": "GET",
    "baseUrl": "<USDataGatewayBaseUrl>",
    "endpoint": "/api/v1/us/data/income",
    "headers": [
        {
            "key": "Authorization",
            "value": "Bearer <access_token>",
            "required": true
        }
    ],
    "queryParams": [
        {
            "key": "lookback_days",
            "value": "180",
            "required": false
        }
    ],
    "pathParams": [],
    "bodyType": "none",
    "requestBody": "",
    "formData": [],
    "rawBody": "",
    "responses": {
        "200": {
            "description": "Success",
            "body": "{\n    \"income\": [\n        {\n            \"income_id\": \"inc_1\",\n            \"amount\": 5100,\n            \"currency\": \"USD\",\n            \"pay_date\": \"2024-09-30T00:00:00Z\"\n        }\n    ]\n}"
        },
        "403": {
            "description": "Missing bearer token or disallowed category",
            "body": "{\n    \"error\": \"TOKEN_REQUIRED\"\n}"
        },
        "409": {
            "description": "Consent revoked or expired",
            "body": "{\n    \"error\": \"CONSENT_REVOKED\"\n}"
        },
        "500": {
            "description": "Verification, consent lookup, or provider failure",
            "body": "{\n    \"error\": \"INTERNAL_ERROR\"\n}"
        }
    }
}

GET /health

Health Check

{
    "title": "Health Check",
    "description": "Returns a plain-text liveness response for the gateway process.",
    "method": "GET",
    "baseUrl": "<USDataGatewayBaseUrl>",
    "endpoint": "/health",
    "headers": [],
    "queryParams": [],
    "pathParams": [],
    "bodyType": "none",
    "requestBody": "",
    "formData": [],
    "rawBody": "",
    "responses": {
        "200": {
            "description": "Success",
            "body": "ok"
        }
    }
}

Feature Flows

Consented Data Retrieval

This is the primary read path for the gateway. The same gate is used by the account, balance, transaction, sync, and income endpoints, with the provider method varying by route.

sequenceDiagram
    participant C as Client
    participant H as Handler
    participant A as authorize
    participant V as TokenVerifier
    participant R as Redis
    participant D as PostgreSQL
    participant P as MockProvider

    C->>H: GET /api/v1/us/data/accounts
    H->>A: authorize
    A->>V: Verify
    V->>R: HGetAll session jti
    R-->>V: session hash
    V-->>A: claims
    A-->>H: claims
    H->>D: SELECT consent row
    D-->>H: consent record
    H->>P: FetchAccounts
    P-->>H: accounts
    H-->>C: JSON response

Service Startup

The bootstrap flow wires the service once at process start.

Step Behavior
1 main() calls config.Load()
2 pgxpool.New connects to PostgreSQL
3 redis.NewClient connects to Redis
4 parsePublicKey loads the JWT verification key
5 Handler is built with DB, Provider, and Auth
6 Routes are attached to http.NewServeMux()
7 ListenAndServe starts the listener

Error Handling

handlers.writeError is the response boundary for all handler failures. It preserves *apiError status codes and emits JSON payloads with an error field, while every other error falls back to 500 INTERNAL_ERROR.

Source HTTP Status Error Code
Missing bearer token 403 TOKEN_REQUIRED
Category rejected by contains 403 CATEGORY_NOT_ALLOWED
Consent inactive or expired 409 CONSENT_REVOKED
Unhandled verification, database, or provider error 500 INTERNAL_ERROR

Data Modeling Notes

JWT Claim Shape

[!NOTE] TokenVerifier.Verify returns ordinary errors for invalid JWTs and Redis failures, so those failures do not preserve the auth-specific status codes. The handler layer only keeps the explicit apiError values.

Claims expects the following JSON claim keys:

JSON Key Go Field
cid ConsentID
pur Purpose
cat Categories

Provider Response Shapes

The JSON response keys emitted by writeJSON are the exact lower-snake-case tags from the provider types.

Type Serialized Shape
Account account_token, name, type, mask
Balance account_token, available, current, currency
Transaction transaction_id, account_token, amount, currency, posted_at, merchant_name, merchant_category
TransactionSync added, modified, removed, next_cursor
Income income_id, amount, currency, pay_date

Testing Considerations

The provided tests focus on the handler error boundary rather than live provider or database behavior.

Scenario Coverage Location Result
Category rejection handlers_test.go 403 CATEGORY_NOT_ALLOWED
Consent revocation handlers_test.go 409 CONSENT_REVOKED

Additional behaviors that are directly encoded in the handlers and should be preserved by future tests:

  • Missing Authorization: Bearer ... returns 403 TOKEN_REQUIRED.
  • Invalid JWTs and Redis failures are surfaced by writeError as 500 INTERNAL_ERROR.
  • Transactions and Income pass optional query values through without additional validation.
  • TransactionsSync returns the provider payload directly, without a wrapper object.

Dependencies

Runtime Services

Dependency Role
PostgreSQL Consent state lookup in Handler.ensureConsentActive
Redis JWT session revocation check in TokenVerifier.Verify
Standard library log Operational request and startup logging

Application Boundary Dependencies

Dependency Role
auth.TokenVerifier Authentication and revocation gate
provider.Provider Consent-scoped data retrieval abstraction
config.Load Runtime configuration assembly

Key Classes Reference

Class Responsibility
main.go Boots the gateway, wires dependencies, and registers /api/v1/us/data/* routes
config.go Loads runtime configuration from environment
auth.go Verifies JWT bearer tokens and checks Redis revocation state
handlers.go Enforces consent and category checks, then returns data payloads
provider.go Defines the read-only data retrieval contract and response models
mock.go Supplies deterministic provider data for the current runtime wiring
handlers_test.go Verifies handler error serialization for category and consent failures