Build an OAuth-based data migration client
Build an OAuth-based Data Migration Client
Build a small command-line program that migrates records from a legacy HTTP API to a new HTTP API.
Required Steps
- Obtain an access token using OAuth 2.0 (Client Credentials flow).
- Call the legacy endpoint using the token to fetch data.
- Transform the data according to the rules below.
- Send the transformed data to the new endpoint.
- Compute and attach a checksum for each payload you send.
API Contracts (Assume these for the exercise)
1) Token Endpoint
POST https://auth.example.com/oauth/token- Body (JSON):
client_id(string)client_secret(string)grant_type="client_credentials"(string)
- Response (JSON):
access_token(string)expires_in(seconds, integer)
2) Legacy Data Endpoint
GET https://legacy.example.com/v1/records?page=<n>&page_size=<k>- Header:
Authorization: Bearer <access_token> - Response (JSON):
data: array of recordsnext_page: number ornull
Example Legacy Record:
{
"id": 123,
"first_name": "Ada",
"last_name": "Lovelace",
"email": "[email protected]",
"created_at_epoch": 1700000000,
"status": "ACTIVE"
}
3) New Destination Endpoint
POST https://new.example.com/v2/records- Headers:
Authorization: Bearer <access_token>Content-Type: application/jsonX-Checksum: <checksum>- (Optional but recommended)
Idempotency-Key: <stable-key>
- Body: One transformed record per request (batching can be justified).
Transformation Rules
Transform each legacy record into the following shape:
{
"legacy_id": 123,
"full_name": "Ada Lovelace",
"email": "[email protected]",
"created_at": "2023-11-14T22:13:20Z",
"is_active": true
}
Rules:
legacy_id = idfull_name = first_name + " " + last_name(trim extra spaces)email = lowercase(email)created_at = created_at_epochconverted to UTC ISO-8601is_active = (status == "ACTIVE")
Checksum Requirement
Compute X-Checksum as:
SHA-256of the exact UTF-8 bytes of the JSON request body you send.- Encode as lowercase hex.
(If you choose to canonicalize JSON to avoid key-order differences, explain your approach.)
Non-functional Requirements
- Handle pagination until
next_page == null. - Refresh/re-acquire the token when expired.
- Retry transient failures (e.g., timeouts, 429, 5xx) with exponential backoff.
- Log progress (counts of fetched/transformed/sent; failures with reasons).
- The program should be restartable without duplicating writes (idempotency strategy).
Deliverables
- A short design explanation (1–2 pages) describing the approach.
- Implementation in a language of your choice.
- Tests for transformation + checksum logic, and a strategy for API mocking.
AI interview prep
Core dilemma
Building a robust, fault-tolerant, and efficient data migration pipeline that ensures data correctness and completeness, balancing operational concerns like restartability and error handling with the practicalities of interacting with external APIs.
3 tradeoffs and 3 stress tests locked
Interview prep
Unlock the interviewer X-Ray rubric, tradeoff matrix, stress tests, clarifying questions, and key concepts.
Build an OAuth-based data migration client
Build an OAuth-based Data Migration Client
Build a small command-line program that migrates records from a legacy HTTP API to a new HTTP API.
Required Steps
- Obtain an access token using OAuth 2.0 (Client Credentials flow).
- Call the legacy endpoint using the token to fetch data.
- Transform the data according to the rules below.
- Send the transformed data to the new endpoint.
- Compute and attach a checksum for each payload you send.
API Contracts (Assume these for the exercise)
1) Token Endpoint
POST https://auth.example.com/oauth/token- Body (JSON):
client_id(string)client_secret(string)grant_type="client_credentials"(string)
- Response (JSON):
access_token(string)expires_in(seconds, integer)
2) Legacy Data Endpoint
GET https://legacy.example.com/v1/records?page=<n>&page_size=<k>- Header:
Authorization: Bearer <access_token> - Response (JSON):
data: array of recordsnext_page: number ornull
Example Legacy Record:
{
"id": 123,
"first_name": "Ada",
"last_name": "Lovelace",
"email": "[email protected]",
"created_at_epoch": 1700000000,
"status": "ACTIVE"
}
3) New Destination Endpoint
POST https://new.example.com/v2/records- Headers:
Authorization: Bearer <access_token>Content-Type: application/jsonX-Checksum: <checksum>- (Optional but recommended)
Idempotency-Key: <stable-key>
- Body: One transformed record per request (batching can be justified).
Transformation Rules
Transform each legacy record into the following shape:
{
"legacy_id": 123,
"full_name": "Ada Lovelace",
"email": "[email protected]",
"created_at": "2023-11-14T22:13:20Z",
"is_active": true
}
Rules:
legacy_id = idfull_name = first_name + " " + last_name(trim extra spaces)email = lowercase(email)created_at = created_at_epochconverted to UTC ISO-8601is_active = (status == "ACTIVE")
Checksum Requirement
Compute X-Checksum as:
SHA-256of the exact UTF-8 bytes of the JSON request body you send.- Encode as lowercase hex.
(If you choose to canonicalize JSON to avoid key-order differences, explain your approach.)
Non-functional Requirements
- Handle pagination until
next_page == null. - Refresh/re-acquire the token when expired.
- Retry transient failures (e.g., timeouts, 429, 5xx) with exponential backoff.
- Log progress (counts of fetched/transformed/sent; failures with reasons).
- The program should be restartable without duplicating writes (idempotency strategy).
Deliverables
- A short design explanation (1–2 pages) describing the approach.
- Implementation in a language of your choice.
- Tests for transformation + checksum logic, and a strategy for API mocking.