feat(ingestion): add Workday HR connector (RaaS report contract, BambooHR parity) by mitasovr · Pull Request #1305 · constructorfabric/insight

mitasovr · 2026-06-11T19:24:59Z

Summary

Adds the Workday HR connector — full parity with the BambooHR pattern: current-state extraction, SCD2 history at the Silver layer, identity feed.

Workday has no fixed bulk-extraction API, so the connector defines a report contract instead: the customer builds two RaaS custom reports (Insight_Employee_Sync, Insight_Leave_Sync) with exact column XML aliases and From_Date/To_Date prompts, shares them with an ISU, and the connector fetches them as JSON. Extra customer columns flow into raw_data without connector changes.

Connector (`src/ingestion/connectors/hr-directory/workday/`)

Declarative manifest (CDK v7.0.4), ISU Basic auth, streams workers + leave_requests (full refresh), Report_Entry extraction, unique_key/tenant_id/source_id/raw_data injection
descriptor.yaml (semver 1.0.0, bronze_workday, tag:workday+), Secret example, README with the full report contract + ISU domain checklist

dbt (mirrors bamboohr model-for-model)

workday__bronze_promoted → RMT promotion
workday__workers_snapshot → SCD2 (Last_Functionally_Updated deliberately untracked to avoid spurious versions)
workday__workers_fields_history → per-field change log
workday__identity_inputs → identity observations; DELETE on Worker_Status='Terminated'
workday__to_class_people / workday__hr_events / workday__working_hours staging views

Specs

docs/components/connectors/hr-directory/workday/specs/{PRD,DESIGN}.md — cpt validate PASS, registered in cypilot/config/artifacts.toml (also fixed the stale workday.md registry path)
workday.md rewritten as the v1 overview (previous draft described a different, effective-dated design)

Local test rig (no public Workday dev tenant exists)

fixtures/ — RaaS-shaped fixtures + mock_raas.py enforcing Basic auth, format=json, and leave prompts

Test plan

source.sh validate — Manifest is valid
source.sh check against mock — SUCCEEDED
source.sh read — 5 workers / 4 leave_requests, 0 errors; all records carry tenant_id/source_id/unique_key; non-contract column (Cost_Center) lands in raw_data; second read deterministic
dbt parse + dbt ls -s tag:workday — all 7 models + tests resolve
cpt validate on PRD/DESIGN — PASS
Customer Sandbox validation at onboarding (no tenant available pre-onboarding by design)

Note: the manifest passes the repaired validate-strict gate from #1311 (pinned CDK 6.60.9 + $ref resolution) — verified locally: "Manifest is strictly valid (Builder-UI compatible)".

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- New Workday HR Directory connector for extracting worker and leave-request data (full-refresh) with Basic auth, two streams, and end-to-end pipeline support into staging/silver outputs.
Documentation
- Added PRD, design spec, setup README, onboarding notes, example secret manifest, test fixtures, and a local mock server for development and validation.

…ooHR parity) Add the Workday connector following the BambooHR pattern end to end: - Declarative manifest (CDK v7.0.4): two full-refresh streams (workers, leave_requests) fetched from customer-built RaaS custom reports with ISU Basic auth. Workday has no fixed bulk API, so the connector ships a report contract (exact column XML aliases + From_Date/To_Date prompts); extra report columns flow into raw_data. - dbt chain mirroring bamboohr model-for-model: RMT promotion, SCD2 snapshot (Last_Functionally_Updated deliberately untracked), field-level history, identity_inputs (DELETE on Worker_Status=Terminated), and class_people / class_hr_events / class_hr_working_hours staging views. - Specs: PRD + DESIGN (cpt validate PASS, registered in artifacts.toml); workday.md rewritten as the v1 overview. - Local test rig: RaaS fixtures + mock server enforcing Basic auth, format=json, and leave prompts. Verified: check SUCCEEDED, read returns 5 workers / 4 leave requests with 0 errors, raw_data passthrough of non-contract columns, deterministic full-refresh re-read. There is no public Workday developer tenant; real-tenant validation runs against the customer Sandbox during onboarding (check gates the report contract). Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

coderabbitai · 2026-06-11T19:26:32Z

📝 Walkthrough

Walkthrough

This PR implements a complete Workday HR Directory connector, migrating from a v1 specification into a v2 system. It includes comprehensive PRD/DESIGN documentation, an Airbyte declarative manifest with two streams (workers and leave requests), a dbt Bronze→Silver transformation pipeline, testing fixtures with a mock RaaS server, and K8s Secret configuration for deployment.

Changes

Workday HR Connector

Layer / File(s)	Summary
Requirements & Technical Design `cypilot/config/artifacts.toml`, `docs/components/connectors/hr-directory/workday/specs/PRD.md`, `docs/components/connectors/hr-directory/workday/specs/DESIGN.md`	PRD (414 lines) specifies scope, functional/non-functional requirements, deduplication keys, and fault-tolerance rules. DESIGN.md (482 lines) details technical architecture: two RaaS streams (workers, leave_requests) with ISU Basic auth, report-contract-driven schema, SCD2/field-history downstream chain, and deployment topology. artifacts.toml migrates from single `workday.md` to dual `specs/PRD.md` + `specs/DESIGN.md` entries.
Setup Guide & Configuration `docs/components/connectors/hr-directory/workday/workday.md`, `src/ingestion/connectors/hr-directory/workday/README.md`, `src/ingestion/connectors/hr-directory/workday/descriptor.yaml`, `src/ingestion/secrets/connectors/workday.yaml.example`	workday.md rewritten as v2 overview (52/-138 lines): condenses Bronze/Silver pipeline into one page, updates identity resolution to `workers.Work_Email` → `person_id`, and refreshes testing/Open Questions sections. README.md (149 lines) documents RaaS custom report contracts (workers and leave with required column aliases and prompts), K8s Secret configuration (base URL, ISU credentials, report paths), local mock-server workflow, and onboarding against Sandbox tenant. descriptor.yaml (14 lines) and K8s secret example (17 lines) provide deployment metadata and secret schema.
Airbyte Connector Manifest & Orchestration `src/ingestion/connectors/hr-directory/workday/connector.yaml`	connector.yaml (376 lines, v7.0.4) declares two declarative streams: workers and leave_requests; both extract from Report_Entry, inject tenant_id/source_id/unique_key via AddFields, enforce unique_key in schemas; leave_requests adds date-range prompts and default `workday_start_date=2020-01-01`; top-level concurrency default is 1. Connection spec requires tenant/source IDs, base URL, ISU credentials, and report paths; streams enable metadata.autoImportSchema.
Testing Infrastructure & Fixtures `src/ingestion/connectors/hr-directory/workday/fixtures/mock_raas.py`, `src/ingestion/connectors/hr-directory/workday/fixtures/workers.json`, `src/ingestion/connectors/hr-directory/workday/fixtures/leave_requests.json`	mock_raas.py (95 lines) implements a standalone HTTP server that enforces ISU Basic auth header, requires `format=json` and leave date prompts, routes by report name, and serves fixture JSON. workers.json (119 lines) and leave_requests.json (48 lines) provide realistic Workday RaaS response structures for testing.
dbt Foundation: Sources & Bronze Layer `src/ingestion/connectors/hr-directory/workday/dbt/schema.yml`, `src/ingestion/connectors/hr-directory/workday/dbt/workday__workers_snapshot.sql`, `src/ingestion/connectors/hr-directory/workday/dbt/workday__bronze_promoted.sql`	schema.yml (214 lines) declares Bronze sources and dbt model documentation with column-level tests. workers_snapshot.sql (25 lines) configures SCD2 append-only snapshot using unique_key and tracked worker fields (excluding Last_Functionally_Updated). bronze_promoted.sql (24 lines) invokes promote_bronze_to_rmt for both tables and returns a marker row.
dbt Silver Layer: History & Identity `src/ingestion/connectors/hr-directory/workday/dbt/workday__workers_fields_history.sql`, `src/ingestion/connectors/hr-directory/workday/dbt/workday__identity_inputs.sql`	workers_fields_history.sql (21 lines) applies the fields_history macro to workers_snapshot tracking named worker fields and custom fields. identity_inputs.sql (24 lines, incremental append) invokes identity_inputs_from_history mapping worker history fields to identity input pairs with termination deactivation logic.
dbt Silver Layer: People, Events & Working Hours `src/ingestion/connectors/hr-directory/workday/dbt/workday__to_class_people.sql`, `src/ingestion/connectors/hr-directory/workday/dbt/workday__hr_events.sql`, `src/ingestion/connectors/hr-directory/workday/dbt/workday__working_hours.sql`	to_class_people.sql (58 lines) transforms workers into class_people SCD2 records, derives valid_from from Last_Functionally_Updated, normalizes status/employment_type, and builds custom attributes. hr_events.sql (32 lines) converts leave_requests into HR events with parsed dates/duration and worker email join. working_hours.sql (27 lines) computes working hours per-week/day with a 40-hour/week fallback.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

🐰 A connector born from Workday's RaaS,
With PRDs, dbt, and fixtures to pass—
Bronze snapshots flowing to Silver so bright,
Identity chains holding data just right! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the main change: addition of a Workday HR connector with RaaS report contract and BambooHR parity. It is specific, concise, and directly related to the changeset.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/components/connectors/hr-directory/workday/specs/DESIGN.md`:
- Around line 414-428: The code block showing the Connection topology in
DESIGN.md is an unlabeled fenced block (the topology fence) which triggers
MD040; update the fence to include a language tag by changing the opening ``` to
```text so the block becomes a labeled "text" fence; locate the topology block
beginning with "Connection: workday-{source-id}" in
docs/components/connectors/hr-directory/workday/specs/DESIGN.md and add the
language tag to the opening fence to resolve the lint and improve parsing.

In `@src/ingestion/connectors/hr-directory/workday/connector.yaml`:
- Around line 257-273: The leave_requests stream is dropping the full report
payload because the transformations only add tenant_id, source_id, and
unique_key; add a field to forward the entire raw payload (e.g., an
AddedFieldDefinition with path [raw_data] and value set to the whole record) in
the same transformations block so the stream preserves all original columns
outside the inline schema; update the transformations list (the AddFields entry
that contains AddedFieldDefinition items) to include this raw_data field
alongside tenant_id, source_id, and unique_key.

In `@src/ingestion/connectors/hr-directory/workday/dbt/workday__hr_events.sql`:
- Around line 26-29: The LEFT JOIN against {{ source('workday', 'workers') }}
(alias w) is missing the connector instance key and can mix identities; update
the join condition in the workday__hr_events.sql join where lr.Employee_ID =
w.Employee_ID AND lr.tenant_id = w.tenant_id to also include lr.source_id =
w.source_id so the join keys are (Employee_ID, tenant_id, source_id) to prevent
cross-instance mixing and duplicate/incorrect rows.

In `@src/ingestion/connectors/hr-directory/workday/fixtures/mock_raas.py`:
- Around line 57-59: The mock currently accepts any Authorization header; update
the authentication check in the handler to require HTTP Basic credentials by
validating that self.headers contains "Authorization" and that its value starts
with "Basic " (reject otherwise), and call self._reply(401, ...) when missing or
not Basic so Bearer or other schemes no longer pass; locate the check around
self.headers and the self._reply call and replace the loose presence check with
this scheme validation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e72811f6-ce80-454a-a9e9-433c2ffbdab5

📥 Commits

Reviewing files that changed from the base of the PR and between a51eaa4 and 20a5afd.

📒 Files selected for processing (19)

cypilot/config/artifacts.toml
docs/components/connectors/hr-directory/workday/specs/DESIGN.md
docs/components/connectors/hr-directory/workday/specs/PRD.md
docs/components/connectors/hr-directory/workday/workday.md
src/ingestion/connectors/hr-directory/workday/README.md
src/ingestion/connectors/hr-directory/workday/connector.yaml
src/ingestion/connectors/hr-directory/workday/dbt/schema.yml
src/ingestion/connectors/hr-directory/workday/dbt/workday__bronze_promoted.sql
src/ingestion/connectors/hr-directory/workday/dbt/workday__hr_events.sql
src/ingestion/connectors/hr-directory/workday/dbt/workday__identity_inputs.sql
src/ingestion/connectors/hr-directory/workday/dbt/workday__to_class_people.sql
src/ingestion/connectors/hr-directory/workday/dbt/workday__workers_fields_history.sql
src/ingestion/connectors/hr-directory/workday/dbt/workday__workers_snapshot.sql
src/ingestion/connectors/hr-directory/workday/dbt/workday__working_hours.sql
src/ingestion/connectors/hr-directory/workday/descriptor.yaml
src/ingestion/connectors/hr-directory/workday/fixtures/leave_requests.json
src/ingestion/connectors/hr-directory/workday/fixtures/mock_raas.py
src/ingestion/connectors/hr-directory/workday/fixtures/workers.json
src/ingestion/secrets/connectors/workday.yaml.example

- Forward raw_data on leave_requests too: the leave report is also customer-built, so extra columns must survive into Bronze (manifest transformation + schema, DESIGN table, README note). - Add source_id to the hr_events workers JOIN to prevent cross-instance identity mixing when one tenant runs multiple workday sources. - Mock RaaS now requires the Basic auth scheme, not just any Authorization header, so connector auth regressions fail the rig. - Label the deployment topology fence in DESIGN.md (MD040). Re-verified against the mock: read returns 5 workers / 4 leave_requests with 0 errors, leave records carry raw_data, Bearer auth is rejected. Signed-off-by: Roman Mitasov <Roman.Mitasov@constructor.tech> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The publish-chart job both patch-bumps the umbrella `version` and rewrites `ingestion.toolboxImage` in charts/insight/values.yaml from the tree checked out at this run's trigger SHA. When two version-bumping PRs merge back-to-back — especially two that touch src/ingestion/** and so both rebuild the toolbox — the second run's trigger SHA does not yet contain the first run's `chore(release)` commit, so both runs: * compute the SAME next umbrella version, and * rewrite the SAME `ingestion.toolboxImage` line to their own build tag. The first run pushes its release commit; the second run's commit-back is then rejected (non-fast-forward). The retry's `git pull --rebase` hits a conflict in charts/insight/values.yaml — specifically on the `ingestion.toolboxImage` line (the version line auto-merges because both runs wrote the same value) — and aborts -> exit 1. A plain job re-run repeats this forever because `checkout` re-pins the same stale SHA. Secondary symptom: both runs `helm push` the same chart version with different contents, so the chart published to GHCR diverges from what main records as that version. Fix: re-anchor the working tree to the live branch tip (`git fetch` + `git reset --hard origin/$GITHUB_REF_NAME`) before any value is computed, so version-compute, the toolboxImage ref and the commit-back are all consistent with the branch tip — and a re-run recomputes against the refreshed tip instead of replaying the stale SHA. Combined with the existing per-ref `concurrency` serialisation the final push fast-forwards, so the fragile rebase-retry is replaced with a fail-loud guard. Reproduced by run 27535189783 (constructorfabric#1306 Figma then constructorfabric#1305 Workday merged back-to-back, both touching src/ingestion/**). Note: bump-descriptors shares the same rebase-retry pattern; left untouched here to keep this fix scoped to the reproduced failure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The publish-chart job both patch-bumps the umbrella `version` and rewrites `ingestion.toolboxImage` in charts/insight/values.yaml from the tree checked out at this run's trigger SHA. When two version-bumping PRs merge back-to-back — especially two that touch src/ingestion/** and so both rebuild the toolbox — the second run's trigger SHA does not yet contain the first run's `chore(release)` commit, so both runs: * compute the SAME next umbrella version, and * rewrite the SAME `ingestion.toolboxImage` line to their own build tag. The first run pushes its release commit; the second run's commit-back is then rejected (non-fast-forward). The retry's `git pull --rebase` hits a conflict in charts/insight/values.yaml — specifically on the `ingestion.toolboxImage` line (the version line auto-merges because both runs wrote the same value) — and aborts -> exit 1. A plain job re-run repeats this forever because `checkout` re-pins the same stale SHA. Secondary symptom: both runs `helm push` the same chart version with different contents, so the chart published to GHCR diverges from what main records as that version. Fix: re-anchor the working tree to the live branch tip (`git fetch` + `git reset --hard origin/$GITHUB_REF_NAME`) before any value is computed, so version-compute, the toolboxImage ref and the commit-back are all consistent with the branch tip — and a re-run recomputes against the refreshed tip instead of replaying the stale SHA. Combined with the existing per-ref `concurrency` serialisation the final push fast-forwards, so the fragile rebase-retry is replaced with a fail-loud guard. Reproduced by run 27535189783 (#1306 Figma then #1305 Workday merged back-to-back, both touching src/ingestion/**). Note: bump-descriptors shares the same rebase-retry pattern; left untouched here to keep this fix scoped to the reproduced failure. Co-authored-by: Roman Mitasov <Roman.Mitasov@constructor.tech> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

coderabbitai Bot reviewed Jun 11, 2026

View reviewed changes

cyberantonz approved these changes Jun 15, 2026

View reviewed changes

mitasovr merged commit 03af566 into constructorfabric:main Jun 15, 2026
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(ingestion): add Workday HR connector (RaaS report contract, BambooHR parity)#1305

feat(ingestion): add Workday HR connector (RaaS report contract, BambooHR parity)#1305
mitasovr merged 2 commits into
constructorfabric:mainfrom
mitasovr:feat/workday-connector

mitasovr commented Jun 11, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jun 11, 2026 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

mitasovr commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Connector (src/ingestion/connectors/hr-directory/workday/)

dbt (mirrors bamboohr model-for-model)

Specs

Local test rig (no public Workday dev tenant exists)

Test plan

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mitasovr commented Jun 11, 2026 •

edited

Loading

Connector (`src/ingestion/connectors/hr-directory/workday/`)

coderabbitai Bot commented Jun 11, 2026 •

edited

Loading