DEVELOPER WORKFLOW DATA
Developer Workflow Intelligence
Engineering coordination data across source control, issue tracking, and data platforms. Unified SDLC schema for coding agents and SWE evals.
See the data
Representative records in the exact shape we deliver. Real provenance and full slices are shared under license.
Merged PR resolving a linked issue (TypeScript)
Representative shape, not real customer code. base_ref_oid / head_ref_oid mirror the evaluator real PR fields; issue_linked is true.
{
"event_id": "dw-04412",
"org_id": "org_01",
"repo": "org/service-api",
"event_type": "pull_request.merged",
"actor_id": "u_07",
"timestamp": "2026-02-04T09:13:55Z",
"commit_sha": "625933e987e4",
"pr_number": 318,
"base_ref_oid": "19608d910738",
"head_ref_oid": "625933e987e4",
"issue_id": "ISSUE-204",
"issue_linked": true,
"changed_files": ["src/webhook/retry.ts", "src/webhook/retry.test.ts"],
"ci_status": "passed",
"test_command": "npm test -- retry",
"release_tag": "v2.14.0"
}Issue opened (problem end of the trace)
Representative. The resolving PR above closes this issue, forming one issue-to-code trace.
{
"event_id": "dw-04388",
"org_id": "org_01",
"repo": "org/service-api",
"event_type": "issue.opened",
"actor_id": "u_11",
"timestamp": "2026-02-03T17:40:02Z",
"issue_id": "ISSUE-204",
"issue_linked": true,
"ci_status": null,
"release_tag": null
}Warehouse query event (data-agent context)
Representative. Data-warehouse activity normalized into the same SDLC graph for data and analytics agent training.
{
"event_id": "dw-09120",
"org_id": "org_01",
"repo": null,
"event_type": "warehouse.query_run",
"actor_id": "u_07",
"timestamp": "2026-02-05T11:02:18Z",
"warehouse_query": "SELECT account_id, SUM(amount) FROM invoices WHERE status='open' GROUP BY 1",
"issue_linked": false,
"ci_status": null
}Record shape
Every field, its type, whether it can be null, and a representative value.
| Field | Type | Constraint | Description |
|---|---|---|---|
| event_id | string | required | Stable id for one normalized SDLC event (commit, PR, issue, review, ci_run, release, warehouse_query). e.g. dw-04412 |
| org_id | string | required | Partner org boundary; entities are never merged across partners. e.g. org_01 |
| repo | string | nullable | Repository the event belongs to (owner/name), entity-resolved. e.g. org/service-api |
| event_type | string | required | One of 50+ normalized SDLC event types. e.g. pull_request.merged |
| actor_id | string | required | Resolved author/actor node, unified across tools. e.g. u_07 |
| timestamp | string · ISO-8601 | required | Event time in UTC for point-in-time reconstruction. e.g. 2026-02-04T09:13:55Z |
| commit_sha | string | nullable | Commit hash for commit/merge events; anchors repo state. e.g. 625933e987e4 |
| pr_number | int | nullable | Pull request number for PR/review events (evaluator field). e.g. 318 |
| base_ref_oid | string | nullable | PR base commit SHA - state before the change (evaluator field baseRefOid). e.g. 19608d910738 |
| head_ref_oid | string | nullable | PR head commit SHA - the proposed change (evaluator field headRefOid). e.g. 625933e987e4 |
| issue_id | string | nullable | Linked issue joined to the resolving commits and PR. e.g. ISSUE-204 |
| issue_linked | bool | required | Whether this PR/commit is joined to a resolving issue (evaluator metric issue_linked_pr_ratio). e.g. true |
| changed_files | string[] | nullable | Files touched by the change. e.g. ["src/webhook/retry.ts"] |
| ci_status | string | nullable | Build/test signal at the event (CI run outcome). e.g. passed |
| test_command | string | nullable | Recorded test command for execution-based verification. e.g. npm test -- retry |
| release_tag | string | nullable | Release that shipped the change. e.g. v2.14.0 |
| warehouse_query | string · sql | nullable | For data-warehouse events: query text for data-agent training. e.g. SELECT account_id, SUM(amount) ... |
Source Control Activity
Commits, pull requests, reviews, merges, and release tags with full diff and review-thread context. Author and repository entity-resolved.
Issue & Project Graph
Tickets, sprints, cycle times, and dependency edges across issue trackers. Joined to source-control activity via issue-PR linkage.
Data Warehouse Activity
Query patterns, dashboard usage, pipeline runs, and model refresh activity across analytics platforms. Useful for data-agent training.
How it is built
- 01
Consented capture across SDLC tools
Founder-owned and partner repositories and their issue trackers are captured with consent. Source control contributes commits with full per-commit diff history plus PRs, reviews, linked issues, CI runs, and release tags.
- 02
Normalization to a unified engineering graph
Tool-specific records from source control, issue trackers, and warehouses are normalized into one SDLC event vocabulary of 50+ types, with author and repository entity resolution.
- 03
Issue-to-code linkage
Tickets are joined to the specific commits and PRs that resolved them, turning each resolved issue into a six-stage trace: problem, change, proposal, scrutiny, verification, outcome.
- 04
Quality filtering
Traces without a clean issue-to-PR join are not promoted to task records, and PRs are filtered against minimum criteria (for example test-file presence) before they become eval-grade.
- 05
Sanitization
Material is processed to remove sensitive content before inclusion; credentialed remotes and secrets are stripped.
- 06
Point-in-time reconstruction
Commit history and release tags recover the repository exactly as it stood just before the resolving change, so an agent sees only what was available then - no look-ahead into the fix.
How we validate
What each evaluation measures and how it is run. Where no benchmark is published, we show the methodology and say so.
Point-in-Time SWE Task
Measures
Whether an agent resolves a real issue the way a human engineer did.
Method
Reconstruct base state from commit history and release tags; pose the linked issue while withholding the resolving commits and PR; score against the gold PR via execution-based verification (run the recorded test command) plus diff comparison.
Result
Methodology-stage. The research is explicit that no benchmark numbers are published; quantifying agent performance is future work.
Task-Quality Rubric (pre-eval filtering)
Measures
Whether an issue-to-PR trace is clean enough to become an eval task.
Method
A reference evaluator (eval_kit v0.6.2) scores each candidate PR on issue clarity, gold-patch clarity, test clarity, and patch/test/issue alignment, and tracks pass-first-filter rate, issue-linked-PR ratio, and PR acceptance rate. PRs without tests are rejected.
Result
Methodology-stage. These are filtering and integrity rubrics run on real repositories, not a published agent-performance benchmark.
Data / Analytics Agent Task
Measures
Whether an agent can write SQL, fix pipelines, and answer analytical questions grounded in real warehouse activity.
Method
Replay query patterns, dashboard usage, pipeline runs, and model-refresh activity from production warehouses as grounded context.
Result
Methodology-stage. No measured outcome is reported.
Ground truth
What correct means for this data, and how it is established.
Ground truth
The human pull request that actually merged and resolved the linked issue - the change that passed the repository own review and CI - together with the repository existing test suite at that point in time.
How it is established
Execution-based verification plus diff comparison against the gold PR. Trace quality is graded by a reference evaluator (eval_kit v0.6.2) on per-PR rubrics and per-repo integrity metrics: issue_linked_pr_ratio, commits_referencing_issues, pr_acceptance_rate, has_ci_cd, and test-framework presence. The harness has been run end to end on real repositories.
Agreement
Correctness is anchored to the repository own tests and the merged human change, not a separate human-rater pass. No inter-rater agreement figure is published at this stage.
Training data for coding agents
Real SDLC behavior - PR reviews, cycle time, issue movement - instead of synthetic benchmarks. Grounded context for coding agents that ship to real teams.
SWE eval harness
Point-in-time snapshots of engineering work for end-to-end evals. Measure an agent against what real engineers actually did, with full review context.
Data and analytics agent training
Real query and pipeline workflows from production warehouses. Grounded context for data agents that write SQL, fix pipelines, and answer analytical questions.
How you load it
Delivery
S3, REST API, Webhook, Restricted license / data room
Formats
JSONL event records, Parquet, Git bundle (per-repo history)
Auth
Restricted access under a signed license; per-recipient export. Raw repositories stay gated; only consented, sanitized material is shared. Source identities are anonymous; full provenance under NDA.
Cadence
Real-time via webhooks or daily batch for live engineering feeds; one-time archive or periodic snapshot for the historical SDLC corpus.
# SDLC point-in-time SWE task flow (mirrors the reference evaluator)# clone -> fetch repo from its git bundle# reconstruct base -> checkout base_ref_oid (state before the fix)# pose issue -> hand issue_title + issue_body to the agent-under-test# agent attempts -> agent reads/edits the repo to resolve the issue# run tests -> execute test_command (execution-based verification)# compare -> diff the agent change against the gold PR# verdict -> pass / fail + per-stage report# Trace quality is pre-graded by eval_kit (issue/patch/test clarity + alignment).
Request access.
Restricted-scope evaluation access for qualified teams. We share real samples, full schema, and provenance under a mutual NDA.