Agents Fail at Office Work Because the Data Never Existed

July 19, 2026Gerra Research5 min read

Frontier models have read most of the public internet: code, papers, documentation, forums, the written record of nearly everything people chose to publish. What they have never read is an organization actually running.

The web describes work; it does not record it. A real decision inside a company is a chat thread that becomes an email that becomes a document that becomes a ticket that updates a CRM record, authored by overlapping people on overlapping teams over days or weeks. That cross-tool handoff structure is the substance of office work, and it has never existed as a training corpus. It sits fragmented across dozens of SaaS tools, each holding one partial view, all of it private.

So when an agent is asked to do office work, it improvises from descriptions of work rather than records of it. This post explains what the missing corpus looks like when it is actually built, using Gerra's Operational Telemetry work and the datasets behind it as the concrete example.

Why synthetic benchmarks do not close the gap

The standard workaround is simulation: scripted tool environments, hand-written task suites, toy companies. These are real contributions. τ-bench demonstrated how sharply agent reliability drops when a simulated user and domain rules enter the loop. But a simulation encodes how its designers imagine work: the tickets are well-formed, the handoffs are clean, the org chart is consistent, and every distractor was placed there on purpose.

Real organizations are not like that. Threads run in parallel, documents go stale, tasks change owners midway, and the conventions that route work are tribal rather than documented. An agent that scores well against a designed environment has demonstrated competence at the designer's model of work, which is a different claim from competence at work.

The code lane already ran this experiment. Coding evaluation became meaningfully harder and more predictive when SWE-bench moved from isolated puzzles to real repositories with real issues and real tests. We made the corresponding training-data argument in Repository-Level Code Data for Coding Agents: the useful unit is the coherent real artifact, not the synthetic fragment. Organizational work needs the same switch, and its coherent artifact is the cross-tool record of a company operating.

What the corpus looks like

Gerra's Operational Telemetry report documents a cross-tool activity graph assembled from 5 consented partner companies, spanning 38 tools across 10 categories. The raw inventory at snapshot:

Tool categoryPrimary recordCount
ChatMessages38.4M
EmailEmails11.2M
File storageFiles3.6M
Project tracking (Jira)Tickets312K
Project tracking (Linear)Tickets248.3K
Source controlRepositories, metadata only2,140
CRMDatabase queries18.6M
CRMContacts2.6M
Service and workflowTickets81K
The raw records are not the contribution; every one of them already exists inside its own tool. The contribution is what connects them. Records are normalized into a single activity graph and entity-resolved across five entity types: user, team, org, document, and task. The same person is one node whether they appear in chat, email, a ticket, or the CRM, so a decision that touched four tools becomes one traversable structure instead of four disconnected exports.

Governance is part of the dataset rather than a disclaimer attached to it. All five partner companies consented. Collection is a one-time snapshot, not a live feed. Personally identifiable information is stripped at the source, before records leave the partner environment. Source control contributes repository metadata only, with no code bodies. And resolution happens strictly inside each partner's org boundary; entities are never merged across companies.

The report is equally explicit about its own limits: it is a data-availability report, and it claims no signal, no model, and no predictive result.

The productized surfaces

Three datasets carry this work in deliverable form:

  • Workplace Activity Graph: the communication and coordination layer. Chat, mail, documents, projects, and calendar activity normalized into 40+ event types, entity-resolved across user, team, org, document, and task.
  • Software Delivery Graph: the engineering layer. Source control, issue tracking, and data-platform activity normalized into 50+ SDLC event types on a commit, pull request, issue, and deploy graph.
  • Company Operating Archive: whole-company depth. Complete operating histories of founder-owned companies, spanning code, business data, communications, documents, and databases, consented and PII-scrubbed.

The first two provide breadth across organizations and tools; the archive provides end-to-end depth within single companies. Which one matters depends on whether you are training routing behavior or reconstructing how a business ran.

Evaluation and training want different things from the same graph

For training, the value is grounded behavior: who hands what to whom, which documents get referenced before a decision, how a task actually moves from report to resolution. An agent trained only on public text has to guess at this. An agent trained on resolved cross-tool structure has seen thousands of real instances of the pattern it is being asked to perform.

For evaluation, the graph is a replayable world. Point-in-time snapshots freeze the state of an organization; an eval hands the agent that state, poses the task the real team faced, and compares the agent's actions to what the team actually did. That inverts the synthetic benchmark. Instead of a designer writing the expected answer, the ground truth is an outcome the organization produced before any eval existed, which makes it very hard for the answer to be gamed or memorized.

The two uses impose different requirements, and a serious supplier should distinguish them. Training wants breadth, normalization, and volume. Evaluation wants frozen state, strict boundaries between organizations, and outcome records the agent cannot see in advance. Timestamped events and per-partner isolation are what make the same graph safe for both.

What to ask before you believe any of it

Any vendor offering organizational data should have immediate answers to six questions. Who consented, and to what? Where is PII removed, at the source or after collection? Are organizations isolated, or does resolution leak across them? Is this a snapshot or a feed? What exactly is entity-resolved? And what is explicitly not claimed?

The last question is the fastest filter. The Operational Telemetry report states plainly that it documents availability, structure, and governance, and nothing else. A vendor who claims agent-performance lift without showing the evaluation is answering a question you did not ask, with a number you cannot check. How to run that inspection in general is covered in How to Read a Data Vendor's Data Room.

Agents will be asked to do office work either way. The open question is whether they learn it from how organizations actually run or from how benchmark authors imagine they run. The data for the first option now exists: consented, normalized, entity-resolved, with its boundary conditions written down.