Three full-sync tools currently flood the MCP client's context window with internal queue bookkeeping. This change replaces a single field — surgically — so responses shrink by two orders of magnitude. Nothing on disk moves.
Each of roxtra_index_full_sync, roxtra_index_full_sync_status, and roxtra_index_manage (action status) currently embeds the entire in-memory progress object. That object carries six internal arrays that grow with the size of the document tree being indexed. On a mid-run roXtra instance, the folder queue alone holds thousands of entries; discovered-file IDs an order of magnitude larger.
Each grows with tree size. None is actionable for the language model. All remain in the persisted sync_progress.json on disk — only the wire shape changes.
Add a pure read-transform createFullSyncSummary(progress) alongside the existing createFullSyncStatus, then swap the progress field for progressSummary at the two return sites. The rename is deliberate — readers can no longer assume the same shape as the persisted progress object, and the change becomes visible at every call site that still depended on the old shape (none in src/).
A pure projection. Reads top-level metadata and the flat counters sub-object. Lives next to createFullSyncStatus.
Every response site swaps the key from progress to progressSummary. The new name flags the new shape.
Nine counter fields surface at the summary's top level, not nested under .counters. One less level for the model to navigate.
The summary keys are fixed at construction. The spec's first scenario asserts equality — extras would fail the gate. Counters are reported bit-for-bit from the persisted object; missing counter keys coalesce to 0.
Top-level metadata · 7 keys
Counters · 9 keys
The response continues to expose documentCount, indexDir, progressPath, and nextRecommendedCall at the same nesting level, with the same names and semantics. No new keys; no removed keys.
createFullSyncSummary(null) returns null — preserving the existing status-on-empty-index response shape for roxtra_index_manage action status against a fresh index.
Illustrative roxtra_index_full_sync_status response, mid-run, against a small folder tree. Real payloads can balloon by two orders of magnitude as the queue and ID arrays grow.
{
"progress": {
"status": "running",
"runId": "7f3...",
"startedAt": "2026-05-21T08:14:02Z",
"updatedAt": "2026-05-21T08:21:18Z",
"completedAt": null,
"rootFolderId": "42",
"lastError": null,
"counters": { /* 9 keys */ },
"folderQueue": ["f-12", "f-13", /* 3,200 more */],
"visitedFolderIds": ["f-1", /* 4,800 more */],
"discoveredFileIds": ["d-9", /* 41,000 more */],
"processedContentFileIds": [/* ... */],
"failedFileIds": [/* ... */],
"unsupportedFileIds": [/* ... */]
},
"documentCount": 3211,
"indexDir": "/.../indices/roxtra-default",
"progressPath": "/.../sync_progress.json",
"nextRecommendedCall": { /* resume args */ }
}
{
"progressSummary": {
"status": "running",
"runId": "7f3...",
"startedAt": "2026-05-21T08:14:02Z",
"updatedAt": "2026-05-21T08:21:18Z",
"completedAt": null,
"rootFolderId": "42",
"lastError": null,
"foldersVisited": 4800,
"foldersPending": 3200,
"documentsDiscovered": 41210,
"documentsIndexed": 12044,
"contentSynced": 11891,
"contentFailed": 12,
"contentUnsupported": 141,
"chromaUpsertedMetadataRecords": 12044,
"chromaUpsertedChunks": 38211
},
"documentCount": 3211,
"indexDir": "/.../indices/roxtra-default",
"progressPath": "/.../sync_progress.json",
"nextRecommendedCall": { /* resume args */ }
}
Counter values are illustrative. Field names and shape are exactly per the spec.
One helper added; two return sites edited. Downstream wrappers pick up the new shape transitively. The persistence layer and resume code paths are untouched by name.
src/server/index_store/IndexStoreCore.ts
createFullSyncSummary(progress) helper. Swap progress → progressSummary on createFullSyncStatus's return (lines 157–177).src/server/index_store/FullSyncOperations.ts
progress → progressSummary on runFullSyncOperation's return (lines 298–321).src/server/index_store/IndexManageFlow.ts
createFullSyncStatus (line 100). No edits needed.src/server/index_store/IndexDebugStatusOperations.ts
createFullSyncStatus. No edits.src/server/RoxtraMcpServer.ts
createFullSyncStatusFromStore. No edits.scripts/dashboard-poc/dashboard.ts
sync_progress.json via file watcher (lines 53–58, 265–270). Not a tool-response consumer.The persisted file is the deliberately complete source of truth for the resume loop. Reviewers should confirm that nothing in the code path below changes shape.
Continue to read and mutate the in-memory progress object, including all six internal arrays. Untouched.
Receives the full in-memory progress, not the summary. sync_progress.json schema is identical to the pre-change shape.
Return the full persisted shape. The resume code path consumes the persisted shape, not the summary. Users with an in-progress sync continue without resetting their index.
Unchanged. Counter values exposed on progressSummary are reported bit-for-bit; the helper performs no recompute, round, clip, or recalculation.
No change to defaults or semantics. nextRecommendedCall.arguments keys and values are reconstructed from the same inputs as before.
Untouched. The single behavioural change is the shape of the tool response object returned to MCP clients.
includeRawProgress input flag. Here's why.progressPath.A repo-wide grep finds no consumer of any raw array off the tool response. The only raw-array consumer is the dashboard POC, which reads sync_progress.json directly via a file watcher — completely independent of MCP tool calls. The persisted file is the documented debug-only escape hatch, and its absolute path is exposed as progressPath on every response.
Each would be welcome — none belongs in this change. Listing them here so reviewers don't ask "why not also …" and so future work can pick them up cleanly.
A roxtra-mcp sync status command that pretty-prints sync_progress.json from disk — high-fidelity progress for humans without re-bloating tool responses.
phase / progress ratioA computed phase field (discovery | drain | content | complete) or a metadataPercent ratio. Useful UX signal — but introduces interpretation rules best validated against live runs first.
includeRawProgressA response-shape opt-out for a single call. Add later if a future automation truly needs raw arrays inline. Today there is no consumer.
progress.counters.folderQueue even when input contains thousands of entries.createFullSyncSummary(null) returns null.0 when missing from persisted counters.pnpm typecheck clean.pnpm gate:parity (typecheck + build + validateTools + validateToolsDist) clean.roxtra_index_full_sync against the live verification instance — end-to-end.roxtra_index_full_sync_status response contains progressSummary; lacks progress, folderQueue, *FileIds.nextRecommendedCall non-null on incomplete responses.sync_progress.json still contains the full raw arrays.