Skip to content

ci: shard node_compat tests into 2 parallel jobs#33298

Open
bartlomieju wants to merge 2 commits intomainfrom
ci/shard-node-compat-tests
Open

ci: shard node_compat tests into 2 parallel jobs#33298
bartlomieju wants to merge 2 commits intomainfrom
ci/shard-node-compat-tests

Conversation

@bartlomieju
Copy link
Copy Markdown
Member

Summary

  • Split the node_compat CI job into 2 shards per OS (3 OS x 2 shards = 6 matrix entries), halving wall-clock time from ~20m to ~10m
  • Uses the same CI_SHARD_INDEX/CI_SHARD_TOTAL env vars and round-robin filter_to_shard mechanism already used by specs and integration tests
  • Report uploads are suffixed with shard index (report-linux-0.json.gz, report-linux-1.json.gz)
  • Summary script merges shard reports, with fallback to the old unsharded filename for historical data

Test plan

  • CI passes -- the node_compat workflow itself doesn't run on PRs (it's scheduled), but the Rust code change is exercised by the existing cargo test --test node_compat path
  • Trigger a manual workflow_dispatch run to validate sharding end-to-end

🤖 Generated with Claude Code

The node_compat CI job now takes ~20m. Split it into 2 shards per OS
(6 total matrix entries) using the same round-robin distribution as
specs/integration tests via CI_SHARD_INDEX/CI_SHARD_TOTAL env vars.

Report uploads are suffixed with shard index, and the summary script
merges shard reports when generating the daily summary (with fallback
to the old unsharded filename for historical reports).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
if (shardReports.length === 1) {
return shardReports[0];
}
// Merge shard reports: combine results, sum counts
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Problem: the summary merge path blindly sums total/pass across shard reports while also overwriting duplicate results keys.

Why here: merged.results[key] = value will silently replace an existing entry if the same test id ever appears in multiple shard reports, but merged.total += shard.total / merged.pass += shard.pass still count both copies. That makes the metadata potentially inconsistent with the merged results object.

Impact: if sharding ever overlaps due to a bug, retry artifact mixup, or future shard-count changes, the monthly summary can overcount totals/passes while hiding the collision.

Ask: can we either assert that result keys are disjoint when merging shard reports, or recompute total/pass from the merged results object after merge? That would make the summary logic more robust to bad inputs.

Recompute total and pass counts from the merged results object after
combining shard reports, rather than summing each shard's counts.
This prevents inconsistency if shards ever overlap due to a bug
or retry artifact mixup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@miracatbot miracatbot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked after the update: recomputing total/pass from the merged result set fixes the inconsistency I had called out in the shard-summary merge logic. This looks good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants