Skip to content

Eager BFS resolution: pre-resolve all reachable actions upfront#4297

Draft
stefanpenner wants to merge 2 commits intoactions:mainfrom
stefanpenner:eager-action-resolution-bfs
Draft

Eager BFS resolution: pre-resolve all reachable actions upfront#4297
stefanpenner wants to merge 2 commits intoactions:mainfrom
stefanpenner:eager-action-resolution-bfs

Conversation

@stefanpenner
Copy link

@stefanpenner stefanpenner commented Mar 13, 2026

Summary

Builds on #4296 (batch + dedup). Before the recursive composite walk begins, a BFS loop eagerly discovers, resolves, and downloads all reachable actions so the recursion makes zero network calls.

The loop iteratively:

  1. Resolves pending actions in one batch API call
  2. Downloads all tarballs in parallel (Task.WhenAll)
  3. Scans downloaded action.yml files for composite sub-action references
  4. Feeds newly discovered actions back into the loop until no new actions remain

Key design detail

Uses separate tracking for downloads (keyed by owner/repo@ref — path excluded, since sub-paths share one tarball) vs scans (keyed by owner/repo/path@ref — so each sub-path's action.yml is scanned for its composite children independently).

Without this distinction, sub-path actions like myorg/monorepo/L2-01@main would be skipped after myorg/monorepo@main was already downloaded, leaving deeper composites undiscovered until the recursive fallback.

Smoke test results

Tested with a self-hosted runner built from this branch vs stock (main), using a 50-action composite tree across 21 repos with 5 depth levels.

Action graph:

workflow → 10× L1 composites + 5 leaf shards (15 top-level steps)
  L1 (×10) → 1 shard (6–10) + 3 L2 composites
  L2 (×10) → 1 shard (11–15) + 3 L3 composites
  L3 (×10) → 2 L4 composites + 1 shard (16–20)
  L4 (×10) → 3 leaf shards (1–5)
21 unique repos, branching factor 3, depth 4

Results (run) (more complex graph, with more nested unique downloads)

Stock runner (main) #4296 (batch + dedup) This PR (eager BFS)
Resolve API calls 311 8 4
Resolution time 101.6 s ~17 s 9.3 s
Speedup vs stock ~6× ~11×

With cached downloads (ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE)

When tarball downloads are instant (local cache / symlink), the bottleneck is purely resolve API calls:

Stock #4296 This PR
Resolve calls 311 8 4
Est. resolve latency ~62 s ~3.2 s ~1.6 s

Where the 4 calls come from

Each BFS wavefront discovers new repos that weren't in the previous wavefront's tarball:

Wave Resolves Downloads
1 checkout@v4 + shard-1..5@main + resolution-test@main (7 unique) 7 repos in parallel
2 shard-6..10@main (5 new, discovered from L1 action.ymls) 5 repos in parallel
3 shard-11..15@main (5 new, from L2 action.ymls) 5 in parallel
4 shard-16..20@main (5 new, from L3 action.ymls) 5 in parallel

After wave 4, L4 composites reference shard-1..5 which are already downloaded → BFS terminates. Recursion finds everything cached.

Test plan

  • All 36 existing ActionManagerL0 tests pass (no regression)
  • New test: PrepareActions_EagerResolution_AllDownloadsBeforeRecursion — verifies all watermarks from earlier BFS wavefronts exist by the time the last resolve fires
  • Smoke test: 21-repo / 5-depth / branching-factor-3 composite tree on self-hosted runners

🤖 Generated with Claude Code

Stefan Penner and others added 2 commits March 12, 2026 21:36
Thread a cache through PrepareActionsRecursiveAsync so the same action
is resolved at most once regardless of depth. Collect sub-actions from
all sibling composites and resolve them in one API call instead of one
per composite.

~30-composite internal workflow went from ~20 resolve calls to 3-4.

Fixes actions#3731

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ons upfront

Before the recursive composite walk begins, a BFS loop iteratively:
1. Resolves pending actions in one batch API call
2. Downloads all tarballs in parallel
3. Scans downloaded action.ymls for composite sub-action references
4. Feeds newly discovered actions back into the loop

This collapses all network I/O into a tight loop so the recursive walk
is purely local (zero API calls, zero downloads).

Key design detail: uses separate tracking for downloads (keyed by
owner/repo@ref) vs scans (keyed by owner/repo/path@ref) so that
different sub-paths within the same repo tarball are each scanned
for their composite children.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant