Skip to content

refactor(lease): centralized LeaseManager decouples heartbeats#401

Merged
v1r3n merged 3 commits intomainfrom
lease-manager-centralized
Apr 20, 2026
Merged

refactor(lease): centralized LeaseManager decouples heartbeats#401
v1r3n merged 3 commits intomainfrom
lease-manager-centralized

Conversation

@v1r3n
Copy link
Copy Markdown
Contributor

@v1r3n v1r3n commented Apr 20, 2026

Replace inline heartbeat logic in both TaskRunner and AsyncTaskRunner with a process-wide LeaseManager singleton. A background daemon thread checks for due heartbeats at ~1s intervals and dispatches them to a ThreadPoolExecutor(4), so heartbeat API calls and retries never block task polling.

Key changes:

  • LeaseManager: singleton with background thread + thread pool, fork-safe via PID check, lazy start on first track() call
  • TaskRunner: delegates to LeaseManager.track/untrack instead of inline _send_due_heartbeats/_send_heartbeat methods
  • AsyncTaskRunner: same delegation; creates a sync TaskResourceApi for LeaseManager since heartbeats run in the thread pool, not the event loop
  • task_handler.py: clean — no LeaseManager knowledge needed
  • 21 unit tests covering track/untrack, heartbeat dispatch, retries, non-blocking behavior, singleton/fork safety, thread safety

…oll loop

Replace inline heartbeat logic in both TaskRunner and AsyncTaskRunner
with a process-wide LeaseManager singleton. A background daemon thread
checks for due heartbeats at ~1s intervals and dispatches them to a
ThreadPoolExecutor(4), so heartbeat API calls and retries never block
task polling.

Key changes:
- LeaseManager: singleton with background thread + thread pool, fork-safe
  via PID check, lazy start on first track() call
- TaskRunner: delegates to LeaseManager.track/untrack instead of inline
  _send_due_heartbeats/_send_heartbeat methods
- AsyncTaskRunner: same delegation; creates a sync TaskResourceApi for
  LeaseManager since heartbeats run in the thread pool, not the event loop
- task_handler.py: clean — no LeaseManager knowledge needed
- 21 unit tests covering track/untrack, heartbeat dispatch, retries,
  non-blocking behavior, singleton/fork safety, thread safety

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@v1r3n v1r3n changed the title refactor(lease): centralized LeaseManager decouples heartbeats from p… refactor(lease): centralized LeaseManager decouples heartbeats Apr 20, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 20, 2026

Codecov Report

❌ Patch coverage is 86.56716% with 18 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...rc/conductor/client/automator/async_task_runner.py 33.33% 12 Missing ⚠️
src/conductor/client/automator/task_runner.py 75.00% 4 Missing ⚠️
src/conductor/client/automator/lease_tracker.py 98.00% 2 Missing ⚠️
Files with missing lines Coverage Δ
src/conductor/client/automator/lease_tracker.py 98.18% <98.00%> (ø)
src/conductor/client/automator/task_runner.py 78.03% <75.00%> (-1.69%) ⬇️
...rc/conductor/client/automator/async_task_runner.py 55.21% <33.33%> (ø)

... and 56 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Three tests against a live Conductor server:
1. Async task with heartbeat → COMPLETED (50s sleep, 10s timeout)
2. Async task without heartbeat → TIMED_OUT
3. Performance: heartbeat tracking adds 0ms overhead on fast tasks
@v1r3n v1r3n merged commit 3cef362 into main Apr 20, 2026
2 checks passed
@v1r3n v1r3n deleted the lease-manager-centralized branch April 20, 2026 21:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants