fix: auto-start workers in TaskHandler context manager#391
Open
nthmost-orkes wants to merge 5 commits intomainfrom
Open
fix: auto-start workers in TaskHandler context manager#391nthmost-orkes wants to merge 5 commits intomainfrom
nthmost-orkes wants to merge 5 commits intomainfrom
Conversation
… tasks - TaskHandler.__enter__ now calls start_processes() so `with TaskHandler(...) as h:` works out of the box without a separate h.start_processes() call. start_processes() is idempotent via _processes_started guard, so existing code that calls it explicitly inside the with-block is unaffected. Fixes conductor-oss/getting-started#42. - WorkflowExecutor.execute() and execute_workflow() now log a WARNING when the workflow returns RUNNING after the wait timeout but a task is already in FAILED/FAILED_WITH_TERMINAL_ERROR state, surfacing the failure reason that would otherwise be invisible to the caller. Fixes conductor-oss/getting-started#41. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The conductor_server.sh one-liner downloaded a 606 MB JAR to the current directory with no progress indicator and ran in the foreground. Replace it with the `conductor server start` CLI approach, which handles download location, caching, and UX correctly. Fixes conductor-oss/getting-started#47 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v1r3n
reviewed
Apr 16, 2026
| logger.info("TaskHandler initialized") | ||
|
|
||
| def __enter__(self): | ||
| self.start_processes() |
Contributor
There was a problem hiding this comment.
what are the implications of auto starting the process?
Contributor
Author
There was a problem hiding this comment.
Good question -- mostly pros in my reckoning but i'll take a second look.
- Fixes the context manager contract. with-as-no-op is a Python anti-pattern. File handles, locks, threads, subprocess all acquire on enter. TaskHandler was the odd one.
- Fixes a real user bug — Python SDK: TaskHandler context manager silently starts 0 workers if start_processes() is not called explicitly getting-started#42. Users wrote with TaskHandler(...) as h: expecting it to run and got silence.
- Idempotency guard preserves every existing caller. Anyone doing with h: h.start_processes() is unaffected — second call is a no-op.
- stop_processes() resets the flag, so stop → restart works. (Onboard repo to PyPI (Python Package Index) #3 from last turn is handled.)
I'll go look at some things that come to mind... will update
If start_processes() raised partway through spawning workers, the flag was already True, causing a subsequent retry call to be a silent no-op. Moving the assignment to the end means a failed start remains retryable.
Python skips __exit__ when __enter__ raises, so an exception mid-startup would leak any workers already spawned. Wrap the call so stop_processes() runs before the exception propagates.
Contributor
Author
|
Changes prompted by this review: Thinking through the implications surfaced two edge cases worth hardening, both now pushed as follow-up commits:
For the main behavior change itself:
|
The extra get_workflow() call on every RUNNING return adds an unnecessary network round-trip. The underlying behavior (workflow stays RUNNING while a failed task awaits retries) is correct Conductor behavior, not a bug — tracked as a docs gap in conductor-oss/getting-started#53.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TaskHandler.__enter__now callsstart_processes()automatically, sowith TaskHandler(...) as th:works correctly without a separateth.start_processes()call inside the block.Changes:
__enter__callsstart_processes()on entry; if it raises, callsstop_processes()before propagating (since Python skips__exit__when__enter__raises)start_processes()gains an idempotency guard (_processes_startedflag) so existing code that calls it explicitly inside thewithblock is unaffectedstop_processes()resets the flag so stop → restart cycles work correctlyFixes conductor-oss/getting-started#42.
User impact
First-time users following the quickstart who used
with TaskHandler(...) as th:without callingth.start_processes()would see their workflow hang inRUNNINGindefinitely — 0 workers were ever started. Now the context manager behaves like standard Python: setup happens in__enter__, teardown in__exit__.Test plan
test_context_manager_enter— verifieswith handler as h: h is handlertest_context_manager_exit— verifiesstop_processes()is called on exitpython3 -m pytest tests/unit/— 440 passed🤖 Generated with Claude Code