Tasks are stuck or failing
A stuck or failing task usually points to an issue in the event pipeline, a tool timeout, or an unhandled error in the execution loop. The worklog and metrics are the best places to start.
Common causes
Section titled “Common causes”| Symptom | Likely cause |
|---|---|
| Task accepted but no response | Worker is not consuming events or Kafka lag is high. |
| Task hangs mid-execution | Tool call is timing out or waiting for human approval indefinitely. |
| Task fails repeatedly | Tool throws an error, secret is missing, or the agent hits a step limit. |
| Task restarts in a loop | Event redelivery or unhandled exception before checkpointing. |
Diagnostic steps
Section titled “Diagnostic steps”-
Check the task state through the runtime API or state store to see whether the task is queued, running, waiting, or failed.
-
Look at Kafka consumer lag. Growing lag means workers cannot keep up or have stopped consuming.
-
Read the worklog for the task. The last recorded event usually shows what the agent was waiting for.
-
Check worker logs for exceptions, secret-resolution failures, or tool errors.
-
Verify the dead-letter queue for events that failed too many times.
-
Confirm timeouts are configured for tools, approvals, and the overall task.
- Scale workers or resolve the event-bus issue if lag is the problem.
- Reduce tool timeouts and ensure tools return errors cleanly instead of hanging.
- Review approval timeout settings so tasks do not wait forever.
- Fix missing secrets or incorrect role permissions referenced in the worklog.
- Replay dead-letter events only after addressing the root cause.