Two ways a task scheduler waits forever

transcript

Your pipeline never finishes. Nothing is running. Nothing has crashed. Two classic bugs. Both leave tasks pending forever, so waiting never returns.

A needs c to succeed first. B needs a. And c needs b. Round and round. And d needs c, from outside the loop. Every task has an unmet dependency. Nothing can ever start.

The fix: run Kahn's algorithm when the batch arrives. Anything never consumed is in a cycle, or behind one. All four rejected.

Bug two. Migrate fails: disk full. It retries. Disk full again. Out of attempts. Failed. Notify depends on migrate. What happens to it? Leave it pending, and the open-task count never hits zero.

Pending forever means the run never finishes. So any ending but success skips each waiting child. Which skips its own children, all the way down.

Reject cycles before anything is stored. Failed or cancelled? Skip everything downstream. Every run ends.

Full breakdown in the lecture.