Scheduling a job before you know who's in it
A greedy scheduler double-booked a user across two pipelines. The bug: downstream jobs have empty membership at scheduling time, so "reserve this job's members" reserves nobody. Fix: reserve the set of users who *could* reach the job — its transitive predecessors — not the ones literally in it yet.
A greedy scheduler packs jobs onto a shared resource one at a time: for each job, find the earliest slot where everything it needs — the resource itself, plus every member (the users that job involves) — is free, place it, mark those members busy until the job ends, repeat.
It worked fine within one pipeline. Then a user appeared in two pipelines feeding the same pool, and the scheduler placed their downstream job in both on the same time slot. One person, two slots, simultaneously.
Why the obvious guard does nothing
The per-job logic looked airtight:
const membersInJob = [...jobInputsA, ...jobInputsB];
const membersBusyUntil = Math.max(
0,
...membersInJob.map((u) => memberFreeAt.get(u) ?? 0),
);The trap: in a dependency pipeline, downstream jobs are created with empty membership. You don’t know who lands in stage 2 until stage 1 resolves — so at scheduling time jobInputsA and jobInputsB are empty arrays. membersInJob is []. The busy-until check reserves nobody, and the post-placement loop marks nobody busy. The cross-pipeline overlap guard never fires because there’s no one to overlap on.
The first stage hid the bug completely: there, membership is fully known, so membersInJob is the real set and everything reserves correctly. The bug only surfaces from stage 2 onward — exactly the jobs you’re least likely to eyeball in a test.
The fix: reserve who could be here
The scheduler already computed, for an unrelated assignment rule, the transitive predecessor union of a job — every user who could possibly flow into it as upstream stages resolve. That set is non-empty even when the literal membership is empty. So drive the reservation off that instead:
// potentialMembers ⊇ (jobInputsA ∪ jobInputsB) always:
// equal at stage 1, the recursive predecessor union downstream
const reservedMembers =
potentialMembers.size > 0 ? [...potentialMembers] : membersInJob;
const membersBusyUntil = Math.max(
0,
...reservedMembers.map((u) => memberFreeAt.get(u) ?? 0),
);The same potentialMembers set drives the post-placement busy-mark, so a user’s other pipelines see the slot as taken.
The trade-off is the interesting part
This over-reserves on purpose. A user who could reach a stage-2 slot but actually drops out upstream still had that slot held for them — time that, in hindsight, was free. So schedules run slightly longer than strictly optimal, and a genuinely resource-starved run can now fail loudly with a capacity error where it previously “succeeded” by silently double-booking someone.
That’s the right direction. A schedule that’s a little longer is an inconvenience; a schedule that puts the same person in two places at once is a broken result. When a greedy algorithm reasons about a future it can’t see yet, reserve against the pessimistic set, and let it fail loud rather than overcommit.
The general shape: any time you guard a resource by inspecting “what’s in this slot right now,” check whether the slot can be legitimately empty at decision time but full later. If it can, the present-tense guard is a no-op exactly when you need it most.