fix(scheduler): skip intervals that overlap unready parent intervals

[breken-ai]

Description

When an upstream model's signal marks an interval as not ready, the scheduler is supposed to hold back every downstream interval that overlaps it. interval_diff in sqlmesh/core/scheduler.py does this, but it only works when the unready interval starts at or before the candidate interval. If the candidate interval is larger and starts first, it takes the index_b += 1 branch. That branch moves past the unready interval and leaves the candidate in place.

This happens whenever a model has a coarser interval unit than a parent that has a signal. Example: a daily model reading from an hourly model whose signal reports 2023-01-01 10:00 and 2023-01-03 05:00 as not ready.

• Jan 1: compared against the Jan 3 unready hour instead of the Jan 1 one, so it is treated as ready. The daily model runs over a day that is missing an hour of upstream data, and the interval is recorded as done, so later runs don't fix it.
• Jan 4: fully ready, but it is dropped by the check after the loop, which compares it with the interval that caused the overlap.

>>> interval_diff([(0, 24), (24, 48), (48, 72), (72, 96)], [(10, 11), (53, 54)])
[(0, 24), (24, 48)]   # expected [(24, 48), (72, 96)]
>>> interval_diff([(0, 24), (24, 48)], [(10, 11)])
[]                    # expected [(24, 48)]

The fix skips the candidate interval on any overlap and keeps the unready interval, because it can also overlap the next candidate. Once the unready intervals run out, every remaining candidate starts after the last one, so all of them are kept. The uninterrupted (depends-on-past) early return is unchanged.

Test Plan

• Added two cases to test_interval_diff for a large interval that contains a smaller unready interval.
• Added test_signals_unready_hourly_parent_blocks_daily_child: a daily INCREMENTAL_BY_TIME_RANGE model downstream of an hourly model with a signal that holds back two hours. On main the child's batches are [(2023-01-01, 2023-01-03)], which evaluates Jan 1 and skips Jan 4. With the fix they are [(2023-01-02, 2023-01-03), (2023-01-04, 2023-01-05)].
• Both new tests fail on main (263723f) and pass with the fix.
• A randomized comparison against a brute-force overlap filter (20k cases, including uninterrupted=True) matches.
• pytest tests/core/test_scheduler.py tests/core/test_snapshot.py: 159 passed.
• tests/core/test_plan.py, test_plan_evaluator.py, test_context.py and tests/core/integration show the same results with and without the change. The only failures are dbt tests that need dbt, which isn't installed in my environment.
• ruff check, ruff format --check and mypy sqlmesh/core/scheduler.py are clean for the changed files.

Checklist

• I have run make style and fixed any issues
• I have added tests for my changes (if applicable)
• All existing tests pass (make fast-test)
• My commits are signed off (git commit -s) per the DCO

I ran ruff and mypy directly rather than the full pre-commit suite. I ran the core scheduler, snapshot, plan and integration suites rather than the full make fast-test.

This fix was found and written with AI assistance (Claude).

[mday-io]

Thanks for the clear write-up and repro. The fix looks right: any overlap now drops the interval while keeping the unready interval for the next comparison, which handles a coarser child containing an unready parent interval. Removing the stale post-loop comparison also stops ready intervals after an unready one from being dropped. The tests cover both cases and fail on main.

One optional suggestion, fine as a follow-up: docs/guides/signals.md doesn't mention that downstream models wait for any overlapping unready upstream interval. A sentence there, e.g. "a daily model downstream of an hourly model with a signal won't run a day until every hour in it is ready", would help users who see a day stop filling after this change.

Approving once CI is green.