Skip to content

fix: reconcile interrupted schedule executions - #6780

Open
BillLeoutsakosvl346 wants to merge 2 commits into
stagingfrom
codex/schedule-recovery-reconciliation
Open

fix: reconcile interrupted schedule executions#6780
BillLeoutsakosvl346 wants to merge 2 commits into
stagingfrom
codex/schedule-recovery-reconciliation

Conversation

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor

Summary

Reconcile claimed schedule jobs after outages from persisted workflow execution logs instead of redispatching them. Untouched database jobs still execute only while pending with zero attempts, while claimed jobs are resolved as success, failure, cancellation, pause, or an indeterminate failure. Schedule accounting uses the existing occurrence claim guard, and cleanup leaves active schedule carriers to schedule recovery.

This intentionally keeps database and Trigger.dev recovery independent. Switching queue providers while schedule jobs are outstanding remains unsupported; deployments must drain outstanding schedule jobs before changing providers.

No schema migration, public API change, or new module is introduced.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other

Testing

  • 72 focused schedule execution and cleanup tests
  • TypeScript type-check
  • Biome on all touched files
  • API validation audit
  • SQL date-binding audit
  • git diff --check

Reviewer focus: at-most-once behavior after a carrier is claimed, claim-guarded schedule accounting, cancellation handling, and cleanup ownership.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

Not applicable; server-side recovery change.

@vercel

vercel Bot commented Aug 17, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 17, 2026 7:50pm

Request Review

@cursor

cursor Bot commented Aug 17, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Changes core schedule at-most-once recovery, occurrence claim guards, and async-job cleanup ownership—incorrect reconciliation could double-run or skip scheduled occurrences.

Overview
After outages, claimed schedule carrier jobs are reconciled from persisted workflow execution logs instead of being redispatched. Outcomes map to success, failure, cancellation, pause, or an indeterminate failure; only pending carriers with zero attempts still run inline.

The schedule tick adds recovery for stale processing, claimed-pending, and unreconciled terminal carriers, with claim-guarded applyScheduleSuccessUpdate / applyScheduleCancellationUpdate / failure accounting and metadata flags (scheduleReconciled, scheduleRecoveryIrrecoverable). Carrier lookup or enqueue uncertainty preserves the occurrence rather than deferring cadence via infra retry.

Stale execution cleanup terminalizes both running and redacting logs, excludes schedule-execution async jobs from generic stale pending/processing handling, and retains schedule carriers until reconciled (irrecoverable tombstones are not pruned).

Reviewed by Cursor Bugbot for commit c7a839a. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit c7a839a. Configure here.

Comment thread apps/sim/app/api/schedules/execute/route.ts
@greptile-apps

greptile-apps Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR changes schedule outage recovery to reconcile already-claimed carriers from persisted workflow execution logs instead of redispatching them.

  • Separates untouched pending database jobs from previously claimed jobs using the attempt count.
  • Reconciles successful, failed, cancelled, paused, and indeterminate outcomes through occurrence-guarded schedule accounting.
  • Preserves claims when queue lookup or enqueue acceptance is uncertain.
  • Transfers active schedule-carrier cleanup to schedule recovery and retains terminal carriers until reconciliation is recorded.
  • Extends stale workflow-log cleanup to interrupted redaction finalization.

Confidence Score: 5/5

The PR appears safe to merge; no concrete blocking or independently actionable non-blocking defect remains.

Claimed carriers are reconciled without redispatch, database settlement and schedule accounting are transactionally coordinated, claim guards prevent stale occurrences from advancing newer schedule state, and cleanup predicates preserve carriers until reconciliation completes.

Important Files Changed

Filename Overview
apps/sim/app/api/schedules/execute/route.ts Introduces log-backed carrier reconciliation, occurrence-guarded accounting, ambiguous-enqueue preservation, and distinct handling for untouched versus claimed database jobs.
apps/sim/background/schedule-execution.ts Extracts reusable claim-guarded success and cancellation accounting helpers used by normal execution and recovery.
apps/sim/app/api/cron/cleanup-stale-executions/route.ts Assigns schedule carriers to schedule-specific recovery, retains unreconciled terminal carriers, and terminalizes stale redaction logs.
apps/sim/app/api/schedules/execute/route.test.ts Adds broad coverage for recovered outcomes, cancellation, released and newer claims, malformed carriers, ambiguous queue operations, and at-most-once execution.
apps/sim/app/api/cron/cleanup-stale-executions/route.test.ts Verifies redacting-log cleanup and schedule-carrier ownership and retention predicates.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Tick[Schedule cron tick] --> Claim[Claim due schedule occurrence]
  Claim --> Lookup[Look up deterministic carrier]
  Lookup -->|No carrier| Enqueue[Create and enqueue execution carrier]
  Lookup -->|Untouched pending, attempts = 0| Execute[Execute database fallback job]
  Lookup -->|Claimed or terminal carrier| Evidence[Read persisted workflow execution log]
  Evidence --> Classify{Classify outcome}
  Classify -->|Completed / paused| Success[Apply guarded success accounting]
  Classify -->|Cancelled| Cancel[Apply guarded cancellation accounting]
  Classify -->|Failed / indeterminate| Failure[Apply guarded failure accounting]
  Success --> Mark[Mark carrier reconciled]
  Cancel --> Mark
  Failure --> Mark
  Mark --> Cleanup[Retention cleanup may delete carrier]
  Enqueue -->|Acceptance uncertain| Preserve[Preserve occurrence claim for later reconciliation]
Loading

Reviews (1): Last reviewed commit: "simplify schedule recovery provider hand..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant