Skip to content

feat: scheduling jobs #12863

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 69 commits into from
Jul 18, 2025
Merged

feat: scheduling jobs #12863

merged 69 commits into from
Jul 18, 2025

Conversation

AlessioGr
Copy link
Member

@AlessioGr AlessioGr commented Jun 19, 2025

Solves #11611 and #12965

Adds a new schedule property to workflow and task configs that can be used to have Payload automatically queue jobs following a certain schedule.

Docs: https://payloadcms.com/docs/jobs-queue/schedules

API Example

export default buildConfig({
  // ...
  jobs: {
    // ...
    scheduler: 'manual', // Or `cron` if you're not using serverless. If `manual` is used, then user needs to set up running /api/payload-jobs/handleSchedules or payload.jobs.handleSchedules in regular intervals
    tasks: [
      {
        schedule: [
          {
            cron: '* * * * * *',
            queue: 'autorunSecond',
            // Hooks are optional
            hooks: {
              // Not an array, as providing and calling `defaultBeforeSchedule` would be more error-prone if this was an array
              beforeSchedule: async (args) => {
                // Handles verifying that there are no jobs already scheduled or processing.
                // You can override this behavior by not calling defaultBeforeSchedule, e.g. if you wanted
                // to allow a maximum of 3 scheduled jobs in the queue instead of 1, or add any additional conditions
                const result = await args.defaultBeforeSchedule(args)
                return {
                  ...result,
                  input: {
                    message: 'This task runs every second',
                  },
                }
              },
              afterSchedule: async (args) => {
                await args.defaultAfterSchedule(args) // Handles updating the payload-jobs-stats global
                args.req.payload.logger.info(
                  'EverySecond task scheduled: ' +
                  (args.status === 'success' ? args.job.id : 'skipped or failed to schedule'),
                )
              },
            },
          },
        ],
        slug: 'EverySecond',
        inputSchema: [
          {
            name: 'message',
            type: 'text',
            required: true,
          },
        ],
        handler: ({ input, req }) => {
          req.payload.logger.info(input.message)
          return {
            output: {},
          }
        },
      }
    ]
  }
})

@philipyong
Copy link

Excited for this PR!

Copy link
Contributor

@DanRibbens DanRibbens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just have some questions.

*
* @default 'manual'
*/
scheduler?: 'cron' | 'manual'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does the 'manual' option do in the code? If it is cron we run the cron schedule, but otherwise the endpoints are just available?

I'm trying to move quick on reviewing this so maybe I'm missing something.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is cron we run the cron schedule, but otherwise the endpoints are just available?

Correct - well, the endpoints are always available. If it is set to cron, we create cronjobs on init to handle scheduling. If it is manual, we do not.

An error is thrown if this is explicitly set and schedules are used

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is effectively the same?

scheduler: 'manual'
// scheduler: 'manual'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct - had it required explicitly like this to ensure the user thinks about what scheduler mode they want to use.

@AdrianMaj
Copy link
Contributor

Hey @AlessioGr any ETA on merging this one? It's crucial to one of my projects 😀

Copy link
Contributor

github-actions bot commented Jul 11, 2025

📦 esbuild Bundle Analysis for payload

This analysis was generated by esbuild-bundle-analyzer. 🤖

Meta File Out File Size (raw) Note
packages/next/meta_index.json esbuild/index.js 748.10 KB ⚠️ +448 B (+0.1%)
packages/payload/meta_index.json esbuild/index.js 1.20 MB ⚠️ +7.78 KB (+0.7%)
packages/payload/meta_shared.json esbuild/exports/shared.js 160.53 KB ⚠️ +120 B (+0.1%)
packages/richtext-lexical/meta_client.json esbuild/exports/client_optimized/index.js 259.22 KB ✅ No change
packages/ui/meta_client.json esbuild/exports/client_optimized/index.js 1.13 MB ⚠️ +5.92 KB (+0.5%)
packages/ui/meta_shared.json esbuild/exports/shared_optimized/index.js 14.10 KB ✅ No change
Largest paths These visualization shows top 20 largest paths in the bundle.

Meta file: packages/next/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules
Unable to render expression.

${{\color{Goldenrod}{ ████████████████████▎ }}}$
81.0%, 601.90 KB
dist/views/Version
Unable to render expression.

${{\color{Goldenrod}{ █▋ }}}$
6.6%, 49.35 KB
dist/views/Document
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.9%, 14.20 KB
dist/views/Root
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.9%, 6.67 KB
dist/views/Versions
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 6.17 KB
dist/views/List
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 5.94 KB
dist/views/API
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 5.90 KB
dist/views/Account
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.7%, 5.33 KB
dist/elements/Nav
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.7%, 4.84 KB
dist/elements/DocumentHeader
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 4.63 KB
dist/views/Login
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 4.38 KB
dist/views/Dashboard
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.5%, 3.68 KB
dist/views/ForgotPassword
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 3.09 KB
dist/layouts/Root
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 3.09 KB
dist/utilities/initPage
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 3.01 KB
dist/templates/Default
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 2.83 KB
dist/views/BrowseByFolder
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.3%, 2.59 KB
dist/views/CreateFirstUser
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.3%, 2.48 KB
dist/views/CollectionFolders
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.3%, 2.46 KB
dist/views/ResetPassword
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.3%, 2.41 KB
(other)
Unable to render expression.

${{\color{Goldenrod}{ ████▊ }}}$
19.0%, 141.52 KB

Meta file: packages/payload/meta_index.json, Out file: esbuild/index.js

Path Size
../../node_modules
Unable to render expression.

${{\color{Goldenrod}{ █████████████████▌ }}}$
70.2%, 841.01 KB
dist/fields/hooks
Unable to render expression.

${{\color{Goldenrod}{ ▊ }}}$
3.3%, 40.06 KB
dist/collections/operations
Unable to render expression.

${{\color{Goldenrod}{ ▋ }}}$
2.8%, 33.32 KB
dist/auth/operations
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.2%, 14.58 KB
dist/queues/operations
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.0%, 11.92 KB
dist/utilities/configToJSONSchema.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.0%, 11.54 KB
dist/globals/operations
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.9%, 11.03 KB
dist/fields/config
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.9%, 10.82 KB
dist/fields/validations.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 9.35 KB
dist/bin/generateImportMap
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.7%, 8.27 KB
dist/database/migrations
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.7%, 7.79 KB
dist/uploads/fetchAPI-multipart
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 7.74 KB
dist/collections/config
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 7.58 KB
dist/index.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 7.07 KB
dist/collections/endpoints
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 6.87 KB
dist/config/orderable
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.5%, 6.25 KB
dist/config/sanitize.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.5%, 5.44 KB
dist/auth/endpoints
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.5%, 5.41 KB
dist/utilities/telemetry
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 5.31 KB
dist/auth/strategies
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 4.68 KB
(other)
Unable to render expression.

${{\color{Goldenrod}{ ███████▍ }}}$
29.8%, 356.63 KB

Meta file: packages/payload/meta_shared.json, Out file: esbuild/exports/shared.js

Path Size
../../node_modules
Unable to render expression.

${{\color{Goldenrod}{ ████████████████████ }}}$
80.2%, 126.19 KB
dist/fields/validations.js
Unable to render expression.

${{\color{Goldenrod}{ █▍ }}}$
5.9%, 9.35 KB
dist/fields/baseFields
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.8%, 2.79 KB
dist/utilities/deepCopyObject.js
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.6%, 2.48 KB
dist/auth/cookies.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.0%, 1.55 KB
dist/utilities/flattenTopLevelFields.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.9%, 1.41 KB
dist/fields/config
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 1.28 KB
dist/utilities/flattenAllFields.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 943 B
dist/utilities/fieldSchemaToJSON.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 917 B
dist/folders/utils
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 881 B
dist/utilities/unflatten.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.5%, 779 B
dist/utilities/sanitizeUserDataForEmail.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.5%, 713 B
dist/collections/config
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 570 B
dist/bin/generateImportMap
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 559 B
dist/utilities/getSafeRedirect.js
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.3%, 423 B
dist/utilities/deepMerge.js
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.3%, 413 B
dist/utilities/getFieldPermissions.js
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.2%, 391 B
dist/utilities/formatLabels.js
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.2%, 380 B
dist/utilities/mergeListSearchAndWhere.js
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.2%, 344 B
dist/utilities/getBestFitFromSizes.js
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.2%, 340 B
(other)
Unable to render expression.

${{\color{Goldenrod}{ ████▉ }}}$
19.8%, 31.16 KB

Meta file: packages/richtext-lexical/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
dist/lexical/plugins
Unable to render expression.

${{\color{Goldenrod}{ ███ }}}$
12.1%, 30.91 KB
dist/features/experimental_table
Unable to render expression.

${{\color{Goldenrod}{ ██▍ }}}$
9.6%, 24.60 KB
dist/lexical/ui
Unable to render expression.

${{\color{Goldenrod}{ ██▎ }}}$
9.2%, 23.47 KB
dist/features/blocks
Unable to render expression.

${{\color{Goldenrod}{ ██▏ }}}$
8.8%, 22.43 KB
dist/packages/@lexical
Unable to render expression.

${{\color{Goldenrod}{ █▊ }}}$
7.4%, 18.99 KB
dist/features/link
Unable to render expression.

${{\color{Goldenrod}{ █▊ }}}$
7.0%, 17.96 KB
dist/features/toolbars
Unable to render expression.

${{\color{Goldenrod}{ █▋ }}}$
6.8%, 17.44 KB
dist/features/textState
Unable to render expression.

${{\color{Goldenrod}{ █ }}}$
4.3%, 10.98 KB
dist/features/upload
Unable to render expression.

${{\color{Goldenrod}{ ▉ }}}$
3.7%, 9.60 KB
dist/features/relationship
Unable to render expression.

${{\color{Goldenrod}{ ▉ }}}$
3.7%, 9.41 KB
dist/lexical/utils
Unable to render expression.

${{\color{Goldenrod}{ ▊ }}}$
3.2%, 8.08 KB
dist/features/debug
Unable to render expression.

${{\color{Goldenrod}{ ▋ }}}$
2.9%, 7.34 KB
dist/features/converters
Unable to render expression.

${{\color{Goldenrod}{ ▋ }}}$
2.7%, 7.04 KB
dist/utilities/fieldsDrawer
Unable to render expression.

${{\color{Goldenrod}{ ▋ }}}$
2.7%, 7.01 KB
dist/lexical/config
Unable to render expression.

${{\color{Goldenrod}{ ▌ }}}$
2.0%, 5.10 KB
dist/features/lists
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.9%, 4.95 KB
dist/lexical/theme
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.6%, 4.01 KB
dist/features/format
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.4%, 3.46 KB
dist/lexical/LexicalEditor.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.3%, 3.23 KB
dist/features/indent
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.0%, 2.50 KB
(other)
Unable to render expression.

${{\color{Goldenrod}{ █████████████████████▉ }}}$
87.9%, 225.29 KB

Meta file: packages/ui/meta_client.json, Out file: esbuild/exports/client_optimized/index.js

Path Size
../../node_modules
Unable to render expression.

${{\color{Goldenrod}{ ████████████▊ }}}$
51.3%, 572.91 KB
dist/elements/FolderView
Unable to render expression.

${{\color{Goldenrod}{ ▋ }}}$
2.7%, 30.35 KB
dist/elements/BulkUpload
Unable to render expression.

${{\color{Goldenrod}{ ▌ }}}$
2.2%, 24.52 KB
dist/elements/WhereBuilder
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.6%, 17.94 KB
dist/fields/Relationship
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.4%, 15.93 KB
dist/elements/Table
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.4%, 15.16 KB
dist/views/Edit
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.3%, 14.78 KB
dist/forms/Form
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.3%, 14.21 KB
dist/fields/Blocks
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.1%, 12.37 KB
dist/fields/Upload
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.1%, 11.94 KB
dist/fields/Array
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 9.29 KB
dist/elements/ListControls
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 9.04 KB
dist/elements/PublishButton
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 8.57 KB
dist/providers/Folders
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.8%, 8.38 KB
dist/elements/LivePreview
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.7%, 8.18 KB
dist/elements/HTMLDiff
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.7%, 7.81 KB
dist/views/CollectionFolder
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.7%, 7.66 KB
dist/elements/ReactSelect
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 6.91 KB
dist/fields/Tabs
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 6.90 KB
dist/views/List
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 6.70 KB
(other)
Unable to render expression.

${{\color{Goldenrod}{ ████████████▏ }}}$
48.7%, 544.64 KB

Meta file: packages/ui/meta_shared.json, Out file: esbuild/exports/shared_optimized/index.js

Path Size
dist/graphics/Logo
Unable to render expression.

${{\color{Goldenrod}{ █████▊ }}}$
23.1%, 3.12 KB
../../node_modules
Unable to render expression.

${{\color{Goldenrod}{ ████▉ }}}$
19.6%, 2.65 KB
dist/graphics/Icon
Unable to render expression.

${{\color{Goldenrod}{ ██▊ }}}$
11.3%, 1.52 KB
dist/utilities/formatDocTitle
Unable to render expression.

${{\color{Goldenrod}{ ██▍ }}}$
9.8%, 1.32 KB
dist/utilities/groupNavItems.js
Unable to render expression.

${{\color{Goldenrod}{ █▌ }}}$
6.0%, 814 B
dist/providers/TableColumns
Unable to render expression.

${{\color{Goldenrod}{ █▍ }}}$
5.6%, 757 B
dist/utilities/api.js
Unable to render expression.

${{\color{Goldenrod}{ █▍ }}}$
5.6%, 756 B
dist/elements/Translation
Unable to render expression.

${{\color{Goldenrod}{ ▉ }}}$
3.7%, 493 B
dist/elements/withMergedProps
Unable to render expression.

${{\color{Goldenrod}{ ▋ }}}$
2.5%, 339 B
dist/utilities/handleTakeOver.js
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.9%, 251 B
dist/elements/WithServerSideProps
Unable to render expression.

${{\color{Goldenrod}{ ▍ }}}$
1.7%, 232 B
dist/utilities/handleGoBack.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.2%, 168 B
dist/fields/mergeFieldStyles.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.2%, 159 B
dist/forms/Form
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.1%, 147 B
dist/utilities/abortAndIgnore.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.1%, 146 B
dist/utilities/hasSavePermission.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.0%, 136 B
dist/utilities/handleBackToDashboard.js
Unable to render expression.

${{\color{Goldenrod}{ ▎ }}}$
1.0%, 129 B
dist/utilities/findLocaleFromCode.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 84 B
dist/utilities/sanitizeID.js
Unable to render expression.

${{\color{Goldenrod}{ ▏ }}}$
0.6%, 77 B
dist/utilities/isEditing.js
Unable to render expression.

${{\color{Goldenrod}{  }}}$
0.4%, 59 B
(other)
Unable to render expression.

${{\color{Goldenrod}{ ███████████████████▏ }}}$
76.9%, 10.38 KB
Details

Next to the size is how much the size has increased or decreased compared with the base branch of this PR.

  • ‼️: Size increased by 20% or more. Special attention should be given to this.
  • ⚠️: Size increased in acceptable range (lower than 20%).
  • ✅: No change or even downsized.
  • 🗑️: The out file is deleted: not found in base branch.
  • 🆕: The out file is newly found: will be added to base branch.

…ecting other tests
…have random auto scheduled jobs in the background in this specific test suite
@AlessioGr AlessioGr merged commit c08b2ae into main Jul 18, 2025
81 checks passed
@AlessioGr AlessioGr deleted the feat/schedule-jobs branch July 18, 2025 10:48
@matthijs166
Copy link

Can't wait to try it out!

@JeffinJ
Copy link

JeffinJ commented Jul 24, 2025

I have updated to the latest version, but adding the scheduler: 'cron', to build config, based on this documentation throws an error 'scheduler' does not exist in type 'JobsConfig'.
Checked the types, and it doesn't exist there.
Has anyone had a chance to try this feature yet?

Version used: "payload": "3.48.0",

@HarleySalas
Copy link
Contributor

This enables us to do a lot of great things. Thank you!

Now it should be trivial to throttle large amounts of background jobs, instead of having to wait a whole minute between each batch.

@bratvanov
Copy link
Contributor

@JeffinJ

Version used: "payload": "3.48.0",

The scheduling jobs feature was released in 3.49.0 just a few minutes ago. Try updating again and give it a go!

@whizzbbig
Copy link

Love you @AlessioGr ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants