Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 17 additions & 12 deletions docs/build/triggers.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,18 +113,16 @@ After the workflow completes, sometimes seconds (or minutes) later.
}
```

## Cron Triggers (formerly timers)
## Cron Triggers

**Cron Triggers** run Workflows based on a cron schedule, and are good for
repetitive tasks that are time-based (e.g., every day at 8am, sync financial
data).
data between two systems).

These Triggers enable users to pull data from connected systems. You can pick
These Triggers enable users to "pull" data from connected systems. You can pick
a standard schedule (e.g., every day, or every month), or define a custom
schedule using cron expressions.

![Cron Trigger](/img/cron_trigger.webp)

:::tip Help with cron expressions

The best way to learn about `cron`, if you're not already familiar, is through
Expand All @@ -137,15 +135,22 @@ Cron Triggers enable Workflows to be run as frequently as once every minute, or
as infrequently as you desire and can be scheduled on very specific dates or
times.

Learn how a workflow's initial `state` gets built from a cron trigger
[here](/documentation/jobs/state#cron-triggered-runs).
### Input `state` for the next run

Every time a cron-triggered workflow is run it will _start_ with the final
output of the last successful run. This allows users to build workflows that
make use of a ["cursor"](/documentation/jobs/job-writing-guide#using-cursors)
that tracks what happened last time the workflow ran. (Only processing data that
changed since that last run, for example.)

![Cron Trigger](/img/cron_trigger.webp)

You can use a Cursor to help build input state when the workflow is triggered:
see the [Job Writing Guide](/documentation/jobs/job-writing-guide#using-cursors)
for more details.
Be default, the input state for the next cron run will be the final output state
of the previous run, but you can configure this to use the output state from a
specific step in your earlier run by changing the "Cron Input Source".

Each time a timed job succeeds, its `final_state` will be saved and used as the
input state for its next run.
More on `state` in cron-triggered runs can be found in the
["Input and Output State"](/documentation/jobs/state#cron-triggered-runs) docs.

### Managing the size of `state` for Cron Workflows

Expand Down
48 changes: 30 additions & 18 deletions docs/jobs/state.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ State is just a Javascript object. It is the means via which Jobs share
information between each other. It also provides a common scope for Operations
to read from and write to.

The final state form a Job must always be a serializable Javascript object (ie,
a JSON object). Any non-serializable keys will be removed.
The final state form a Job must always be a serializable Javascript object
(i.e., a JSON object). Any non-serializable keys will be removed.

![Job State Overview](/img/state-javascript.webp)

Expand All @@ -35,16 +35,16 @@ State objects tend to have the following keys:
name.

At the end of a Job, the configuration key will be removed, along with any other
non serialisable keys.
non serializable keys.

Adaptors will occasionally write extra information to state during a run - for
example, database Adaptors tend to write a `client` key to state, used to track
the database connection. These will be removed at the end of a Job.

## Input & output state for runs

Depending on whether you're running Workflows locally via the CLI or on the app, the input
state for a Run must be generated differently:
Depending on whether you're running Workflows locally via the CLI or on the app,
the input state for a Run must be generated differently:

- When manually creating a work order, you must select or generate your input
manually (e.g., by creating a custom `Input` on the app or `state.json` file
Expand All @@ -54,24 +54,25 @@ state for a Run must be generated differently:

The final state of a Run is determined by what's returned from the last
operation. Remember that job expressions are a series of operations: they each
take state and return state, after creating any number of side effects. The final returned
state controls what is output by the run at the end of all of these operations.
take state and return state, after creating any number of side effects. The
final returned state controls what is output by the run at the end of all of
these operations.

Best practice is to include a final state cleanup step that removes any data
that should not persist between runs or be output (like PII), for example:

```js
// get data from a data source
get('https://jsonplaceholder.typicode.com/users')
get('https://jsonplaceholder.typicode.com/users');

// store retrieved data in state for use later in job
fn(state => {
state.users = state.data;
state.users = state.data;
return state;
});

// get more data from another data source
get('https://jsonplaceholder.typicode.com/posts')
get('https://jsonplaceholder.typicode.com/posts');

// store additional retrieved data in state for use later in job
fn(state => {
Expand All @@ -89,10 +90,10 @@ fn(state => {

// cleanup state at the end before finishing job
fn(state => {
state.data = null
state.users = null
state.posts = null
state.data = null;
state.users = null;
state.posts = null;

return state;
});
```
Expand Down Expand Up @@ -148,20 +149,31 @@ The input state looks like this:

### Cron triggered runs

When a run is triggered by a cron job, its input state will be the output of the **first step** from the previous run. This allows each subsequent run to know about previous runs. In other words, you can pass information from one run to another even if they happen days apart.
When a run is triggered by a cron job, its input state will be the final state
of the previous run. This allows each subsequent run to know about previous
runs. In other words, you can pass information from one run to another even if
they happen days apart.

**Example scenario**: You have a **daily sync at 9 AM** with a workflow that has 3 steps: (1) fetch patient records, (2) transform data, (3) send to database. On Monday, the **first step** processes records up to ID 1000 and outputs `{ lastProcessedId: 1000 }`. Even though steps 2 and 3 modify the state further, only the **first step's output** gets saved for the next cron run. On Tuesday at 9 AM, the cron job starts again with `{ lastProcessedId: 1000 }` from Monday's first step, so it knows to fetch records starting from ID 1001.
**Example scenario**: You have a **daily sync at 9 AM** with a workflow that has
3 steps: (1) fetch patient records, (2) transform data, (3) send to database. On
Monday, the workflow processes records up to ID 1000 and outputs
`{ lastProcessedId: 1000 }` as its final state. On Tuesday at 9 AM, the cron job
starts again with `{ lastProcessedId: 1000 }` as its input, so it knows to fetch
and process records starting from ID 1001.

The first time the workflow runs, the initial state will simply be an empty JavaScript object: `{}`
The first time the workflow runs, the initial state will simply be an empty
JavaScript object: `{}`

#### Overriding cron input

You can always manually run a cron-triggered workflow with:

- **Empty input**: `{}` - starts fresh without previous state.
- **Custom input**: Your own data to test specific scenarios.
- **Default input**: Uses the same input as the scheduled runs.

If the manual run succeeds, the next scheduled cron run will start with whatever output state your manual run produced.
If the manual run succeeds, the next scheduled cron run will start with whatever
output state your manual run produced.

## Input & output state for steps

Expand Down
Binary file modified static/img/cron_trigger.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading