Notifications: email, webhooks and Slack
We tell you when a pipeline needs you: by email, by a signed webhook, or in a Slack channel. Every notification is about one condition (a pipeline that stopped, a sync that is late, your database holding too much log…), sent once when it begins and, if you want, once more when it ends.
What you are told about
Section titled “What you are told about”Event (trigger) |
Severity | Sent when | Ends when |
|---|---|---|---|
new_error |
error | a pipeline stops and will not restart on its own (one notification until it recovers: starting it again without a recovery sends nothing more, and the next stop after a recovery is a new one) | it makes progress, or a run stays up an hour |
retrying |
error | a pipeline keeps failing to start, from the second failed attempt | it makes progress or a run stays up an hour; a stop takes it over |
out_of_credit |
error | your account runs out of credit (sent whenever an address is set, whatever your switches say) | the wallet no longer paused: credit brings the balance above zero, the account becomes invoiced monthly, or, for an account that has never had a top-up, a credit grant or a paid invoice, refunds of runs that did not finish bring it back to zero; the notice says which, and if a charge has taken the balance back to zero or below since the wait ended, it names the balance and says the account is paused again at its next hourly check unless credit brings the balance above zero first (or, for an account that has never had a top-up, a credit grant or a paid invoice, unless refunds of runs that did not finish bring it back to zero). Credit smaller than the amount below zero ends nothing |
low_balance |
warning | a prepaid account’s balance lasts fewer days than your setting at the last week’s spend (7 by default), or is at or below your amount (sent to every owner, whatever their switches say) | the balance lasting at least 1.25 times the days at the recent spend (or no recent spend to measure by), above zero and above the amount, or the account becoming invoiced monthly |
spend_threshold |
warning | this UTC month’s billed charges pass 50% or 80% of your monthly budget, or 100% of a budget that only notifies; a budget set to stop new runs is told no step once the month reaches it, because budget_reached says so (sent to every owner, whatever their switches say) |
the next UTC month, the budget removed, or a higher step taking it over |
budget_reached |
error | a budget set to stop new runs is reached: scheduled runs wait, and the notice names the pipelines that stream continuously, with what each was charged this month, as pipelines that stream continuously are never stopped by a budget (sent to every owner, whatever their switches say) | the budget’s stop lifting: next month (UTC), an owner raising the budget above this month’s charges, an owner removing the budget or setting it to tell owners only, or refunds of runs that did not finish bringing this month’s charges back under it |
spend_spike |
warning | a pipeline’s metered spend on a UTC day is 3 times or more the median of the 7 days before it, at least $1.00 above that median, and above every one of those 7 days (so a weekly job whose last week already had a day this large is not told). Metered is the usage the meter measured, priced at the rate card in force that day and rounded down to the cent as the charge is, leaving out runs that had already ended in failure: it is not your bill, and a run of that day that fails later makes the day cost less. Each check judges the last 3 closed UTC days once the meter has finished them, so it usually comes the next morning (UTC), and a day measured late still comes. Not judged: a pipeline’s first 8 days (it has no week to compare), a week that mostly metered nothing, and a week in which our rate card changed (a change could be our price, not your usage). A pipeline that ran little while paused and catches up afterwards can be told, because that day is large against its week. One notice per spike, however long it lasts, even when it returns only some days a week. A rate card that changes | |
| inside every week would keep the check from judging at all; ours changes far less often. Emailed even with warnings off, until you switch warnings yourself; choosing it off per event, or muting the pipeline’s notifications, silences it. It changes nothing on the pipeline | a whole week after the spike, once the meter has finished it, in which EVERY day metered less than 3 times the median the notice named (of the 7 days before the spike). One low day, a day of failed runs or a day the account was held does not end it: a spend that stays that high, even some days a week, is one notice, never ended | ||
sync_delayed |
warning | a scheduled sync is 60 minutes past due (45 minutes after the grace you set, 15 by default). Never for a scheduled pipeline a reached budget holds or one waiting for credit: those read budget_reached or out_of_credit, and budget_reached or out_of_credit tells you once. Once the pipeline is no longer held, it says the hold skipped the runs, and what has happened since: nothing tried (it runs at its next scheduled time), a run that is still going (including one whose start has only just begun, or a restart after a failed attempt), or a run tried that failed (a start refused, a restart the service is waiting to make, a launch whose saved credentials could not be read, or a give-up) |
a sync completes |
keeps_stopping |
warning | a pipeline keeps stopping and restarting | a run stays up an hour or completes; a stop takes it over |
rows_discarded |
warning | your destination accepted rows and did not keep them | a day with no new report |
values_fitted |
warning | values were changed when written: cut, written as NULL, rewritten as your destination stores them, or a NUL character removed | a day with no new report |
source_log_held |
warning | your database holds a lot of log (WAL) for a pipeline: the slot holds half of its room (what it holds plus what it may still keep before your database gives it up), or 20 GiB when it sets no limit. Both can be changed per pipeline; a smaller percentage warns earlier. Emailed even with warnings off, until you switch warnings yourself: a full disk stops your own database | a measurement below 80% of the threshold or our watch finding the slot gone (stopping or pausing the pipeline does not end it: the slot keeps holding log, and while the pipeline does not run we read the slot about every 10 minutes). Every message says when the figure was measured, and a figure that has not been measured since says so |
source_log_unmeasured |
warning | we cannot tell how much log (WAL) your database holds for a pipeline: the newest measurement is no longer current (5 minutes old for a running pipeline, 25 for our watch of a stopped one) and was below the threshold, or our watch of a stopped pipeline’s slot could not read it: it could not reach your database, or your database turned the read down (its sign-in, the connection’s settings or its certificate), or the SSH bastion or proxy agent it is reached through would not forward the read (the message names which, and what to check there: the bastion’s AllowTcpForwarding and permitopen, or the agent’s allow list). A stale figure is never read as safe. Sent once it has stood 30 minutes, so a restart is never a message. Emailed even with warnings off, until you switch warnings yourself, for source_log_held’s reason |
a current measurement, or our watch finding the slot gone from your database (stopping or pausing the pipeline does not end it) |
source_slot_invalidating |
warning | your database will invalidate a stopped pipeline’s replication slot because nothing reads it: PostgreSQL 18’s idle_replication_slot_timeout is set, and our watch predicts the moment (the slot’s inactive_since plus the setting; your database acts at its first checkpoint after it). Told whatever the slot holds, since the setting exists to keep slots small, and after it the pipeline’s tables must be copied again. Emailed even with warnings off, until you switch warnings yourself, for source_log_held’s reason |
the pipeline reading its slot again, our watch no longer predicting the invalidation, or the slot gone (a stop does not end it) |
source_slot_lost |
warning | our watch found a stopped pipeline’s replication slot invalidated by your database, or gone from it after we had measured it: the changes your database made since the pipeline last read it cannot be streamed, and the pipeline’s next start says what it needs, usually copying its tables again. A slot nothing ever measured is not reported missing. Emailed even with warnings off, until you switch warnings yourself | the pipeline reading its source again, or our watch measuring the slot again (a stop does not end it) |
no_data_moved |
warning | no data has moved for longer than the threshold you set (off until you set one) | a batch is applied |
data_stale |
warning | what your destination holds is older than the freshness threshold you set, counted beyond the hold your schedule gives the pipeline (off until you set one, 5 minutes to a week). Only for a PostgreSQL source read by logical replication (the only kind that reports its clock; the threshold is refused for any other), while it runs and nothing else is wrong with it: a scheduled sync that is late is sync_delayed. A quiet source does not raise it: once replication passes its end of log, it is proved current, and a worker restart keeps that proof (How current your destination is). On a standby source it is measured to the last change the standby replayed, so a primary with no changes for the threshold raises it too. The message says when the data was last current and how far behind that is |
newer evidence showing the data within 80% of the threshold beyond the hold, or the threshold removed |
source_agent_offline |
error | a proxy agent’s pipelines have waited five minutes for it (it is not connected, or its key is not approved): one message per agent, about the account, naming the agent, since when and how many pipelines wait | no pipeline waits for the agent any more: it connected and they started again (the resolved message says the agent is back) |
agent_flapping |
warning | a proxy agent dropped its connection three times in thirty minutes (its own drops: never a deploy, a replacement, a deletion or a key change); one per agent | thirty minutes without a drop of the agent’s own |
agent_duplicated |
warning | a proxy agent was replaced three times in ten minutes: two copies of it run, or its key file was copied; one per agent | ten minutes without the agent being replaced |
table_parked |
error | a table of a pipeline stopped and waits for you: an answer to a change it met, a statement to run on your destination, or anything else a table stops for (its other tables may carry on). Sent two minutes after the newest table stopped, so one migration that stops several tables is one message naming them all | no table waits any more; another table stopping meanwhile takes it over, with a new message naming every waiting table |
table_gone |
warning | a table was dropped or renamed at the source; its destination table keeps its rows. Sent two minutes after it stopped | no dropped or renamed table waits any more |
schema_changed |
warning | the pipeline followed a change to a source table’s shape (a column added, dropped or renamed, a type changed, a truncate) as its change policy says | a day with no new change |
old_slot_holds_log |
warning | a full re-sync gave a pipeline a new replication slot, and the old one is still on your source, keeping its log (WAL) for ever: the message names the slot and the statement that drops it. Emailed even with warnings off, until you switch warnings yourself, for source_log_held’s reason |
the pipeline finds the old slot gone from your source (stopping the pipeline does not end it, and while it is stopped nothing looks) |
source_certificate_expiring |
warning | a certificate of a pipeline’s source ends within 30 days: its client certificate, an uploaded CA certificate or the certificate you approved (a server certificate under a CA is not watched: its CA renews it). The message names each one, its subject, fingerprint and end date, and is sent once at 30 days and once more at 7. Emailed even with warnings off, until you switch warnings yourself: when it ends the pipeline stops, and only you can renew it | the source no longer has the certificates it named (you replaced them on the source); a certificate crossing into its last 7 days, or another one nearing its end, takes it over with a new message |
channel_disabled |
warning | we switched one of your channels off because it kept failing or its receiver said it is gone (emailed whenever an address is set) | you switch it back on, or delete it |
Nothing ends because an alert disappeared. One alert can hide another (running out of credit hides a delay), and our own restarts clear some for a moment, so each condition ends only on its own fact, above. A condition that ends because you stopped or deleted the pipeline is told to the webhooks that heard it begin and have recoveries on (so an incident a machine opened is closed), and not to email or Slack.
Billing notices (out_of_credit, low_balance, spend_threshold, budget_reached) go to every owner
whatever their switches say, and each says what holds your pipelines at that moment: credit, this month’s
budget, or both (then scheduled runs wait for both, and each notice lists what ends each wait). Credit lifts the account’s pause only when it brings the balance above zero (an account that has never had a
top-up, a credit grant or a paid invoice stops waiting once refunds of runs that did not finish bring it back to zero, and the
notices say so; an account given a credit grant is not one of them), and every
billing email sent while the account is out of credit names the balance. The notice that one is over is a recovery like
any other: it is emailed only while recoveries are on for your account, and the billing notice itself says
whether it will be. A webhook’s details.held_by carries the same fact: credit, budget,
credit_and_budget, or null.
Who is told
Section titled “Who is told”- Email goes to the address in Settings → Notifications. Errors are on and warnings off
until you change them; recoveries (one more email when something you were told about ends) are
on. Choose per event sets any event to always or never be emailed, whatever its switch says,
except running out of credit and a channel we switched off: those are always emailed while an
address is set, and the API answers a choice for them with
422. An event that began while you had it off is not sent when you turn it on. - Each person’s own email (Settings › Notifications, the Your notifications section, or
PUT /v1/me/notifications) goes to their sign-in address: the pipelines they create (the default), every pipeline, or none, with their own errors and warnings switches. Running out of credit goes to every owner, whatever they chose here (None included), and never to members; owners are also always told when an owner is added or removed, about the account’s API keys (API keys), and about single sign-on’s security events: a sign-in refused because it came from a different identity at your identity provider, a person answering No, it wasn’t me to a confirmation code (naming an owner invitation when the sign-in would have accepted one), a recovery owner signing in with an email link while single sign-on is required, single sign-on being turned off, allowed or required (with how many email-link sessions a switch to required ended and kept), and a verified domain whose DNS record the daily check did not find (on the first day, and again on the fourteenth, when the domain lapses). A confirmation code nobody entered tells nobody. Nobody gets the same email twice: a person whose address is the account’s address gets the account’s copy when the account’s switches send it, and their own when they do not (the account’s warnings off, their own on). A pipeline made before people could be told apart has no creator, so “the pipelines I create” does not cover it. Only an owner can change the account’s address, its switches and its channels, or resend. See Members and invitations. - Channels (Settings → Webhooks and Slack) each have their own errors, warnings and recoveries switches, per-event choices, and a pipeline filter. A channel hears about what begins after you add it. A notification waiting for a channel is dropped if the channel is off when we come to send it; switch it off and on again before then and it is still sent.
- Muting a pipeline (on its page, Settings › Notifications and when to warn me: Mute for an hour, Mute for a day or Mute, until Unmute) stops who is told about it, by email, webhook and Slack alike, never the alerts. It also stops a notification still being retried when you mute. Unmuting does not send what happened during the mute; a recovery of something you were already told about is still sent, and so is a Resend you ask for.
- Delivery history (Settings › Delivery history, or from Alerts) lists every notification of the last 30 days and where it went, with Resend for a delivery that finished.
Webhooks
Section titled “Webhooks”Adding one
Section titled “Adding one”Add a webhook in Settings → Webhooks and Slack (or POST /v1/notification-channels). Your browser
generates the signing secret, whsec_ followed by 32 random bytes in base64, and shows it once,
before you save: store it with your receiver, tick “I have stored this secret”, then save. We never
show it again, in the app or the API. Through the API you generate your own: whsec_ and 24 to 64
random bytes in standard base64.
The URL must be https, with no user name, password or # part, and its host must be public: we judge
the address we actually connect to, every time, and do not follow redirects. Any port is fine.
What we send
Section titled “What we send”Every delivery is a POST with a JSON body:
{ "id": "evt_6xq2m4w7t5xgz5v3b6n2k3rjkk", "type": "pipeline.alert.raised", "version": 1, "timestamp": "2026-10-04T07:12:03Z", "test": false, "data": { "account": {"id": "t_acme"}, "pipeline": {"id": "orders", "name": "orders", "url": "https://app.example.com/pipelines/orders"}, "alert": {"id": "alr_…", "kind": "new_error", "severity": "error", "phase": "raised", "started_at": "2026-10-04T07:12:03Z"}, "details": {}, "summary": "Syncing is blocked on orders." }}typeispipeline.alert.raised,pipeline.alert.resolved,account.alert.raisedoraccount.alert.resolved. A resolve carriesresolved_atandreason(recovered,stopped,deletedorsuperseded).alert.idis the same on a raise and its resolve.timestampis when the condition began (a raise) or ended (a resolve).detailscarries numbers per event (for exampleattemptandnext_attempt_atforretrying,held_bytes,threshold_bytes,measured_at,measured_byandcurrentforsource_log_held), never free text from your data.- A raised
spend_spike’sdetailscarryday(the UTC day measured,YYYY-MM-DD),currency,metered_minor(that day’s metered spend, not billed),median_minor(the median of the 7 days before it),multiple(3) andbaseline_days(7), amounts in minor units ofcurrency; its resolve carries none. Itsstarted_atandtimestampare the start of the day measured, and a resolve’sresolved_atthe start of the last day of the calm week that ended it. - A raised
budget_reached’sdetailscarrycurrency,monthly_budget_minor,month_net_minor(this month’s charges),month(YYYY-MM, UTC),budget_action,resumes_at(the start of the next UTC month),held_by, andwallet_paused_atwhenheld_byiscreditorcredit_and_budget. Itscontinuous_pipelinesare the pipelines it names as streaming, up to 20, each with itsid,nameandspend_minor(what it was charged this month).continuous_pipelines_truncatedistruewhen more were known than it lists, andcontinuous_pipelines_partialistruewhen the list may miss a running pipeline it could not tell apart as one that streams, also when it names none. With neithercontinuous_pipelinesnorcontinuous_pipelines_partial, no pipeline that streams is running. - A test (from the Send a test button or
POST /v1/notification-channels/{id}/test) is the real body of the event you chose, about a sample pipeline namedexample-pipeline, with"test": true. - Version 1 only grows: new optional fields may appear, so ignore fields you do not know.
Verifying a delivery
Section titled “Verifying a delivery”Each delivery is signed twice with your secret, so you can use our scheme or any Standard Webhooks library:
| Header | Value |
|---|---|
Reyansh-Webhook-Id, webhook-id |
the event’s id, the same on every retry and resend |
Reyansh-Signature |
t=<unix seconds>,v1=<hex>: HMAC-SHA256 of "<t>.<body>", keyed with the whole secret string (whsec_…) |
webhook-timestamp, webhook-signature |
Standard Webhooks v1: v1,<base64> HMAC-SHA256 of "<id>.<t>.<body>", keyed with the bytes after whsec_, base64-decoded |
Verify one signature against the raw body, in constant time; refuse a timestamp more than five minutes from now; and dedupe on the id. During a secret rotation both secrets sign, so either verifies. In Python:
import hmac, hashlib, time
def verify(secret: str, body: bytes, header: str) -> bool: parts = dict(p.split("=", 1) for p in header.split(",")) t = int(parts["t"]) if abs(time.time() - t) > 300: return False expected = hmac.new(secret.encode(), f"{t}.".encode() + body, hashlib.sha256).hexdigest() sigs = [v for k, v in (p.split("=", 1) for p in header.split(",")) if k == "v1"] return any(hmac.compare_digest(expected, s) for s in sigs)Delivery, retries and order
Section titled “Delivery, retries and order”- A delivery is retried after 5 seconds, 5 minutes, 30 minutes, 2, 5, 10, 14, 20 and 24 hours (each
with a little jitter), ten attempts in all, honouring
Retry-Afterup to an hour. Answer 2xx within 10 seconds (we wait that long for your response’s headers); a redirect is a failure. - A delivery that got no answer from your receiver is shown as one reason, “the receiver could not be reached, the secure connection to it failed (its certificate, for example), or no answer came back”, on purpose: saying which would let the test button scan someone’s ports. It covers an address that cannot be reached; a secure connection that fails (a certificate we do not accept, plain HTTP on the https port, a receiver that speaks only TLS older than 1.2); and no answer (none within 10 seconds, or the connection closed before one). Your receiver needs TLS 1.2 or later and a certificate a public authority issued for its host name; a self-signed one is not accepted.
- At least once: a delivery can arrive twice. Dedupe on the id. Order is not guaranteed: use
timestampanddata.alert.phase. - A receiver that answers 410 is switched off at once; one that fails for five days, after failing
for 12 hours, is switched off as failing (a delivery it answered in the minute or two before the very
failure that crosses the five days can come too late to keep it on). Either way we email you
channel_disabled, and you switch it back on in Settings. A429never counts against your receiver. - Rotating the secret (
POST /v1/notification-channels/{id}/rotate-secret) keeps the old one signing for up to 24 hours (you choose; 0 ends it at once). Sending the current secret again changes nothing. To replace a secret that leaked during an overlap, pass"drop_previous": true.
Events
Section titled “Events”A webhook can also be told what happened, not only what is wrong: a sync started or ended, a table
was copied or stopped, a source table’s shape changed, a connection test passed, and every change in
your audit log. In Settings › Webhooks and Slack, press Choose events for events with
POST or PUT /v1/notification-channels. GET /v1/notification-event-types lists every type.
A subscription is a list of exact types (sync.ended), every type of one noun (pipeline.*), or every
type (*). It is sent only what happens after you save it: subscribing is not a request for history
(the pipeline’s history and its export are). When you edit it, a type you keep keeps arriving without a
gap, whatever pattern takes it now (sync.ended widened to sync.*, or * narrowed to sync.*); a type
you add starts from the edit; a type you remove stops, including what happened before but was not sent yet.
The webhook’s pipelines work the same way: a pipeline you add to them is sent only what happens after you
add it (the webhook’s row in Settings › Webhooks and Slack, and events_pipelines_since, say from when
each is sent). A webhook you switch
back on is sent nothing that happened while it was off. Slack channels do not take events: a person reads
Slack, and a sync a minute would bury every alert there. We recommend a separate webhook for events: a
webhook’s events wait in their own queue, so they never hold up its alerts, but it has one health, so a
receiver that keeps failing on events is disabled for its alerts too (the channel_disabled email says
whether the failures were on events).
| Type | Fivetran’s name | When |
|---|---|---|
sync.started |
sync_start |
A run began, or a continuous pipeline’s session did. data: run_id, run_url, shape (run or session) |
sync.ended |
sync_end |
A run or session ended. data: run_id, run_url, status (succeeded, failed, stopped), rows, bytes, duration_ms (null when they could not be counted: never a smaller number), and reason, the pipeline’s own words, for a failure or a stop |
connection_test.succeeded |
connection_successful |
A connection test, or a stored destination’s own test, passed. data: the audit row (below): changes.tested is connection or destination, and changes.kind the source, or the destination for its own test |
connection_test.failed |
connection_failure |
A connection test, or a stored destination’s own test, failed, or a step of it got no answer (changes.outcome unanswered) |
table.copied |
— | A table’s existing rows were copied, and it streams now. data: table, rows |
table.parked |
— | One table stopped and waits for you; the others carry on. data: table, cause, held |
schema.changed |
— | A source table’s shape changed, and what the pipeline did. data: table, change, column, from, to, outcome (applied, waiting, …) |
Every action of the audit log is an event type of the same name (the audit log guide lists them):
pipeline.created, pipeline.started, pipeline.stopped, pipeline.deleted,
pipeline.schedule_changed, table.recopy_requested, api_key.created, member.removed, and so on.
Fivetran’s create_connector, resume_connector, pause_connector, delete_connector,
force_update_connector (pipeline.sync_requested), resync_connector (pipeline.resync_requested) and
resync_table (table.recopy_requested) map one to one; its edit_connector is the pipeline’s own change
actions (pipeline.schedule_changed, pipeline.discarded_rows_changed, …), which pipeline.* takes all
of. Each thing is sent once, as one type. An audit event’s data is audit_id (as
GET /v1/audit-events names the row), action, actor (kind and label: a person’s address, a key’s
name), via, target and changes. A changed secret is {"changed": true}, never a value, and an event
never carries a person’s IP address or browser (those are for owners reading the audit log).
An event’s body:
{ "id": "evt_6xq2m4w7t5xgz5v3b6n2k3rjkk", "type": "sync.ended", "version": 1, "created_at": "2026-10-16T07:12:03Z", "test": false, "account_id": "t_acme", "pipeline": {"id": "orders", "name": "orders", "url": "https://app.example.com/pipelines/orders"}, "sequence": 4182, "data": {"run_id": "…", "run_url": "https://app.example.com/pipelines/orders/runs/…", "status": "succeeded", "rows": 1200, "bytes": 480000, "duration_ms": 95000}}created_atis when it happened, as the pipeline recorded it; for async.endedthat a give-up ended (below), the give-up’s moment, when the ending became known.pipelineis null for an account’s event (an API key created, a member removed).- At least once, and no order is promised, as with alerts: dedupe on
id. To order one pipeline’s events, usesequence: it increases with each thing the pipeline recorded, with gaps (only these events are sent). An audit event has nosequence; order those bycreated_atandaudit_id. - A body is under 16 KB. An audit row with a very large change set keeps every field’s name, as
{"changed": true}, with"truncated": true(the audit log’s own rule). - It is signed and retried as an alert is, and resendable from the notification history, also after you
switch a disabled webhook back on (what the
channel_disabledemail tells you to do). The history lists events only when you ask: the app’s Events view,GET /v1/notifications?phase=occurred, or one event type (?trigger=sync.endedlists its events and its tests). The API’s default page lists alerts and tests, as it always has. - When it is sent. What happened is recorded within a minute or so, and sent as fast as your receiver answers: each webhook is sent its events one at a time, oldest first, for up to 40 seconds of every minute. So one webhook’s ceiling is 40 seconds divided by your receiver’s answer time plus about 10 ms of our own work per event: about 150 events a minute for a receiver that answers in a quarter of a second, 78 to 80 for one that takes half a second, under 40 for one that takes a second. A burst larger than that is sent over the next minutes. Its alerts are sent apart from its events and never wait behind them. After a deploy, or behind a burst of hundreds of rows, recording can take a few minutes more.
- Your webhooks share the sending fairly. Up to four webhooks of an account are sent at once, and when one finishes early its turn goes at once to the next, so more than four share those four in the same minute. A webhook with nothing due takes no turn at all (one whose deliveries all wait for a retry is skipped until one is due), and a webhook whose receiver is failing is sent only after every webhook whose receiver answers. So neither kind can slow down one that answers: beside nineteen failing webhooks, an answering one is still sent everything every minute. A failing webhook is still tried every minute, after the others, while your answering webhooks leave one of the four free; when they need all four (more than four of them busy at once), it waits its turn behind them: beside eight busy ones it was tried in 11 of 15 minutes in our tests, never more than two minutes in a row untried. Its first answer puts it back among them.
- Keeping up. A webhook keeps up while it is sent no more than its ceiling, the webhooks of your account
that answer need no more than four at once, and (on our cloud) the platform’s need no more than its 24
lanes at once (self-hosted:
REYANSH_NOTIFY_CHANNEL_POOLsets the channel slots, 32 by default, of which three quarters send events). At the design’s sizing, forty events a minute per webhook (a hundred pipelines on a five-minute schedule), a receiver answering within half a second keeps up with room; at twice that, one answering in a quarter of a second still does, and one taking half a second is at its ceiling: measured, it was sent 80 of 80 a minute on one run and 79 of 80 on another, so it keeps up or falls behind by about one event a minute, its events arriving later by about a minute for every hour that lasts. A slow receiver costs a lane, not only time: each event holds one of your account’s four lanes for as long as the receiver takes to answer, up to 15 seconds (it must start answering within 10), so three events a minute to one that takes 14 seconds hold a lane for the whole 40 seconds, and four such webhooks hold all four, leaving your other webhooks to wait for one of them to finish. Something first seen more than four days after it happened is never sent, and an event’s delivery is retried for up to four days from when it was recorded: within those limits a receiver that answers keeps up, so the four days meet only one that has not answered for days, or one sent more than its ceiling for days. - A run that copies at its start (a pipeline’s first run, or a worker that starts with a table waiting to be
copied, a full re-sync applied at its start included): its
table.copiedevents come before itssync.started, because the run starts once the start’s copies are applied. A table copied while a run goes on is copied inside that run, after itssync.started. - A worker that stops without finishing its run. When a pipeline’s worker process is killed (the system’s
out-of-memory killer ends a process with SIGKILL), crashes, or stops on an error it could not continue past, it
says nothing about its run. Since 2026-11-16 (IST) we end that run ourselves once we see the worker stop (before we
restart it, and also when you stop or delete the pipeline after it stopped): its
sync.endedis told withstatus: failedand our sentence saying how the worker stopped (an exit code, or the signal) as itsreason, at the moment we saw it stop, before any restart’s ownsync.started. The run history shows the run ended with the same words, and the run is not charged (a failed run is free, the whole run). A run that said how it ended itself (it finished, was asked to stop, or refused to start and said why) is never ended a second time. - One failed
sync.endedper restart, and no email until the restarts are given up on. Every worker start that ends that way is told: a pipeline whose worker crashes and is restarted, again and again, sends your webhook a failedsync.endedfor each start, including the first starts of a refusal that is told only from a later start (those name only an exit code). No notification email is sent for a restart: the email is for the give-up. So a pipeline that crash-loops below the give-up’s limits (for example one that delivers rows between its crashes, which starts its count of stops again) sends no email at all, and its page may not sayretryingeither; its webhook events and its run history show every stop. - A pipeline whose worker keeps crashing. Each restart is a run of its own, with a new
run_id, and each one we restart is ended as the bullet above says. When the restarts are given up on, every attempt still open has ended, and each is told its ownsync.ended, withstatus: failed: the last attempt (the one whose worker process started last, also when it died before itssync.started, copying a table or waiting for one) with the reason, the earlier ones with none (they stopped without saying why), each dated at its own last event in the run history: the last event that carries its ownrun_id. What we write ourselves carries norun_id(a relaunch, a scheduled sync, your own edits and requests while it crash-loops), so it is shown in the run of the attempt whose worker process started last, if that one has not ended (in no run otherwise). Of those rows only the give-up ends a run: the last attempt, at the give-up’s moment, with its reason. None of the others moves any run’s start or end. These are exactly the runs the pipeline’s run history shows as ended then; a first copy’s counts are null, because its start is not a run’s. - A run that began long before the give-up. The give-up’s
sync.endedevents are read from the run history’s own window up to the give-up (its newest 20,000 events within 30 days). A run that had already ended before that window (its own stop, say) is never told a second time. A run that began before the window and ended at the give-up is told with its counts andduration_msnull unless the window holds itssync.startedand every row it moved: absent, never a smaller number. - An attempt left open by an earlier restart. Since 2026-11-16 (IST) an attempt we restart is ended when we
restart it (above). One that died before then, or whose end we could not write (our own database not answering
at that moment), or whose worker stopped together with our service (no supervisor saw it stop), says nothing, so
it reads as running in the run history and is told no
sync.ended. The next time the pipeline’s restarts are given up on (within 30 days), it is ended too: toldsync.ended,status: failed, with no reason, then, so itscreated_atis when that became known (the give-up’s moment) and can be days after it stopped; itsduration_msruns from its ownsync.startedto its own last event (the last one carrying itsrun_id), which is also its end in the run history. - Test it with
POST /v1/notification-channels/{id}/testand{"event_type": "<a type it subscribes to>"}: a sample of that event, with"test": true. The app offers the types the webhook’s subscription takes.
Create an incoming webhook in Slack (Apps → Incoming Webhooks → Add to a channel) and paste its URL
(https://hooks.slack.com/services/…) into Settings → Webhooks and Slack. The URL is the credential:
we store it sealed and never show it again. Messages never mention anyone, whatever a table is named.
Slack has no dedupe of its own, so a crash at the wrong moment can post a message twice.
Each message ends by saying why this channel got it, from the channel’s own settings, never your email’s: the end of a condition reaches a channel only while that channel’s recoveries switch is on, whatever your account’s recovery email says, so it says “recovery notifications were on for this channel when it ended”; any other message says that the channel’s settings include it (a billing notice: that it concerns your account’s billing).
Security notes
Section titled “Security notes”- We never return a URL or a signing secret: the API shows a hint (
url_hint,signing_secret_hint) and the secret’s id (signing_secret_id). - The page that shows a new secret cannot protect it from browser extensions that read pages or from clipboard history; store it and clear your clipboard.
- Self-hosted: notifications never reach private addresses unless you list them in
REYANSH_NOTIFY_ALLOWED_PRIVATE_CIDRS(separate from the list your databases may be reached on).
Limits
Section titled “Limits”20 channels per account; each minute, per account, 10 test emails, 10 channel tests and 30 resends (three separate limits); the history keeps 30 days.