Runaway & Storm Control

Sometimes a single problem won't stop shouting. A scheduled job set to retry every minute keeps failing, or a flapping monitor fires over and over. Without a brake, that one issue can bury your team in hundreds of identical alerts β€” and burn through a month of SMS and voice allowance in an afternoon. Runaway control (also called storm control) is that brake: it notices when the same alert is repeating faster than expected, collapses the flood into one alert you can act on, and quietly holds the rest.

Why this matters: the goal isn't to silence alerts β€” it's to make sure a runaway loop doesn't drown out everything else (or exhaust your SMS and voice allowance) while still telling you, clearly and once, that something is stuck.

Always on β€” nothing to enable

Runaway control runs on every flow, on every plan, automatically. There's no switch to turn it on. All you can tune is how chatty a given flow normally is and what to do when a storm trips β€” see Per-flow settings below. Sensible defaults mean legitimate alerting never trips it; only an actual runaway does.

How it works

JAMS Incident Management groups repeating alerts into a series β€” "the same thing happening again." A series is matched by its source:

SourceA "series" is…
JAMS Schedulerthe same job (by name/folder), even across separate runs
Azure Monitor / ServiceNow / Zendeskthe same incident
Email monitorthe same subject + sender
Webhook / API / manualthe flow itself

Two independent levers protect each series:

  1. Cooldown β€” after an alert goes out for a series, repeats are held for a short window (default 15 minutes) so the same issue can't page you every minute.
  2. Storm threshold β€” if a series still exceeds a repeat count within a window (default 10 times in 15 minutes), it's declared a storm: JAMS sends one collapsed "this is repeating" alert and then mutes that series for a few hours (default 4). If it's still failing when the mute ends, it storms again β€” so you're never permanently in the dark, but you're also never flooded.

Series are isolated. Because matching is per-series, a storming job never suppresses a different job on the same flow. If NightlyBackup is storming, a separate PayrollExport failure still alerts normally.

Per-flow settings

Open a flow (New flow or an existing one) and find the Rate & runaway control card:

  • Cooldown between repeats β€” how long to wait before the same alert can fire again.
  • How chatty is this flow? β€” a simple preset that sets the storm threshold. Pick what matches the flow's normal behavior:
    • Quiet (~5 / 15 min) β€” backups, batch jobs that rarely fire.
    • Normal (~10 / 15 min) β€” a typical operational flow. (default)
    • Chatty (~25 / 15 min) β€” a busy monitor with expected bursts.
    • An Advanced option lets you set the exact repeat count and window if you need to.
  • When a storm is detected:
    • Collapse & notify (recommended, default) β€” keep the flow running, send the single storm alert, hold the rest.
    • Pause the flow β€” automatically disable the flow entirely and notify recipients to re-enable it.

The single storm alert

When a storm trips, the one alert JAMS sends explains what's repeating and how often, and β€” on the interactive channels (voice, SMS, Microsoft Teams, Slack) β€” offers quick responses:

  • Mute β€” keep the series suppressed for the mute window.
  • Pause this flow β€” stop the whole flow until an admin turns it back on.

Email & push storm alerts are informational β€” use the interactive channels or the dashboard (below) to mute or pause. Whichever way you respond, the alert's group is acknowledged so escalation stops.

Active storms on the dashboard

Admins get an Active storms strip at the top of the dashboard whenever one or more series are being suppressed. Each row shows the alert, the flow it belongs to, how many alerts have been held ("N alerts suppressed"), and how long it's muted β€” with three controls:

ControlWhat it does
Mute 4hExtend the quiet window for this series.
ResumeClear the mute now and let this series alert again immediately.
Pause flowDisable the whole flow (all its series) until you re-enable it.

There's no "dismiss" button on purpose β€” the strip mirrors what's currently being suppressed, so it stays honest:

  • It clears itself when the mute window ends. If the storm has genuinely stopped, the row simply drops off; if the issue is still firing, it re-mutes on the next alert and reappears.
  • Resume clears a row immediately (and lets alerts flow again).
  • A paused flow stays listed until you re-enable it β€” that's a deliberate "off" state, not a transient storm.

Tip: if you see a flow in Active storms, that's your cue that something is stuck at the source. Muting buys quiet time; fixing the underlying job (or pausing the flow) is the real resolution.