AI systems
Pulse
Private repository — the publicly agreed part is shownA watchdog over a dozen services that doesn't become noise
- 6
- real addresses exercised
- 40
- tests
- 3.11–3.13
- Python in CI
Problem
Polling a URL every minute and pinging Telegram on every 500 is an hour of work. A day later that bot is a stream nobody reads, which is worse than nothing: it creates a sense of control that isn't there.
System
The HTTP, Railway GraphQL and Vercel REST probes are plumbing; the core is the alerting policy. An incident is confirmed only after N consecutive failures, so a single network timeout wakes no one. While an incident is open no repeat is sent — only a rare reminder on a cooldown. A separate "unknown" health separates a service being down from the probe being unable to tell: an expired token neither looks like an outage nor closes an open incident. State lives in SQLite, otherwise a container restart would re-announce failures already reported. On a confirmed outage the deployment's log tail is pulled and the model names the cause against a schema — cause, the line it rests on, first action, confidence.
Role
Solo build: architecture, alerting policy, provider integrations.
Outcome
The policy was exercised against six real addresses rather than mocks: the first failure passes in silence, the second opens an incident, the third is swallowed by deduplication, and recovery closes it — while the live services stay quiet throughout, as they should. A separate barrier surfaced in the order of operations: a one-off run demanded a Telegram token, so "can I even see my services" could not be answered without first creating a bot. It now runs with no token at all. 40 tests, CI on Python 3.11–3.13. A failing triage never drops the alert — knowing the service is down matters more than knowing why; restarting a deployment from chat takes a separate confirmation.