The situation
Dispatchly sells shipping labels to online stores. Standup is at 09:30, and the first ten minutes used to go on the same question: did anything break overnight? Someone opens the error tracker, someone else scrolls the deploy log, and the person who would know is still on the train.
A report answers that question before anyone asks it. It's a playbook's standing brief, run on a schedule, and every run lands on the Reports page as a new edition.
What we set up
On the desktop, an admin opened the Production playbook and set a schedule: every weekday at 09:00, Europe/London. The brief says what each edition should cover:
- label failures by carrier since the last edition, against the week before;
- new Sentry issues, and any with a sharp rate change;
- deploys since the last edition;
- webhook endpoints that were disabled overnight.
The report runs with the organization's own identity and model key, not a person's, and only with the tools that playbook allows. Each run shows up on the Usage page under playbook_report.
What it found
Tuesday's edition was on the Reports page at 09:03, with an amber dot for unread. It led with one finding:
UPS labels to addresses outside the US have failed since 17:40 yesterday, every one of them. 389 failed overnight with UPS_ADDRESS_VALIDATION_FAILED, carrier code 264002. The onset matches carrier-adapter 4.2.0, deployed at 17:40, whose changelog moved address validation to UPS's street-level API for all destinations. UPS domestic, FedEx, USPS and DHL were normal. Sentry had one new issue, the same UPS error, and nothing else moved.
Second, smaller: Bloom Botanics' webhook endpoint was disabled automatically after 50 consecutive 401 invalid signature responses, and their events are queuing.
How this report was made lists every query behind those lines, so nobody has to take the summary on trust.
At 09:12 the developer on call asked under the report: is it every UPS international label, or only some countries? The answer appeared below it with its own tool calls: every one since 17:40, across six countries, and 251 of the 389 from one customer, Northloop Cycles in Canada. Follow-ups run under the same playbook and tools as the report, and everyone who can see the playbook sees them.
What changed
The lead pressed Create issue on that answer. The draft already held the finding, the counts and the error, and the title got one edit before filing it to dispatchly/carrier-adapter. Standup opened on a filed bug instead of on "did anything break?". The adapter went back to 4.1.3 by 09:50, and the forward fix is to call street-level validation only for US and Puerto Rico addresses.
Wednesday's edition will say whether the failures stopped. Schedules are set on a playbook, and the Reports docs cover editions and follow-ups.