Overview
Alfred is the Discord bot of the CodeLounge team. It lives in the team’s #monitoring channel, where the alerts about our servers and services arrive, and takes care of the chores around them: reacting to each alert, calling in the team member on maintenance duty when something breaks, keeping track of who is on duty each week, and reporting on the state of the monitored services on demand.
Alfred started as one of my slack projects, and I am its lead developer.
Monitoring Alerts
Alfred follows the Icinga↗ alerts posted to #monitoring, and tells their severity apart by the color of the alert:
- a critical alert gets a 😱 reaction and a reply that mentions this week’s maintenance assignee, asking them to take care of it. On weekends and on public holidays in Ticino, the reply leaves the issue for the next working week instead of pinging anyone;
- a recovery gets a ❤️;
- anything else gets a 🤔.
Whenever it starts, Alfred greets #monitoring with its version number.
Maintenance Rotation
Every calendar week, one team member is responsible for maintenance. Slash commands let the team:
- ask who is on duty this week;
- assign the duty for the week to a team member, mentioning them;
- show the maintenance leaderboard, i.e. how many weeks each team member has been on duty;
- refresh the list of team members, which Alfred rebuilds from the authors of the last 500 messages in
#monitoring.
Icinga Reports
On request, Alfred queries the Icinga API and posts a report of every service that is not in the OK state, listing its state, host, and check output, most severe first. The report shows errors and critical issues by default. An option sets the minimum severity to include, down to warnings. When nothing reaches that threshold, the report says so, and notes whether issues of lower severity remain.
Under the Hood
Alfred is written in Scala 3. It builds on Discord4J↗, whose reactive event streams it consumes as fs2↗ streams: login, startup, channel messages, and slash commands are merged into a single stream of events, each of which knows how to react. The team members and the weekly assignments are stored in an SQLite database. Alfred is packaged as a Docker image, which a GitLab CI pipeline builds, publishes, and deploys to the lab’s server.
