alwaysOn

Photo by Christian Holzinger on Unsplash The ants are doing well. That’s not a hedge or a “we’re learning a lot” consolation line — they’re genuinely good at the job now. Have they ever publicly shamed themselves in front of an audience, say the SVP of IBM Consulting? Maybe. However, that nightmare day aside, what I’ll say is that they have new tools, they’re solving patterns we never explicitly taught them, and the base architecture has settled to the point where adding a n

Photo by Егор Камелев on Unsplash Recently, it was suggested that I read a book called Children of Time by Adrian Tchaikovsky. It was an excellent read, and it turned out to be a very timely suggestion as, my big plan for this weekend was to upgrade my AI ants from a colony that works together to produce complex outcomes, to a colony that learns from experience so that it drives better complex outcomes over time. In this vein, I've been thinking a lot about how to have the

Photo by Thomas T on Unsplash We, the SRE live in a world of black swans and thundering herds . Cascading failures and unknown unknowns are our territory. In this world orchestration is fragile and doesn't scale. We don't just need resilience, we need adaptability. Adaptability does not come from control, it comes from the ability to assess and react in novel situations and that is the realm of choreography. For us to lead in the AIOps space we need to cede the control that

Photo by Jorge Coromina on Unsplash Over the past week or so I have been working on coding an agent ant colony that restores service to a running application. The agents don't do complex RCA, log tickets, interact with engineers etc.. They just keep the application up and running. It was definitely fun, and I learned some interesting and useful things. What I built: A simple python based web ordering application with a front end supported by two micro-services each with