They see me rolling (back)

Moving backward is a feature too!

We go through a lot of pain to make sure that Canary deployments are quick and painless. It’s worth remembering that even though the deployment happened in minutes, a bunch of stuff has happened in the background. (Your bird created a crypto key-pair, exchanged the public key with your console, and registered itself as one of your birds).

From that point on, all communication between your bird and your console is encrypted (with a per-device key) and goes out via valid DNS requests. This makes sure that deployments are quick and simple, even on complex networks.

Once your bird is successfully deployed, it’s completely configurable via your Canary Console.
So with a few clicks, a user is able to change a deployed Canary from a Cisco Router, to a Windows Server



However mistakes happen and, as anyone who has remotely configured network interfaces over SSH can attest, remote network changes aren’t kind to missteps. How does your Canary react if you configure it with broken network settings? Your console will already warn you of certain obvious network misconfigurations that fail sanity checks. 


But what if someone enters settings that pass sanity checks but are simply wrong for the network in question? (For example, providing static IP settings which are incorrect for your Canary’s network location.)
Previously, this would simply mean that the Canary would apply the new settings, and promptly lose connectivity with the console as the IP settings aren’t valid. The fact that it could no longer reach the console, would mean that it couldn’t be “fixed” from the console, and some poor admin would need to trudge on over to the device to reconfigure it.


This sucks, so we introduced “Network Rollback”. If a customer applies a config that prevents the Canary from getting back to the Console, the Canary figures this out, and rolls back to its last known working settings.


Of course, we then give you a quick notification that “something bad happened”. This incident can also be sent via your regular notification channels, such as email, text message, syslog or Slack.


When you configure a bird which has recently been rolled back, you’ll get a warning too, so you know the settings have been rolled back.




We try as hard as possible to make sure people do the right thing by default, but when you get it wrong, we will try to give you a mulligan.

Comments

Popular posts from this blog

Simple Graphs with Arbor.js

Small things done well¹

Using the Linux Audit System to detect badness