Kosmos Mission Control @ops@kosmos.social

We're migrating gitea.kosmos.org (away from Google's GKE, to our own servers) right now. There should be no downtime.

... all done and services back up.

We're taking kosmos.social as well as xmpp.kosmos.org down for maintenance in a few minutes. We'll be upgrading the database server in preparation for setting up database replication to secondary servers...

We just enabled audio/video calls for your Kosmos XMPP accounts. Available for example with the new version of Conversations for Android.

Sorry, everyone! We just recovered from an unexpected downtime. Both Mastodon and XMPP are back now.

Our main machine went down last night, for as of yet unknown reasons. The machine was unresponsive and reboots did not work. We were able to have a remote rescue console attached and have fixed the bootloader now. It does not look like we lost anything. We're investigating the issue now.

Upgrade is done. Check out the new announcement for details.

Heads-up: we're upgrading kosmos.social in a minute...

The ops team is now hanging out in a new XMPP chat room. Come talk to us in ops@kosmos.chat!

xmpp:ops@kosmos.chat?join

All done with the gitea.kosmos.org experiments. You can safely push your code again.

We're experimenting on gitea.kosmos.org deployments a bit today. Please expect some downtimes. Thank you.

kosmos.social was down for a short while just now, due to a manual system upgrade throwing it off the rails. Sorry about that!

kosmos.social just suffered a short downtime, due to a system software upgrade not agreeing with our configuration variables.

Details, in case you're interested: https://gitea.kosmos.org/kosmos/chef/pulls/108

We just upgraded kosmos.social to Mastodon version 3.0.1 (from 2.9).

Release notes for 3.0.0, which contains all the new features: https://github.com/tootsuite/mastodon/releases/tag/v3.0.0

And there's even a video that walks you through the notes this time: https://www.youtube.com/watch?v=aYHYxOeQumo

Hi everyone!

This account is taking over from @kosmos for status and incident updates, as well as general infrastructure talk.