I took the server down for maintenance on Monday morning to update to a newer kernel version. I rebooted, ran the "start server" script, and thought nothing of it. Then I get a message asking why the server is on 1.11.2. I had no idea why -- I initially thought the reporter was accidentally trying to connect to the wrong server. I open up the console and I see it's running the old version.

Thinking somehow the new map had somehow just been opened with a new version of Minecraft, I hurriedly shut it down and start poking around the filesystem. Except...everything recent is gone. A change I made Sunday night isn't there anymore. All of the logs from this map are gone. The backups are gone.

Panic sets in.

Where is everything? Computers just don't work this way -- files aren't magically replaced with other files. It's as if all of the changes we made for the past few months just ceased to exist. It was completely mystifying. As I said in the staff chat at the time, "this is 1 1=3 territory". Eventually, I determined the files on disk reflected the state of the server at some point between December 16th and 19th -- about nine months ago. Not only did we lose the 1.14 map, we went back in time on the 1.11.2 map.

Eventually, I discover there's a second, unmounted filesystem, with all of the newer files, in complete working order. My theory for what happened:

• We run two SSDs mirrored, for safety.

• I upgraded the OS version during that time period in December.

• I believe that during the OS upgrade, the mirroring was somehow disabled. We were then left with one filesystem that was updated over the following nine months, and one that was left unchanged.

• During the kernel upgrade on Monday morning, the updater decided that the other drive was the one we should boot to.

I attempted to fix this by carefully re-mirroring the drives, but I was worried that I would inadvertently overwrite the good data with the bad data rather than the other way around. To be safe, I took a complete backup of the entire server, copied it to my computer, wiped the server and reinstalled the OS from scratch, and moved the Minecraft server back. That's where we are today. You'll notice the website isn't running yet -- I'll get to that in a few days, but getting the Minecraft server up and running was my first priority.