I got to work on Thursday morning to open the office. I printed out the first two reports and started to go to work. On the first phone call I'm trying to make a reservation, but it's incredibly slow.
And then it goes down completely.
Oh well. No big deal. Sometimes the network goes down with power failures. It happens. I figure that the system will reset itself and then I'll be able to go back to work.
Nine hours later I go home, and the system still isn't back up. Nine hours is a long time, and so we're starting to get worried.
The next day we learn that the entire triple redundant database system is shot. Everything that we've done since January 7th is gone, backward and forward. Uh . . . that's not good, but there's a backup, right?
Apparently, they go to pull the backup off the external drive and find that the backup has been running perfectly. Every night it's been backing up everything that they told it to backup. Too bad they forgot to tell it to backup the database. It's been backing up the system logs every night religiously, which doesn't help us one bit.
So they're sending the server drives to California, where they're hoping that we can recover the database (for around $25k, incidentally). However, this is the fourth day since the system failed. Everyone that works in my department has been pulling massive amounts of overtime trying to manually rebuild the system from our hard copy backups as best we can, but it isn't going well.
It turns out that our paper backups aren't nearly as complete as we would have liked to believe.
We really don't know who's coming in every night, and some of the people that we think are coming in aren't. It turns out that we don't store data on who cancels at all, except in the computer.
After two 16 hour days by nearly the whole department, we managed to reconstruct the month of May. Only the month of May, with five people working solid trying to type fractional data into the computer.
We're still about 9 thousand entries behind, including reservations as far out as January.
So, I've been missing my free time at work. I don't have time to check my email, or look at fark or listen to music, or anything. I go in, grab a pile of physical records, and start typing.
It's mind numbing, and if you're missing me online recently, that's why.