Friday, July 17, 2009

Raid 0 + medical data + disk failure = EPIC FAIL

I work as a contractor for the hospital, and some server we don't have control over or support failed. I was asked to go take a look at it despite the fact that it's an unknown and unsupported environment (family doctor office not part of actual Hospital infrastructure).

So I check it out, and it's an old dell power edge with internal IDE drives that are hooked to a raid card. This is basically just a normal desktop computer with a fancy case and a raid card. When it boots up it just beeps loudly, and I isolated the beep to be the raid card and not the motherboard, then managed to get it to POST so I checked the raid settings and sure enough a disk failed. So at first I figured no biggie, there must be an option to push the disk out of the raid 1 array and run it as a degraded array until a new drive is added to rebuild. Then I realize, it's not a raid 1, it's a raid 0!!! WTF?!?! Who the hell makes a raid 0 on a server containing medical records?

I go see the doctor about it and he says he has backups, now the IT manager drags me further into this and wants me to try to rebuild his environment from the backups.

Now imagine this: You're a contractor, someone brings you to their house that had a fire and tells you to build it the way it was. That's exactly what this IT manager is asking from me. Backups or not, if the one who setup that environment is not around and can't support it, then tough luck, not much can be done. There is zero documentation or anything on this server and I'm not aware of how it was setup. i can only make speculations based on the rest of the environment such as the fact that DHCP stopped working and DNS is set to that server's IP, or at least, what I assume is that server's IP. Small things like that can give me small clues, but that's it.

It's been about a week now and I'm still trying to get it to work from the backups. The backups appear to be somewhat corrupted or incomplete (may of tried to backup files that were in use such as the DBs) so the restore ain't working well. The IT manager is on my back more and more as time progresses and is extremely agitated that I can't get it to work. What do you expect me to do, pull a 100% working backup out of my ass? Not only that but he expects me to rebuild the environment like it was with all this (unknown to me) programs. Give me a break. Seriously this guy is a nut case and is very degrading. He can GFH.

I took two half days off as I had mold in my house to remove, but part of the reason I took those half days off was to limit the time I'm stuck with him pestering and attacking me.

Is it illegal to release poisonous spiders in his office? My new house has plenty to spare.

Sunday, July 12, 2009

More house progress

Still in the demolition stage, but finally getting to a point where we can start painting soon.

The kitchen tiles are also in. This is why I want to learn to do this stuff myself, it costed me 600 in labour + about 400 in materials. I will make sure to NEVER drop something on that floor. :o



Did quite a lot of work and here is some of it:

Mold in the ceiling. $$$

Have to move the electrical boxes which may be a task as it seems most of the kitchen is fed from that box. Debating on installing pot lights instead of two fixtures, but think I'll stick to track lights just to simplify things. Now would be a good time to put pot lights though, given the ceiling is open, and I will need to remove and replace the insulation.

Did lot of work in the crawlspace such as started adding more light sockets, and removed all the bricks. I still need to conquer the dungeon though, that is a sub crawlspace which has tons of spider webs and is overall nasty.

Almost done cleaning crawlspace of spider webs (still need to go through again to broom it better)

The dungeon

Heater in the dungeon.

This heater needs to be there to keep the foundation from freezing up and collapsing. Back then they only dug 6 feet for foundations, now they dig 8 feet which is below ground freezing level.

Light box. 4 total have been added, to make a total of 6 100w lights in there!

One light installed, 3 more to go.

All 4 are now installed as of today, just one remains to wire and the two others are wired but the center light box has so many wires and wire nuts that it is unsafe to close. I will look at it further tomorrow to figure what will be the safest way to deal with this box. May need a junction box. It almost looks like the previous owners used this box as a junction box for a completely unrelated circuit.

Bricks gone!

Stacked the bricks in the garage.

Removed the old dryer few days ago.

So got to clean up there. Looks quite nasty, there was about 3 dead spiders on the ground from all the raid I sprayed previously.

I'm also very anal when it comes to labeling at work. There is so much stuff that is not labeled properly or undocumented. Well, I'm the same way here. Labeled a few things such as the wire that goes to the heater and added a note that it needs 240v and 30a. I need to verify that it really needs 30a as it probably only needs 15. Also labeled the breakers that are off, and why they are off (so I don't go and turn them on at a bad time!)

Lastly, we even started to remove the wall for the washroom enlarging. This was also done yesterday, and started the week before.

None of the work posted was done in posted order. We just kind of jump around and get things done.

Right now what's left to do is

-Fix the mold problem by removing old insulation and cleaning or replacing that part of the vapor barrier, then move the light sockets, then re put the drywall.

- Clean the dungeon area so it can be safely walked in without having spider webs near by (spider webs creep me out, it's a phobia of mine). I kept a 10 foot curtain rod for this reason.

- Finish crawlspace electrical and also figure out what some mystery wire is for. There is also another mystery wire in the kitchen where a plug used to be. It's just hanging there, hope it's not live. :o

And tons more, but the above is priority at this point.

Wednesday, July 1, 2009

House renovation progress

Happy Canada day eh!

Been a bit over a week since I got my house. Did lot of demolishing to prepare for the renovations.

Removed all baseboards (still a few left here and there), all kitchen cabinets, all screws and most curtain fixtures, and cleaned a lot of the walls. Kitchen was nasty.

All pictures can be found here.