Wednesday, December 31, 2008

Happy New Year!

I have lot of stuff planned for 2009 such as the start and completion of a total revamp of Age of Valor, as well as the completion of ServerWeave and the start of a small hosting company on the side for extra cash.

I might even open up a SBBS file in notepad++ and MAYBE change some code.

Seriously, I need to finish that one day. It has lot of features other free forums don't have. I just got tied up with other application projects and lost interest.

As for tonight, it turned into a super boring NYE. I even took an earlier shift (usually work 12:30-9:00) so I can be home early. But I should of just sticked with the 9:00pm shift. We're doing absolutely nothing this year. Normally we at least rent a movie, or maybe visit family or go out to the restaurant. This year, my mom is cleaning the house. AGAIN! She is obsessed, it's brutal - HOURS of vacuuming not to mention all the chemicals. Nothing worse then hearing a 10,000rpm AC motor for hours on end, combined with the torturous smell of all sorts of chemicals.

The house was cleaned not even a week ago, it's already clean, sit down or something. geez.

But other then that, all is good, and I wish everyone a happy new year! Don't drink too much, and if you drink anything at all, don't drive!

Tuesday, December 30, 2008

Serverweave progress update

Some good news!

I just got my first sign of functionality out of the system. There is now a task that runs every 15 minutes to regenerate the config files for apache (and all other daemons once I get there). It works. I can now set virtual hosts and they get written to the config file. Later on I can add more options to assign vhosts to a port/IP but for now I'm staying basic.

I will soon be able to move on to DNS, which should be fairly easy.

The tough one will be email. Once those 3 work, I will pretty much have a basic web hosting system going. From there I'll setup FTP and SSH. Mysql in there somewhere.

The system is at 3448 lines of code and 122 files now.


In other news I registered www.clientweave.com as well which in the future may be used for a client portion of serverweave. I will eventually add in some features such as deployment, backup and other tasks that could affect workstations, so there may be a lightweight system that will be installed on windows clients to make them easier to manage. We'll see. No official plans yet.

Friday, December 26, 2008

Merry Christmas!

Hope everyone had a good one!

Christmas time is great to visit family that we don't see often, and enjoy giving gifts, and of course, to remember the #1 reason we even celebrate it: Jesus our savior being born so he could die on the Cross to take our place so we can go to heaven. For all have sinned and fall short to the glory of God (Romans 3:23), but God's gift to us is Jesus taking our place. For God so loved the world, that He gave His only begotten Son, that whoever believes in Him shall not perish, but have eternal life. (John 3:16)


Tuesday, December 23, 2008

How to turn a 2 lane road into a 4 lane road

Q: How to turn a 2 lane road into a 4 lane road

A: Remove the snow banks!

I was going to post pictures of the snow banks, but they kind of removed them this morning. I managed to get a pic of some of the equipment they use. Basically a giant snow blower and line ups of dump trucks. I've seen as many as 5 dump trucks lined up so they just seamlessly move to next as one fills up.

Have no idea where they bring this snow but it must be a seriously huge mountain after doing all the streets. They usually do our street about 3-4 times a year. Some streets randomly get neglected for kicks.



Sunday, December 21, 2008

Sunny with cloudy periods

It's sunny with cloudy periods right now.

Sounds like a nice day doesn't it...

Weather Report

No snow either!

That's not the coldest it's been recently.

Saturday, December 20, 2008

More snow pics

For you southerners that think -5 is cold. :P

More to come later, and I'll try to get some of the 6 foot snow banks during the day tomorrow or Monday but the snow blower might pass, since it's overdue. We'll see.



Thursday, December 18, 2008

Serverweave update!

Been caught up writing a date/time class. Now that the class is usable, I have some nicely enhanced output. ;)




Recompiling [g++ -o serverweave serverweave.cpp -g -pthread -save-temps -lmysqlpp -L/usr/lib/mysql -lmysqlclient -I/usr/local/include/mysql++ -I/usr/include/mysql] ...
*******************************************************************************


*******************************************************************************
Completed. Relaunching...


Using host libthread_db library "/lib/i686/nosegneg/libthread_db.so.1".
[Thread debugging using libthread_db enabled]
[New Thread -1208203552 (LWP 27596)]
Dec-18-2008 11:58:48pm (1) Starting the program...
Dec-18-2008 11:58:48pm (2) Checking for config file...
Dec-18-2008 11:58:48pm (2) Config file found. Initializing...
Dec-18-2008 11:58:48pm (2) Using module: apache
Dec-18-2008 11:58:48pm (2) Using module: bind
Dec-18-2008 11:58:48pm (2) Loading Core...
Dec-18-2008 11:58:48pm (2) MySQL[1] Successfully connected to DB (srv: [localhost] usr: [root])
Dec-18-2008 11:58:48pm (2) MySQL[1] Selecting DB serverweave
Dec-18-2008 11:58:48pm (2) Loading modules [2]...
Dec-18-2008 11:58:48pm (3) Modload: apache
Dec-18-2008 11:58:48pm (4) MySQL[1] Performing SQL query: SELECT * FROM generalsettings WHERE gmodule='apache';
Dec-18-2008 11:58:48pm (3) Modload: bind
Dec-18-2008 11:58:48pm (4) MySQL[1] Performing SQL query: SELECT * FROM generalsettings WHERE gmodule='bind';
Dec-18-2008 11:58:48pm (3) Module load completed
Dec-18-2008 11:58:48pm (1) Listening on port 10000
Dec-18-2008 11:58:48pm (2) Added task, type:Core Save
[New Thread -1208206448 (LWP 27600)]
Dec-18-2008 11:58:48pm (1) Program loaded, now in main program loop.
Dec-18-2008 11:58:53pm (4) Running task: Core Save; (last run: [never])
Dec-18-2008 11:58:53pm (2) Saving all configuration...
Dec-18-2008 11:58:53pm (3) Module: apache
Dec-18-2008 11:58:53pm (3) Module: bind
Dec-18-2008 11:58:53pm (2) Done saving all configuration.
TEST:Dec-19-2008 12:13:53am
Dec-18-2008 11:58:53pm (4) Finished task: Core Save; (Next run: Dec-19-2008 12:13:53am, interval: 00:15:00)

Saturday, December 13, 2008

Uptime: It's not rocket science, volume 2

Ok another rant about uptime. More specificly, Server/Application maintenance planning.

I see this all the time. Someone decides to perform an upgrade to a system, be it a forum, game server, or any other service. Great, new stuff. Cool.

But WHY should any type of upgrade require the service to be down while the upgrade is being performed? I've seen sites or part of sites go down for MONTHS for upgrade. I can understand a data transition phase, but that should not take more then a few minutes in most cases, if done properly. Anything that requires no data transition phase should be instant or if it requires a recompile, then only take a few minutes depending on how long it takes to compile.

How it should be done
I run several websites and a game server. NEVER have I taken any of those services down for an upgrade. I do the upgrade on a local test server, when it's ready to go, I upload it all in one shot, replacing the old site. Someone can be on the site, click on a link, BOOM, they see the new site and everything is functional. At worse, they may get a php or other error if it's not done uploading. Now sure it happens where something wont work out as expected but this should not mean the entire service is down. (ex: uploaded wrong config file so paths are wrong - just upload the right one and good to go)

As for my game server Age of Valor, it is scheduled to reboot every day at 8:00 then comes back up after a few minutes. This does two things. 1: clears any possibly leaked memory, 2: Performs any required upgrades. So the server gets a few minutes of downtime per day, and that's constant and expected by players, they know it's coming and know it will be back up very shortly.

So say I do a major change to the game server, such as totally revamping a core system or something simple like adding a new monster. I do it all locally, test it, validate it, and test it some more. Once it's ready I do a patch, upload it to the proper folder on the live game server (which is running normally during this whole time). When it reboots it applies the patch, et voila! performed server upgrade with no irregular downtime. Well scripts have to recompile, so add maybe a minute or two but if compiles took longer I could easily arrange so I don't even have to have that happen.

What about data migration
Ah yes, glad you asked. Yes, that is a bit trickier.

Basically say you have a HUGE sql database that needs to be migrated to work with a brand new system, you can't really have the migration happen while it's live or it may cause problems. BUT, the system that USES the data CAN be done while server is live. Use test data (such as a backup copy of the database from live) to do testing and coding locally.

Once it's ready to go, here's what you do. If it takes less then 5 minutes I say just go ahead and do it. In most cases a forum site or what not should not take that long.

Now if it takes more then that, say a few hours, one option is to set the old system in a read only mode so people can at least view the data, and put a notice up that it's being upgraded. Perform data migration.

For stuff where read only does not apply (such as a game server) then downtime may be the only way to go, but this is strictly migrating the data over - the actual system should be done already. I really can't picture any data migration taking too long if it's properly done. Mysql is very fast, even flat files can be fast. A forum with millions of topics being converted to a different table format should take less than an hour I would imagine.

But out of all instances where I seen a service go down, from a forum to a game server, there was no data migration required, its just a system change but the data is not changed in any way. Those type of upgrades should be instant.

What about physical server move?
Ok you really got me now, maybe there IS a reason for a service to be down.... NOPE. There are ways to prevent downtime even in a server migration.

1: Get new server ready to go, with old data from the existing server's system.
2: Test it very well
3: Since we're talking servers the key here is DNS. Set TTL very low on server's zone (do this a few days in advance)
4: Put old server systemin read only mode if applies
5: move data to new server (This really depends on connection and amount of data but should not take THAT long - in case of mysql, dump db, compress to tar.gz, transfer over, uncompress, apply to db)
6: Switch DNS IP to new server, remove system from old server and leave with a placeholder page that DNS will propagate shortly, possibly instructing users to put something in their host file to force the DNS to change for them.

This applies more or less to a website move but really could apply to anything such as a email server. I'm also assuming there is no failover/cluster setup. IF there is, well this is 10 times easier... A AD Domain controller for example is very easy to migrate to a new server if you have two servers. Just take it offline, as easy as that.

Some specialized applications may also have something that makes it easier to transfer to a new server. It may not be meant for that but it's just a design "feature" that will help. Use such things to your advantage. Use your imagination.

True story

When I moved Age of Valor from a P4 to a quad Xeon, a lot of preparation was involved, most if all of it happened while players were playing normally, as if it was a normal day.

These are in brief, steps I had to take, all doable while server is fully operational:

1: Get server provisioned
2: Log on to server, configure security stuff like firewall, root password, user accounts etc... In most cases this is just copy and paste from old server, but here I was actually migrating from pure windows server to Linux, using a VM for windows. (game server only runs on Windows).
3: Install VMware on server, create a fresh Windows VM.
4: Configure secure remote access to the windows VM. This involves setting to NAT network mode, forwarding the ports on the virtual router, then establishing a SSH tunnel to that private port. RDP can only be accessed if I'm SSHed in the server.
5: With windows up, and me RDPed to it, start configuring windows specific aspects of security and other "new install" tasks.
6: To avoid possible latency by running game packets through the virtual vmware nat, create a new interface that is bridged.
7: Get a second static IP provisioned from provider and use it for this new nic.
8: Forward proper ports through the windows firewall so game server is accessible through this IP.
9: Take latest copy of game server system and data and begin uploading. Now this takes a fair while as the maps alone are almost 1GB. I did it from old server to new, within same data center, so I got decent speeds.
10: Run new game server, fully test to ensure all is working.
11: Configure tasks such as backups and test while game server is running.

Now this is where it got VERY interesting. This is a good example of a very well planned service/data move. Given I don't have the latest data I obviously can't just point DNS to the server right now, right? No problem.

12: I shut off game server for now on new server and I waited for the scheduled server reboot on the old server.
13: Old server rebooted (by reboot, I mean just the game server engine itself, not the OS). Stop game service from starting back up.
14: With both game servers stopped, transfer data from old server to new, while it transfers, switch over DNS, AND modify old server scripts to point to new server should someone be connecting by IP instead of hostname.
15: An extra minute of downtime later, start up server on NEW machine.
16: Watch in amazement as everyone starts logging on, as if nothing ever happened.

This is a good example of using a specific application design at full advantage. The way Ultima Online works is when you log in, you are presented by a server list. That list is simply a list of name, IP, and port. Yep, you guessed it. That script change I made on old server while the files were copying to new, was to make it so the list points to the NEW ip.

I have to admit, that was by far the smoothest server migration I ever did. Literally, it took no less then 5 minutes, as far as user's are concerned. I worked for a few days behind the scenes on this, but all while the server was running live.

I also don't want to sound like I'm bragging because I'm not. This is just how it SHOULD be. There is little to no reason to take a service offline for so long while performing an upgrade or migration.

And this is how the cookie crumbles.

Mmmm, cookies.

Speaking of food, If I need to transfer all the food to a new fridge, should I make all the food inaccessible during this time?

Wednesday, December 10, 2008

Why are people such retards?

I'm sure if you have half a brain you are aware of the Hallmark fake egreeting card virus that's going around.

I work at a Help Desk and one of our customers is a school board. We have gotten many cases of this virus. It's not the kids opening them. It's the TEACHERS. Yes, they are supose to be the people who educate today's children, but yet they cannot operate a computer using some half common sense.

Viruses are so 1990. Why do people STILL fall for them?

Funny thing is, I got this email in my inbox, one of those chain letters.


You should be alert during the next few days. Do not open any message with an attachment entitled 'POSTCARD FROM HALLMARK,' regardless of who sent it to you.


It is a virus which opens A POSTCARD IMAGE - it 'burns' the entire hard disc C.


This virus will be received from someone who has your e-mail address in their contact list.


This is the reason why you must send this e-mail to all your contacts. It is better to receive this message multiple times than to receive the virus and open it.


If you receive mail called' POSTCARD,' even though sent to you by a friend, do not open it! Shut down your computer immediately


This is the worst virus announced by CNN. It has been classified by Microsoft as the most destructive virus ever.


McAfee discovered this virus yesterday, and there is no repair yet for this kind of virus.


This virus destroys the Zero Sector of the Hard Disc, where the vital information is kept.


COPY THIS E-MAIL TO YOUR FRIENDS.

REMEMBER: IF YOU SEND IT ON, YOU WILL BENEFIT ALL OF US




Now those type of emails are exagerative and all, but you'd think that with those being sent, people would not open viruses. But they STILL open them.

Out of all people, my Grandma is the one that sent that email. She's not that computer savvy as computers are fairly new and complicated for someone that age, but even SHE knows not to open a virus, yet teachers that are suppose to be up to date and educated, still open viruses.

Not to pick on teachers, there are many other retards out there who still open viruses, though teachers should be the last people to do that...

Saturday, December 6, 2008

Hancock

Just watched the movie. Was decent, BUT, why is it every single good action movie has to be ruined with a love story line?

The movie was great at the start when there was lot of destruction and action and even the part where he goes to jail then the girl steps in and ruins it! Isn't that what always happens? Even Star Wars was ruined like this. Keep love out of action movies (unless it's a different kind of action, the type that falls in xxx category, then it's ok :P)

My $0.03 (Our dollar is worth less now so should be worth $0.02 USD or so)

More snow!



Now it's looking a bit more like Christmas time!

Thursday, December 4, 2008

Oops!

They make crappy software, they also have a crappy website. It's Galileo!

Give me my data!

Ok...

Tuesday, December 2, 2008

Uptime: It's not rocket science

I've always been stumped at how some people just have trouble keeping a server up, worse yet, those that don't have a proper plan if it goes down, which happens.

In the past, 2 days, I've witnessed two Ultima Online servers go down, for hours. :O

There is no reason for a server to be down that long. If a HDD fails and it's not a raid, then there should be a very good plan to recover as fast as possible. 1-2 hours MAX. With raid, there is no reason why it should take more then an hour if something else goes wrong.

I will admit, my online production server has NO raid, however, I have everything documented in the case of a failure, so I can be up and running ASAP.

Also, backups are your friends. Automatic backups. No excuse.

That said, this is my uptime.



I'm aiming to pass 1 year. This server has been rebooted once, and it was the day after it was deployed. I did a reboot to test that everything works properly after a reboot, so I don't get any surprises in the future if I do have to reboot. (such as a service not set to start, and it reboots when I'm not around).

This server is hosted at The Planet and their service and support have been great.

Sunday, November 30, 2008

Yay, new work shift

They introduced a new work shift and chances are I'll get it as people who have kids/wife wont want it.

The shift is from 12:30pm to 9:00pm.

My current shift is 7:00-3:30. It's nice as I finish early, BUT, I'm not a morning person. Also, I'm not a go to bed early person, so it's very hard going to bed at 10:00.

Now I'll be able to go to bed late (starting tonight) AND sleep in a little. I will also get much more work done on ServerWeave as I tend to code way better at night. In the day I tend to loose focus and spend more time browsing the internet then actually code. But at night I'm more focused.

Expect a beta for SW by end of December :)

Once I am satisfied with the state of SW I will then start revamping Age of Valor. Even debating on recoding the entire thing in C++.

We're talking over 417,000 lines of code though, so that would be a huge task. Mind you RunUO has tons of redundant code so I could probably cut it down to like 100 thousand lines. True story. Hardest part would be figuring out some of the packet stuff like encryption and compression.

Right now SW is at a measly 2,651 lines of code. :P

Friday, November 28, 2008

Every cup tells a story

As office workers, Tim Horton's is a big part of the work environment. With that, lot of trays get accumulated. After a bit of messing around, we found a great use for them after having enjoyed many coffees, hot chocolates and other beverages.

Having that said, here is our Christmas tree:



I also bought out all the plastic candy canes at Wal-Mart to decorate the rest of the office. At 94 cents, they are a steal.

Sunday, November 23, 2008

Network info

Just for those curious, these are some stats on Age of Valor's server. This server also does mail and DNS for all my sites. The bulk of the bandwidth is actually offsite backups.

Bandwidth usage (speed)
Bandwidth usage (totals)
Network Layout

Friday, November 21, 2008

Network troubleshooting 101, the epic fail way

I found this on a forum and thought it was funny. I reworded the whole thing to protect the innocent.

user: "My internet is always slow, I lag in games all the time. The cable company checked and everything looks ok at their end."
other user: "Try to do an ipconfig /release then ipconfig /renew"

me: FAIL

Yeah, renewing your IP is going to speed up your connection. Suuuuuure.

Thursday, November 20, 2008

Yay! Snow!

It's finally here! Looks like it's staying. It's not much yet but it's still coming down nice and slowly. I say it's probably about 2 or so inches so far.

Snow really helps bring a Christmas spirit. Christmas is just not the same without lot of snow.

Pics:







Monday, November 17, 2008

Windows Vista... for a SERVER?!

Yes, I've seen people do it. WHY?! Why do people even THINK about doing that?

It's like paying 2 grand more then you should for a low end server when you can get a higher end one for that price. A server OS should have very low foot print so it can run the applications and give them the most resources possible.

Vista wont do this. It will hog them all instead.

Windows for a server is bad enough in most cases, but at least use Windows Server if you must use Windows. Some server applications are simply better in Windows, and that's what Windows Server is for. Win2k3 is actually not too bad of an OS. But it all depends on the needs. Something like web or mail will be better in Linux. Mail in Windows is actually pretty bad, they really kill you with all their licensing restrictions. It's YOUR server, Microsoft should not have a say in how many mailboxes or users you have. Linux won't limit you like that.

My "rant" of the day.

Sunday, November 16, 2008

Yay snow!

It's very late, and it's not a lot, but at least it's finally here! Snow!

It's not Christmas season without at least 3-5 feet of snow! Global warming has taken it's toll on our weather in past years though as our winters have been starting later and later every year. Instead, we get non stop rain. In fact, it has not stopped raining since June. Literally, we had maybe 10 days from June to September, of no rain. From September to now it's been maybe 80% rainy days. SICK OF IT!

Hopefully this cold weather sticks around for good till spring, so we can at least enjoy a few months without rain.

Thursday, November 13, 2008

ServerWeave sneak preview / screenshots

I added lot of packet handling and UI stuff to the web client so far. I'm not really concentrating on prettiness at this point, only consistency like assigning right CSS class to right elements, and so on. I will probably come up with better colors then gray, though it's kind of growing on me... that logo has to go though. :P


You will also notice on the left, the debug info outputted while displaying that page. At debuglevel=5 all packet contents is shown. In fact, I just noticed now, my log is 9MB big.

Anyway, on with the screenshots!

Login
Module list - this will probably show up as icons (sorta like cpanel)
Errors - packet handling issue
Table - the heart of the system: tables.
More stuff in a table

These are all generated in the back end, the client simply knows how to read the packets and how to display them. The beauty of this is that a C++ class will be able to generate tables on the fly and send to client to display variable data.

The downside is that all data will be stored in memory during this time... but I can probably make it so variable data has pages, so for example when displaying a list of virtual hosts, it will only show 50 at a time, or something.

All in all, there is good progress going on. Tomorrow I should be able to work on data submission so server can get data from the client.

Tuesday, November 11, 2008

Money money money!

This is what happens when you accumulate change and then come to realize: it's time to roll it up...



I asked my cat to help, but she just wanted to play with the paper rolls and she gave me the most evil look...




Then she went to bed. She looks so peaceful there.




After rolling all the money, this is the end result:




That totals to $325.


In other news, I hope this was safe to eat, because I ate it anyway:




Some minor technical difficulty at the smarties factory eh.


Lastly and most importantly, a big Thank You to those who served for the country to give us the freedom we have today! Many lost their lives, were physically and emotionally injured to give us the freedom we have today. Shall we never forget!

Monday, November 10, 2008

Crappy software

Why is it that so many businesses use some of the crappiest software? It's seen in all sorts of industries from telecom to travel agencies. Usually they are very hard to install apps, very complex, badly documented, non standard (unlike many MS apps that people tend to hate - at least it's standard and has lot of support)

Today I had to deal with an app suite called Galileo, used by a travel agency that is our new customer for IT managed services. Transfering this thing to a new PC is quite a beasty task. First off, they have no install files for it and their support says it's on their site, but it has about 20 "sub apps" that are required and they are nowhere to be found.

A full day, hardly no progress done. I pretty much went nuts, literally...



It field can be nice, but it sure comes with some nightmares sometimes.

Wednesday, November 5, 2008

Irate customers

If you've worked in tech support you've talked to these people at least once I'm sure.

There are several types, some are just plain assholes, others are very clueless about life, and are assholes about it. Some are just plain crazy and don't even know what they even want. They'll just call and rage.

Today I had one of the clueless ones. This guy calls in saying someone keeps faxing his phone every 5 minutes. He used *67 to trace the call which ended up coming from somewhere in Toronto area. WAY outside our territory. This guy is getting pissed off at us because it's happening. I advise him there is simply nothing we can do as it is not even coming from our systems. It's not possible to do anything about it but wait, from a technical side of things, and even if we could, we would not legally be able to do anything about it. Something like this, you get the police involved.

So he hangs up. Later on he calls back saying it's still doing it, I advised him again, there is nothing we can do. He started raging and wanting to talk to my supervisor. This was at 7:30am. Managers along with 95% of every other person out there starts at 8:00 or 8:30. This guy rages even more when I tell him he is not in.

He starts raving about how he's losing business over this.

First of all, who in the world will be calling a business at 7:30am? This is in a town with a population of like 300. I doubt they even get more then 2 calls a day. Go to bed!

Then this is the funny part. I told him I'm guessing it's someone who set their fax to auto send and left for coffee or something, then he's all "what? so you guys only deal on a guess basis and not on facts?" Oooooooooo Kkkkkkkkkk, so I'm suppose to just KNOW, what some guy 900km from me is THINKING. I did not realize mind reading and other magic tricks were a requirement for tech support, but apparently people think we have it.

Then you got people that call and think we can see their screen, or their desk. WTF people? Or people think that we can remotely fix a physical network issue such as an unplugged cable. "Can't you just remote in and do it?" Yes, I can remotely move a physical object I can't even see. If I send the right signal to your modem it will generate the proper atomic reaction that will cause the molecules around to move in a certain way to plug the cable back in. I don't even know the IP of the modem because you're too stupid to know how to give me it and unwiling to do ipconfig, but if I just say a few magic words while moving my wand, it will automatically get punched in my program! The joys of modern technology!

Maybe if they took warning labels off stuff, we could get rid of people like this. These are the people that keep this job interesting though. It may be frustrating while on such a call but it's something to laugh about later.

Tuesday, November 4, 2008

Misc ServerWeave progress stuff

I'm still in slacking mode and not coding at 100% speed but getting somewhere at least.

I've completed the logon process handling. Everything works great and is very stable so far.

You can see [here] a sneak preview of the (still very basic looking) login screen and some logging on the side of a few attempts which I purposely failed.

There is about 1500 lines of code so far in the back end. As far as the client goes I probably wont be running an inventory system to track the development of it as it is not really counting. I am also trying to make the client as simple as possible so all the work happens on the back end portion. The idea is I want to make it possible for people to write 3rd party clients such as a windows/linux GUI one that connects directly, just to keep the options open, and stuff.

The client will basically receive packets on how the data should be displayed and what data should be displayed, then client will send how to change it (ex: changing a setting and submitting it). I may even throw in some ajax at some point. I want to make it slick and easy to use.

There's one major bug I have not had a chance to look at yet though. Say the SQL server connection is lost, you're dead in the water and have to restart the daemon...

CRASH

Nasty stuff. I can probably find a function to check connectivity before performing a query though.

Friday, October 24, 2008

ServerWeave progress update

I did some major shifting around as to how header files are organized. Using a custom tool I coded called cpinventory which inventories includes, I made it easier and faster for me to add new files without having to start including them all over the place. Basically this inventory program does a few things:

1: Adds a header to each file indicating what it's for
2: When it was last modified and who, number of lines,
3:keeps track of the description if I put a Description: header before running the inventory.
4: and most importantly creates two files, includes.h and sources.h which contain include directives of all the files. In some cases I have to include within includes but I have it setup so there wont be duplicates and I try to use forward declarations as much as possible. In most cases just seperating prototypes from source helps a lot

As far as the actual progress not much functionability yet, but these last organization changes will really speed things up. So far the inventory shows this:


======================================
Inventory and header tagging completed

Files: 30
Folders: 10
Lines of code: 1088
Previous lines of code: 1041
Lines added: 47
======================================


I plan to write lot of the packet handlers tonight and possibly the packets themselves too. Working on the base class as I speak.

Lastly, this cppinventory program will be usable for Age of Valor scripts too so we can keep track of who a file was modified by and when. Well the when part is tricky as I have not coded a date/time class yet but will be easy to add it in later.

In other news I bought some cookie dough so I'll probably bake some cookies tonight. I don't think I've ever done that before hopefully I don't make them extra crispy and extra dark. :P

Wednesday, October 22, 2008

Rebuilding raid!

Borg's raid is currently rebuilding. I ordered 2 1TB hard drives of same model as one of the existing drives so I will be replacing two of the different drives so that all 3 match up. I swapped out one drive so far, and it's currently rebuilding.

This is the part where I really don't want one of the drives to fail. :P Running in degraded mode until the rebuild is complete. Once it's done I will then swap out the other drive and put in the new one and do another rebuild. At that point all 3 drives will be Hitachi Deskstars 1TB. Those have a bad history back in the day with laptop drives, but that was a specific case incident. I should be ok. :P



[root@borg raid1]# mdadm --misc --detail /dev/md0
/dev/md0:
Version : 00.90.03
Creation Time : Sat Sep 20 02:15:28 2008
Raid Level : raid5
Array Size : 1953519872 (1863.02 GiB 2000.40 GB)
Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
Raid Devices : 3
Total Devices : 3
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Wed Oct 22 12:09:18 2008
State : clean, degraded, recovering
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1

Layout : left-symmetric
Chunk Size : 64K

Rebuild Status : 2% complete

UUID : 11f961e7:0e37ba39:2c8a1552:76dd72ee
Events : 0.95544

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
3 8 32 1 spare rebuilding /dev/sdc
2 8 48 2 active sync /dev/sdd
[root@borg raid1]#

Tuesday, October 14, 2008

Packets!

So I started on the network handling portion of ServerWeave. It now listens on a specified port. Does not really do anything with the packets yet, but the general code is there. I'm actually slacking TBH. It's been a few weeks and this is all I got...


======================================
Inventory and header tagging completed

Files: 31
Folders: 8
Lines of code: 842
Previous lines of code: 842
Lines added: 0
======================================


Getting there though. The core is the part of a program that holds the biggest challenges and requires lot of planning. So there's more thinking then actual coding going on. So far I have a rather solid design framework and I'm always trying to think of ways to improve it before I keep adding more components.

I also setup the inventory program to auto gen a .h file that has all the .cpp files so I'm spending less time manipulating what files to compile and stuff.


In other news, some retard from 70.106.62.191 was spamming dumb crap on anythingforums.com and just flooding the whole forum. Little did he know I have a tool in place that cleans up a mess like that in about 0.1 seconds, AND bans the account. He did it 4 times right in front of me. So I reported him to his ISP. Writing that email took longer then cleaning up his 100+ posts. He'll have to try harder.

Friday, October 10, 2008

Disk space!

It sure is nice having lot of Disk space. I don't have much more to say, just wanted to show off my new raid 5 array. :P With how cheap drives are now I'm sure some people have me beat, but I can say its 5 times more space then I had before!

Also keep in mind in that screenshot the shared drives that have the same size are the same disk, so no, they're not individual 2B arrays.

Tuesday, October 7, 2008

Introducing: ServerWeave

I finally thought of a name for my new control panel: ServerWeave. I also registered the domain before someone cybersquats it off me: http://www.serverweave.com/.

I already have a cool idea for a logo. The site is not yet priority though. Once the product is semi usable and that I've started using it for hosting, I'll probably then actually code the site for it.

Saturday, October 4, 2008

Control panel progress

I've officially started on my hosting control panel. Still not sure what I want to call it yet. I want something generic but good. Look at cpanel, its as generic as you can get as far as a name, but that's how it's remembered and known. I have a few list of rather interesting names, some of which the .com is not yet taken and show low results in Google, so I got some possible winners.

I have a decent logging system going on already, and the data/class organization is starting to come together. Sample log entry:


DEBUG: [date] Starting the program...
DEBUG: [date] Checking for config file...
DEBUG: [date] Config file found. Initializing...
DEBUG: [date] Using Apache2 module
DEBUG: [date] Loading core...
DEBUG: [date] Loading modules [2]...
DEBUG: [date] Modload: apache2
DEBUG: [date] Modload: generic
DEBUG: [date] Error Loading modules:Error loading module [generic], did you forget to inherit ConfigModule::Load()?
DEBUG: [date] Error loading core: Error loading module [generic], did you forget to inherit ConfigModule::Load()?


My plan is to have each server component be a module of the control panel. So in a typical hosting environment you'd want to have the apache module, mysql module, php module (though that would maybe be built into apache as a setting to turn on/off) a dns module, postfix module, dovecot module etc....

When enabled, users would see the module as a menu entry in the control panel and be able to configure settings based on their access level (aka, their own account). Each module would be responsible for storing the settings, and retrieving those settings, as well as applying them to their respectful program.

This setup enables me to later on add more modules, so say I want this control panel to also support sendmail, well I add a sendmail module.

When this program is first started it looks for the startup config file, if it can't find it, it starts the setup wizard which is where you choose to enable/disable modules.

So far I think I got a nice framework going on. Still a lot of core stuff go do though, like I still need to make a date/time class and what not, and still debating on if the settings will be stored in a mysql database, or if I want to use flatfiles. The beauty of flatfiles is the simplicity. If I later on add non webhosting features to this then maybe I don't want mysql to be running, or even installed. So its one dependency less to worry about. Also this program wont be storing all that much data so I can probably serialize everything into files or come up with a basic built in DB system. Server migration will be easier too, just copy and paste the files, and done.

Monday, September 29, 2008

Not so bad after all

Server load stats can be deceiving. I'm running 7 VMs and the system is barely taking any hit.


(click to enlarge + view more)


I played with the drives yesterday and the one that was reporting slow is no longer reporting slow. But now another one is.... I was starting to fear a back plane issue but it turned out to be a drive name mix up so it would not pickup any new drives in the slot I was trying.

Reboot wins again!

Think I can safely leave the server at that. I can deal with this i/o issue considering its only affecting server stats, not so much performance. The backup jobs kill the server though but those happen at night anyway...

So think all is good, and God was with me as I struggled to get this working. It could of been much worse.

Now I need continue to code my hosting control panel. I still need to find a good name for it. It's hard to code something that does not have a name yet. I've been refering to it as "controlpanel" in my code so far. I officially started on it yesterday before I did that hdd swap thing to see if the back plane was at fault.

Sunday, September 28, 2008

Yeah, so looks like I got a failing hard drive...

Problem is, it's not really technically failed. It's just VERY VERY slow. My raid has been performing horrendously so was wondering why. With help of people at Anandtech we ruled that it was /dev/sdc that was slow. That turns out to be one of my newest seagate drives too! SMART is not showing any signs of failures.

So figured I'd rule out the backplane slot so I put it in another slot and proceeded to rebuild the raid. Lets just say that did not go too well... Though I'm fearing it may even be the backplane, I was moving drives around to see and does not look too good.

But either way this is going to cost even more money. This is madness!

Never again am I building my own server. A dell poweredge sounds more affordable, considering you take it out of the box and don't have to spend more money because half the stuff does not work.

Saturday, September 27, 2008

Borg Server Upgrade + Nightmares!

Sometimes, newer is not always better.

I decided I'd upgrade my old home server, Borg, to newer hardware, and with the price of sata being very low, add tons of expandability so I can make some raid arrays. Now Borg is a "all eggs in one basket" type server. It does domain services, file, VMware, DNS, email, some custom applications, and it even powers the pump for the kitchen sink drainage system. (ok, maybe not the last part)

Went to Tigerdirect.ca and looked at prices for motherboards, cpus, ram, and hard drives (only needed 2 as I could reuse two of my satas that are raid 1 in the old server). "Prices are just too good not to upgrade" I figured.

Things really looked good. *add to cart* *search* *add to cart* Oh right, I'll need a bigger power supply too *add to cart* and come to think of it I'll need this, and that... *add to cart* ... oh wow, $1,700, a bit more then I hoped, but this will be worth it. *check out*

Later on the parts arrive...



And the cat was happy too. It's like Christmas!




I also got two of these for the drive arrays with intention of buying a disk controller in the future (which I ended up buying now just to save trouble in the future)





Once all put together I decided to power on and see if it boots, or if there are sparks, or if something is terribly wrong and 25,000 volts arcs from the PSU and kills me.



It POSTED. Now What I expected to see was a disk boot failure of some sort, given there was no OS.

Black screen with blinking cursor...

Reboot!

Mem test freezes half way

Reboot!

Wont power on, keeps turning itself on, then off, then on

Reboot!

Does same but then boots into black screen again

Ok time for memtest.

Boots on it, its ok... not the ram. Hmm well lets try to boot off a CD, maybe that black screen is just equivalent to the boot error.

Ignores the CD (even though memtest worked, and was a CD).

Hmmm ok weird. Anyway I spent a few days, turned out the IDE controller did not work so I went to the store to try and find a sata dvd rom drive. No luck, so I ordered it along with some Y molex cords which I realized I'd need.

A week later I get the molex cords but not the drive. They might of screwed up, I figure I'll call them later and check. I forgot one store, so I go check and they have sata dvd drives so I go pick one up go to install it. Ok weird, only some of the sata ports work. But I do manage to get it to boot and install CentOS 5.2 on the HDD.

So far so good.

Ok, time to start configuring stuff. Set the IP and stuff, do a few other things, reboot. hmm IP wont stick, and... wtf, the nic is dead!

It would pickup the nic, but it would eventually die off 5 minutes into the OS.

Ok, that's it, this MSI motherboard is crap, RMA! Undo everything, repackage it, send it back etc.... So I order an intel board instead, given the cpu is an Intel Core2Quad it should be pretty stable right?

When I sent the mobo off I realized I forgot to remove the plastic X for the heatsink. Oh great, where in the world will I find one of those? I end up contacting Masscool to see if I can get one. They are based in China so I was not sure if they understood my problem or if they were even willing to send one. It was after all, my own mistake. They seemed to be sending one but was still unsure.

A week or so later I get the mobo (AND that sata dvd drive, I have two now!) so decided to just buy another heat sink given I was waiting for a plastic X so I can actually build the server.

Another thing I also noticed the leds on those HDD enclosures are kinda messed up...



Some light up weird like that (notice not the full HDD image is lit) Some don't light up at all. Not consistent. They DO flash though, like when there's hdd activity the orange led gets brighter. But imo it should be OFF when no activity, not half off on some drives and off on others, but anyways....


So I install it and set everything up.

Ok first problem I notice when I boot up is the 3 pin fan connectors and the cpu 4 pin connector does not have enough voltage. [b]I WANT MORE POWER![/b] This is a server. I want 6000 RPM 120 fans, and I want tons of noise!

Bios does not have much options for it. I realize it goes by temp, but geeze, don't wait till my cpu is 100C before giving an extra 2 volts to the fan that's turning slower then my eyes can follow the blades.

I get the solder iron, scissors, and a glue gun. This is serious business.

So I spliced the two case fans and cpu fan to connect to a regular molex connector. For the case fans it went well, for the cpu fan, well lets just say I did not realize it ran on 5 volts, not 12. That smoke looked pretty cool though.

I REALLY don't want to undo the whole mobo again so I can change the fan, which uses the same screws that go through the mobo. (I hate Intel's heat sinks for that - need to remove the mobo to remove the heat sink)

So I break off the blades of the fried fan to clear air flow and glue gun another fan on top, and this one has speed, and runs on 12 volts. Glue gun is FTW. You'd think it would melt inside a PC, but its not that sensible to heat as some may think. I've glue gunned a fan directly on a heat sink before and it never fell off (except for when I forgot to turn it on!) In this case I'm gluing it on the old fan so its glue on plastic. Pretty solid. Most of my mods are ghetto like that. If it works, it's good enough for me!

I go to install CentOS 5.2 but turns out it does not like that hardware for w/e reason so ended up going with Fedora Core 9. As I feared, VMware server 1.0 did not install so had to go with VMware server 2.0 RC2. I'm not a huge fan of server 2.0, I prefer the basic client of 1.0. The next day the release non beta version of 2.0 is released! What luck, so I go ahead and install that. Configured mail and all that stuff, took a few days and everything seems to be running.



That's also with the two sata cards I bought which I had a little fun with, their instructions are not exactly, English. I managed to figure it out and got the 4 ports to work. The 4 on the back don't, its one or the other. At this point I had already done most of the software side of things. That was a whole other nightmare, but configuring linux mail server, samba, dns and stuff all at once is usually always a nightmare. But got it working fairly well. The 1.8TB raid array is quite nice too.

**random cat picture**



Now today was an interesting day. At this point the server config is pretty much done and it's running live. I deployed the server to it's rightful location as it was sitting in my room this whole time, generating noise and heat. It's in the minuses out so the window being opened helped.

I had to unjoin and rejoin the two PCs to the domain. Dealing with the profile issues is a pita, but managed to get my profile to pickup again. Redoing my windows profile would be a pure nightmare. So much stuff to configure.

I go to start coding on one of my projects, and it so happened a backup job kicked in. Normally this is not an issue, but something with this server, its not good at multitasking. The backup job set the load to 4.0 :O Now my program, which is a basic server app that just listens for connections and writes the packets ti disk, skyrocketed the load to 20! I've never seen a server load go even past 2 without me doing something very abnormal like powering on 5 VMs that are DoSing each other.

I decided to reboot to see what happens and it *seems* to have solved it. Load is now 0, and goes to around 5ish when my application has 100's of connections blasting data. To be expected.

I still got some nitty gritty stuff to do like setup the UPS software and ensure all backups are working properly, but in general, it's DONE DONE and DONE!


Ready for deployment:



Deployed:

Yeah its a weird setup. My parents did not go for my server rack idea in the basement. Maybe when I have my own house. :)


Specs:
Fedora Core 9 64-bit edition
Intel Core2Quad 6700 @ 2.66Ghz
8GB of OCZ ram
OS drive: 250GB
Data drive: 3x 1TB Drives in software raid 5 configuration - total of 1.8TB of space

Spare parts:
1 dvdrom drive (was impatient since order did not come in so found one in town)
1 PCI video card (the replacement board has onboard video)
1 plastic X retainer and heat sink (yes Masscool DID send it, was nice of them, did not cost me anything)

price:
probably a little over 2 grand... I lost track. I often had to go to store to get stuff.

I plan to sell the old server as well as my backup server (and the UPS that goes with it) to make up for some of the money. I'm saving up for a house and I spent way too much on this!

Moral of story is, if you don't NEED to upgrade, save your money.

In the end though, it was sorta worth it. Rather then all the /data1 /data2 /data3 drives I have /raid1 and everything is more organized, and I'm running a later OS that is easier to get support for, and the yum repositories work! (FC5 was unsupported so I was unable to use yum to install/update anything)

Next time I build a server I may just buy a dell or other pre-made one. At least that takes away the hardware hassles, and if something breaks they have a pretty good warranty.

Ahhh cobras!




Yeah I decided to make a blog so I can bore people with server and programming stuff.

I'll also post stuff here about my various projects such as Iceteks.com and uovalor.com. I also have some plans to start a hosting company on the side, but thats still in planning/coding stages.