Archive for October, 2008


vmware nightmare

Firstly, I love virtualisation. ESX, Hyper-V, Xen Server, VirtualBox anything.
But I when its not working as designed (due to whatever cause), I just have to scream out that I am utterly disappointed.

Just two days ago, we have had SAN Array upgrade on the mirrored cage. SAN is mirrored hence we were able to run the whole system without shutting down ESX as it fall back to 2nd array without issue (thanks to Datacore SANMelody)

Problem became apparent when ESX failed to log back in to the primary SAN Array and only logged in to the secondary SAN Arrays.

Ok… what do we do.

1 panic
2 panic
3 panic

well lets put a jokes aside, conclusion with speaking to vendors were to reboot ESX server or to reset SAN switch.

well we have V-motion so shutting down ESX werent big issue (as we thought)

Moral of the story – dont perform V-motion when system is unstable.

First we perform migration on test server without issue, (in fact 4 times) then we perform migration on the mail(exchange) and it failed. By this time, VC had lost connection to ESX and we were force to connect directly to ESX via VI or SSH.

well lucky enough we still had 1 stable ESX which manged to get mail server migrated and started up ok in 40min.

Aftermath – complaints from executives, blame from vendors that SAN swtich firmware is old, vmware update (3.5u2) is not installed.

I guess we got a lot of work to do…


give me your key.

104bit WEP key can be obtained within 1min with 20MB of sniffed data.

Wow… now WEP isnt safe at all ?
I knew with enough data (like 2Gig) you can calculate and pull out WEP key but only 20MB now?

University of Kobe and Hiroshima will be releasing the details soon including WINDOWS based program as well. I guess we just have to keep our eyes open for now.


sorry, its out of warranty

There are people who thinks they will NEVER have problem.
There are people who thinks support is perfect.
There are people who thinks software never breaks.

I know its bad assumptions but usually decision maker just take the risk and say “I’ll take the risk and move on for now”

Ok… What exactly is a risk ?
“I wont be here when its broken”?
“I dont care, Im not using it”?
“It works great why worry something later?” ?

I’ve seen way too many software roll outs that can potentially cause issues in the future that will suffer end-users badly. People who makes the call usually arent the end-users, they just make decisions and sometimes, bad one.

Point is, I just discovered that we’re running an application that designed for Win9x and running it on W2k3 T/S Environment. (great..) Why worry? Because vendor has no idea about the software’s fault and the usual request has came back. Guess what… REINSTALL.

1 I have error msg that I believe it helps to debug, why cant you tell us the meaning of error code?
(ans: I dont know, reinstall)

2 I have backup of file that has been restored but still failes, what makes you think reinstall is the fix?
(ans: I dont know, reinstall)

Im afraid when we ran out of support choices, we have to inform user that software is just no longer working. and there is no ways to FIX them.

well well well. now who is going to pay for bad support and bad app? Im 100% sure its not the manager who made the decision to use this piece of S**T software..


operation timed out but you cant do anything!

I mean… shouldnt program give OPTIONs when operation failed? or if there are some previous task?
This is about VMware, Virtual Infrastructure for ESX..

I have asked my colleague to take snapshot the server then install the .net framework 3.5 on a corporate web server.
Shutdown the server, take the snapshot, start the server, and install, reboot.

All should be 5min task.


snapshot has failed and miserably we have spent more than an hour trying to recover stuck server to start up. I knew immediatly that vm task or vmserver pid requires to be KILLED on the ESX but since my colleage was more GUI guy than CLI he never thought we requires to logon to ESX via CLI and issue KILL command.

Ok.. I like GUI, I do, but when CLI provides more options (in fact REQUIRED operation in this case) but GUI doesnt, I must say that GUI sux.

I think VI should

1 list all queued task
2 list all process that can be killed
3 provide all command wrapper (instead of cancel option thats greyed out..)

I know.. im asking too much…
I’ll keep using putty for now..

for others who may have suffered same issue and think you need to manually KILL the virtual guest machine, read this