Posts Tagged ‘san


vmware nightmare

Firstly, I love virtualisation. ESX, Hyper-V, Xen Server, VirtualBox anything.
But I when its not working as designed (due to whatever cause), I just have to scream out that I am utterly disappointed.

Just two days ago, we have had SAN Array upgrade on the mirrored cage. SAN is mirrored hence we were able to run the whole system without shutting down ESX as it fall back to 2nd array without issue (thanks to Datacore SANMelody)

Problem became apparent when ESX failed to log back in to the primary SAN Array and only logged in to the secondary SAN Arrays.

Ok… what do we do.

1 panic
2 panic
3 panic

well lets put a jokes aside, conclusion with speaking to vendors were to reboot ESX server or to reset SAN switch.

well we have V-motion so shutting down ESX werent big issue (as we thought)

Moral of the story – dont perform V-motion when system is unstable.

First we perform migration on test server without issue, (in fact 4 times) then we perform migration on the mail(exchange) and it failed. By this time, VC had lost connection to ESX and we were force to connect directly to ESX via VI or SSH.

well lucky enough we still had 1 stable ESX which manged to get mail server migrated and started up ok in 40min.

Aftermath – complaints from executives, blame from vendors that SAN swtich firmware is old, vmware update (3.5u2) is not installed.

I guess we got a lot of work to do…