Log in

No account? Create an account
entries friends calendar profile Previous Previous Next Next
SANs and cables - Ed's journal
SANs and cables
Today I have mostly been sorting out hte aftermath of our SAN switch debacle last weekend - over the last week, we've been trying to piece together the puzzle of quite what happened when our switch was moved.

And the cable monkeys, monkeyed the cables. So I've spent the day doing an analysis of some 160 connections, and quite what's supposed to be where, and how. Without being able to see it physically, and having to work remotely.

We've ended up with one switch 'bridging' the two fabrics - this is bad, because it means the config of both 'merges' into one, and so you've got to disentangle it again after (not that hard, just annoying).

We've found a couple of extra switches, which were connected by accident despite not actually being in use.

A few more that were just plugged into the wrong side of the fabric, and so in a 'Segmented ISL' state. ISL is 'inter switch link' and ... SAN switches each have a unique id number, called a 'domain ID'. When this number clashes, they cannot join the fabric domain, and so show as 'segmented'.

Of course, one of the 'extras' we got randomly, had a number that clashed with an existing switch, effectively displacing that from the config (when there's a segmented ISL, it's first come first served) and thus 'disappearing' leaving us to try and figure out quite where those servers had vanished to.

THe reason this is a nuisance, is that this puts the port offline, so you can't actually check the WWN of the thing that's connected to it. WWNs are the SAN equivalent of MAC addresses - hardware addresses for the host bus adaptors (HBAs).

So having pieced together that, I've now got two spreadsheets - one with a list of what is, and one with a list of what _should_ be. I now need to translate this into 'monkey proof' instructions, and raise a change for it.

All good fun really, and gives me a lovely chance to show off my SAN-fu. I just would rather have some people who knew what they were doing first time around.
Leave a comment