Sunday, March 03, 2002

Day 1 - Continue the reserach

This is, surprisingly, an extremely comfortable project. It's not
like we're idle, or that we don't have anything to do. but I'm this
strange twilight zone where there always seems to be someone to do
whatever needs to be done. Probably the most difficult part of the day
was organising Lunch. In Sydney, and especially in Chinatown, which is
where the convention center is mostly located, it has become a traditional
pastime to have Yum Cha (Dim Sum) for a Sunday lunch/brunch we were well
aware of this and had planned to make the day worthwhile by finding a
Good restaurant, but somewhere between overwork and apathy we didn't
actually book anywhere and instead just wandered our way into Chinatown
roughly around lunchtime looking for a feed.

Unfortunately either a misdirected desire to not offend the fellow
workers or just sheer ignorance meant that not only did we not know
where to go, but no-one really wanted to take charge and just pick a
place. The fascist in me kicked in early, but my out of date lack of
knowledge meant that our first choice was closed, so I waved my arms
and deferred tot the team. Glen, the Worldcom systems guy "Had a plan"
but somehow this reverted to 10 people squeezed into an elevator where
we quickly concluded that there was "no fucking way there is fucking
Yum Cha joint on the 5th floor of this place...". Glen admitted out of
date knowledge (or a complete lack of spatial orientation) and finally
we ended up in somewhere that actually a) sold food and b) sold beer.

Getting back to the real work for a moment, we had another productive
day. The sparkie who had promised me power the night before hadn't
followed through but assured me he would do it right now. I went back to
the NOC to check on the status of the racks and realised that we probably
needed to optimise our hardware inventory and rather than keep all these
extra UPS's in boxes we could put them out in the racks. Of course this
meant that needed to go back to the sparkie and now tell him that I needed
2 circuits and not one. Karma being what it was, as I passed one of the
rack locations I noticed that he had actually given me 2 circuits so,
as usual, the real work had actually been done.

However, with real power we were able to roll out the peds onto the
show floor and try our luck at patching the whole shooting match into a
functional network. Some time earlier Glenn and I attempted to try and
test our fibre links, but the fibre test kit we'd been given had a few
personal problems. Since we seemed to be unable to set any reference other
than "Really Dark" there was this wonderful moment of self realisation
where (after about a 15 second silence) we both concluded that we should
"Bugger the fibre test, lets just patch the bastards in and see what
happens".

I only thought of this a long time later when casually asked if we'd
patched in the peds. "Yes," was the reply, "All the links came up
just fine".

Glenn and I tried to focus and turn our addressing/patching data into some
sort of reality. Extreme, because they wanted to try this new 'Super VLAN'
technology required us to develop a stable list of IP address to Port
matchup. so I had to find some way to resolve my hacked up addressing
programme with the patching spreadsheet and still keep our sanity. This
mostly was resolved by a lot of silence by Glenn as he wrestled the data
into submission. But he emerged at the end with a coherent patching list
and sent the team off to ping the bejesus out of the network.

Our moment of irony was when the ONE booth we actually required fluke
tools for was, of course, the Fluke booth.

Extreme 1, NOC 0.

At about 4pm we re-convened for a state of the nation and decided
that since "The damn thing just works" we should deliver addressing
sheets and, thus, the network. This was when Paul (extreme) chimed in
and reminded us that he hadn't actually dropped in the final configs
(with the Super VLANS) and that would probably be in our best interests
to re-ping all the drops. I made a mental note to inform Paul about my
plans in advance and admitted that, yes, we should re-test the entire
network if we respected our handiwork.

So while the team was out pinging away and delivering drop sheets we
distracted ourselves with the finishing touches. Sanjeev explained how
the two external routers failed over:

"I couldn't get VRRP stable enough for my liking" He started "So
what you do is disconnect this yellow cable, here and re-patch it
into the second router, here..."

And Bernard decided that he couldn't wait until the morning and would
the fix our NOC door. The Actual fixing was not really the amusing part
(mostly a lot of grunting and shaking of the booth). What was more
entertaining was watching everyone work out the problem for themselves

"What's wrong with it ?" They'd ask
"Try and lock the door"
"Yeah, it has lock, I can see that"
"No," We'd all chant, "Try and LOCK the door"

This was when they finally discovered that we could very easily and
securely lock ourselves INTO the NOC, but anyone from the outside could
just turn the latch.

Safe with the knowledge that the outside world could be kept behind
locked doors, it was Beer O'Clock.