Note: TFTS' spam filter seems to hate me. Oh, well.

Preface: I work in telecom. However, this story takes place during my days at [Data Center]. I had left [Telco] and was cross training IT. What’s unfortunate is that many of the tech guys were doing the opposite.

$BT – Me

$DT – Data Center Tech

$CU – Customer

$CU2 – Another customer

$1LN – [Data Center’s] front line (phone) support

I don’t mind working with people.

I don’t.

Despite my normally brisk outer shell, it’s the social aspect of my job that makes the nights go faster. It allows me to have a sense of normalcy, and it helps keeps me sane even when I deal with insane things.

Well that and an IV line of dark roast coffee.

I guess I should suffix my previous sentence and say that I don’t mind working with people willing to learn. One person doesn’t know everything about a job, but a man (or woman) who knows how to find the information they don’t know, and is willing to learn it is someone I (typically) don’t mind being around.

This night was the exception.

You see, at the time I had a coworker, we’ll call him $DT. Now, $DT was a clever chap. He had his A , Network , Security , and CCENT certifications, and was (at the time) only a few weeks of studying away from completing his CCNA Routing and Switching. But data centers require much more than just book knowledge.

You’re dealing with customers’ livelihoods.

You also need to know cabling, infrastructure, racking and stacking techniques, telco procedures (and troubleshooting), as well as a host of other odds and ends. You need to know the difference between a drill and dremel and when not to use them.

In short, the one thing $DT was lacking was in actual, physical experience.

So I was tasked to help him.

One evening, we were assigned a maintenance event like any other. Go to a Juniper MX960 on the floor (in the customer’s cage), call the customer, have them redirect traffic off the card (if it hadn’t been already), and then swap the card and optics out. It was a simple gig.

Side note:

This is a Juniper MX960 Edge Router. It is expensive. You’ll notice the vertical cards. Those cards can easily cost more than the $80,000 chassis. Fuck, the fantray alone is something like $3,000.

There are plenty of used parts on the market now, but at the time these things were cutting edge.

And of course, since $DT had previously worked for an IT company that had used Juniper equipment, management wanted him to tag along and assist. And he, wanting to impress management, happily volunteered to take point.

At the time, I thought nothing of it.

[Customer’s] cage was messy. Their cable management was, for lack of a better word, total and utter shit. We had been careful to demarcate our cable trays from theirs, in order to provide them with a dedicated space to run their jumpers, but for some reason they insisted on running their stuff inside the same trays as our stuff. And since they had decided to keep everything color matching with our jumpers, doing work in their cage was always a hassle.

It stood to reason then, that having a second set of hands to help me was a blessing of sorts.

Side note 2:

Why they didn’t have their own dedicated/semi-dedicated local tech, I’ll never know. Maybe they just liked paying out the nose for high priority remote hands work. Maybe they got a tax write-off. Either way, it was job security.

After picking up the box with the new card from our delivery room, and arriving at the cage, I had made clear to $DT the gravity of the situation. Replacing cards in an MX960 required a bit of finesse. There are hundreds of very closely spaced pins involved, and if one is not careful those pins would bend. And unlike a serial connector there is no way to bend them back.

After assessing the situation, it was time to begin.

$CU – [Customer], this is [Name] speaking, how can I assist you.

$DT – Hi, this is $DT with [Data Center] we have a ticket here to replace a card, and to call you before we begin work.

$CU – You’re in [City]?

$DT – Yes, sir.

$CU – Okay. Let me take a quick look.

After a few seconds of clicking (he was on speakerphone) he came back on.

$CU – Alright. Looks good. Go ahead and swap it out.

$DT – Give us a minute and we’ll get it done.

It was clear from the way he was looking at me, that $DT expected me to guide him.

$BT – Let’s go ahead and start pulling optics and jumpers, and tagging them with the port ID’s.

Before we could finish, the wireless phone I was required to carry with me rang.

$BT – Yo, $DT, you good man?

-Ring-

$DT – I can handle it. Everything’s tagged, so it should be as easy as mounting the card and plugging the optics in.

$BT – Sounds good. Nice and easy. Don’t rush it.

-Ring-

$BT – [Data Center] this is-

$CU2 – I’M HAVING TROUBLE WITH MY CIRCUITS AND I NEED YOU GUYS TO FIX IT!

Side note 3:

We had rules at the data center. Customers were unable to call us directly. They had to call our frontline support associates who gathered documentation about a problem, and then they had to warm transfer the caller to us. It was absolutely not acceptable to just cold transfer or, “dump,” a customer on us.

That was what happened here.

$BT – Ma’am, I’m going to need to collect some information from you before I can proceed.

After several minutes of yelling and threatening to go to another data center for her service, I finally obtained enough of her information to verify her identity, her ability to make decisions for the account, and the trouble she was having.

After solving her trouble for her, I ended up back at operations.

Side note 4:

It turned out to be a switch they were using for remote monitoring just needed to be rebooted. She had zero remorse for her attitude, despite it being an issue that had nothing to do with us.

And there before me was $DT.

$BT – You get that buttoned up?

$DT – Easy as can be.

$BT – Okay…

-Ring-

I looked at $DT, now drinking a Monster and eating a Twinkie.

-Ring-

Please, no.

-Ring-

$BT – [Data Center] this is BT.

$1LN – Hi $BT, this is [Name] with [Data Center’s] support, how are you?

$BT – I’m good. What’s up?

$1LN – I have a rep from [Customer] on the line, calling about an MX960 maintenance event y’all worked on down there. Would you be able to assist him with that?

$DT was now giving me the thumbs up.

$BT – Sure, patch him through.

To be continued.