The Weight of Technical Liberty…Cutting the Cruft

Over the next few months, it’s my sincere desire to share with you some of the amazing technology accomplishments currently underway at First Data and how we are attempting to change the industry.  In any conversation about the future, you must begin by framing the past.   As you may or may not know First Data is a company that was founded in 1971.   It is a company hallmarked in its early years by significant technology innovation with a number of ‘firsts’ in the enablement of credit card processing across the globe. 

Throughout the years the company grew both organically as well as through large numbers of mergers and acquisitions on a global scale which ultimately enabled it to become the international leader it is today.  I will spare folks a deeper commercial of the company only to state that today it has more scale and technology reach than any other company like it in the #Fintech space. 

I share this information because it’s that unencumbered growth over decades of acquisition, an evolving and changing regulatory and compliance field of requirements, and a historically growing list of platforms and services that ultimately led to the largest trove of ‘Cruft’ I have ever been challenged with in my personal career. It’s a challenge 45 years in the making. 

As you may recall I first defined ‘Cruft’ while engaged at the Turn-Around at AOL:

Cruft is defined as years of build-up of technology, processes, politics, fiscal orphaning, and poor operational hygiene that ultimately impede technical agility and operation.  Additionally, Cruft can create an acidic cloud of lethargy or apathy in the workforce that ultimately sucks the energy out of innovation from within.

When I originally defined the term I was referring to the work we accomplished attacking the Cruft in a different organization which ultimately led to the company winning the Uptime Institute’s “Server Round-Up” Award. That award was created to promote full IT and Facilities integration and improve overall energy efficiency.  While recognized for the energy efficiency improvement, it was really a by-product of other technological and organizational wins for the company.

Our work on ‘attacking the Cruft’ at First Data has resulted in similar, in fact, greater energy cost savings, but more importantly it has reduced and continues to reduce the operational complexity of our environments.  Attacking the Cruft problem along the technology, process, and hygiene axes have resulted in some very powerful and significant results.  While we are far from completing the task, the last twenty-four (24) months have yielded some mind-numbing progress.

Is this really my metric? So Not Technical…

The first challenge I had was trying to find a way to truly quantify the reductions in a metric everyone could understand.   Simply counting servers was not enough, it could not account for other devices like storage equipment, network equipment, and other kit that does not easily fold into that definition.   Measuring power usage decreases, while absolutely telling the effort from a purely technical perspective, obfuscated the tremendous amount of work and passion the teams poured into modernizing our plant.  Many of the consumers of the information of our modernization efforts are not technology or energy wonks.   We had to come up with a metric that was universal.  That everyone, even non-technical people could understand and visualize.   In the end, we settled on the ‘ton’. 

I know what you are thinking…the ton?  As in… like..weight?

Yes. 

It’s not as cool as measuring in megawatts, or measured computational capacity, or MIPS, or IOPs, or whatever metric is fashionable these days, but it is universal.  Additionally, the scale of the work output would just get lost.   So what did we achieve over the last 24 months?

  1. We removed 220+ tons of IT Equipment from our global data centers.
  2. We consolidated and shutdown 5 data centers across the world; and have an aggressive plan to continue to consolidate more.
  3. We employed large-scale internal virtualization technology, open source cloud technologies, and are building a hybridized cloud controller that has resulted in moving nearly 75% of our physical distributed server environments to a virtualized footprint. (I will share more on that in a different post).

There were significant other achievements as well which we can discuss at a later date.  But as I said, we had to set the framework of what the starting position was.   We still have a mountain of work in this space to do but the momentum has started and passions have been ignited.  Those passions are blowing away that “acidic cloud” that results from Cruft.  The results speak for themselves.  That is an incredible amount of work to achieve in just 24 months.  It’s not just about establishing a set of technical goals for an organization to achieve.  As a leader it’s about ensuring that you have created the fertile soil for those changes to take place and have empowered your people to make decisions along that alignment. 

Of course, none of this could have been achieved if the firm from the top down was dedicated to driving this kind of significant change.   First Data is truly blessed with a board and leadership team who not only understand technology, they have lived it, they have managed it, they have won with it.   It’s a very unique set of variables that have been toggled.

While tonnage may be an easier metric for non-techies to understand how much equipment was removed,  it is hard to grasp just how much 220 tons actually represents.  As these efforts over the last two years have created more operational simplicity giving us the freedom and liberty to expand and explore new technology approaches it is only fitting to associate it with the Statue of Liberty.  Which by coincidence also weighs 220 tons.  Visualize that.

\Mm

Dinner with @FirstData CTO at GATech Career Fair on 9/13

gaworkforfd

We will have representatives from my First Data technology teams at the Georgia Tech 2016 Career Fair on September 13th, 2016.   If you are a current Georgia Tech student or alum looking for some incredible opportunities its definitely worth stopping by to see what we are all about!  We have a bunch of great positions open in the Atlanta, Denver, New York and a host of other markets.

In addition we will be choosing 10 resumes at random for a private dinner with me, First Data’s Chief Technology Officer.  Think of it as an opportunity to grow your network, ask career related questions, or an opportunity to just get a free bite to eat.   Just you, me, food, and drink.   All you have to do is ensure that you drop your resume off at the First Data booth to enter.

You can tweet or follow #GATechWork4FD for more information as well.

\Mm

Just like the NASA Stereo-B Probe…

On August 23rd 2016, NASA announced that its DSN (Deep Space Network) had re-established connection with the Stereo-B probe whose joint mission (with its counterpart probe Stereo-A) was to orbit the sun.  The purpose of the twin probes managed by the Johns Hopkins University Applied Physics Laboratory in Maryland was to use the two probes to see the entire star. Not just the side facing our planet, but the other side as well.  The folks lost communication with the probe, but never gave up.  Two years later (just a few days ago in fact) they were able to re-connect to the silent space adventurer and the team is hard at work trying to get things back into working order.

How does that have anything to do with the LooseBolts blog you ask?  There are some interesting corollaries to our prodigal space mission and my ramblings here on Loosebolts.   The obvious one of course is that its been about two years since my last post.   I didn’t time it that way but it is kind of spooky.   Similarly just as Stereo-B had its counterpart in the Stereo-A mission, its not like I have been really silent as I have been maintaining a blog internally to my work at First Data.  That internal blog has been blasting out all kinds of interesting, pithy, technical, optimistic, critical, business-relevant, and some non-business-relevant posts throughout the entirety of the time.

My internal blog is called Random Thoughts.  The dual blog system (one focused internally and the other focused externally) is a mechanism I have used for a very long time throughout my career.  Like our twin solar satellites, Random Thoughts has been buzzing away on the far side of the sun accomplishing its half of the mission.   Loosebolts however, had to take a backseat until I could get some of my house in order. 

To be honest my ‘Random Thoughts’ are not really all that random and are almost exclusively focused primarily on the work at hand inside the firm.  In my opinion Communication is the number one factor whether you will have an effective organization.  This importance is magnified 1000 times if you are actively engaged in the turn-around of the business.    Communication defines expectations. Communication defines concerns.  Communication defines Priority and Importance.  Communication defines direction.  Most importantly – Communications defines culture. 

This last bit is incredibly important when engaging in re-inventing a company which essentially defined the first and second generations of an industry. A company that has been in operation almost 40 years and whose next evolution maintains as a basic precept that technology will be a defining factor in its paradigm shift. The target of so many disrupters trying to out disrupt itself and everyone else at the same time.  A company that has scale beyond any other in the #Fintech space in terms of reach but one that also has to contend with significant legacy technology decisions. 

Communications are in full effect. In addition to the internal blog, I also send out a personal note to the entire technology organization summarizing the work accomplished and challenges discovered each week every Friday.  We have even deployed our own internal Global Technology TV Station focused around our execution.

On the Technology side, we have been hard at work doing some incredibly cool things. Things you cannot do without scale to start.   Problems, opportunities, and a vantage point really unimaginable for most.  Of course it will all be about execution. The execution (both looking ahead and looking behind) is well underway. 

While I have personally participated in some very large turn-around’s and technology strategy shifts before – the learnings and challenges here have truly changed my perspective across a number of areas.  The strange result of dropping an Internet minded technologist in the middle of Financial Services with the executive support required to make the change.  I hope to share many of these with you here again on Loosebolts.  We are getting ready for some interesting transitions and revelations. 

Communication initiated to Loosebolts blog….

Connection Established.

Sending queued messages…. 

\Mm

On the Move: First Data

Well after a bit of time in stealth I am finally able to announce that I have taken the position of Chief Technology Officer at First Data. 

After being asked to join the Turn-Around team at AOL and driving some amazing results over the past four years, it was time for a change.  I absolutely loved my time there and the people were amazing.  Success has a quality all on its own and it was an incredibly personally rewarding experience for me to be a part of something that unique.

The move to First Data is an incredibly exciting move for me for many different reasons but one of the key drivers for me is that I feel that this industry is ripe for change.  It’s a move for me from running and building large scale Internet products and infrastructure to the Financial Services Industry.  

For those of you who may be be unaware of who First Data is, or what they do, its probably easiest to think of it this way – one out of every two credit card or debit card transactions across the world touches our infrastructure at some point in the transaction process. From a transactional scale perspective its very similar to what I have been used to companies likes AOL, Microsoft, and Disney.  The difference being of course that these transactions are a little more important than checking your favorite sports scores, or getting your e-mail. 

My challenge of course is to drive automation. To build a platform that makes infrastructure a decisive and differentiating platform to launch products and services.  To create a learning infrastructure and software eco-system that gets smarter over time.   In large part how do you blend the agility of the Internet with the maturity and complexity of the Financial Services Industry.   Sure it’s a complex technical problem space, but it has some very interesting business and regulatory challenges as well.   In many respects dealing with Safe Harbor, Regulatory and tax has been part of my job for many years.  The challenge now is to take that automation to the next level.  

To that end I have to say that First Data is assembling an amazingly formidable team to drive this change.  I will be reporting to the President of First Data, Guy Chiarello.  Guy is a universally respected Technology leader in the Financial Services industry with top posts at Morgan Stanley and JP Morgan Chase.  Technology will be key to the success of the company and the leadership team is a unique blend of technology savvy leaders from across the world. 

The new adventure begins!

You can follow the link to the official press announcement.

Along with the initial pickup from the Wall Street Journal.

\Mm