Industry Impact : Brothers from Different Mothers and Beyond…

My reading material and video watching habits these past two weeks have brought me some incredible joy and happiness. Why?  Because Najam Ahmad of Facebook is finally getting some credit for the amazing work that he has done and been doing in the world of Software Defined Networking.  In my opinion Najam is a Force Majeure in the networking world.   He is passionate.  He is focused. He just gets things done.  Najam and I worked very closely at Microsoft as we built out and managed the company’s global infrastructure. So closely in fact that we were frequently referred to as brothers from different mothers.   Wherever Najam was-I was not far behind, and vice versa. We laughed. We cried.  We fought.  We had alot of fun while delivered some pretty serious stuff.  To find out that he is behind the incredible Open Compute Project advances in Networking is not surprising at all.   Always a forward thinking guy he has never been satisfied with the status quo.    

If you have missed any of that coverage you I strongly encourage you to have a read at the links below.   

http://www.theregister.co.uk/2013/11/12/facebook_cisco_killer_ocp_networking/

http://www.wired.com/wiredenterprise/2013/11/facebook_network/

http://www.datacenterknowledge.com/archives/2013/11/12/open-compute-reports-progress-open-network-switch/

http://www.businessinsider.com/facebooks-plan-for-the-network-market-2013-5

http://gigaom.com/2013/11/11/facebooks-hardware-vp-says-were-very-close-to-open-source-switches/

This got me to thinking about the legacy of the Microsoft program on the Cloud and Infrastructure Industry at large.   Data Center Knowledge had an article covering the impact of some of the Yahoo Alumni a few years ago. Many of those folks are friends of mine and deserve great credit.  In fact, Tom Furlong now works side by side with Najam at Facebook.    The purpose of my thoughts are not to take away from their achievements and impacts on the industry but rather to really highlight the impact of some of the amazing people and alumni from the Microsoft program.  Its a long overdue acknowledgement of the legacy of that program and how it has been a real driving force in large scale infrastructure.   The list of folks below is by no means comprehensive and doesnt talk about the talented people Microsoft maintains in their deep stable that continue to drive the innovative boundaries of our industry.  

Christian Belady of Microsoft – Here we go, first person mentioned and I already blow my own rule.   I know Christian is still there at Microsoft but its hard not to mention him as he is the public face of the program today.  He was an innovative thinker before he joined the program at Microsoft and was a driving thought leader and thought provoker while I was there.  While his industry level engagements have been greatly sidelined as he steers the program into the future – he continues to be someone willing to throw everything we know and accept today into the wind to explore new directions.

Najam Ahmad of Facbook – You thought  I was done talking about this incredible guy?  Not in the least, few people have solved network infrastructure problems at scale like Najam has.   With his recent work on the OCP front finally coming to the fore, he continues to drive the capabilities of what is possible forward.  I remember long meetings with Network vendors where Najam tried to influence capabilities and features with the box manufacturers within the paradigm of the time, and his work at Facebook is likely to end him up in a position where he is both loved and revilved by the Industry at large.  If that doesn’t say your an industry heavy weight…nothing does.

James Hamilton of Amazon – There is no question that James continues to drive deep thinking in our industry. I remain an avid reader of his blog and follower of his talks.    Back in my Microsoft days we would sit  and argue philosophical issues around the approach to our growth, towards compute, towards just about everything.   Those conversations either changed or strengthed my positions as the program evolved.   His work in the industry while at Microsoft and beyond has continued to shape thinking around data centers, power, compute, networking and more.

Dan Costello of Google – Dan Costello now works at Google, but his impacts on the Generation 3 and Generation 4 data center approaches and the modular DC industry direction overall  will be felt for a very long time to come whether Google goes that route or not.   Incredibly well balanced in his approach between technology and business his ideas and talks continue to shape infrastructre at scale.  I will spare people the story of how I hired him away from his previous employer but if you ever catch me at a conference, its a pretty funny story. Not to mention the fact that he is the second best break dancer in the Data Center Industry.

Nic Bustamonte of Google – Nic is another guy who has had some serious impact on the industry as it relates to innovating the running and operating of large scale facilities.   His focus on the various aspects of the operating environments of large scale data centers, monioring, and internal technology has shifted the industry and really set the infancy for DCIM in motion.   Yes, BMS systems have been around forever, and DCIM is the next interation and blending of that data, but his early work here has continued to influence thinking around the industry.

Arne Josefsberg of ServiceNow – Today Arne is the CTO of Service Now, and focusing on infrastructure and management for enterprises to the big players alike and if their overall success is any measure, he continues to impact the industry through results.  He is *THE* guy who had the foresight of building an organiation to adapt to this growing change of building and operating at scale.   He the is the architect of building an amazing team that would eventually change the industry.

Joel Stone of Savvis/CenturyLink – Previously the guy who ran global operations for Microsoft, he has continued to drive excellence in Operations at Global Switch and now at Savvis.   An early adopter and implmenter of blending facilities and IT organizations he mastered issues a decade ago that most companies are still struggling with today.

Sean Farney of Ubiquity – Truly the first Data center professional who ever had to productize and operationalize data center containers at scale.   Sean has recently taken on the challenge of diversifying data center site selection and placement at Ubquity repurposing old neighorbood retail spaces (Sears, etc) in the industry.   Given the general challenges of finding places with a confluence of large scale power and network, this approach may prove to be quite interesting as markets continue to drive demand.   

Chris Brown of Opscode – One of the chief automation architects at my time at Microsoft, he has moved on to become the CTO of Opscode.  Everyone on the planet who is adopting and embracing a DevOps has heard of, and is probably using, Chef.  In fact if you are doing any kind of automation at large scale you are likely using his code.

None of these people would be comfortable with the attention but I do feel credit should be given to these amazing individuals who are changing our industry every day.    I am so very proud to have worked the trenches with these people. Life is always better when you are surrounded by those who challenge and support you and in my opinion these folks have taken it to the next level.

\Mm

Through an idea and force of will, he created an industry…

This week the Data Center Industry got the terrible news it knew might be coming for some time.   That Ken Brill, founder of the Uptime Institute had passed away.  Many of us knew that Ken had been ill for some time and although it may sound silly, were hoping he could somehow pull through it.   Even as ill as he was, Ken was still sending and receiving emails and staying in touch with this industry that quite frankly he helped give birth to.  

I was recently asked about Ken and his legacy for a Computerworld article and it really caused me to stop and re-think his overall legacy and gift to the rest of us in the industry.  Ken Brill was a pioneering, courageous, tenacious, visionary who through his own force of will saw the inefficiencies in a nascent industry and helped craft it into what it is today.

Throughout his early career experience Ken was able to see the absolute silo’ing of information, best practices, and approaches that different enterprises were developing around managing their mission critical IT spaces.    While certainly not alone in the effort, he became the strongest voice and champion to break down those walls, help others through the process and build a network of people who would share these ideas amongst each other.  Before long an industry was born.   Sewn together through his sometimes delicate, sometimes not so delicate cajoling and through it all his absolute passion for the Data Center industry at large.

In that effort he also created and permeated the language that the industry uses as commonplace.   Seeing a huge gap in terms of how people communicated and compared mission critical capabilities he became the klaxon of the Tiering system which essentially normalized the those conversations across the Data Center Industry.   While some (including myself) have come to think it’s a time to re-define how we classify our mission critical spaces, we all have to pay homage to the fact that Ken’s insistence and drive for the Tiering system created a place and a platform to even have such conversations.  

One of Ken’s greatest strengths was his adaptability.   For example, Ken and I did not always agree.   I remember an Uptime Fellows meeting back in 2005 or 2006 or so in Arizona.  In this meeting I started talking about the benefits of modularization and reduced infrastructure requirements augmented by better software.   Ken was incredulous and we had significant conversations around the feasibility of such an approach.   At another meeting we discussed the relative importance or non-importance of a new organization called ‘The Green Grid’ ()and if Uptime should closely align itself with those efforts.   Through it all Ken was ultimately adaptable. Whether it was giving those ideas light for conversation amongst the rest of the Uptime community via audio blogs, or other means, Ken was there to have a conversation.

In an industry where complacency has become commonplace, where people rarely question established norms, it was always comforting to know that Ken was there acting the firebrand, causing the conversation to happen.   This week we lost one of the ‘Great Ones’ and I for one will truly miss him.  To his family my deepest sympathies, to our industry I ask, “Who will take his place?”

 

\Mm

Pointy Elbows, Bags of Beans, and a little anthill excavation…A response to the New York Times Data Center Articles

I have been following with some interest the series of articles in the New York Times by Jim Glanz.  The series premiered on Sunday with an article entitled Power, Pollution and the Internet, which was followed up today with a deeper dive in some specific examples.  The examples today (Data  Barns in a farm town, Gobbling Power and Flexing muscle) focused on the Microsoft program, a program of which I have more than some familiarity since I ran it for many years.   After just two articles, reading the feedback in comments, and seeing some of the reaction in the blogosphere it is very clear that there is more than a significant amount of misunderstanding, over-simplification, and a lack of detail I think is probably important.   In doing so I want to be very clear that I am not representing AOL, Microsoft, or any other organization other than my own personal observations and opinions.  

As mentioned in both of the articles I was one of hundreds of people interviewed by the New York Times for this series.  In those conversations with Jim Glanz a few things became very apparent.  First – He has been on this story for a very long time, at least a year.   As far as journalists go, he was incredibly deeply engaged and armed with tons of facts.  In fact, he had a trove of internal emails, meeting minutes, and a mountain of data through government filings that must have taken him months to collect.  Secondly, he had the very hard job of turning this very complex space into a format where the uneducated masses can begin to understand it.  Therein lies much of the problem – This is an incredibly complex space to try and communicate it to those not tackling it day to day or even understand that technological, regulatory forces involved.  This is not an area or topic that can be sifted down to a sound bite.   If this were easy, there really wouldn’t be a story would there?

At issue for me is that the complexity of the powers involved seems to get scant attention aiming larger for the “Data Centers are big bad energy vampires hurting the environment” story.   Its clearly evident reading through the comments on the both of the articles so far.   Claiming that the sources and causes have everything to do from poor web page design to government or multi-national companies conspiracies to corner the market on energy. 

So I thought I would take a crack article by article to shed some light (the kind that doesn’t burn energy) on some of the topics and just call out where I disagree completely.     In full transparency  the “Data Barns” article doesn’t necessarily paint me as a “nice guy”.  Sometimes I am.  Sometimes I am not.  I am not an apologist, nor do I intend to do so in this post.  I am paid to get stuff done.  To execute. To deliver.  Quite frankly the PUD missed deadlines (the progenitor event to my email quoted in the piece) and sometimes people (even utility companies) have to live in the real world of consequences.   I think my industry reputation, work, and fundamental stances around driving energy efficiency and environmental conservancy in this industry can stand on its own both publicly and for those that have worked for me. 

There is an inherent irony here that these articles were published in both print and electronically to maximize the audience and readership.  To do that, these articles made “multiple trips” through a data center, and ultimately reside in one (or more).  They seem to denote that keeping things online is bad which seems to go against the availability and need of the articles themselves.  Doesn’t the New York times expect to make these articles available on-line for people to read?  Its posted online already.  Perhaps they expect that their micro-fiche experts would be able to serve the demand for these articles in the future?  I do not think so. 

This is a complex eco-system of users, suppliers, technology, software, platforms, content creators, data (both BIG and small), regulatory forces, utilities, governments, financials, energy consumption, people, personalities, politics, company operating tenets, community outreach to name a very few.  On top of managing through all these variables they also have to keep things running with no downtime.

\Mm

Rolling Clouds – My Move into the Mobile Cloud

As many of you saw in my last note, I have officially left Digital Realty Trust to address some personal things.   While I get those things in order I am not sitting idling by.   I am extremely happy to announce that I have taken a role at Nokia as their VP of Service Operations.  In this role I will have global responsibility for the strategy, operation and run of infrastructure aspects for Nokia’s new cloud and mobile services platforms.

Its an incredibly exciting role especially when you think of the fact that the number of mobile hand-held’s around the world are increasingly becoming the interface by which people are consuming information.  Whether that be Navigation-based applications or other content related platforms your phone is becoming your gateway to the world. 

I am also very excited by the fact that there are some fierce competitors in this space as well.  Once again I will be donning my armor and doing battle with my friends at Google.   Their Droid platform is definitely interesting and it will be interesting to see how that develops.  I have a great amount of respect for Urs Hoelze and their cloud platform is something I am fairly familiar with .  I will also be doing battle with the folks from Apple (and interestingly my good friend Olivier Sanche).  Apple definitely has the high end hand-held market here in the US, but its experience in Cloud platforms and operations is not very sophisticated just yet.  On some levels I guess I am even competing against the infrastructure and facilities I built out at Microsoft at least as it relates to the mobile world.  Those are some meaty competitors and as you have seen before, I love a good fight.

In my opinion, Nokia has some very interesting characteristics that position it extremely well if not atop the fray in this space.   First there is no arguing about Nokia penetration of hand-held devices across the world.  Especially in markets like India, China, South America, and other emerging Internet-using populations.    Additionally these emerging economies are skipping past ground-based wired technologies to wireless connectivity.   As a result of that, Nokia has an incredible presence already in those markets.   Their OVI platform today already has a significant population of users (measured at least in the 10s of millions) and so scale at the outset is definitely there.    When I think about the challenge that Google has in getting device penetration out there, or Apples high-end (and mostly US) only approach you can see the opportunity.    I am extremely excited to get going.

Hope you will join me for an incredible ride!

\Mm

A Digital Adieu

Those who follow the news from Digital Realty Trust closely may have recently read that I have decided to leave the company to focus a bit more on some personal work/life balance issues.  With this move comes a new role that I will talk more of in the coming days and weeks.

I would like to take a moment and talk about my time and experience at Digital and what I believe to be some industry ground breaking work that is being done there.   The first thing that strikes me about the company is the quality and dedication of the people.  The staff within the organization are incredibly committed to both providing the best product  (in terms of engineering and construction) along with an obsessive regimen around Operations.  In my role running all aspects of design, construction, and operations, this passion showed through every single day.    It was a delight to work with such motivated people. 

From the outside it might be difficult to gauge just how significant this operation truly is.   As many of you know I have run some large programs before, but they all pale in comparison to the size, scope, and complexity of the work happening at DLR.  Its one thing to be building a couple of very large facilities and quite another to be building out tens upon tens of data center construction initiatives across the world.   There simply is no organization in the world that has to construct, manage and operate more data centers, period.   In addition to these “block and tackling” items there is also a healthy focus on modularization and evolving data center design and prototyping.   This focus is not just about driving additional efficiencies in power and cooling, but also in cost, and time to deploy.  A true intersection of business requirements.  On top of all this you add the Pod Architecture Services program and Build to Suit program which additionally extend Digitals capabilities to those looking to build “Do it Yourself” (DIY) Data Centers.   In short, it was a ton of fun with incredible opportunities for growth.

In my time at the company I have focused on driving additional streamlining efforts and operational rigor across the board and have helped set the engineering direction of the company.   This work has already begun to pay some significant dividends and I am sure will likely continue well into the future.   But let me be clear – The success of these initiatives will be delivered by a top rate team with few peers in the industry.  

In short, Digital was a great experience and I feel blessed in having made some life-long friends there as well.   So as I start a new chapter in my life, a bid fond adieu to a Data Center Juggernaut and look boldly forward to what is to come, for me and for Digital.

\Mm

A look back and a look forward…

For those of you who are not on the Digital Realty Trust email distribution for such things, I recently did a video for them on some reflections of the past and looking ahead with regards to the data center industry, technologies, and such.  You can find the video link here if your interested.   

I for one would never trust some data center dork in a video.

\Mm

Live Chiller Side Chat Redux

I wanted to take a moment to thank Rich Miller of Data Center Knowledge, and all of those folks that called in and asked and submitted questions today in the Live Chiller Side Chat.   It was incredible fun for me to get a chance to answer questions directly from everyone.   My only regret is that we did not have enough time!

When you have a couple of hundred people logged in, its unrealistic and all but impossible to answer all of the questions.  However, I think Rich did a great job bouncing around to clue into key themes that he saw emerging from the questions.    One thing is for sure is that we will try to do another one of those given the amount of unanswered questions.  I have already been receiving some great ideas on how to possibly structure these moving forward.  Hopefully everyone got some value or insight out of the exercise.  As I warned before the meeting, you may not get the right answer, but you will definitely get my answer.  

One of the topics that we touched on briefly during the call, and went a bit under-discussed was regulation associated with data centers or more correctly, regulation and legislation that will affect our industry.    For those of you who are interested I recently completed an executive primer video on the subject of data center regulation.  The link can be found here:

Data Center Regulation Video.

Thanks again for spending your valuable time with me today and hope we can do it again!

\Mm

Data Center Dynamics – San Francisco July 17th

For those of you interested I will be at Data Center Dynamics in San Francisco on July 17th.   I am scheduled to be on a panel moderated by James Staten of Forrester Research at 11:05AM along with Tom Furlong of Facebook, John Haas of Intel, and Bill Mazzetti of Rosendin Electric entitled ‘The Data Center Efficiency Schism…New Realities in Design’.  I may also do a guest appearance during the Chris Crosby keynote.    When I am not speaking I am likely to be found wandering around the event from session to session so please feel free to stop me and say Hi.

I am also planning on attending a Green Grid event at the conference later in the day as well. 

If you would like more information or are looking for registration information it can be found here.  See you there!

\Mm

Its the Law of Unintended Consequences – Some Clarity around My thoughts on Data Center Regulation

I have gotten a lot of response from the post on my thoughts on Data Center regulation.   Many of the comments in a response to an Infoworld article focused on the disbelief of regulations particularly targeting data centers.  A Greener Computing article felt that because the current administration is very tech-savvy they wouldn’t do anything to hurt data centers.  In fact the exact quote was:

I can understand Manos’ concerns, but I think he’s on the wrong track. The federal government is very unlikely to issue strict green regulations related to data centers. And if they do regulate them in some way, the regulations will no doubt be reasonable. The current administration is very technology-savvy — after all, the current Secretary of Energy Steven Chu was recently the director of the Lawrence Berkeley National Laboratory, whose work was heavily dependent on its data center. Chu did some great work related to Green IT when at the labs. He knows what can and can’t be done — and will make sure that data centers aren’t hamstrung with unnecessary regulation.

I guess for clarity sake I should state unequivocally that I do not believe that Data Centers will specifically be targeted or singled out for regulation.   Domestically here in the United States the EPA has kicked off its Energy Star Data Center evaluation which looks to study data centers as a sector, and something may come out of that, but in all honesty that wont be for some time.  I think the more immediate threat is in the efforts around Carbon Cap and Trade.  As the Greener Computing Article calls out, it was front and center at the G8 meetings.   With the UK leading the charge and the only real legislation on the books in this space, it would be hard for the other countries not to use it as the base for their programs.   My previous post focuses specifically on the fact that Data Centers will end up being significant contributing factors to Carbon metrics for companies.  Data Center Managers just aren’t thinking about it, and wont be until its far too late.  

While I am hopeful that leaders like Steven Chu and the Obama administration will weigh all possible aspects in a Carbon Cap and Trade program, the fact remains that they will need to legislate to the least common denominator and data centers are unlikely to be called out unless there is a group specifically calling attention to it.  Ergo my call for an industry wide group lobbying on its behalf.     I have doubts they will altruistically incorporate all possible sub cases into the mix without that kind of pressure.   President Obama frankly has bigger problems to be thinking about in my opinion.   

I am reminded of a quote from another excellent communicator and activist president, Ronald Reagan:

"The nine most terrifying words in the English language are: ‘I’m from the government and I’m here to help.’"

Its those times more than any other that you should put your guard up even higher.  I guess only time will tell, but one thing is certain Data Centers and IT departments will have a role to play in Carbon Reporting. 

\Mm

Coming Soon to a Data Center near you, Regulation.

As an industry, we have been talking about it for some time.  Some claimed it would never come and it was just a bunch of fear mongering. Others like me said it was the inevitable outcome of the intensifying focus on energy consumption.   Whether you view this to be a good thing or bad thing its something that you and your company are going to have to start planning for very shortly.  This is no longer a drill.

CRC – its not just a cycle redundancy check

I have been tracking the energy efficiency work being done in the United Kingdom for quite some time and developments in the Carbon Reduction Commitment (CRC).  My recent trip to London afforded me the opportunity to drive significantly harder into the draft and discuss it with a user community (at the Digital Realty Round table event) who will likely be the first impacted by such legislation. For those of you unfamiliar with the initiative let me give a quick overview of the CRC and how it will work. 

The main purpose of the CRC is a mandatory carbon reduction and energy efficiency scheme aimed at changing energy use behaviors and further incent the adoption of technology and infrastructure.  While not specifically aimed at Data Centers (its aimed at everyone) you can see that by its definition Data Centers will be significantly affected.  It was introduced as part of the Climate Change Act 2008.

In effect it is an auction based carbon emissions trading scheme designed to operate under a Cap and Trade mechanism.  While its base claim says that it will be revenue neutral to the government (except of course for penalties resulting from non-compliance), it provides a very handy vehicle for future taxation and revenue.  This is important, because as data center managers you are now placed in a position where you have primary regulatory reporting responsibilities for your company.  No more hiding under the radar, your roles will now be front and center.                             

All organizations including governmental agencies who consume more than 6000 MWh in 2008 are required to participate.  The mechanism is expected to go live in April 2010.  Please keep in mind that this consumption requirement is called out as MWh and not Megawatts.  What’s the difference? Its energy use over time for your whole company.  If you as a data center manager run a 500 kilowatt facility you account for almost 11% of the total energy consumption.  You can bet you will be front and center on that issue. Especially when the proposed introductory price is £12/tCO2 (or $19.48/tCO2).  Its real money.  Again, while not specifically focused on data centers you can see that they will be an active contributor and participant in the process.  For those firms with larger facilities, lets say 5MW of data center space – dont forget to add in your annual average PUE – the data centers will qualify all to themselves.

 

 

For more information of the CRC you can check out the links below:

• A Good Deck with a high level overview can be found here
• The wikipedia definition
• For alot of additional detail the CRC user guide is a great tool

While many of you may be reading this and feel poorly for your brothers and sisters in Great Britain while sighing in relief that its not you, keep in mind that there are already other mechanisms being put in place.  The EU has the ETS, and the Obama Administration has been very public about a similar cap and trade program here in the United States.  You can bet that the US and other countries will be closely watching the success and performance of the CRC initiative in the UK. They are likely to model their own versions after the CRC (why invent the wheel over again, when you can just localize to your country or region).  SO it might be a good idea to read through it and start preparing how you and your organization will respond and/or collect.

I would bet that you as a Data Center Manager have not been thinking of this, that your CIO has not thought about this, the head of your facilities group has not thought about this.  First you need to start driving awareness to this issue.    Next we should heed to a call to arms.

One of the items that came out during the Roundtable discussions was how generally disconnected government regulators are to the complexities of the data center.   They want to view Data Centers as big bad energy using boxes that are all the same.  When the differences in what is achievable from small data centers to mega-scale facilities are great.  Achieving PUEs of 1.2x might be achievable for large scale Internet firms who control the entire stack from physical cabling to application development,  banks and financial insitutions are mandated to redundancy requirements which force them to maintain scores of 2.0. 

Someone once decried to me that data centers are actually extremely efficient as they have to integrate themselves into the grid, they generally purchase and procure the most energy efficient technologies, and are incented from an operating budget perspective to keep costs low.  Why would the government go after them before they went after the end users who typically do not have the most energy efficient servers or perhaps the OEMs that manufacture them.  The simple answer is that data centers are easy high energy concentration targets.   Politically going after users is a dicey affair and as such DCs will bear the initial brunt.

As an industry we need to start involving ourselves in educating and representing  the government  and regulatory agencies in our space.   While the Green Grid charter specifically forbids this kind of activity, having a Data Center industry lobby group to ensure dumb things wont happen is a must in my opinion.  

Would love to get your thoughts on that.

/Mm