Best of Luck to a Great Guy…

I just read that Chris Crosby has announced his departure at Digital Realty Trust. As an alumni of that great firm I can definitely tell you that Chris’ hand-prints are all over that company.  I had the pleasure of interacting with him, before I joined,  heavily in my role there, and have maintained our relationship since leaving.   Chris was an influential force in defining how that company ran, operated, and ultimately succeeded in dominating the wholesale data center market.  Not to mention that he was always a charismatic tour-de-force as one of its primary faces and pitchmen.

I don’t know what Chris is up to next but I wish him the greatest success and happiness.  If his near term goal is a little time off, Lord knows he has earned it.  I do know that his tornado like energy wont keep him out of the fray for long.


A Digital Adieu

Those who follow the news from Digital Realty Trust closely may have recently read that I have decided to leave the company to focus a bit more on some personal work/life balance issues.  With this move comes a new role that I will talk more of in the coming days and weeks.

I would like to take a moment and talk about my time and experience at Digital and what I believe to be some industry ground breaking work that is being done there.   The first thing that strikes me about the company is the quality and dedication of the people.  The staff within the organization are incredibly committed to both providing the best product  (in terms of engineering and construction) along with an obsessive regimen around Operations.  In my role running all aspects of design, construction, and operations, this passion showed through every single day.    It was a delight to work with such motivated people. 

From the outside it might be difficult to gauge just how significant this operation truly is.   As many of you know I have run some large programs before, but they all pale in comparison to the size, scope, and complexity of the work happening at DLR.  Its one thing to be building a couple of very large facilities and quite another to be building out tens upon tens of data center construction initiatives across the world.   There simply is no organization in the world that has to construct, manage and operate more data centers, period.   In addition to these “block and tackling” items there is also a healthy focus on modularization and evolving data center design and prototyping.   This focus is not just about driving additional efficiencies in power and cooling, but also in cost, and time to deploy.  A true intersection of business requirements.  On top of all this you add the Pod Architecture Services program and Build to Suit program which additionally extend Digitals capabilities to those looking to build “Do it Yourself” (DIY) Data Centers.   In short, it was a ton of fun with incredible opportunities for growth.

In my time at the company I have focused on driving additional streamlining efforts and operational rigor across the board and have helped set the engineering direction of the company.   This work has already begun to pay some significant dividends and I am sure will likely continue well into the future.   But let me be clear – The success of these initiatives will be delivered by a top rate team with few peers in the industry.  

In short, Digital was a great experience and I feel blessed in having made some life-long friends there as well.   So as I start a new chapter in my life, a bid fond adieu to a Data Center Juggernaut and look boldly forward to what is to come, for me and for Digital.


Private Clouds – Not just a Cost and Technology issue, Its all about trust, the family jewels, corporate value, and identity

I recently read a post by my good friend James Hamilton at Amazon regarding Private Clouds.   James and I worked closely together at Microsoft and he was always a good source for out of the box thinking and challenging the status quo.    While James post found here, speaks to the Private Cloud initiative being what amounts to be an evolutionary dead end, I would have to respectfully disagree.

James’ post starts out by correctly pointing out that at scale the large cloud players have the resources and incentive to achieve some pretty incredible cost savings.  From an infrastructure perspective he is dead on.  But I don’t necessarily agree that this innovation will never reach the little guy.  In my role at Digital Realty Trust I think I might have a pretty unique perspective on the infrastructure developments both at the “big” guys along with what most corporate enterprises have available to them from a leasing or commercial perspective.  

Companies like Digital Realty Trust,  Equinix, Terramark, Dupont Fabros, and a host of others in the commercial data center space are making huge advancements in this space as well.   The free market economy has now placed an importance on low PUE highly efficient buildings.   You are starting to see these firms commission buildings with Commission PUEs Sub 1.4.   Compared to most existing data center facilities this is a huge improvement.  Likewise these firms are incented to hire mechanical and electrical experts.  This means that this same expertise is available to the enterprise through leasing arrangements.  Where James is potentially correct is at that next layer of IT specific equipment.

This is an area where there is an amazing amount of innovation happening by Amazon, Google, and Microsoft.   But even here in this space there are firms stepping up to provide solutions to bring extensive virtualization and cloud-like capabilities to bear.    Companies like Hexagrid have software solutions offerings that are being marketed to typical co-location and hosting firms to do the same thing.  Hexagrid and others are focusing on the software and hardware combinations to deliver full service solutions for those companies in this space.    In fact (as some comments on James’ blog mention) there is a lack of standards and a fear of vendor lock-in by choosing one of the big firms.  Its an interesting thought to ponder if a software+hardware solution offered to the hundreds of co-location players and hosting firms might be more of a universal solution without fear of lockdown.  Time will tell.

But this brings up one of the key criticisms that this is not just about cost and technology.   I believe what is really at stake here is much more than that.   James makes great points on greater resource utilization of the big cloud players and how much more efficient they are at utilizing their infrastructure.   To which i will snarkly (and somewhat tongue-in-cheek) say to that, “SO WHAT!”  :)   Do enterprises really care about this?  Do they really optimize for this?  I mean if you pull back that fine veneer of politically correct answers  and “green-suitable” responses is that what their behavior in REAL LIFE is indicative of?    NO.

This was a huge revelation for me when I moved into my role at Digital.  When I was at Microsoft, I optimized for all of the things that James mentions because it made sense to do when you owned the whole pie.   In my role at Digital I have visibility into tens of data centers, across hundreds of customers that span just about every industry.  There is not, nor has there been a massive move (or any move for that matter) to become more efficient in the utilization of their resources.   We have had years of people bantering about how wonderful, cool, and how revolutionary a lot of this stuff is, but world wide Data center utilization levels have remained abysmally low.   Some providers bank on this.  Over subscription of their facilities is part of their business plan.  They know companies will lease and take down what they think they need, and never take it down in REALITY.   

So if this technology issue is not a motivating factor what is?  Well cost is always part of the equation.   The big cloud providers will definitely deliver cost savings, but private clouds could also deliver cost savings as well.   More importantly however, Private clouds will allow companies to retain their identity and uniqueness, and keep what makes them competitively them –Them.

I don’t so much see it as a Private cloud or Public cloud kind of thing but more of a Private Cloud AND Public cloud kind of thing.   To me it looks more an exercise of data abstraction.   The Public offerings will clearly offer infrastructure benefits in terms of cost, but will undoubtedly lock a company into that single solution.  The IT world has been bit before by putting all their eggs in a single basket and the need for flexibility will remain more key.    Therefore you might begin to see Systems Integrators, Co-location and hosting firms, and others build their own platforms, or much more likely, build platforms that umbrella over the big cloud players to give enterprises the best of both worlds. 

Additionally we must keep in mind that  the biggest resistance to the adoption of the cloud is not technology or cost but RISK and TRUST.  Do you, Mr. CIO, trust Google to run all of your infrastructure? your applications?  Do you Mrs. CIO, Trust Microsoft or Amazon to do the same for you?    The answer is not a blind yes or no.   Its a complicated set of minor yes responses and no responses.   They might feel comfortable outsourcing mail operations, but not the data warehouse manifesting decades of customer information.     The Private cloud approach will allow you to spread your risk.   It will allow you to maintain those aspects of the business that are core to the company. 

The cloud is an interesting place, today.  It is dominated by technologists.  Extremely smart engineering people who like to optimize and solve for technological challenges.  The actual business adoption of this technology set has yet to be fully explored.   Just wait until the “Business” side of the companies get their hooks into this technology set and start placing other artificial constraints, or optimizations around other factors.  There are thousands of different motivators out in the world.  Once they starts to happen earnest.  I think what you will find is a solution that looks more like a hybrid solution than the pure plays we dream about today.

Even if you think my ideas and thoughts on this topic is complete BS, I would remind you of something that I have told my teams for a very long time.  “There is no such thing as a temporary data center.”  This same mantra will hold true for the cloud.  If you believe that the Private Cloud will be a passing and temporary thing, just keep in mind that there will be systems and solutions build to this technology approach thus imbuing it with a very very long life.  


Must Have Swag…..

I try not to post much business related stuff (ala Digital Realty Trust) on Loosebolts as its my own place to rant and rave.   To be clear-none of the things I say on here represent the views of the company what-so-ever.   But sometimes, there are a things that come along that really make me smile and I have to comment on them.

As you know I am huge fan of modularization in the data center.  Modularization in construction, modularization in operation, modularization is just all-around goodness from a technical perspective through the business side of things.   That’s why the newest marketing campaign from Digital has me smiling ear to ear.  image   The new Data Center Construction kit brings back memories from when I was a kid and built giant structures for my little people to generally live, die, and party in.    It was of course a modular approach that led to endless hours of fun and imagination.   Applying these fond remembrances of youth and combining it with both the modular data center movement, and general fun will make this the MUST-HAVE piece of swag in the industry.   Data Center Knowledge posted a video about the toys a few weeks ago.    I can definitely tell you it will lead to hours of fun and wasted time at work putting it together.   I should know, my completed “data center” sits proudly in my office! 

After all we are all just kids at heart, aren’t we?


Live Chiller Side Chat

I am extremely excited to be participating in a live (webcast) Chiller-Side Chat hosted by none other than Rich Miller of Data Center Knowledge.   The event is scheduled for Monday, September 14th from noon to 1pm Central Standard Time.  You can register for the online event at this link.

I think perhaps the most interesting aspect of this to me is that this will be a live event and focused on answering questions that come in from the audience.   As you know I usually use my ‘Chiller Side Chat’ posts to discuss some topic or other that interests or frustrates me.    Sometimes, even others think they may be interesting or relevant too.    I am planning on meeting up with Rich and doing the webcast from Las Vegas, where I am speaking at the Tier One Hosting Transformation Summit.

I am incredibly excited about this event and hope that if you have time you will join us.  While I will endeavor to give you the right answers – one thing you can be sure of is that you will get MY answers.  :)

See you then!



Its the Law of Unintended Consequences – Some Clarity around My thoughts on Data Center Regulation

I have gotten a lot of response from the post on my thoughts on Data Center regulation.   Many of the comments in a response to an Infoworld article focused on the disbelief of regulations particularly targeting data centers.  A Greener Computing article felt that because the current administration is very tech-savvy they wouldn’t do anything to hurt data centers.  In fact the exact quote was:

I can understand Manos’ concerns, but I think he’s on the wrong track. The federal government is very unlikely to issue strict green regulations related to data centers. And if they do regulate them in some way, the regulations will no doubt be reasonable. The current administration is very technology-savvy — after all, the current Secretary of Energy Steven Chu was recently the director of the Lawrence Berkeley National Laboratory, whose work was heavily dependent on its data center. Chu did some great work related to Green IT when at the labs. He knows what can and can’t be done — and will make sure that data centers aren’t hamstrung with unnecessary regulation.

I guess for clarity sake I should state unequivocally that I do not believe that Data Centers will specifically be targeted or singled out for regulation.   Domestically here in the United States the EPA has kicked off its Energy Star Data Center evaluation which looks to study data centers as a sector, and something may come out of that, but in all honesty that wont be for some time.  I think the more immediate threat is in the efforts around Carbon Cap and Trade.  As the Greener Computing Article calls out, it was front and center at the G8 meetings.   With the UK leading the charge and the only real legislation on the books in this space, it would be hard for the other countries not to use it as the base for their programs.   My previous post focuses specifically on the fact that Data Centers will end up being significant contributing factors to Carbon metrics for companies.  Data Center Managers just aren’t thinking about it, and wont be until its far too late.  

While I am hopeful that leaders like Steven Chu and the Obama administration will weigh all possible aspects in a Carbon Cap and Trade program, the fact remains that they will need to legislate to the least common denominator and data centers are unlikely to be called out unless there is a group specifically calling attention to it.  Ergo my call for an industry wide group lobbying on its behalf.     I have doubts they will altruistically incorporate all possible sub cases into the mix without that kind of pressure.   President Obama frankly has bigger problems to be thinking about in my opinion.   

I am reminded of a quote from another excellent communicator and activist president, Ronald Reagan:

"The nine most terrifying words in the English language are: ‘I’m from the government and I’m here to help.’"

Its those times more than any other that you should put your guard up even higher.  I guess only time will tell, but one thing is certain Data Centers and IT departments will have a role to play in Carbon Reporting. 


Coming Soon to a Data Center near you, Regulation.

As an industry, we have been talking about it for some time.  Some claimed it would never come and it was just a bunch of fear mongering. Others like me said it was the inevitable outcome of the intensifying focus on energy consumption.   Whether you view this to be a good thing or bad thing its something that you and your company are going to have to start planning for very shortly.  This is no longer a drill.

CRC – its not just a cycle redundancy check

I have been tracking the energy efficiency work being done in the United Kingdom for quite some time and developments in the Carbon Reduction Commitment (CRC).  My recent trip to London afforded me the opportunity to drive significantly harder into the draft and discuss it with a user community (at the Digital Realty Round table event) who will likely be the first impacted by such legislation. For those of you unfamiliar with the initiative let me give a quick overview of the CRC and how it will work. 

The main purpose of the CRC is a mandatory carbon reduction and energy efficiency scheme aimed at changing energy use behaviors and further incent the adoption of technology and infrastructure.  While not specifically aimed at Data Centers (its aimed at everyone) you can see that by its definition Data Centers will be significantly affected.  It was introduced as part of the Climate Change Act 2008.

In effect it is an auction based carbon emissions trading scheme designed to operate under a Cap and Trade mechanism.  While its base claim says that it will be revenue neutral to the government (except of course for penalties resulting from non-compliance), it provides a very handy vehicle for future taxation and revenue.  This is important, because as data center managers you are now placed in a position where you have primary regulatory reporting responsibilities for your company.  No more hiding under the radar, your roles will now be front and center.                             

All organizations including governmental agencies who consume more than 6000 MWh in 2008 are required to participate.  The mechanism is expected to go live in April 2010.  Please keep in mind that this consumption requirement is called out as MWh and not Megawatts.  What’s the difference? Its energy use over time for your whole company.  If you as a data center manager run a 500 kilowatt facility you account for almost 11% of the total energy consumption.  You can bet you will be front and center on that issue. Especially when the proposed introductory price is £12/tCO2 (or $19.48/tCO2).  Its real money.  Again, while not specifically focused on data centers you can see that they will be an active contributor and participant in the process.  For those firms with larger facilities, lets say 5MW of data center space – dont forget to add in your annual average PUE – the data centers will qualify all to themselves.



For more information of the CRC you can check out the links below:

While many of you may be reading this and feel poorly for your brothers and sisters in Great Britain while sighing in relief that its not you, keep in mind that there are already other mechanisms being put in place.  The EU has the ETS, and the Obama Administration has been very public about a similar cap and trade program here in the United States.  You can bet that the US and other countries will be closely watching the success and performance of the CRC initiative in the UK. They are likely to model their own versions after the CRC (why invent the wheel over again, when you can just localize to your country or region).  SO it might be a good idea to read through it and start preparing how you and your organization will respond and/or collect.

I would bet that you as a Data Center Manager have not been thinking of this, that your CIO has not thought about this, the head of your facilities group has not thought about this.  First you need to start driving awareness to this issue.    Next we should heed to a call to arms.

One of the items that came out during the Roundtable discussions was how generally disconnected government regulators are to the complexities of the data center.   They want to view Data Centers as big bad energy using boxes that are all the same.  When the differences in what is achievable from small data centers to mega-scale facilities are great.  Achieving PUEs of 1.2x might be achievable for large scale Internet firms who control the entire stack from physical cabling to application development,  banks and financial insitutions are mandated to redundancy requirements which force them to maintain scores of 2.0. 

Someone once decried to me that data centers are actually extremely efficient as they have to integrate themselves into the grid, they generally purchase and procure the most energy efficient technologies, and are incented from an operating budget perspective to keep costs low.  Why would the government go after them before they went after the end users who typically do not have the most energy efficient servers or perhaps the OEMs that manufacture them.  The simple answer is that data centers are easy high energy concentration targets.   Politically going after users is a dicey affair and as such DCs will bear the initial brunt.

As an industry we need to start involving ourselves in educating and representing  the government  and regulatory agencies in our space.   While the Green Grid charter specifically forbids this kind of activity, having a Data Center industry lobby group to ensure dumb things wont happen is a must in my opinion.  

Would love to get your thoughts on that.