Coming Soon to a Data Center near you, Regulation.

As an industry, we have been talking about it for some time.  Some claimed it would never come and it was just a bunch of fear mongering. Others like me said it was the inevitable outcome of the intensifying focus on energy consumption.   Whether you view this to be a good thing or bad thing its something that you and your company are going to have to start planning for very shortly.  This is no longer a drill.

CRC – its not just a cycle redundancy check

I have been tracking the energy efficiency work being done in the United Kingdom for quite some time and developments in the Carbon Reduction Commitment (CRC).  My recent trip to London afforded me the opportunity to drive significantly harder into the draft and discuss it with a user community (at the Digital Realty Round table event) who will likely be the first impacted by such legislation. For those of you unfamiliar with the initiative let me give a quick overview of the CRC and how it will work. 

The main purpose of the CRC is a mandatory carbon reduction and energy efficiency scheme aimed at changing energy use behaviors and further incent the adoption of technology and infrastructure.  While not specifically aimed at Data Centers (its aimed at everyone) you can see that by its definition Data Centers will be significantly affected.  It was introduced as part of the Climate Change Act 2008.

In effect it is an auction based carbon emissions trading scheme designed to operate under a Cap and Trade mechanism.  While its base claim says that it will be revenue neutral to the government (except of course for penalties resulting from non-compliance), it provides a very handy vehicle for future taxation and revenue.  This is important, because as data center managers you are now placed in a position where you have primary regulatory reporting responsibilities for your company.  No more hiding under the radar, your roles will now be front and center.                             

All organizations including governmental agencies who consume more than 6000 MWh in 2008 are required to participate.  The mechanism is expected to go live in April 2010.  Please keep in mind that this consumption requirement is called out as MWh and not Megawatts.  What’s the difference? Its energy use over time for your whole company.  If you as a data center manager run a 500 kilowatt facility you account for almost 11% of the total energy consumption.  You can bet you will be front and center on that issue. Especially when the proposed introductory price is £12/tCO2 (or $19.48/tCO2).  Its real money.  Again, while not specifically focused on data centers you can see that they will be an active contributor and participant in the process.  For those firms with larger facilities, lets say 5MW of data center space – dont forget to add in your annual average PUE – the data centers will qualify all to themselves.

image 

 

For more information of the CRC you can check out the links below:

While many of you may be reading this and feel poorly for your brothers and sisters in Great Britain while sighing in relief that its not you, keep in mind that there are already other mechanisms being put in place.  The EU has the ETS, and the Obama Administration has been very public about a similar cap and trade program here in the United States.  You can bet that the US and other countries will be closely watching the success and performance of the CRC initiative in the UK. They are likely to model their own versions after the CRC (why invent the wheel over again, when you can just localize to your country or region).  SO it might be a good idea to read through it and start preparing how you and your organization will respond and/or collect.

I would bet that you as a Data Center Manager have not been thinking of this, that your CIO has not thought about this, the head of your facilities group has not thought about this.  First you need to start driving awareness to this issue.    Next we should heed to a call to arms.

One of the items that came out during the Roundtable discussions was how generally disconnected government regulators are to the complexities of the data center.   They want to view Data Centers as big bad energy using boxes that are all the same.  When the differences in what is achievable from small data centers to mega-scale facilities are great.  Achieving PUEs of 1.2x might be achievable for large scale Internet firms who control the entire stack from physical cabling to application development,  banks and financial insitutions are mandated to redundancy requirements which force them to maintain scores of 2.0. 

Someone once decried to me that data centers are actually extremely efficient as they have to integrate themselves into the grid, they generally purchase and procure the most energy efficient technologies, and are incented from an operating budget perspective to keep costs low.  Why would the government go after them before they went after the end users who typically do not have the most energy efficient servers or perhaps the OEMs that manufacture them.  The simple answer is that data centers are easy high energy concentration targets.   Politically going after users is a dicey affair and as such DCs will bear the initial brunt.

As an industry we need to start involving ourselves in educating and representing  the government  and regulatory agencies in our space.   While the Green Grid charter specifically forbids this kind of activity, having a Data Center industry lobby group to ensure dumb things wont happen is a must in my opinion.  

Would love to get your thoughts on that.

/Mm

Dinner and fireworks

Last night I attended my first Digital Realty Round Table Discussion in London and it was a fantastic treat and topper for my trip to the UK.   For those of you not familiar with these events, its an opportunity to discuss the challenges and issues facing the industry in an informal setting.  The events are hosted by Bernard Geoghegan, who is the General Manager for European Region who does a great job  of MC’ing the dinner and ensuring that conversation flows.    As the dinner begins attendees introduce themselves but are not required to mention the firms they work at.   The purpose of this meeting is real unfiltered conversation.  Selling and product positioning is not strictly not allowed, most especially from Digital attendees. 

As I sat around the largest round table in London (literally!) and scanned across the 25 or so attendees I really did not know what to expect.  I was pretty confident that if no-one bothered to offer any conversation points up,  Jim Smith (who also attended) and I could probably find some aspect of technology to argue and debate about.  But It didn’t take long for the fireworks to come out.   In fact the first person to introduce himself also listed out some of the things concerning him and that process flowed on to each participant.   By the time we got around the table of introductions we had healthy list of issues, challenges, and topics to talk about.  So much so, that there was absolutely no hope of getting to all of them.

After introductions I kicked off the conversation by diving into Data Center Management measurements.   Currency per kilowatt.  It was a great conversation with those that agreed that this was a good metric and those that did not.  I am not going to go into the topics we discussed in this post. They ranged from data center metrics, data center industry challenges, PUE, Data Center Tiering, Cloud Services, Managed Services, and a host of others. There is way too much to cover, and each will likely end up being its own post.   Lets just say there was no lack of opinion or fervor behind most topics.    Most interesting to me was the variation and representation of the firms around the table.  While many did not identify their specific firms, they did mention that they worked for a bank, a hosting provider, a large retail chain, etc.   It really highlighted to me how diverse our industry is and the technology applications we need to solve for.   The pervading thought as I left was that the current regulatory attempts to govern this space are going to be downright disastrous or ineffectual unless those agencies began to start reaching out to our industry in specific.   I have a whole post in mind on this, but fair warning – IT IS COMING (its already here), IT WILL AFFECT YOU – and YOU CANNOT IGNORE IT ANY MORE.

More on that to come.   I would strongly suggest that if you havent attended one of these events you think about doing so.   Quite a few of the attendees shared that they learned a great deal through this kind of group therapy.  It was a blast.

 

/Mm

Schneider / Digital New York Speaking Engagement

new-york-city.jpg

Just in case anyone wanted to connect – I wanted to highlight that I will be co-presenting the keynote at the Schneider Symposium at the Millenium Broadway hotel in New York City with Chris Crosby of Digital Realty Trust.  I will also be giving a talk on practical applications of Energy Efficiency and sitting on an Energy Efficiency  panel led by Dan Golding from Tier One Research.   The program kicks off at 8am on Wednesday.   Feel free to stop and say hi!

/Mm

Upcoming Webinar on Modular Data Centers

On June 22, 2009 I will be hosting a free webinar for Digital Realty Trust on Modular Data Center approaches. Its a topic I have some expertise on and quite a bit of passion around.  So I hope to not embarrass myself too thoroughly. If you have interest in the future of where data center construction, application, support and maintenance is going this might be something worth attending.   We will also touch on emerging technologies such as modular IT configuration and IT Containers and their where they might be of benefit.  

As I mentioned its a free event and you can register for it here

Digital actually has some very good taped webinars on a variety of Data Center topics you might find interesting.  The like to their video library can be found at this link.

The official blurb on the talk follows:

Digital Realty Trust would like to invite you to join us for another in our series of informational webinars. The Industrialization of the Datacentersm has given birth to a variety of modular development methods. From PODs to containers, end users are overwhelmed by a number of options and are often unsure of the best method to use for their particular application.

Monday, June 22, 2009

12:00 p.m. – 1:00 p.m. Central

In this webinar Digital Realty Trust will present the various modular alternatives that are available to today’s datacenter customers and the strengths and weaknesses of each. This presentation will also help provide attendees with a clear understanding of the potential uses for each modular approach and which datacenter requirements each is best designed to address.

Space is limited! Click here to reserve your space.

This webinar will be presented by Michael Manos, Senior Vice President of Technical Services at Digital Realty Trust. Mr. Manos is a 16-year veteran in the technology industry and most recently was responsible for the global design, construction, and operations of all of Microsoft’s datacenter facilities.

Hope to see you there!

/Mm

Chiller-Side Chats : Is Power Capping Ready for PrimeTime?

I was very pleased at the great many responses to my data center capacity planning chat.  They came in both public and private notes with more than a healthy population of those centered around my comments on power capping and their potential disagreement on why I don’t think the technology/applications/functionality is 100% there yet. So I decided to throw up an impromptu ad-hoc follow-on chat on Power Capping.  How’s that for service?

What’s your perspective?

In a nutshell my resistance can be summed up and defined in the exploration of two phrases.  The first is ‘prime time’ and how I define it from where I come at the problem from.  The second is the definition of the term ‘data center’ and in what context I am using it as it relates to Power Capping.

I think to adequately address my position I will answer it from the perspective of the three groups that these Chiller Side Chats are aimed at namely, the Facility side, the IT side, and ultimately the business side of the problem. 

Let’s start with the latter phrase : ‘data center’ first.  To the facility manager this term refers to the actual building, room, infrastructure that IT gear sits in.   His definition of Data Center includes things like remote power panels, power whips, power distribution units, Computer Room Air Handlers (CRAHs), generators, and cooling towers.   It all revolves around the distribution and management of power.

From an IT perspective the term is usually represented or thought of in terms of servers, applications, or network capabilities.  It sometimes blends in to include some aspects of the facility definition but only as it relates to servers and equipment.   I have even heard it used to applied to “information” which is even more ethereal.  Its base units could be servers, storage capacity, network capacity and the like.

From a business perspective the term ‘data center’ is usually lumped together to include both IT and facilities but at a very high level.  Where the currency for our previous two groups are technical in nature (power, servers, storage, etc) – the currency for the business side is cold hard cash.   It involves things like OPEX costs, CAPEX costs, and Return on Investment.

So from the very start, one has to ask, which data center are you referring to?  Power Capping is a technical issue, and can be implemented at either of the two technical perspectives.   It also will have an impact on the business aspect but it can also be a barrier to adoption.

We believe these truths to be self-evident

Here are some of the things that I believe to be inalienable truths about data centers today and in some of these probably forever if history is any indication.

  1. Data Centers are heterogeneous in the make up of facilities equipment with different brands of equipment across the functions.
  2. Data Centers are largely heterogeneous in the make up of their servers population, network population, etc.
  3. Data Centers house non-server equipment like routers, switches, tape storage devices and the like.
  4. Data Centers generally have differing designs, redundancy, floor layout, PDU distribution configurations.
  5. Today most racks are unintelligent, those that are not, are vendor specific and/or proprietary-also-Expensive versus bent steel.
  6. Except in a very few cases, there is NO integration between asset management, change management, incident management, problem management systems between IT *AND* facilities systems.

These will be important in a second so mark this spot on the page as it ties into my thoughts on the definition of prime time.  You see, to me in this context, Prime Time means that when a solution is deployed it will actually solve problems and reduce the number of things a Data Center Manager has to do or worry about.   This is important because notice I did not say anything about making something easier.  Sometimes, easier doesn’t solve the problem. 

There is some really incredible work going on at some of the server manufacturers in the area of power capping.   After all they know their products better than anyone.  For gratuitous purposes because he posts and comments here, I refer you to the Eye on Blades blog at HP by Tony Harvey.  On his post responding to the previous Chiller-side chat, he talked up the amazing work that HP is doing and is already available on some G5 boxes and all G6 boxes along with additional functionality available in the blade enclosures. 

Most of the manufacturers are doing a great job here.  The dynamic load stuff is incredibly cool as well.    However, the business side of my brain requires that I state that this level of super-cool wizardry usually comes at additional cost.   Lets compare that with Howard, the every day data center manager who does it today, who from a business perspective is a sunk cost.   Its essentially free.   Additionally, simple things like performing an SNMP poll for power draw on a box (which used to be available in some server products for free) have been removed or can only be accessed through additional operating licenses.  Read as more cost.    So the average business is faced with getting this capability for servers at an additional cost, or make Howard the Data Center manager do it for free and know that his general fear of losing his job if things blow up is a good incentive for doing it right. 

Aside from that, it still has challenges in Truth #2.  Extremely rare is the data center that uses only one server manufacturer.  While its the dream of most server manufacturers, its more common to find DELL Servers, along side HP Servers, alongside Rackable. Add to that fact that even in the same family you are likely to see multiple generations of gear.  Does the business have to buy into the proprietary solutions of each to get the functionality they need for power capping?  Is there an industry standard in Power Capping that ensures we can all live in peace and harmony?  No.  Again that pesky business part of my mind says, cost-cost-cost.  Hey Howard – Go do your normal manual thing.

Now lets tackle Truth #3 from a power capping perspective.   Solving the problem from the server side is only solving part of the problem.   How many network gear manufacturers have power capping features? You would be hard pressed to find a number on one hand.   In a related thought, one of the standard connectivity trends in the industry is top of rack switching.  Essentially for purposes of distribution, a network switch is placed at the top of the rack to handle server connectivity to the network.     Does our proprietary power capping software catch the power draw of that switch?  Any network gear for that matter?  Doubtful.  So while I may have super cool power capping on my servers I am still screwed at the rack layer –where data center managers manage from as one of their base units.   Howard may be able to have some level of Surety that his proprietary server power capping stuff is humming along swimmingly, he still has to do the work manually.  Its definitely simpler for Howard, to get that task done potentially quicker, but we have not actually reduced steps in the process.   Howard is still manually walking the floor.  

Which brings up a good point, Howard the Data Center manager manages by his base unit of rack.  In most data centers, racks can have different server manufacturers, different equipment types (servers, routers, switches, etc), and can even be of different sizes.    While some manufacturers have built state of the art racks specific for their equipment it doesn’t solve the problem.  We have now stumbled upon Truth #5.

Since we have been exploring how current power capping technologies meet at the intersection of IT and facilities it brings up the last point I will touch on regarding tools. I will get there by asking some basic questions as to the operations of a typical data center.  In terms of Operations does your IT asset management system provide for racks as an item of configuration?  Does your data center manager use the same system?  Does your system provide for multiple power variables?  does it track power at all?  Does the rack have power configuration associated with it?  Or does your version of Howard use spreadsheets?  I know where my bet is on your answers.  Tooling has a long way to go in this space.   Facilities vendors are trying to approach it from their perspective, IT tools providers are doing the same, along with tools and mechanisms from equipment manufacturers as well. There are a few tools that have been custom developed to do this kind of thing, but they have been done for use in very specific environments.  We have finally arrived at Power Capping and Truth #6. 

Please don’t get me wrong, I think that ultimately power capping will finally fulfill its great promise and do tremendous wonders.  Its one of those rare areas which will have a very big impact in this industry.   If you have the ability to deploy the vendor specific solutions (which are indeed very good), you should. It will make things a bit easier, even if it doesn’t remove steps.   However I think ultimately in order to have real effect its going to have to compete with the cost of free.   Today this work is done by the data center managers with no apparent additional cost from a business perspective.   If I had some kind of authority I would call for there to be a Standard to be put in place around Power Capping.  Even if its quite minimal it would have a huge impact.   It could be as simple as providing three things.  First provide for free and unfiltered access to an SNMP Mib that allows access to the current power usage information of any IT related device.  Second, provide a Mib, which through the use of a SET command could place a hard upper limit of power usage.  This setting could be read by the box and/or the operating system and start to slow things down or starve resources on the box for a time.  Lastly, the ability to read that same Mib.    This would allow for the poor cheap Howard’s to take advantage of at least simplifying their environments.  tremendously.  It would still provide software and hardware manufacturers to build and charge for the additional and dynamic features they would require. 

\Mm