We’d like to thank Infoworld for selecting us, our users and customers for their support, our developer community for their contributions, and the hard-working Hyperic engineering, quality assurance, and customer support teams. We couldn’t have done this without you!

We have some more news coming down the track soon, so keep your eyes on this blog as well as to our Twitter stream.

According to the report, published on March 17, 2009, “cloud computing is becoming one of the most visible sourcing option for infrastructure and operations professionals. I&O professionals, thus, must be aware of how they will manage this environment from an availability, performance, disaster recovery and service aggregation point of view”.

The Gartner research report suggests IT operations organizations that are using cloud services provided by Amazon or Google should consider Hyperic products to monitor this environment. In addition, development teams desiring to better understand the performance of applications being designed for the cloud should investigate Hyperic’s offerings. We couldn’t agree more. To access the report, you’ll have to talk to Gartner first.

Not so long ago, when a business decided to solve a problem with software, the calculus was simpler: make or buy. Software products came from software vendors, who competed with each other, and with internal development teams to get business.

It’s different now. Pick a problem that can be solved by software, and you’ll find multiple open source offerings. Certainly, the lean and the mean look to the  open source world first. But corporate behemoths are down with it too: acceptance of OSS by the man is old news. And when even the UK government buys in, open source is mainstream.

The “you get what you pay for” adage doesn’t necessarily apply to open source.  Many software companies, including Hyperic, provide robust and rich open source versions, in addition to enterprise editions that customers pay for.  We do pack extra, high-value features into our enterprise version, but we have plenty of users that manage smaller infrastructures quite successfully with the OSS version.

But, back to Mr. Burton’s plea for help. He asks what open source monitoring solution he should use instead of Nagios. Clearly, he should consider Hyperic HQ.   But, perhaps the broader issue is, how do you choose the right open source product, whether you’re looking for IT monitoring or an application to identify weed species in crop systems in the Indo-Gangetic Plains”?  (Not a made up example.)

Cursory research (using a proprietary search engine) indicates there’s a lot more chat about why to choose open source software than how to choose it. One of the few on how to shop around is this ZDNet article; it focuses on how to choose a CMS, but the recommendations apply to any software choice.  The author suggests checking out freely available information for clues about the software publisher’s process, quality, and community.

Here are eight additional ideas to include in your software evaluation checklist:

1. Do you have to build the product to be able to use it? I’d rather this were an option than a requirement, and (personal opinion) the fact that I have to build it myself seems to indicate that ease of implementation and use is not a high priority to the publisher.
2. Is information about the product as free and available as the software itself?  Is there documentation? Or is the information you need on page 200 of 10,000 pages returned by your search query?  If there is documentation, is it current? It should correspond to the current version of the software. Is it documentation free? If it’s not, how do you like paying for information that is required to make the product work?
3. Can you find a statement that unambiguously describes the product’s functionality?  It’s funny how long it can take to figure out that the behavior or feature you’re trying to use isn’t actually supported.
4. Is there a clear definition of what the product is “best” for? Can you figure out what type of environment it was designed for, what its limits are, in terms of scalability, security, or other criteria?
5. Can you locate information about the implementation process, like how much configuration is required, and what’s involved?  For instance, to configure a behavior, do you use a web UI, edit a property file, or modify the source code?
6. How big, active, and satisfied is the community. User forums tell you a lot.  Do questions get answered?  Do many responses suggest “reading the code”?  (Danger Will Robinson.) Are there actual contributions – new features, plugins, extensions, etc. – from community members?
7. Will you get the support that you need? Obviously, this depends on your appetite for self-support and the impact of unsolved problems on your mission.  If you don’t have a high tolerance for unsolved bugs or unanswered questions, is there somewhere to go for support, even if you have to pay for it?
8. Is there a growth path? Assuming the software has some hard limits, in terms of scalability, security, or functionality, what will you do when you hit them? Is there a fuller-function offering you can migrate to easily?;-)

Bottom line, Mr. Burton should have no trouble finding a good OSS monitoring solution. Perhaps he’ll use take this advice and make the right choice. :-)

Developers like the cloud since it lets them bypass operations (i.e., the control agents) and serve their environment needs quickly. In the cloud, developers can build an application and launch it themselves, on their own, without waiting for hardware to be purchased, racked & stacked.  Without waiting for hardware or virtualization capacity planning, electricity consumption plans, cooling or green discussions to finish. If you’re a developer, the cloud offers speed! The cloud is way cool. You’re the man. But developers too often forget to ask themselves what happens afterwards?  Who will make sure the app is always available?  Apps in the cloud will still have performance issues and will still break.

The first cloud dilemma for developers… For better or for worse (up for debate), by publishing to the cloud devs become responsible for all three phases of the application – build, deploy and manage – whereas typically they are only responsible for the build stage.  Red flag – few developers have an operations background. And they usually know little about application performance monitoring, or the tools of this trade.

Developers often misstep by putting a web application up on Amazon’s EC2 cloud computing environment assuming that bad code, bad programming or a poor choice of architecture is somehow not a problem, or that it will somehow work out in the end. Wrong. Wrong. Wrong.  Can the ease of spinning up a new AMI in a nanosecond mask application performance problems like malformed queries, memory leaks and runaway processes?  Maybe, but the issues will catch up with you and you’ll be running up lots of unnecessary charges on AWS.

The second cloud dilemma for developers is subtle but should frighten any business that has to stand behind a service or application that it hosts in the heavens. This dilemma is centered around service level agreements (SLAs) that developers must consider to avoid becoming become trapped between a rock and a hard place.

The Rock: The SLA the developer provides to customers
The Hard Place: The SLA that the cloud provides to the developer

Almost never shall the two meet. What developer wants to be on the hook for a customer SLA that REQUIRES MORE THAN the cloud platform’s SLA? You’d be surprised how many developers make this bad decision and don’t even know it.

With the cloud, It comes down to this — performance, cost and availability are at the center of a very important job that rests in the hands of someone who typically doesn’t have the experience to make those decisions alone, the developer. That’s a dilemma no business wants to find itself in.

We believe that companies have two options.  One: continue to leverage our friends, the ops guys, to make sure your applications perform — wherever they may live.  And two: wait for the inevitable evolution of the role of developers  –  some will learn new skills and address the monitoring and management challenges head on.

How about you? Are your apps in the cloud rock solid? Do you see any “epic fails” in the cloud that you don’t see on-premise?  Who manages your apps in the cloud?

“A recent IDC survey of IT executives, CIOs, and their line of business (LOB) colleagues shows that cloud services are ‘crossing the chasm’ and entering a period of widespread adoption. Moreover, IDC expects the cloud adoption trend to be amplified by the current financial crisis. The cloud model offers a much cheaper way for businesses to acquire and use IT – in an economic downturn, the appeal of that cost advantage will be greatly magnified. This advantage is especially important for small and medium businesses, a sector that will be key target in any plan for recovery.”

The report goes on to cite that spending on IT cloud services will triple in the next 5 years, reaching $42 billion and capturing 25% of IT spending growth in 2012 and nearly a third of growth the following year. With he overall IT market spend being $383 billion, this kind of growth validates that the opportunity deserves the attention it is getting.

IDC is also first to point out that its not just cloud vendors – the Amazons, Googles and Salesforces of the world will not get this pie exclusively. In fact, just as important are all the offerings that support the development, delivery and deployment of those consuming these services, which is the market Hyperic is in – bridging the gap for monitoring between the datacenter and the cloud

In fact, in the survey, two of the top 4 demands are areas that Hyperic is solving:

CloudStatus – providing real-time visibility into the health and availability of cloud providers, giving users a third-party perspective into SLAs

Hyperic HQ – providing real-time trending, analysis and control capabilities to maintain the performance of your web applications. This can run the gamut of possibilities from helping to improve the performance of your application, to making critical infrastructure architecture decisions quickly (e.g., spawning new virtualized resources or shifting loads from the cloud to your datacenter and vice-versa based on demand or performance changes).

I am sure there will continue to be pundits of all kinds of caliber, and while cloud computing may seem at times too much like inflated marketing-speak, the delivery model and innovative services that are powering this growth are unquestionable. Its nice to have some independent analyst numbers to back it up. Thanks Frank & all the rest of the IDC team for sharing this.

Maybe I’m an idiot, but I have no idea what anyone is talking about. What is it? It’s complete gibberish. It’s insane.

And of course, Richard Stallman, founder of the Free Software Foundation, who told the Guardian his opinion of cloud computing:

It’s stupidity. It’s worse than stupidity: it’s a marketing hype campaign.

I wasn’t surprised at Larry’s reaction, as the more classical IT enterprise that his business serves is by definition going to be a late adopter. I was surprised at Stallman, however.

His arguments went for a general theme of cloud computing being too much risk for the CIO. Funny, as open source suffered the same start. Open source weathered many years of questions on what it is, how could businesses accept the indemnification risks, and of course how mature were the products being developed under its banner.

Hyperic has embraced both. We open sourced our software in 2006, and have roots in open source starting in the nineties from Netscape to Apache from our founders. It’s also no secret that Hyperic, and its users, are early adopters of the cloud. We’ve been part of this trend unfolding, visibly so with our CloudStatus service.

We didn’t do either because they were cool, or for any desire to be some sort of fashionable IT software company. We did it because it makes sense for our users, and both are an approach to building more affordable, useful, and scalable IT.

Open source is a business model for us to work with our users – they are building modern web-driven applications and need better monitoring and management for these dynamic, custom-built environments. Since they are web-afficianados when they have a problem they consult, that’s right – the web. They do a Google search, they look for how others solved it – and they look for the straightest line to applying the same solutions to their unique situation. They want self-service. They want quick results. Open source lets them download and use the software without unneccesary limits on time, scale or its application. Once our software establishes its value, if there is more opportunity to help them more, we have an educated, successful, in-house champion to help work through the sale. We save a lot on the cost of sales and support this way. The customers in turn save time and money in completing their solution. Open source is way to package our products and services to ease adoption so everyone benefits.

Cloud computing is much the same. Cloud computing is a system of technologies and services that have commoditized IT to make it more readily consumable, scalable, and cost-effective for everyone. It has leveraged the innovation and expertise of Internet giants like Amazon and Google, and is making it accessible to anyone with the next big idea. It removes the investment in physical and human resources to scale up a business. It affords more folks to try their ideas and vet its worth in the market. It also affords these same businesses to scale out as quickly as their business demands. Cloud computing, same as open source, is a way to package products and services to ease adoption so everyone benefits.

Open source was sexy because it was toppling the big guys by eroding their market shares. They *mostly* now all get that it is a better way to do business with a larger, more unpredictable market that prefer to leverage open components to construct their own inventions.

Cloud computing is sexy because it taps into the entrepreneur’s “CIO envy”, as the451group’s Rachel Chalmers called it when we last spoke. This “CIO envy” channels the aspirations for anyone to be the next Facebook. It removes the need for deep pockets and a deep technical bench to scale up their business to go to market, and scale out to capitalize on customer demand. It will come in all shapes and sizes, from infrastructure services to software application services to development platform services and all the surrounding implementation support services that typically surround IT. Regardless of the form, its purpose will be the same – to reduce IT complexity to create scalable, building blocks that can be consumed and paid for based on real usage.

In short, the term may evolve to become not as sexy, but the concept, just like open source, is too big and attractive to ignore.

Amazon AWS Loves CloudStatus.com, Here’s The Proof (Hyperic Video 2/2) from Toon Vanagt on Vimeo.

My favorite is Virtualization.com’s for a few reasons:

1. Trav still had the mohawk in that video. Yes, people – the girlfriend won, and he is now back to the short haired Nebraska boy his momma knew well.
2. A couple typos turned Javier into a Xavier and Trav got promoted to a CTO!
3. Last, and definitely mostly – the videographer, Toon Vanaght managed to catch Amazon’s CTO Werner Vogels the next day at Structure and get it on tape that Amazon thinks CloudStatus is important and good for their customers!

So, here’s to you, Mr. Underappreciated Sys Admin Guy (or Gal).

If you’ll be joining us at OSCON this year, be sure and check out the Evolution of Community panel featuring yours truly.

Over the past ten years, nothing has impacted business more than community. Whether through the openness of software development spurred by Linux or the dismantling of media empires through blogging, the rise of communities has been the driving force in how we work and live today. But what’s next? For open source developers, what has to happen to maintain and grow the communities they’ve built? What happens to communities when successful projects are acquired by big corporate behemoths? What happens to communities when their projects fail?

Join a panel of those who get a first-hand look at what it takes to manage some of the highest-profile communities in open source: Joe ‘Zonker’ Brockmeier from OpenSUSE, Ross Turk from SourceForge.net, Jono Bacon from Ubuntu, John Mark Walker from CollabNet and Jeremy Hogan from Hyperic. What trends are they seeing across their communities? What advice can they give other community managers? What’s worked and what hasn’t for them? What’s on the horizon for each of their communities?

More info here. If you register now, you can get 15% off using this coupon code: os08s15. ]]> http://blog.hyperic.com/the-evolution-of-community/feed/ 0 http://blog.hyperic.com/hyperlinks-july-8-2008/ http://blog.hyperic.com/hyperlinks-july-8-2008/#comments Wed, 09 Jul 2008 01:34:06 +0000 Hyperic Press Desk http://blog-staging.hyperic.com/hyperic/?p=336 Very interesting headlines in today’s HyperLINKS. Here are Tuesday’s tidbits:

As the world moves to cloud computing, service level insight gets foggier. Don’t let your performance get lost in the clouds.

www.cloudstatus.com

This afternoon we get another data point showing that despite the exciting promise of cloud computing, the realities of managing large scale infrastructure insist on rearing their ugly heads. TechCrunch is reporting that Google Apps had an outage today which caused the service to be completely down.

There’s two serious problems with this outage:

1- The reports of service outages arrive long after anyone who depends on the services can possibly do anything to mitigate their effect.

2- The services themselves seem incapable of providing any visibility into the circumstances that might lead to future outages.

On the first point, despite my affinity for TechCrunch, I think it’s rather sad that that is the place where the outage is most prominently chronicled. After all, TechCrunch is a news site. Counting on news sites to explain why your cloud dependent app is broken is like finding out from CNN that your house is on fire.

The second point just shows how these cloud services seem dependent on blogs (again, news technology) as a means to report the status of their services. Even TechCrunch points out that the Google Apps blog doesn’t even mention the outage. Other clouds rely on blogs such as this one, this one, or maybe even this one (from our good friends at Mosso). These are all places where outages can be discussed, but not the right means for people to find out whether it was their application that crashed, or the cloud that it depends on.

Without this type of visibility and transparency (as TC’s article points out) it will be difficult for serious , mission critical applications to be built using clouds. Even if the company providing the cloud is as famous and respected as Google and Amazon both are.

Marty Messer, our esteemed Director of Customer Success has been recently deluged with the same request from some of our top customers, “do you know anybody we can hire?”

So, I thought I would take the liberty to help them out and post the job reqs to our community. If anyone is interested, go ahead and respond to these Dice postings:

LDS (Mormon Church of Latter Day Saints), Salt Lake City
Senior Systems Engineer (Monitoring)

CNET Networks, San Francisco
Senior DBA/Systems Administrator

Three Rings Design, San Francisco
Senior UNIX Network Administrator

Cypress HCM, San Francisco
NOC System Administrator (3-6 months contract)

If you have any others, I started a forum post on job requests as well. Hopefully the power of the community can help attract good help for our other community members!

They even outline the important monitoring tasks you need, and stress that it will help when dealing with their own world class support.

Monitoring involves a number of essential tasks, including those listed below:

• Monitoring log files.
• Checking for HTTP-availability and performance (e.g. by getting the same page every five minutes and displaying the time on a graph).
• Looking at many different parameters such as load, connections, IO, database-trends, and so on.
• Charting long-term trends.
• Keeping an access log of requests to the web server. This is vital, especially when requesting performance-related support from Atlassian.

They even pass on a screenshot of their Hyperic HQ deployment, which if you notice, they are hosted by Contegix, another Hyperic customer.

We agree 100% with Atlassian that it is critical to monitor their apps, which is why we’re also working with them to build application specific management plugins for Confluence and JIRA.

Systems Management is a tried and true habit for IT. It is sort of like brushing your teeth every day. Its not glamorous, but its something you do if you don’t want your teeth to fall out. Same thing, you “do” systems management if you want your IT infrastructure to be available. What IT Admin wouldn’t want that?

From this new report, it appears that virtualization has caused formerly trusty, change-adverse SysAdmin to throw caution to the wind, gather up all their lucky pennies, four leaf clovers and wear their lucky socks every day for the rest of eternity. Or maybe (more likely) their bosses are forcing the cost savings/next biggest IT trend down their throats before they have adequate time to form a proper management strategy.

Hence, 98% of IT employing virtualization seems to have a heavy case of VMotion Sickness. Joe describes a common scenario of VMotion Sickness through his interview with Hyperic customer, Mosso (a division of Rackspace):

Mosso’s operations staff quickly learned that the native tools that had performed adequately in development and test environments were not up to the task of responding to production requirements. Company officers likened the experience to a game of whack-a-mole, as Web traffic spikes drove IT to manually VMotion guest instances to accommodate demand.

These VMotions sound great, they are a quick fix to a current problem, but they introduce a host of issues such as:

• A performance penalty is incurred across the entire virtualized infrastructure as the virtualization software uses serious resources to facilitate the VMotion. All running applications now have less resources to use. The more you do this, the more resources you need to allocate to virtualization software and reduce the cost savings of improved resource usage.
• Performance history of the guest operating system is lost, or disrupted. There is no way to see if the problem is in the physical or the virtual and if this is a growing trend. It is always just a temporary fix.
• Capacity planning becomes skewed and unpredictable.

Before writing this report, Joe had a chat with Hyperic CTO, Doug MacEachern and myself. Joe reached out to us since he’s covered Hyperic in the past and new we had a particular focus on managing virtualization environments. We discussed our experience in managing these virtualization environments, and described the issues covered in our whitepaper “Bridging the Virtual Divide” and subsequent webinar with Hyperic customer, Mosso. This webinar is actually where Jonathan Bryce and I first introduced the term “VMotion Sickness”.

Since then, Joe has learned a lot – as his research shows. It looks like the largest vendors – cloud providers, like Mosso, and ISPs alike – are the ones leading the pack in actually managing their virtual sprawl. This makes sense, as these companies are executing virtualization in production at scale, and consequently suffering the repercussions of daily operations at scale as well. As Joe describes:

Large virtualization deployments are surprisingly complex. Because of gaps in management toolsets, everyone from VMware to third-party vendors to large organizations recommended engaging an experienced outside partner to resolve current VM sprawl issues or when planning for large-scale VM conversions and deployment. VMware, for example, strongly recommends that customers with more than 100 VMs work with a channel partner or the company’s own services arm.

Fortunately, for those that haven’t found it yet – there is hope. As Mosso’s Jonathan Bryce summed it up:

While Mosso is now able to efficiently manage its large virtualized hosting infrastructure with a stable pool of operations employees, it reached this level of maturity only after partnering with outside expertise from Hyperic to address VM management concerns.

It sounds like the other 98% of you should try Hyperic out. Its free, and it works a heckuva lot better than those smelly “lucky” socks.

Once again, SourceForge.net is holding its annual community choice awards. As a former SourceForge.net worker bee, it’s great to be able to utilize what we built at VA Linux all those years ago.

So, if you’re a fan of Hyperic HQ – and if you’re not, you really should be – nominate us in one of the categories. Click the image to the left, and the rest is up to you!

мебели

Perhaps the greatest irony of working in the management software business is the fact that the problems our products solve are the same ones that keep people from finding us. After all, who has time to look for new software when you’re busy fighting fires, right?

This is the reason why we’ve historically made such a big deal about the role search plays in helping people find Hyperic. Interestingly, people dont tend to search explicitly for Hyperic… instead they search for things like:

- “total nursery size weblogic”

- “how to find replication transactions in mssql 2005″

- “exchange 2003 monitor total ops per user”

- “freebsd open files per process”

Each one of these items (found in our huge search logs) came from someone out there, looking to troubleshoot a problem with part of their infrastructure. Even more interesting, each of these search results points to either documentation about the problem, a reference to someone else looking for the answer, or a product that can help you find the answer (namely ours!). This pattern repeats itself thousands of times a day and is a key part of how more and more people will find out about HQ and how it might help them.

Google’s search engine has proved to be the best tool for people to use to solve these problems not just by finding software, but by pointing them at the potential answers to some potentially obscure questions. The skill of going from an obscure error message in your Apache logs to a solution to a production issue is essential in any web ops or admin role these days and your success rate is highly dependent on how well you construct the query in the first place.

It’s also an incredibly important skill for developers. I’ve long maintained that the skill that Doug (our CTO/co-founder) has for finding references to poorly documented management APIs or references to strange errors (here’s looking at you, Webs-Fear) is one of our many secret weapons.

That skill itself is a key ingredient for success of sysadmins, and even developers and something I would argue should be measured as part of the interviewing process for either position. Seems reasonable, right? Talk to any of our developers (or folks who have interviewed here) and they’ll tell you we have a pretty grueling interview process with a lot of code-related questions. It’d be great if companies could develop an equally useful method to measure how effective a candidate may be at finding the answer to some challenging troubleshooting questions.

I’m curious if anyone has figured out a way to measure and evaluate this skill as part of their interviewing process… perhaps one day we’d have a whole methodology for testing this like we test for coding skills.

• Oracle Application Server
• Oracle Containers for Java (OC4J)

As a relative new-comer to the Hyperic Engineering team, I would like to use this opportunity to introduce myself to the community. I spent the previous two years as a platform engineer working on the Oracle Communication and Mobility Server. At Oracle, I helped integrate the J2EE and SIP application servers, exposing management and configuration features via Java Management Extensions (JMX).

So, as you can imagine, I was very excited to take on the project of developing a Hyperic plugin for the Oracle application server, a product I am so intimately familiar with. So, for you Oracle users out there – please check out the HyperFORGE and take the new management plugins for a spin. I’ll be watching for feedback in the community as we prepare to make this part of the standard distribution in 4.0.

Many of you know Barry as the co-founder and longtime CTO of JasperSoft. Barry recently took leave of JasperSoft to find something that let him enjoy his family more. However, for those of you that know Barry well – his big brain is always cooking, and he can’t stay still for long. We’re excited that Barry is joining us at Hyperic on a part-time basis to do some more work on our JasperSoft integration. He’s actually cooking up a whole new ad-hoc reporting tool that is sure to cause millions of Hyperic-lovers out there to swoon.

Welcome, Barry!

Removing this “undifferentiated heavy lifting” (hereto referred as UHL) from your cycles is supposed to free operations to spend more time actually operating. As I was listening to the talk, I wondered how many folks believe there is some competitive advantage in UHL? Certainly choice of hardware, network architecture, data center setup all fall under UHL and can mean the difference between success at scale versus utter failure. Nonetheless, the argument is that you shouldn’t reinvent the wheel and should take advantage from those providers (Amazon in this case) who do the UHL for you, right? Well, a lot of people certainly think so based on how much press and use is being directed towards the cloud.

This begs raises the question: “If racking boxes, configuring OS’s and so forth is UHL today, what will be UHL tomorrow?” That question is material to companies in the systems management market because getting caught on the wrong side of UHL means your future is (as our favorite magic 8 ball would say) “Outlook Not So Good”.

Luckily, UHL today is mostly about the pain associated with hardware and network provisioning and configuration. These problems are bounded just enough to make it feasible for someone to simply delegate them to a cloud provider (as many already have). Of course, the implications to management vendors focused on managing UHL tasks are not pleasant if you buy the idea that most applications will move into this sort of an environment. If you buy this vision, then conceivably the future will require one GIANT Tivoli license for the One-Cloud-Provider-To-Rule-Them-All, right? Heh, not quite, but it’s fun to think about it that way!

The reality is that this trend is forcing both service providers as well as application developers to rethink their operation strategy. Providers want to be more like clouds, developers want to run inside them.

Why rethink their ops strategy? Because the stuff that sits above the UHL layer… the middleware, the databases, and most importantly, the code that makes up a given application present the most daunting challenges due to their complexity. This leaves a meaty management problem yet to be solved. One that has more to do with managing complex software stacks with components which might reside inside or outside of the cloud and with an ‘elasticity’ (to borrow another one of Werner’s terms) which demands equal agility in the management layer than what is found in the software stack.

This is where the next big wave of innovation will happen in the management space, and we’re excited to be a part of it. Come visit our booth at the Web 2.0 Expo this week, and you’ll see what I’m talking about.

This year definitely had a different feel, and of course that had a lot to do with Sun’s influence. It felt like it was almost a new event, a chapter 2 for MySQL, and its ecosystem of vendors and customers. There were more people – I don’t know exact numbers, but it felt appeared to be twice as packed. The exhibit hall was the same, but we took up a bit more space than last year and certainly there were much fancier booths – ours included! We even gave away multiple prizes this year – our fun 8-ball tshirts, and a couple remote control helicopters. Scott Baird and Mike Hogan were the lucky winners this year.

The one thing that hasn’t changed is our fit with the MySQL customers. This year we met several of our own customers and users face-to-face – including an entire legion of the Rackspace/Mosso guys. I had dozens of conversations with MySQL users managing the LAM-* stack, and showed them how Hyperic helps wrangle all the moving parts. Several of them were lined up when the exhibit hall opened the next day to tell me how their deployment went the night before! Very cool. We spend a lot of time talking to users – via the forums, email and phone – so to see their faces the next day after they deployed is a really cool experience.

And this experience wasn’t mine alone, Hyperic has grown quite a bit in the past year, almost tripling in size. So we had many more people from all departments hanging out on the floor and interacting live with users. Check out our own Mark Deadder, sales guru, wooing a small crowd while flashing his broken wing. (Minutes before, Mark fatally crashed the demo helicopter – perhaps if we could monitor it with HQ, the wing would still be attached!)

Thanks, MySQL – we’re looking forward to next year already!

I’ve been lucky to hear Marten speak at a number of events, and have found that one of the most consistent themes can be distilled down to this (which we at Hyperic fully subscribe to):

Build a great product, Empower your Users, Build a great business

He also reiterated the importance of the GPL as an essential element to drive empowerment of users. This lets them adopt products and participate while creating an opportunity for businesses to deliver value people are willing to pay for. A virtuous cycle, right? Sun seems to think so also (a billion dollars is not the type of money any company throws around lightly)

Ironically, as I was in the middle of writing this post, Slashdot’s editors let out this gem in a post claiming that MyS^H^H^HSun had “begun to close source MySQL”. Luckily, Marten was able to set the knee-jerk cable-news-inspired Slashdot post straight by again explaining to the readers that MySQL is a business which has been able to build a great free, open, GPL database by creating value they don’t necessarily give away. He’s candid about the fact that they are experimenting trying to arrive at the best business model that balances all the elements of the above equation. Obviously, it’s not easy and someone’s undoubtedly going to get upset with the result. Apparently some people on Slashdot think MySQL’s database is written by monks who are morally opposed to any compensation.

Why is this important? Well, because software doesn’t build itself. Another admirable (if slightly more outspoken) guy named Marc Fleury made a big point of that with JBoss’ “Professional Open Source” business model. Everyone got what they wanted: the community got a great, free, open J2EE app server, and JBoss got paid to hire more people to continue to build it.

Recently there were a few examples of failed companies in the management space which either directly used open source or had some connection to it in their businesses. Many wonder if the failure had something to do with open source, their choice of license, or the business model. Few (if any) looked at the more obvious part: the products offered by those companies. Just because something is free and open source, it doesnt mean people will flock to it. Without that first element of the equation, there’s no adoption and no opportunity to create value people will pay for. The result is a failed business. The formula for creating that value is still being developed and is different for every company. We’ve chosen one which so far has brought us much success, but might not work for others.

Much like MySQL, we’re eager to experiment with different ways in which we can deliver the best products to the largest audiences while delivering financial success for our employees and investors. Over the course of this year, you’ll see us continue to enhance and deliver more features and functionality to our open source platform, while continuing to enhance the value of our various subscription offerings. Also like MySQL, we’re eager to engage our community directly as we experiment and learn how to continue to build the best solution to managing large scale web infrastructure.

Hyperic will have a booth in the expo. We’re at booth #12, so stop by and see us. In keeping with the theme of the event – informal and irreverent – we promise to have a fun booth for everyone, with some great shwag giveaways and prizes.

Use this:

To warm this:

The article from the AP states:

In a few cases, the heat produced by the computers is used to warm nearby offices. In what appears to be a first, the town pool in Uitikon, Switzerland, outside Zurich, will be the beneficiary of the waste heat from a data center recently built by IBM Corp. for GIB-Services AG.

Now, the question is are folks inside the data center monitoring the temperature in the pool, or are they all at the pool monitoring the temperature of the data center? Perhaps they can use Hyperic to monitor their Senatronics temperature sensors?

SAN FRANCISCO, Calif. – March 26, 2008

• Hyperic today announced a new version of both the open source and enterprise versions of its flagship software, Hyperic HQ
• The centerpiece of the new release is a plugin framework, HQU, that makes it easy to script and automate tasks so that operations teams can better organize and manage their infrastructure
• Hyperic HQU plugins can be applied in three ways:
  • Application #1: Custom UI views allow improved tracking and automation of specific applications or hard-to-manage tasks
  • Application #2: Web Services integration allows Hyperic to easily syndicate key data across all systems that manage IT infrastructure
  • Application #3: A Groovy script console leverages templates to simplify manual, error-prone tasks through automation
• While the majority of the HQU plugins will be available exclusively through the HyperFORGE, some will be bundled directly in the HQ application

Supporting Quotes

• “85% of our customers update their technology weekly to keep up with changing demands. As a result, their environments are so highly customized and prone to change that they are extraordinarily difficult to manage. Our mission is to help this new generation of online companies overcome this challenge. Using the new HQU framework, users can deploy a ‘zero touch’ management strategy that perfectly fits their custom environment – tailoring their deployment and automating tasks so they can solve problems fast, even when operating at extraordinary scale.” – Paul Melmon, SVP of engineering at Hyperic
• “With an extremely diverse infrastructure, meteoric customer demand and some of the most demanding SLAs in the business, we were facing an enormous management challenge. Hyperic was the only solution that made sense for us. The new HQU plugin framework will make it even easier to manage such a complex environment with a scale like ours. We plan to use it aggressively to automate tasks and simplify finding and fixing problems so we can better serve our users.” – Matthew Porter, CEO of Contegix
• “Open Source Software’s roots lies in open cooperation with the community that supports it. HQU Plugin Framework delivers a serious development advantage to Hyperic’s users and its community. Our community can now develop full feature functionality in an extremely modular, easy way and share their achievements with other HQ users. Our forums are positively buzzing with anticipation – I can’t wait to see what they come up with first.” – John Mark Walker, Hyperic Community Manager
• “Every sysadmin has existing scripts or products that interact with their infrastructure. HQU was designed to combine these scripts with the power of HW – to bring your management interface together with Rails’-like programming ease. Using Groovy with HQU, I wrote a new plugin UI view which shows JIRA trouble tickets right next to HQ’s resource information in about 15 minutes. Building something that useful, that quickly – it’s just cool.” – Jon Travis, Hyperic Principal Engineer

Supporting Resources

• More about Hyperic HQ
• More about Hyperic HQ Enterprise
• Contegix Website
• Contegix Case Study
• Groovy Website

Over the course of the last 9 months or so, my perspective has changed a bit due to two reasons. First, Hyperic is fortunate to count some of the largest players in the social networking space (Hi5, CNET, plus a few we haven’t announced yet) as well as some of the most significant enterprise and software-as-a-service players (Comcast, Dice, Intuit, and many others) as customers. This puts us in the unique position of being able to see how both of these seemingly different types of companies are tackling some of the same problems. The second reason is because I’ve spent quite a bit of time talking to my good friend, advisor, and Hyperic board member Bob Bickel. Bob’s new company Ringside Networks, launching this week, is focused on building technology that bridges the gap between social and business applications and does so through an open source product offering.

Rather than provide another recap of what Ringside does, I encourage folks to check out Matt Asay’s great writeup on his blog. Instead, I’ll offer up some commentary on why the “Social Application Server” concept makes sense, and how it might relate to Hyperic at some point in the future.

There’s a lot of talk these days about social networking and SaaS ‘platforms’. Companies expose API’s to their services and encourage people to build integrated apps which leverage the power of a CRM (like Salesforce) or social (like OpenSocial or Facebook) platforms into their apps. The nature of this integration typically involves two applications running on separate infrastructure, talking through the Internet through some variant of REST or web services. Aside from Salesforce’s Force platform, the majority of the integration between these sorts of apps is done on the outer layers of an application. An app can devote an IFrame or some other piece of screen real estate so it can display content from a platform, or perhaps use Javascript calls to embed functionality. Even in cases like Force (where there’s a richer API which can be embedded at any layer of the application), the interests of the platform provider (especially those of ad-driven social networking sites) restrict the ability of a customer of the platform to more deeply integrate the content from a social app into theirs. Ringside’s open source Social Application Server is meant to remove that restriction.

What does this have to do with Hyperic? Well, for a long time we’ve been looking at the emerging problems of managing large scale web applications whose content and functionality is not entirely under control of a single provider. If you’re a downstream customer of one of these platforms, how do you react to it’s availability or proper performance? If you’re a provider of a platform, how do you measure the usage of your API such that you can scale effectively without compromising your own site’s performance? There’s a lot of infrastructure in between the two applications that can fail. If the integration is done at the wrong level of the application, the entire experience of on both sides of the application might be compromised. These are the types of scenarios which will become more and more common, and will benefit from having an open piece of middleware which can help build richer, more robust applications which tie into social platforms. Developers can build richer integration, customers get a richer experience using the application, and providers of the application can build integration into these services that lets them maintain control of their customers experience.

We wish Bob and the Ringside team the best of luck, and are eager to see how our customers can benefit from their technology!

I’m going to say this now – Javier deserves a ton of credit for incorporating community into Hyperic very early on. How many other CEO’s can say that one of their earliest hires was the Open Source community guy? He recognized from the beginning that a key to Hyperic’s success was our community. Speaking of which, look for some very interesting announcements coming down the pipe later this week :)

See the full interview with Javier here.

Mark Hinkle actually delved into the OSI by-laws and reported what he found.

Reuven Lerner opened the question of who should lead the Open Source community.

Both of these attack the problem from different angles, but the question remains: is the current setup the best we can manage? I would argue no.

1. an introductory act or step; leading action: to take the initiative in making friends.
2. readiness and ability in initiating action; enterprise: to lack initiative.

When I think of the OSI, I think of an organization that started 10 years ago to help define a market around Open Source. That market is now booming, so what is their current “Initiative”? The protection of the term “Open Source”? This is contradictory in that to protect it, they chose echo-chamber myopia as their methodology.

I deeply respect every single one of the individuals associated with the current OSI. That bears repeating – I have only the utmost respect for all of the individuals currently working within the OSI. I’ve known most of them for quite some time, and I’ve never had much reason to differ in opinion. Last year, when the brouhaha over badgeware came to a head, I was asked for a quote on what I thought Open Source meant, and I replied that the OSI had every right to protect the definition. I still feel that way – I just wish the OSI would change with the times.

Like it or not, the term “Open Source” occupies a niche as the iconic open technology standard-bearer, so when the OSI refuses to play with the non-compliant software vendors, they end up tainting said vendors’ works by not supporting it. Thus, instead of SugarCRM simply having a badgeware license, they get branded and labeled as “fake” Open Source and inauthentic. So strange to see such strident action from an organization that purports to be the “business-friendly” face of Free Software. So bizarre to act against a company, SugarCRM, who sits far, far away from the proprietary pole on the openness spectrum. And why did this occur? Why, because SugarCRM dared to use a license, before it migrated to GPL v3, that didn’t strictly comply with the OSD. And where did the OSD come from? Was it handed down directly to Moses with the other 10 commandments? Well, no, unless there’s something about Bruce Perens I wasn’t aware of.

Frankly, I don’t get it. The OSI is, at its heart, an ideological institution masquerading as a pragmatic one. Viewed in the context of the split from the Free Software movement in 1998, this actually makes some logical sense. After all, the purpose of creating Open Source in the first place was to put a business-friendly face on Free Software. However, many Open Source newcomers wouldn’t know the difference between Richard Stallman or Richard Simmons, and the story of the Open Source – Free Software split is as relevant to them as historical documents from the protestant reformation. The bottom line is this: the Free Software Foundation and Free Software movement are far more relevant to us in 2008 than the OSI.

Technology and the market have come a long way from 1998 to 2008. Open Source is prevalent and growing. The thought leaders from 1998 need to reinvent themselves, their definition of open source and their initiative – just like all the free software companies are reinventing technology and business faster than their predecessors.

So what to do now? As I chuckle over the recent scorched earth campaign by Bruce Perens in an attempt to regain former glory, I’ve been thinking about some recent articles and blog posts which mention an “identity crisis” for Open Source. Upon further thought, I think there is an identity crisis at work, but it’s not merely the result of coming of age or losing authenticity, as suggested elsewhere. Rather, it is the result of a narrowing of thought, a failure to comprehend, and a willingness to remain blind to new information. It is the same type of thought that will lay down the law of what “open” means and reward those – and only those – who adhere to the OSD, without considering the possibility that new developments demand thoughtful reexamination of the rules. After all, because of the success of the term “Open Source”, the OSI *must* have been absolutely correct in its formulation and governance, right?

It’s time for a new initiative, one that takes a more flexible approach to the idea of open software and open technology. I would suggest using the Creative Commons as a possible template with its recognition that you can adjust the licensing terms based on use case, eg. stipulating non-commercial use downstream and precluding downstream commerce. A commenter on one blog^* calls this “weakening their standards,” a willfully ignorant statement if I ever saw one.

No, I don’t call it “weakening their standards.” I call it “common sense.”

^* Edit: I originally (and mistakenly) attributed this to Simon Phipps. That was incorrect.

Sys-Con just announced that Hyperic HQ has been nominated for this summer’s Reader’s Choice Awards. The category we were nominated for is Best Virtualization Platforms Monitoring & Reporting. The nomination process is open until Sunday June 22nd, and the voting period will be until November 15. The final results will be at the upcoming 4th International Virtualization Conference & Expo.

It’s of course always nice to be recognized as a contentious player in any market segment, but in this case I find the exact category really interesting. You see, there are several Virtualization Management Categories:

1. Best Virtualization Configuration Management
2. Best Virtualization Management Tools
3. Best Virtualization Platforms Monitoring & Reporting
4. BestVM Lifecycle Management

I think this is perhaps the first time in the history of awards for the “Systems Management” category that actually breaks out these very different disciplines instead of lumping them in together. This can be for one of two reasons:

1. Virtualization Journal Readers are smarter than the average bear
2. Virtualization as a technology and an industry has successfully disrupted a tired, old industry plagued by incumbents & realizes that the application of these disciplines to the complex environments leveraging virtualization have no expectation of being satisfied by a monolithic approach.

The second, longer point is pretty interesting. It also puts in perspective how important all disciplines of management are for virtualized environments. 4 out of 18 categories are on management alone. This is something Hyperic has been talking about for some time (check out our whitepaper or our virtualization best practices webinar for more). But it is really good to see the industry is putting the focus on it too.

Javier will appear in the panel I’ll Show You My Forge If You Show Me Yours at 10:30am on Wednesday, March 26. The topic of discussion will be how you engage users and developers and sell them on the forge concept, and there will be a whole lot of expertise on hand to discuss it. Besides Javier, there’s also Rod Johnson, CEO of SpringSource, Dave Rosenberg, CEO of MuleSource, and Joe “Zonker” Brockmeier, OpenSUSE’s Community Director. Moderating is Eric Knorr, Editor-in-chief of InfoWorld. Now *that’s* a high-powered panel.

As for me, I’m apparently the only person crazy enough to go up against Mark Shuttleworth, John Roberts and Marten Mickos in the 10:30am timeslot on Tuesday. I am moderating the panel The Community Imperative: Building and Leveraging Community into IT at 10:30am on Tuesday, March 25. I’m blessed with a very intriguing panel featuring Russ Danner from the Christian Science Publishing Society, Gautam Guliani from Kaplan Test Prep & Admissions, Paolo Juvara from OpenBravo, and Gianugo Rabellino from Sourcesense. Look for some interesting back and forth, as these guys all have different perspectives on community building.

I’m really looking forward to this, as I’ve always been a fan of OSBC. If you’re going to OSBC and have already seen Mickos, Roberts and Shuttleworth speak multiple times, then this is the panel for you!

As Jay Lyman of The 451 Group notes, this result is hardly a surprise:

…the GPL is not some exotic, first-of-its kind license, document or legal doctrine. Actually, it is based largely on U.S. copyright law, particularly in the case of GPLv2, which is the BusyBox license. It amazes me that some people think the GPL will be refuted, defeated or ‘thrown out of court.’ That would mean ‘throwing out’ U.S. copyright law, and I don’t see that happening, ever.

It amazes me how much misunderstanding of the GPL still exists. No, the GPL does not cede your intellectual property to the public domain – as a matter of fact, it does a pretty good job of protecting it. In fact, the GPL is a pretty good compromise between granting rights to all parties and protecting IP. This case is another demonstration of that. Verizon knew they couldn’t win, so they settled. Makes sense to me.

There’s a reason we chose the GPL v2 when releasing Hyperic HQ under an Open Source license. As Eben Moglen himself has been known to say, It’s Good Not To Be Your Competitor’s Free Lunch.

The answer? We feel fine :)

On paper, it looks like a fine marriage, but of course the devil is in the integration details. What is of interest to us is that, for all of BMC’s posturing about Open Source about a year ago, there isn’t much to show for it, and the news today hasn’t changed that. Of course, it’s not an easy trick to change a company’s processes or mindset, but the silence is deafening.

“The Commission services use OSS where a clear benefit can be expected.

“The Commission will consider OSS solutions the same way as proprietary ones in IT procurements. Contracts will be awarded on a ‘value for money’ basis. Not only licence costs ,but also setup, maintenance, support and training costs must be considered.

“For all future IT developments and procurement procedures, the Commission shall promote the use of products that support open, well-documented standards. Interoperability is a critical issue for the Commission, and usage of well-established open standards is a key factor to achieve and endorse it.”

In light of the EU’s recent anti-trust action against Microsoft, one wonders if there’s any reading between the lines to do here.

But perhaps most intriguing, at least from a commercial Open Source vendor perspective, is this quote from Aslett:

What appears to be changing in terms of the Commission’s position on open source is a wider recognition that promoting the use of open source software can be a means of promoting the European software and services industries.

Which is all well and good, but as an American company, one hopes that we will be included in the mix, as well.

Jonathan was recently interviewed on Inc.com. The “pithy quote” in this interview is:

Having a server is like having a car. Having 300 servers is like having 300 cars. There’s always one that has a flat tire or needs an oil change. There is always something wrong with one of them. Being able to separate the server software and everything running on a server from the actual physical hardware, it really helps with the management side of things.

And Hyperic HQ is the web infrastructure management software they use to help separate the physical from the virtual and all the software running in between.

Love the metaphor, Jonathan!

1. Systems Management Taxonomy – Systems Management is a BIG term. It has many disciplines within it. We have application management, systems management, provisioning, network management, configuration management, and the list goes on. The reality is most people think it encompasses application, system and network management. Hyperic does a great job up in the application and systems level – especially for custom-built, heterogeneous stack web-based infrastructure. We do some level of Network management, but it is small potatoes compared to OpenNMS. A public announcement and partnership (not to mention blog posts) will make more people understand quickly that we’re very different – and they need us both!
2. Customers need Hyperic & OpenNMS – smart customers have realized how complimentary we are for a while. They’ve been implementing our products side by side the hard way – trying to figure out how to work with each company and who to engage to get the work done. And paying, either with dollars or time & effort spent, integrating our products. We’ve seen it enough times, and we’ve responded to the community at large – we’re committed to making this easier.
3. Why Hyperic & OpenNMS are seen so often together – At conferences, in our offices, at customers and maybe even over a few margaritas… we’ve gotten to know Tarus, Dave & the OpenNMS gang pretty well over the past year. Personally, I have found them to be incredible veterans of the industry – equally part sage and sarcastic, they keep working the hardest problems because they have such tremendous passion for it. I have had a chance to meet with several of their customers as well, and they share that enthusiasm for all things OpenNMS. And being such masters of their domain, I view their endorsement of Hyperic HQ as one of the biggest public compliments we can have. And to that, on behalf of Hyperic, I say thank you!

And if you’d like to see and hear more about what this partnership has in store for our communities, please join Dave & Charles next Tuesday at 11 am PST for our HyperCAST. Or read Dave’s whitepaper on the integration.

While I know nothing first hand of this particular case, and have no corroborating evidence to add to the proof that this did or did not happen, I generally find Tarus a wholly believable individual and not one to go spouting off on things he can’t back up. Ok, so he’s not exactly someone to hold his mouth in check, and we love him for that, but he wouldn’t make this accusation without strong evidence.

In his blog post, Tarus mentions the idea of contacting an attorney and deciding it’s not worth the hassle. Tarus: contact the SFLC and explain to them your situation. I bet they can give you the lowdown on your options.

This is the sort of thing that the community-at-large deserves to know, hence the reason I’m posting this. It deserves our attention and I hope it is resolved quickly. The GPL is not a license to rip people off, and you guys deserve your just credit and recognition for your hard work and donations.

And if they don’t give you a clear solution – let us know, I am sure we can band together and get any of the other open source monitoring and management vendors like Zenoss, Groundwork, and Nagios to post your credit publically. If Cittio won’t tell them, we will.

The HQ Agent operates independent of the HQ Server, which is necessary because the network link between the two cannot be guaranteed to be up 100% of the time. Due to this, the HQ Server must trust the HQ Agent when it receives metric data from it. The HQ Agent stamps each metric with a timestamp before it is sent off to the HQ Server. When the HQ Server receives the metrics, it looks at the timestamp on the metric to determine when it was collected and inserts the metric into the appropriate “slot”.

From this scenario, it should be apparent now how important it is the system time be in sync on both the platforms. If the system time on the platforms are not in sync, the HQ Server will think the metrics from the HQ Agent are from a different time period than they were actually collected. This can cause metric data to be put into the wrong “slot” or even lost. The main symptom is seemingly false alerts being fired.

The false alerts are almost always from alert conditions triggered by the Availability metric. This is because there is a process within the HQ Server which looks for missing Availability metrics and fills them in with “DOWN”. The reason this process exists is because if the platform running the HQ Agent goes down, there is no way for the HQ Agent to communicate this fact to the HQ Server. Therefore if a metric is missing for long enough, the HQ Server assumes the HQ Agent is down and marks it as such. This will obviously cause alerts that trigger off that Availability metric to fire.

If you ever find yourself in the situation where you are getting what appear to be false Availability alerts on your resources, the first thing to check is the system time on all your hosts. Hyperic highly recommends running NTP on all hosts in your environment. Virtualized environments, in particular can struggle with this time synchronization issue. There are more discussions and solutions on time synchronization in virtual environments than we can go into here. Whatever type of environment you run with, it is very important – and not just for Hyperic HQ – that the system time on all your hosts be in sync.

1. Just cool – not to be confused with useful for society at large (yes, the wave powered boat falls in that category).

2. Useful – either to improve my business or my customers business. Today I am in love with LoopFuse’s LeadFlow Programs, its helping our sales people know what leads (read: enterprise registrations) are serious and which ones are just kicking the tires.

3. Unusual – this bucket is the most amusing. But bizarre technology and activities are usually at the expense of others or technology itself. Microsoft has fallen in this bucket more days than most lately. Microsoft is usually everyone’s favorite whipping child, and yet …. miraculously the favorite ally when they choose to endorse you. Today I am amused with NimSoft’s press release as a lead pitch to Microsoft. They had an outtage yesterday, I know. I couldn’t read personal email for hours. Clearly this is a target for a snarky monitoring and management comment. Maybe even a phone call from a sales rep. However, it is an odd use of money to put out a PR for a glorified cold call sales pitch. It’s can’t be a strategy to actually sell. Any sales person worth their salt knows that one of the first things you did with cold calling was a background check. Microsoft, as any professional software provider does, lives and dies by eating their own dog food. Anything purely Microsoft, is run by Microsoft’s own monitoring and management software. They must know this though, and not care. The fact that it comes from marketing (unless does sales own PR there?), says they are desparate for news. They must be trying to pick a fight with them, but reality is that Microsoft won’t care. They get picked on by a lot bigger bullies so regularly this one won’t make the radar.

I guess it got enough attention for me to write… but somehow I don’t think that’s really going to help all that much. Hmmm. So they have that going for them, …which is nice.

So, there you have it. The results is that they made it onto my radar as the most unusual desperate act of marketing on the net today.

Oh, and Nimsoft – in case you actually thought Microsoft was interested, anything in the Windows Live division not actually run yet on .NET is happily being managed by Hyperic…. ;-)

The article seemed innocent enough until I ran into this quote from the CEO of Dallas-based startup Mile Meter:

During the first dot-com boom, he said, “It was a badge of strength to have as much as possible in house. “Now, unless that is your core business…it’s a liability.”

Hmmm… I think the dot-com era statement was a bit over the top. Realistically, there was no real alternative to in-house infrastructure management. The hosting business was itself developing almost simultaneously and it was a dumb idea to simply hand over management of your entire setup to a 3rd party. Of course, you also needed tons of expensive gear to run stuff too. Well, it turns out today you still need tons of gear (assuming you’re successful anyway) and while it’s cheaper, the gear itself is only part of the problem.

But it’s the second part of the statement that I have deep issues with (and I am perfectly open to the possibility of it being taken out of context). The idea that you have to be in the business of managing technology assets in order to justify not going the EC2/S3 route (and it must be stated that EC2, while tres-cool is still in beta) seems short sighted. In fact, it counters completely the idea that operational excellence delivered through a talented ops team and a good infrastructure is a competitive advantage. It certainly is for the big, established players in the business — including Amazon.com. Tim O’Reilly wrote about this a while back when discussing how operations is the new “Secret Sauce” of companies doing business over the web.

Of course, perhaps part of the value of the S3/EC2 offering is that you tap into Amazon’s ops excellence. From what I have seen from playing with it, all you get is virtualized storage and virtual machines which have some lifecycle tooling around them. In effect, hosted gear. No monitoring, no application management, no person to call to debug why your memcache setup isn’t performing, and perhaps most important… no SLA (not for EC2 yet, anyway).

EC2/S3 save you hardware and storage costs. Everything else regarding operations skill, capacity planning, scalability knowledge, and good setup techniques still hold true for any business which derives its revenue from the web.

The award was granted by Andi Mann, a vice president of research over at EMA. Hyperic has a direct relationship with EMA, so I got a chance to review Andi’s write-up, which was quite a treat. Andi’s perspective on the crowded and somewhat turbulent management space is always spot on. (I didn’t need this award to know that either!) We talk often and discuss big picture issues of the evolving needs of this market space we work in. Particularly, Andi loves talking about how virtualization and the increasing need for agility in the data center are changing the basic requirements for management. To that end I am very flattered he recognized us for leading with innovation (part of the write-up, but copyright prevents me from copy-pasting it here – so you are just going to have to believe me!).

In fact, we’re getting pretty excited to start innovating even more — 3.2 is about ready to be shipped, and the engineers here are starting to push the ideas for 4.0 (our summer release) really hard. I won’t get too far ahead of myself here – but rest assured, and Andi and his compatriot, Steve Brasen shall be part of that endeavor. Perhaps during that time I shall also have to ask Andi what other details the survey unveiled. I hear we scored the highest in customer satisfaction for Performance and Availability… but I don’t know what other categories there were!

First, the huge news about MySQL being acquired by Sun Microsystems… then the news about BEA being acquired by Oracle. The first deal is significant given the ‘imminent’ IPO that was expected by MySQL later this year. The second is significant if only because of the amount of bickering between BEA’s management team, Oracle, and their respective shareholders.

Perhaps the most revealing thing about both of these huge transactions is how they are presented on their websites! MySQL has the Sun deal front and center (as one would expect). BEA’s website has the same SOA-logic-gene-liquid thing they’ve been talking about for months, along with the now extremely overused (and BLUE!) ink floating on air image that’s part of their “Think Liquid” campaign. Where’s the blurb about the $8 BILLION acquisition of its soul by Oracle? Well, it’s in small print… down on the left hand corner of the site. Look closely! You’d think that for that amount of money, Oracle would have demanded a big Oracle Logo smack in the middle of the main page.

Makes you wonder how excited those guys are about joining the Oracle machine…

On a more positive note, Sun’s comments about MySQL continue to validate the importance of open source, as well as the unique role that MySQL plays in large scale web infrastructure.

From Jonathan’s Blog:

Both sets of customers confirmed what we’ve known for years – that MySQL is by far the most popular platform on which modern developers are creating network services. From Facebook, Google and Sina.com to banks and telecommunications companies, architects looking for performance, productivity and innovation have turned to MySQL.

This serves as a great example of why we here at Hyperic are focused on helping solve the ops challenges of large web applications, most of which use MySQL. Hearing Sun talk about MySQL in these terms rather than simply saying it’s the cheap/free database that you can use instead of Oracle is refreshing, and a great validation of a market that will continue to demand open, innovative solutions that arent simply rehashes of old technology delivered under a lower cost business model.

Mosso, as you may recall, is a Hyperic customer and was the subject of a recent case study.

Total innovation opportunity (TIO) as described by Aslett refers to the advantage inherent in building on the knowledge of others and not re-constructing every wheel, thus freeing developers to focus on new innovations. I should note that building on the knowledge of others does not necessarily mean code re-use, although it can. Solving the same problem using a different means, language, or algorithm but using a previous solution as a model will invariably take less time than implementing a “clean-room” solution. Regarding those core developers for a project, it would seem that the way to maximize innovation opportunities is to ensure regular interaction between them and outside community members. After all, one of the worst things that can happen to a developer group is to develop tunnel vision and work within an echo chamber. It’s better to let them see what outside users, customers and developers have to say about project direction. To illustrate this concept, I’ll make use of a comparison between older, proprietary models, and modern open source processes.

1. Older model – Engineering works with Product Management and small focus groups, usually a subset of customers, to fashion new product features. In this scenario, outside involvement is minimal, with the possible exception of a beta period that takes place well after a code freeze. The beta stage is usually designed to squash bugs, not introduce or kill product features
2. Newer Open Source model – Depending on how much of the engineering process is conducted out in the open, the product “touches” a much wider audience at each stage of development, from inception to implementation, to QA, and to ultimate release.

The key to (2) above is to maximize the number and diversity of outside touches, whereby outside community members can participate in the initial features discussion, outside engineers help validate the usefulness of the new code, and users from a variety of industries are able to put the proto-product and its features to the test. Sheer numbers of outside touches, ie. users, are indeed essential, but the diversity is actually more important – the user base should consist of developers, system architects, sysadmins, managers, and an assortment of end user types. From reading several articles on a given project’s community health, one thing is clear: just increasing downloads says nothing about the diversity of touches. If only end users download your software, you are not maximizing the potential value from your project’s community.

Each user type will touch the project in a different way and at a different stage: end users will be more likely to engage with a project that’s feature complete, sysadmins will have a higher tolerance for breakage, and ditto for developers. This will result in multiple types of data streaming in through the interfaces you’ve built, such as bug trackers, forums, wikis, etc., and often in parallel. If you accept the premise that community diversity is a key to your project’s success, then surely you must agree with the idea that maximizing your ability to handle the data throughput will determine how well you capitalize on your community’s value. Failing to close these parallel feedback loops will necessarily mean less than maximum value from your community. Thus, it’s important that each community layer have clearly defined methods of interacting with the project maintainers. Can developers easily find your code repository? Can a sysadmin locate your bug tracker? Will they understand how to find and edit, if necessary, your documentation?

Each community layer should be addressed depending on their strategic value to your project. If your priorities are structured such that your project’s community does not reach the ideal level of diversity, then a re-weighting of priorities might be in order.

Case in point. Slashdot reported this morning that the game ICO, an excellent (if moderately popular) game for the Playstation 2, contains GPL code in the form of libarc and zlib. In both cases, they are pieces of both of these libraries with modifications. Nonetheless, it suggests to me that the developer of the game was relying on the idea that it’s actually quite hard/impossible/illegal to dissasemble a game binary to hide their use of open source code. Perhaps there wasnt any malice here, and the developer(s) simply didnt know about the terms of use of this code. Regardless, it makes me wonder how many other places this is happening. Not just in games, but also in all kinds of other software.

I personally dont believe that all software should be open source. That’s a matter of choice for the developer of the software. However, the prospect of having people ripping OSS projects off by taking their code (as has happened even in our own segment of the industry) and packaging it up without abiding by the license is… well… crap. It’s something I recall Marc Fleury from JBoss being quite vocal about in the past… the idea that open source developers shouldnt get ripped off.

Too bad not many people cared about ICO. I bet if it was a higher profile game, this’d be bigger news.

The problem gets further compounded when one of these “Gatekeepers” is also a platform or stack vendor. See, it’s hard to resist the temptation of delivering the absolute best management for IBM products from a Tivoli solution while shortchanging non-IBM ones. Or, to lay this on one of the aspiring members of the big 4… how about getting support for SQL Server on Oracle’s Enterprise Manager. Hmmm… I’m gonna guess it sucks because Oracle wants you using their database. Besides, who uses OEM that isnt already an Oracle db customer?

Lucky for us, Hyperic has always aspired to be a completely independent management software company. We figure its important for us to deliver the best management solution for whatever technology you’ve chosen. Case in point, Mosso, a Hyperic enterprise customer was recently interviewed regarding their use of virtualization technology from VMWare. The key point from this interview was the fact that they’re open to switching to a different virtualization platform based on their requirements. Lucky for them, Hyperic will be delivering Xen support (as well as OracleVM) as soon as the Xen folks finalize their management API’s (c’mon guys, we’ve been waiting a long time) so their management solution will be in lockstep with their technology choices.

It was definitely a welcomed change. These machines were beefy enough for us to actually compile and run on the laptops, while running a host of other applications for daily use. I probably saved enough time in not having to search and hunt for a document, application, or email (hello, Quicksilver anyone?) to fix an extra bug or two per day just for our loyal users. In fact, the original support for Intel-based Macs for Hyperic HQ came from building on one of the Macbook Pros.

While we were extremely satisfied with our productivity improvement, we began to find areas of complaint as we became more familiar with OS X. So when Apple released Leopard, just two weeks ago, several of us at Hyperic gleefully upgraded our systems right away in anticipation of Apple removing those last remaining grumblings on an otherwise satisfied experience with their products.

Well, you know what they say about best laid plans. There were snags right off the bat. One of the machines suffered the Blue Screen of Death. Once Leopard booted up, the first question was, “What have they done with the Dock (gratuitous transparency and does anyone know why there is a crosswalk)?” Then the list of incompatible applications began to emerge: Parallels 2.5, GroupCal, Fireworks 8, Growl Mail notification, etc. We knew that Java 5 was questionable on Leopard, and Eclipse (Europa) was certainly proof of that, requiring several force kills a day. Okay, so these are annoyances and nothing one couldn’t live without. However, Mail was the big one. On Tiger, Mail was painfully slow and unreliable, especially with Exchange. However, Mail on Leopard is even worse than Tiger, if that is possible. I suppose the implementation of HTML stationary outweighed stability for the Apple developers. We began to get duplicated email messages, Mail process would hang, rules don’t always apply, and sometimes emails simply would not be received for several hours. This is obviously a huge impact to our productivity.

However, there are bright spots: Spaces, Time Machine, and … Hyperic HQ! Yup, HQ server and agent both worked without modification. It detected the O/S and version correctly.

In fact, SIGAR appears to function even better with Leopard than with Tiger. With Tiger, there were issues with auto-discovery and platform services due to long execution paths and permissions. In order for the agent process to communicate with other application processes, you had to set the security policy for all accounts, even if you had an administrator account.

sudo sysctl -w kern.tfp.policy=1

These issues appear to have been resolved with Leopard. I can easily create a process service to monitor any arbitrary application or process, such as Mail.

All of my applications get properly discovered and automatically imported into my inventory. So now, I can properly monitor my Mail application when it decides to quit on me.

Even though I can’t prevent Mail from dying, at least I can now be notified of it.

All in all, the transition to Leopard has been worthwhile. And for all those early adopters of Leopard out there, rest assured, Hyperic HQ will continue to keep an eye on your infrastructure for you.

We have procured the big spotted cat and will run HQ through its paces on said operating system. Since HQ does not require Java 6, we don’t foresee any major issues, but expect a writeup in the next couple of days on the topic of HQ and Leopard. If anything pops up, we’ll make sure to mention it there.

Today, my good friend Andi Mann of Enterprise Management Associates (EMA) confirms what I’ve known for some time, and why I bought into the Hyperic value proposition and the Open Source business model.  A press release from EMA describes a survey which found the following:

Data Center Automation (DCA) technologies can effectively avoid Data Center staffing  increase requirements by 100%, saving an average of $750,000 per year on resource costs alone. (this is a 50% reduction in costs on average)

In addition to substantial cost savings, 75 percent of respondents stated that DCA delivers measurable increases in profitability. The study also found that DCA prevents Mean Time to Repair (MTTR) increase for system problems by an average of 250%. (this is a 66% reduction in MTTR on average)

(Note: The original publication had some errors in the way this data was presented, at Andi’s request I am attempting to alter this with the correct information, but it’s no longer an exact quote.)

Additionally, Andi uncovered this startling fact:

…cost reduction, though listed in the top five, was not the primary driver for DCA deployment. Rather, EMA found that the primary drivers for DCA deployment were mostly strategic — such as reducing complexity, improving response time and reducing firefighting. When asked what stopped them from deploying (or deploying more) DCA solutions, respondents were concerned primarily about purchase costs. However, EMA found that the big issues were really internal and political — like lack of available people, processes or time.

This further supports the finding that:

 When polled on the most important deciding factors for choosing a DCA solution, respondents noted ease of use and ease of implementation as the two most important factors, outranking requirements such as purchase cost and feature set by a small but significant margin.

I translate this to people, especially the ops teams, want to make their jobs and lives easier, more predictable. They want something that is implemented easily, and allows them to make better decisions earlier. They will spend money to save time (and eventually money), but they need to be very clear on what that value proposition is early.

This is where Open Source comes in. By removing those artificial barriers to experiment, and prove to themselves the value proposition, while at the same time providing it at a lower cost - those same ops teams now have access to these types of solutions in an acquisition model that better suits them, Thus proving Andi’s final conclusion:

“Open Source and appliance-based solutions are two growing trends that will continue to shake up the DCA market,” stated Mann. “They will definitely become even more important over time as data centers look to deploy DCA solutions in less expensive form factors.”

I couldn’t have said it better, Andi. Thank you for putting hard data behind what we at Hyperic have been working for all along.

(For more information on this report, visit http://www.emausa.com/ema_collateral/EMA_DCA-2007-RR-OV.pdf. For pricing information or to purchase this report, please contact Kevin Hecht at khecht@enterprisemanagement.com or (303) 543-9500 x124.)