Price Per Ounce: Normalizing the Download Counter

Two things happened today that got me thinking. One, I read Alex Fletcher’s blog post on Open Source and the Numbers Game and two, I went grocery shopping (you’ll see how they’re related in a minute). As I was wondering the aisles trying to remember what I was there to buy, it occurred to me that Safeway already has figured out a way to solve the ever present, always annoying, download number game. Yes, folks, a game. This means that while its somewhat indicative, and mildly amusing, tracking download counts is in no way a real diagnostic of a project’s success. When there’s money involved (as there always is in Commercial Open Source companies), there’s a built-in incentive to use raw download metrics as evidence that one project is ‘taking off’.

For those of you who may not be as entertained, or irritated, by the lunatic process of watching the download numbers for projects as I am, let me enlighten you on how irrelevant these numbers really are by giving you some of the key plays of the game.

1. Too famous not to acknowledge, there’s the legend of the download server, where someone sets up a machine that continuously downloads their own project files, drastically inflating the numbers. This is a popular rumor, and maybe an urban legend, but it is so easy to conceive that it has to add a certain speculation of doubt to the entire process.
2. Next is the file number strategy. How many files do you need to download to get your software going? Project A may have 2 files for a complete installation, while project B has 4 – perhaps they force their users to download check files to even get that many. Project B could have double the downloads and appear the market leader, but really, they are playing at about even.
3. Then, you need to think about how often the project publishes new code. 100 people downloading a new build once a week for a month would look about the same or better then 400 people downloading a single build a month. But one project has a 100 users and the other 400.

I believe that community is the better measure, and should be the focus of any project’s success. How many people are using and working on the project is a real measure of the projects overall relevance and traction in the open market. Measuring how many questions are community asked and answered. Measuring contributions for code fixes, enhancements and, plugins – by both number and complexity. Documentation contributions – including best practices on how to implement, use or extend the project also have variations on complexity. How many people are using the product actively, and how broadly used and useful is it. These are the real questions and measure of success. Unfortunately, while somewhat transparent, these measures are not easily aggregated. Solid numbers are an objective measure that people like and unless the industry figures out how to compare community data better, download numbers are always going to be a part of the mix.

So, back to Safeway. When trying to figure out whats the best price from everything from soup to shampoo, pricing and packaging are typically meant to confuse the user. Lets take cold medicine as an example, a name brand has a bottle shelved right next to the generic store brand. The prices are off by pennies. But take a closer look, for the same relative price, the store brand offers 50 more pills. The bottles look the same, but the consumer has to look really hard at the bottles to figure out any differences. Its usually found with the quantity and dosage information – 1 pill of one equals 2 pills of the other, and one bottle comes with 50 more pills. In order to demystify this buying process for the shopper, Safeway put that clever little price per ounce on the pricing label below the products. This is the instant normalizer that lets you know when you’re getting ripped off or fooled.

This is what the open source community needs. Price per ounce for downloads. A normalized download counter. The first step will be to defraud the numbers as they stand. The second, and more important one, is to start measuring the heartbeat of the community. We’re eager to work with folks like Sourceforge.net, other OSS project hosting providers and projects themselves to help make this process as transparent as our businesses are.

If you like this post then please consider subscribing to our RSS feed. You can also subscribe by email and have new articles sent directly to your inbox.