Measuring Content Effectiveness

First off apologies to anyone that checks the blog regularly, I know I’ve been behind on my posts but it’s been tough to keep up recently with work and life on the go!

Recently I’ve had to do a few sessions at work for our editorial teams around measures for content effectiveness and I thought I’d share the basic idea from those sessions.

Why care so much about content?

Content is King CrownWell if you’re in the media business you’re already well aware that content is king!

I like to think about it by comparing media or content driven sites to an e-commerce site. On an e-commerce site, your products or services are the most fundamental pieces of content on your site, 9 times out of 10 (warning: completely fictitious statistic) that’s the reason why people even showed up in the first place.

Similarly, on media or content-driven sites, the articles, videos or audio streams provided are the most fundamental and most important pieces of information on a website. Without interesting, engaging content what incentive do you provide your visitors to even come to your site in the first place?

Why then do many of us in the media or content-driven industry measure content effectiveness strictly by considering only page views?

The need for better metrics

Sample Dashboard - Need for Better Metrics!Don’t get me wrong page views for any given media site are crucial, as a matter of fact due to the revenue models (see my posts on net present value as well as important metrics for media sites overall) that come with traditional media sites, page views is one of the most important metrics.

But when you really dissect exactly what a page view is as a metric, it really isn’t sexy at all. A page view simply tells us that a page did in fact load, that’s it. I have no idea if you liked what you saw, read it, told a dozen friends about it, came to it by accident, visited 15 more pages afterwards, found it fascinating enough to discuss it…you can see where I’m going with this.

I like to think of page views at a wonderful proxy metric, if everything goes right then yes we absolutely would expect to see eventual increases in overall page views. What page views fail to provide us with however is what I’ll refer to as measures of content potential. Page views give us a bit of a measure for what is working, but fail to let us know what will work or even why something worked.

Measures of Content Engagement & Potential

USA Today Article ExampleSo what are the deeper measures we require to really understand content potential? To start, how about we look at a current leader in online media, USA Today.

As an example let’s take a look at one specific article.

Why did I pick USA Today? Because of the pretty enhanced number of story tools that they have. Let’s break it down.

Commenting

USA Today Commenting Example 1USA Today Commenting Example 2USA Today encourages its users to comment on their articles. Why is commenting important to measuring content effectiveness? Ever heard the phrase, “there’s no such thing as bad press?” That pretty well sums up my feelings on commenting.

Discussions surrounding content, whether they’re positive or negative indicates a keen interest of a group of people visiting your website. Understanding both the number of comments on any specific piece of content (or groups of those pieces) receives as well as the average number of comments per piece of content is critical to understanding overall site performance.

An experiment I’d love to be able to run is one where you toy with a content comment velocity threshold. When a given piece of content receives a number of comments (x) over a certain amount of time (y) a content management system (working in tandem with a web analytics tool) automatically identifies this as “hot” content. What action is taken? Perhaps the system automatically starts to place the piece of content in an “up and coming” content widget that’s placed on pages. Perhaps the story is automatically featured on a homepage and content vertical page. Perhaps the system integrates some user segmentation to align this story with other users that have commented on similar stories. The possibilities are endless (well almost)!

Finally, at a more fundamental level, commenting leverages the interactive power of the Internet. Most other media distribution channels are one-way. I send you my message and you take it whether you like it or not. Obviously in the online world we can actually have a conversation with our audience.

Taking the time to actually analyze the content of comments on popular or “hot” articles themselves provides invaluable insights. Many times editorial staff is able to analyze the content of these comments to gain ideas for future editorial segments on the site. While this shouldn’t be taken to an extreme and brands should always retain some editorial control (just because people want more Britney Spears doesn’t mean the New York Times should be doing homepage featurettes everyday), there’s nothing wrong with having a tool that essentially becomes an idea generator for future stories.

As a quick example, what if on a news article relating to the upcoming US presidential election, a few comments stated they were curious about an aspect of Barrack Obama’s childhood and remembered reading something on the subject but had since forgotten. This is a perfect opportunity for the editorial staff to respond in real time to the visitor’s request by generating content or linking to existing content that answers the query as well as posting a response on that article for others to see.

Rating Star Rating

E-commerce sites were (I believe) the first sites to pick up on the cue that allowing their audience tell you what they like and don’t like was a good thing. I feel that many online businesses underestimate the power of rating and its importance, particularly for media sites (though many are picking up the trend now).

Having a rating widget on every page of your website is essentially an informal user survey on every piece of content you offer. It’s a friendly way of asking, “Hi there! What did you think about what you just read? Could we do better?” Rating widgets on media sites similarly to e-commerce websites will become what arguably is the most important metric as a general rating of editorial quality.

Keeping in line with the idea of this post, for a media site, our product is information don’t we want to have some idea of how good or bad our product is?

Today’s Parent Rating ExampleOne more thought on the rating front, rating doesn’t always have to be as boring as a little rating box that allows users to rate a piece of content out of 5. What about tailoring the rating engines on web sites to match the actual content there?

If I’m writing a politically charged blog / website why don’t I have separate ratings for the Most Liberal/Democratic and Most Conservative/Republican articles? If I have a comedy site why not the Most Slapstick, Most Ironic or Most Anecdotal? There are a lot of possibilities here but you’ll need to think through two things first:

  1. How do your users want their content organized?
  2. How does your business want to rate their editorial content?

Sharing

Social BookmarksSharing a piece of content whether it be via a social bookmarking site (like Digg, del.icio.us or Newsvine) or via an e-mail send to a friend feature is also an important measure for a media site.

These features are indications both of engagement with content as well as potential audience growth. If an article I just read had me so interested/disgusted/outraged/happy that I decided to send it to a few friends of mine, those are a few more people that could potentially become loyal readers afterward.

Helpful hint: When adding an e-mail / send to a friend feature on your website make sure to tag the links in those e-mails that go back to your site so you can evaluate this source of traffic.

Saving

A new feature on some websites being able to save your content to a sort of favourites list is on it’s surface a great idea. Users save their favourite articles and can even share these articles with friends encouraging greater engagement and loyalty from your audience.

Hang on a second though this sounds a lot like social bookmarking!

(As a quick side not here I absolutely love the guys at Common Craft, very simple idea for a web site that has always helped me when I’ve needed to explain tricky things to executives who may not be so web savvy).

My point with pointing out the analogy is along the lines of “stick to what you’re good at”. If entire sites are being created that are devoted to this concept, chances are they are 1) probably better at it than you are and 2) have way more resources to devote to improving upon them. Unless you have a pretty serious business reason to invest in this functionality, I’d say don’t worry about it.

Printing

PrinterIf you’re anything like me then for any article you’ve read online that you found genuinely fascinating you can’t be bothered to sit in front of your computer and scroll page-by-page. Instead, you need to print it off, read it a bit to get used to it. Perhaps you post it on your cubicle/home office cork board and refer to it from time to time.

Granted, most of that engagement with that content is now taking place offline and we’ll never know that it occurred, but what we do know is you found that content interesting enough to print it so you could take it with you and that’s important.

Warning: No web analytics tool will actually be able to tell you if someone successfully printed an article, the best you can get is that a tool can tell you how many times people opened up their print dialog box (Ctrl+P or Control+P for Mac users).

The “New” Top Content Report

So what does all of the above mean? The old “Top Content” report that we have is dead! We need something better! My personal suggestion?

The New Top Content Report

Check here

Google Analytics Report ViewsI’m a big fan of the views that Google Analytics supports the number of times they’ve helped me in doing some quick analysis are countless. In an ideal world, I’d take all four of these views (not just the two shown in my report) and simply add the additional metrics I’ve discussed above.

I’d love to know what people think about this so please feel free to provide some feedback. :)

Related Posts

Posted in content based, strategy | Leave a comment

Tips on Visually Measuring Your Traffic Sources

Measuring what sources are driving the highest quantity and quality visits to your site is one of the fundamentally most important (and easiest!) things a web analyst can do.

But from many that I’ve spoken to, their traffic sources report usually doesn’t look much different than this:

Example of a Traffic Sources Report

Then if you’re really lucky you get a slick visual summary of that data in the form of a pie chart!

Pie Charts - WOW! :)

I’ve got no problem with either of the two ways of displaying data – they both serve to summarize data very well. I’ve only got one question…

Where are my insights!?

The observations I can make from the information above:

  • Looks like a lot of our traffic comes from some other site
  • Some people come directly
  • Not many from search engines

But the questions I’m left with:

  • What do they do from these sources?
  • What about paid versus organic search?
  • What about RSS feeds / e-mail / affiliates? Can’t I see these separately?
  • And a whole lot more!

The point is, the presentation of the information above probably creates more questions than answers. But there is a solution!

Here are some useful tips I’ve found for really getting insight out of your traffic sources reporting:

1. Decide on your sources

Your organization is likely complex with a lot of different things going on. You may have e-newsletters, e-blasts, Facebook applications, RSS feeds and who knows what else! All of these efforts may be bringing significant traffic to your website but you may not know.
Talk to people in your organization (marketing is a good start) and try to get a list of all the on-going efforts right now that potentially drive traffic to your site. Try to group all of these efforts into 7-8 main groups. Anything more than that begins to get a bit too detailed and prohibits you from seeing the forest from the trees.
For example you may have:

  1. Direct Traffic
  2. E-mail (Newsletters, eBlasts)
  3. RSS / Atom Feeds
  4. Paid Referrals (aka Content Alliances)
  5. Non-Paid Referrals (all referrals that you didn’t pay for)
  6. Paid Search
  7. Organic Search

Make sure that for each group you define the group well so that everyone is on the same page about what’s being included where.

2. Set up tracking on your sources

Now that you know what you want to measure you’ll likely need to put a bit of effort into getting things tagged.
For most analytics tools, tracking these sources independently is as simple as adding a little tag to links in each of these sources so that the tool can identify it.

For example, in my RSS feed, all links have Google Analytics tracking codes

Google Analytics tracking in RSS Feeds

The text that’s underlined above tells Google Analytics, “Hey! This visitor came from the RSS feed so group them in there!”

You’ll have to work with your analytics vendor to know what parameters you’ll have to use, but the method should be similar.

3. Report and find those insights!

Now that you’ve got all that awesome data pulled into your favourite web analytics tool you need a concise way to report on it so that those insights just scream out at anyone looking.

Here’s the most important nugget (in my humble opinion) of this entire article. This is all based on experience, but after a few times of people throwing my reports in the trash, I found what worked for them!

Traffic Sources / Marketing Dashboard

Let’s take a look at the structure of the report first. As you can see from the two biggest headings the only two things I’m really trying to keep track of with this dashboard are:

  1. Traffic Quantity and
  2. Traffic Quality

Basically all I want to know is:

  1. Am I getting traffic and if so, what source do I have to thank for it?
  2. Of my ways to improve traffic, which are the most profitable for me?

With that being the “big picture” let’s drill into a few of these graphs.

Am I getting traffic? And if so, who do I thank?

Traffic Source Volume by Source

Traffic Contribution by Source

I love these staked bar graphs, I think of them as a time-series pie graph. I can quickly tell if traffic has been increasing or decreasing from month to month and within seconds I know exactly who I have to thank.

While this is great, I still don’t have any idea if the quality of traffic from these sources was any good. Did they convert? Did they bounce? Did they puke?

Which of my sources are providing the highest quality traffic?

Traffic Quality by Source

My absolute all time most favourite (that’s right favourite – I’m Canadian eh?) graph. What I’m looking at here is

  • Pageviews per Visit along the horizontal axis
  • Bounce rate along the vertical axis and
  • Visits determine the overall bubble size

Now I’ve chosen Pageviews per Visit and Bounce Rate here because those are the three metrics that currently provide the best indicator of traffic quality. If you were an e-commerce or lead generation site you may graph Bounce Rate along the vertical and Conversion Rate along the horizontal.

Generally I like to ensure the bubble size is still related to a traffic quantity metric like visits or unique visitors. This is simply to ensure you don’t spend a lot of time optimizing a high quality source that can only bring in 1% of your overall site traffic.

Traffic Profitability by Source

This second graph may require a bit of explanation. Essentially the optimal goal of analyzing your marketing sources (or traffic sources) is to understand return on investment.

So to understand ROI, we need to know our cost per visit and revenue per visit by source. You can literally devote a whole blog post to understanding how to calculate cost per visit and revenue per visit for different websites.

For my blog, my value equations are pretty simple:

Revenue per Visit = (Pageviews per Visit x Ads per Pageview) / 1000 x CPM

Cost per Visit = $0! (The costs in the chart above are added just to illustrate what it could look like – actually the revenue numbers are fake too :) )

(In case you’re wondering CPM is available for AdSense customers but is listed as eCPM as Google doesn’t actually provide publishers with any money for the number of impressions they provide.)

When I spoke at eMetrics a lot of people asked me how they can get through to their boss or senior executives about web analytics. If you really want to connect with them, stop talking about web analytics, talk about what’s on their mind – revenues and costs!

4. Go ask for that raise!

Now that you’ve given your marketing team and organization the tools they need to properly understand where to allocate their budgets for maximum profitability, head up to your favourite HIPPO and ask for a raise!

I’d love to hear from you! What are your thoughts on this dashboard? What’s missing? What would you add?

Posted in suggestions | 4 Comments

New Blog KPI Dashboard!

For those of you that have visited my old “Blog Metrics” page, you may have noticed that it’s a pretty sad excuse for a dashboard.

Old Blog Dashboard

No use of KPIs or anything that really lets you know how the blog is performing.

Well I’m slowly but surely aiming to change all that.

Please take a look at this blog’s new and improved dashboard!

New and improved blog performance dashboard!

The KPIs that I’ve selected are largely based on a previous post relating to the most important metrics for content-based sites. I’ve also made the dashboard downloadable in Excel if you’d like to compare it to your own results (not to mention adding a few slick sparklines for basic metrics :) ).

My reasons for producing this dashboard are really two fold:

  1. Help other fellow bloggers with relative metrics in order for them to know if their performance is “good” or “not so good” :)
  2. Provide a demonstration of the power of storing your KPI information in a relational databaseOn that second note, all the data you see in the dashboard is stored within a database on my web server. The benefit of this of course is that I’m able to pull from this datasource anywhere I want! The web-based dashboard you see is only one incarnation, I also am pulling this information into an Excel spreadsheet I plan to publish soon.

Thoughts and opinions are always welcome!

Posted in kpis, news | 4 Comments

eMetrics Summit – Web Analytics on Steroids!

eMetrics Summit - See me speak!If you’re any kind of veteran (and by that I mean 2-3 months :) ) in the web analytics field you’re probably already well aware of eMetrics, but if you’re not this is for you.

Here’s a short shpeel on what the conference is and why I personally think you should be signing up.

What is the eMetrics Summit?

The eMetrics summit is the location to learn more about web analytics and really – how to use online data to improve your website for your visitors or customers. Learn about the terminology, the tools, the best practices and the pitfalls to avoid from industry gurus as well as people just like you!

Who should attend the eMetrics Summit?

Basically anyone who interacts with and is responsible for a portion of the online channel. Whether you manage content, usability, paid search marketing, e-mail marketing or your entire website – this conference is for you.

Why should I attend the eMetrics Summit?

You mean besides the fact that yours truly will be speaking? :)

To be honest, the only summit I’ve attended as of yet was last year’s in Washington D.C. but I’ll go on record saying if you have the budget for only one conference all year, make it eMetrics.

Knowledge

The knowledge you end up with at the end of the day can honestly almost make you dizzy (in a good way!).

I came back from Washington with presentations to share with many people in the organization. In fact, I recall that within Rogers, eMetrics filled me with info to share with: executives, paid search marketers, content creation and management folk, our business analysts and even our finance guys!

What I’m trying to say is that the conference isn’t just about web analytics, it’s about online optimization.

Contacts

The people you’ll meet and connections you come away with are incredible after an eMetrics summit. Here are a few people I’m happy to say I’ve now had the pleasure to correspond with (some on a fairly often basis!):

  • Eric T. Peterson – CEO – Web Analytics Demystified
  • Avinash Kaushik – Author / Blogger /Analytics Evangelist, Google
  • Brett Crosby – Senior Manager Google Analytics, Google
  • Rachel Scotto – Executive Director, Sony Pictures Entertainment
  • June Li – Managing Director, ClickInsight
  • Eric J. Hansen – President, SiteSpect
  • Braden Hoeppner – Director Web Sales, Coastal Contacts Inc.
  • and more!

Just think about it – the people at eMetrics are literally the brightest in the industry and you’ll meet all of them, share your stories and at the end of the day, have a new buddy to call on!

Where is the eMetrics Summit?

The eMetrics summit is actually in a number of cities: Toronto, London, San Francisco, Washington D.C., Stockholm and Munchen! Check out the website for more info. Right now, I’ll be most concerned with Toronto :) .

Toronto Marriot Downtown Eaton Centre Hotel

525 Bay Street

Toronto, ON, Canada

M5G 2L2

When is the eMetrics Summit?

The Toronto summit is March 31 – April 2, 2008 but the early bird registration date (save $200) is
March 9!

Booked that ticket yet?

What are you waiting for!? Book your ticket and flight if you need to and make sure you’re there! :)

If you have any questions at all please feel free to contact me at mike.sukmanowsky@oddinteractive.com.

Posted in news | 6 Comments

Web Analytics and Net Present Value – Part 1

Companies with projects that deal with the internet in someway always surprise me.

Say a company like Ford was evaluating the purchase of a major piece of machinery that was expected to increase plant efficiencies. These efficiencies resulted in the plant being able to produce x more number of cars per day.

There’s no way executives at Ford would approve the purchase without their finance department performing some analysis.

Specifically, at a bare minimum I’m sure executives would want to know the project’s net present value (NPV), payback period (PP) and return on investment (ROI).

None of the above calculations are difficult (in fact, a few seconds in Excel produces the results!), the work all lays in the underlying assumptions that feed the simple formulas.

For most web firms however ROI is often the only metric ever discussed (and often only at a very high level). It isn’t that NPV or PP is impossible to calculate for web-based projects, but the analysis can be slightly more difficult. Often the difficulty is only rooted in the fact that many people don’t understand the relationships between revenues and costs behind online businesses.

I’ve taken a stab at writing a business case that demonstrates how to use web analytics to evaluate a project’s net present value. Please check out the following two files and let me know what you think!

Download the business case and the background! Download the NPV analysis and the answer!
Posted in strategy | Leave a comment