Beginner's Guide to Web Analytics and Measurement

Beginner’s Guide to Web Analytics and Measurement

Author: Andrew Maier

Because each website appeals to its audience differently, the prudent user experience designer takes a measured approach when communicating, especially when they do so on behalf of their client. No matter what the vision and no matter how it’s executed, a design can always communicate more effectively.

Online and off, we gauge the effectiveness of design—of communication—by its affect; in other words: what action(s) do people take after they give us their attention? Properly utilized, Web Analytics and Measurement helps us answer this fundamental question.

In the early days of the Internet, webmasters used hits (remember counters?) to gauge their website’s success. The logic went like this: if people liked what was written on a site, they would request that content more often. This made sense because, at the time, the web was largely state-driven. People navigated the Web one page at a time.

Today, however, that’s far from true.

As a consequence, analysts have turned their attention towards the constant in the Internet + User equation: the User. Instead of simply tracking hits, analysts track user behavior. Emphasis has appropriately gone from answering the question “what is the web server doing?” to “what is the user doing?” (Joshua Porter details this trend in his post User Engagement Metrics.)

Both the metrics and methods required to illustrate what our users do are nuanced. In this article we’ll take a closer look at how these methods inform our design process.
 

What is Web Analytics?

It’s nearly impossible to understand why someone does something online. How, then, can we possibly hope to evaluate trends across (potentially) thousands of viewers? As it turns out, it’s not so difficult.

Even with only a modicum of traffic, web servers generate a tremendous amount of data. Web Analytics tools were created to collate and refine this data. Typically manifested as web-based applications, Web Analytics tools (such as the popular Google Analytics) take data and, through a variety of computations, generate insightful charts and reports.

Unlike research methods—which are typically qualitative in nature—web analytics methods are decidedly quantitative. So instead of dealing with warm, fuzzy descriptions of problems (practically written by our users) web analysts look at reports based on cold, hard data about them. Where’s the love?

Louis Rosenfeld explains the conundrum:
 

…analytics tell us what is happening, not why. After detecting data patterns, we might guess what’s going on with reasonable accuracy. But we can’t know for sure unless we conduct qualitative analysis, such as actual user testing, where we can ask people why they do what they do.
 

Louis Rosenfeld

As a consequence, Avinash Kaushik motions to combine the two heretofore disparate disciplines with his definition of “Web Analytics 2.0:”
 

“[Web Analytics 2.0 is] the analysis of qualitative and quantitative data from your website and the competition, to drive a continual improvement of the online experience that your customers, and potential customers have, which translates into your desired outcomes (online and offline).”
 

Avinash Kaushik, Web Analytics 2.0

Marko Hurst calls this behavioral metrics. He explores this amalgamation in his recent presentation, User Experience by the Numbers.
Back to topWhat is Web Measurement?
 

What is Web Measurement?

If Analytics provides the tools, Web Measurement is the process by which those tools are utilized. Thus, Web Measurement helps us make and act on inferences from the aforementioned tools. Marko Hurst makes an uncanny parallel in his presentation Analytics & Gambling—How Similar They Really Are.

Because analysis is rooted in mathematics, it’s typically accomplished following a logical, deductive process. We start by defining outcomes, then we proceed to measure, monitor, and act on our analysis. Let’s briefly cover each:

Outcomes

We begin measurement by establishing a desired outcome, or goal. In other words, what do we want to accomplish?

For example, say you want to improve your company’s next email blast. Their mailing list contains 150,000 members; what action do they want recipients to take? Fortunately for us, this email contains a coupon—perfect.

Outcomes are best when they are both specific and quantifiable. Let’s set a hypothetical goal of six percent. That is, six percent of our 150,000 members should convert (9,000). Next, let’s set a deadline: if we send this email out on Friday, let’s give users 1 week to determine whether or not they found our offer valuable to them.

Measure

We begin measurement by determining what’s quintessential to our desired outcome. In other words, a metric is anything we can track, but we’re looking for what we should track. For email, we’ll want to take a look at delivery rate, open rate, bounce/invalid rate, click-through rate, etc. Based on our desired outcome, though, we’ll put supreme emphasis on conversions. Thus, conversion (using our coupon) becomes our Key Performance Indicator (KPI).

Monitor

Monitoring keeps you aware of and lets you know three things at all times:

1. Where you are at—1,000 successes after 3 days.
2. Where do you want to be?—9,000 in 7 days. At the current rate you’d need 27 days to achieve your goal.
3. How you will get there?—Your current path and actions are not going to cut it, so you’ll need to change something to still have a chance to achieve the desired outcome of 9,000 conversions in the next four days. This is your output or delivery item for this phase. It tells what item or items can and should be acted upon to still achieve your goal.

Remember—the greatest gift monitoring can give you is time to make adjustments and manage expectations before it’s too late, in this case after the seven days.

Act

Action helps us refine measurement endeavors based on the data we’ve received thus far. In the case of our hypothetical email blast, we could:

• inform stakeholders of this campaign’s outlook.
• send a blast to a new set of subscribers (in case we’re not approaching our goal).
• send a blast to the same set of subscribers (minus those that have already converted) as a reminder.
• change the title of the email or the call-to-action in the email—the equivalent to an A/B Test.

Tools of the Trade

Many different tools enable Web analysts to do their jobs. Here’s a selection of some of the most popular: