Skip to content

3 Ways to Test Effectiveness of Mobile Content

Mobile content is hot. There’s no denying it. Over the next 12 months, four out of five marketers intend to increase their mobile spending, according to the new “Mobile Marketing: Plans Trends and Measurability” study by King Fish Media. Just like other content marketing, mobile content should be findable, readable, understandable, actionable and shareable (See the Creating Valuable Content Checklist). So what do you need to know to make sure your mobile content is valuable and effective?

Testing content in any format gives you a better opportunity to get it right. Testing mobile content should answer the following questions:

  • Is the mobile content hitting its mark?
  • Do your customers and potential customers understand what you are trying to tell them?
  • Is the communications vehicle (e.g. blog, Facebook, newsletter, print magazine) the right one for the mobile environment?
  • Do your users have the technological savvy to navigate your mobile format?

Why is testing mobile content different from testing traditional content?

There are three main reasons why testing mobile content (both websites and apps) is different from traditional content:

  1. Needs change depending on context—Users have fixed attention because they probably need to find a piece of information or perform a task quickly, while still managing many sensory inputs or distractions.
  2. Devices change on a yearly basis—Users are constantly learning new mobile applications and navigation choices, creating “navigation aggravation.”
  3. Personalization complicates delivery—The small screen necessitates knowing exactly what content to deliver to which customer to avoid both navigation aggravation and context issues.

Let’s break down each of these priorities to understand how to better test mobile content.

Needs change depending on context

When you work from your desktop, tablet, laptop or even from your smartphone in a fixed environment, like your office, family room couch or kitchen table, your attention and inputs are stable. This means that your environment remains mostly unchanged and you are able to focus on the task at hand. However, because your environment is relatively static, you may be more open to exploring a website or application because your need preferences are less focused than in a mobile environment. Many mobile users’ needs tend to be more focused on completing a task or obtaining a piece of information.

Recommendation: Test content while your targeted customer base is in a mobile, fluid environment. Give users tasks to perform what most mimic what you are testing.

For example, if you represent an electronics website, you need to know your mobile content persuades users about your competitive pricing, even while they are physically in a big box store and comparing prices. Therefore, you will want to test your users while they are in the same type of store, so you can accurately measure if your content helps them in real-time situations. You will be able to ascertain with greater certainty if they can find your content and rank its value even though inputs are constantly shifting.

Testing this way would include the following tasks:

  1. Create a set of short simple tasks (2-5) you want your users to complete.
  2. Pick an environment that most closely mimics where and how you think your users are accessing your content.
  3. Give the users a prescribed amount of time to perform the tasks.
  4. Stand next to them and ask them what they are thinking about. Prod them to talk out loud about how the content is helping or deterring them from finishing the assigned task.
  5. Watch them as they access the content using the mobile device. (Make sure the users are using their own mobile device or something that most closely mimics theirs. If they need to learn an iPhone during the test, the test won’t get at what you really want to know, which is how they are interacting with the content).
  6. You may want to consider videotaping them as they perform the tasks, but have someone else do it. You want to act as the moderator so you can ask them pertinent questions about how the content is satisfying—or not satisfying—their needs.

Devices change on a yearly basis

When is the last time you had to relearn how to “use” your laptop? Or, even a desktop computer? Yes, applications change a little bit with each new version, but the basics are the same.  And yet, with the introduction of new mobile devices and operating system updates every week, the learning curve can be steep. You cannot know that your content is accessible and valuable unless you test it using the same devices your customers  use.

Recommendation: Use your analytics to determine the most popular devices your customers use to access your content, and then test your content on those devices.

Personalization complicates delivery

With such a small amount of space available compared to your website’s home page, more marketers are personalizing content for mobile devices. Think of the home page of many mobile sites you go to.  They are either far too complicated—a mirror of the traditional home page—or far too simple.

There are a number of ways to personalize content. Customers can set up a mobile device the first time they access a website to deliver the content most valuable to them. Or, even better, technology by companies like Maxymiser that run alongside a mobile site tracks user data to deliver the right content.

Think of people who go to their bank’s online site or app using a mobile device. If it’s 2:00 a.m., they probably are looking for the location of an ATM. If it’s 3 p.m., they may want to know what time the bank closes. By aggregating real-time data, you can understand context and return  what is possibly the most valuable content, thereby saving the customer a lot of navigation aggravation

Recommendation: Create smart, accurate user scenarios so you know the hows and whens of your customers’ interactions with your content. This will allow you to make sure you are delivering the right types of content at the right time.

So tell me about your adventures with mobile content testing: Do you have any more advice to give?