Joomla.org Usability Testing Report
They’re all doing Usability Testing — and no wonder.
Usability Testing is probably the most underrated and overlooked method for improving the user experience of a site. It’s also the most powerful and the most cost-effective method. You can do it for very little cost or even for no cost at all.
What is Usability Testing?
Usability Testing asks users to complete a number of set tasks on a website. The ease with which they complete those tasks, the time it takes to complete them, and the user’s perception of the experience all help the publisher to improve the site.
So what are the goals in a Usability Test for Joomla.org? I wanted to find out:
- Whether the Tester understands what Joomla is;
- Whether the tester understands that Joomla is free;
- What the tester wants to click on the home page.
Joomla Day West was a perfect opportunity to do some Usability Testing on both Joomla.org and Joomla 1.6 because it was full of, well… Joomla users! The results were fascinating and eye-opening.
In this post I’ll focus on the Joomla.org home page. Future posts will look at the Joomla 1.6 usability testing that I’ve done.
To ensure the integrity of the test, it was important to test the Joomla.org home page with someone who had never even heard of Joomla —not easy to find at a Joomla conference. So I had to do this test when I got back home. I found Aaron, a 20-something English major student. When I asked him if he’d ever heard of Joomla, his reply was “Huh?” Perfect! In return for a bottle of beer, he agreed to sit with me for a short usability test.
Here’s the video test, watch it first and then read on:
I first asked Aaron to tell me his initial impression of the site. His reaction was positive — he thought it had a “clean and modern” design. I then asked him to tell me what is “Joomla”? Here he had more difficulty. He figured it was software, but he couldn’t tell me what it does. “I have no idea,” he said. It took a visit to the “About” page to figure it out.
That’s a problem. A site has less than eight seconds to grab someone’s attention before they move on. Visitors need to understand what Joomla is right on the home page, and as quickly as possible.
Once Aaron understood that Joomla allowed him to create websites, he was interested in downloading it. Expecting to find an executable file, he was surprised to receive a bunch of files in a folder. He didn’t know what to do with them, but he was willing to read the manual, although he described it as “a pain.” He also decided to try the demo but didn’t expect it to take up to an hour. Finally, he concluded that the software was not free because it came with a 30-day trial.
The second tester, Will, had been using Joomla but wasn’t very familiar with the Joomla.org website, since he knew what Joomla was, we didn’t focus on the home page as much as the other tester, we focused mainly on the “Demo” feature. He was expecting to find an actual demo site that he could browse without having to create a site. He said he didn’t want to commit to something he wasn’t sure he needed. We also had a few problems with FireFox 2, the browser we used to conduct the test.
Note: sometimes it’s hard to understand the tester on this video because of the recording the typing sounds, my apologies.
My recommendations based on these tests are:
- Include a clear explanation of Joomla right on the home page ;
- Offer a live demo, without having to register or create a site;
- Explain that Joomla is free and can be installed anywhere, and explain how to install it easily (with tools like Fantastico) ;
- Explain what the 30-day trial does;
- Explain what users are downloading and what they have to do with the downloaded file ;
- Test with Firefox 2.0.
Here’s a mockup for the home page, click on the image to see it in full size:
As you can see, the top of the page clearly says: “A free and powerful way to create websites (1).” “CMS,” or “Content Management System” might be a better description but most people are not familiar with the term.
Three sections then follow a user’s logical sequence. First, users want to know what Joomla is and what it can do for them(2); second, they want to see a live demo to figure out if they like it (3); and third, once they have plenty of information, they are ready to set up their site (4).
Notice I didn’t use the word “demo” on the third section because what Joomla.org offers is not really a demo. It allows them to set up their site easily — which is really cool — but it’s not a demo.
The power of Usability Testing is that it’s a very accurate insight into the tester experience. Let’s try to make it a great one!