Proposing Navigation Changes

I just made a ticket on Trac with a proposal to address navigation in the WordPress dashboard. Here is the description.

1. Explaining the difference between Posts and Pages to new users is time consuming and often frustrating. We’ve all done it, have our best/fastest version of the talk down pat, but it still takes longer than it should to get many new users to the point of understanding the difference.

2. Back in 2.7, when we set up the left navigation we put Pages at the bottom of the content nav section because in testing 2.5/2.6 so many people complained about accidentally clicking Posts/Pages by accident because they were close together in the old UI and both started with P (blame it on capital_p). Because of this, Pages falls below the less-frequently accessed areas of Media and Links, and people don’t necessarily see it right away because they expect it to be higher up.

I’ve been testing out two changes to the left navigation aimed at reducing these two issues on my test blog for some time now, and have been using it during demos with both new and existing users to great success, so I think it’s time to propose it for core.

Change 1: Change the Posts label to Blog. All Posts can remain as is, or could be reduced to just Posts, since the reason we added the All in the first place was that Matt thought it looked weird to have the same word shown twice.

This change reduces the amount of time it takes me to get a new user really understanding the difference between posts and pages by about 75% (very informal testing, have kept track with about 30 new users by just keeping an eye on the computer clock to see how long it is before we move on). The dynamic blog/static site difference is much easier to grasp when they see that familiar word Blog instead of Posts because “posting” is an action that applies even to static content, and even posts are displayed in web pages (vs Pages).

Change 2: move Pages up the menu to sit below Blog, so the two most important content types are at the top. Since they wouldn’t look similar (ha ha capital_p) there would be much less risk of accidental misclick based on letter shape (poor manual dexterity would not be affected, but in that case those people are already clicking the wrong things, right?)

I’ve attached a screenshot showing what the navigation would look like with these changes.

Testing 1, 2, 3: Usability and WordPress

I did some lo-fi testing of WordPress 3.3 pre-beta on Sunday/Monday with local users, but since in the next set I want to get more multisite users, finding good participants will be tougher. Savannah has an art school, a music scene, and a burgeoning tech culture, so I can’t spend an hour at The Sentient Bean without overhearing someone tell their co-caffeineators about some new WordPress site they’re working on — a flyer posted for half a day gets plenty of individual users.

Multisite superadmins, though, are a bit harder to come by without effort, so I’m going to test out using some fancy web technology called “the skype” + “quicktime recording” as an experiment.

A Bit of Testing History

Once upon a time, usability testing was done in a lab, and was very expensive. Costly software like Morae was used, whole research teams were needed to recruit participants, moderate test sessions, and analyze results, and sometimes things went really high-end and lasers were involved. We’ll call this “formal usability testing.” The testing done in Spring 2008 on WordPress 2.5 and the Crazyhorse prototype was formal, included laser eye-tracking, and led to/strongly influenced the 2.7 admin redesign. Read all about it.

If you couldn’t afford formal testing, lo-fi setups involving a camcorder and a tripod allowed capturing the screen and participant speech, but even with a decent camera, the screen images were never fantastic. Also, unless you set up two cameras, you would miss things like facial expressions (which are hugely informative during observational testing). Although you could set it up anwhere and didn’t have to rent a lab, it was a clunky solution.

It is hard to believe it’s been so long, but I have been doing usability testing –both formal and informal — for more than 12 years now.

In July 2008, Clearleft released their Silverback app, and popularized the concept of  ‘guerrilla’ usability testing. Armed only with your Mac and their app, you could get screen capture and user video/audio thanks to your Mac’s built-in iSight. This was a game-changer. Though it didn’t have the hardcore analysis features of Morae, simply getting a high fidelity screen cap on your own was huge… and it was only about $50! Suddenly, you didn’t need to rent a big lab, you could use your regular office or even set up shop at the local caffeine vendor. Just stick people in front of your laptop and you were good to go. Suddenly testing costs could be a tenth of what they had been, and guerilla testers cropped up everywhere. By the time 2.7 was ready for testing in Autumn 2008, that’s how I was doing it, too.

I had grand plans to introduce a distributed testing model to create an avenue for usability testing professionals to have a way of contributing to WordPress that would be analogous to writing patches or helping in the forums. I put up a post about it, corresponded with potential volunteers, and tried to work out the logistics. Having each volunteer running Silverback and then uploading their videos to a central repository for analysis was the plan, but infrastructure was a problem in two ways:

1. How would we match volunteer test moderators/usability professionals with volunteer participants? Well, there were some more grand plans around building a volunteer database tied to the .org profiles, but when the person who was working on it left, that basically sputtered, so finding participants remained a normal logistical nightmare.
2. It would mean only Mac users could conduct testing, and non-Mac users would be using an unfamiliar OS during the test.
3. We didn’t have a good way to collect the session videos.

So that plan never really got off the ground.

Skip to Today

Guerilla testing is still going strong, and if the number of people proposing sessions about it at SXSW is anything to go on, the fact that it’s been more than 3 years since Silverback was introduced hasn’t made it any less exciting a concept. That said, it’s still not a perfect solution. Let’s look at pros and cons…

• Pro: Output video content and quality on par with tools like Morae
• Con: Lacking in the post-test analytic tools of Morae.
• Pro: Works on Macs! Morae is still PC only.
• Con: Works on Macs! 3 years later, Silverback is still Mac only, excluding all our PC users from participating in tests in a familiar environment, and preventing non-Mac usability pros from getting involved this way.
• Pro: Much cheaper than buying Morae.
• Con: If the plan is to have many people doing testing all over, each moderator needs a copy of Silverback. While markedly less expensive than copies of pro software like Morae, adding up all the copies of Silverback would wind up being more than one person or team running Morae.
• Pro: Can conduct testing in convenient locations such as an office, a coffee shop, etc. or travel to participant location. Great flexibility.
• Con: Limited to in-person-only testing unless test participants download and install Silverback on their machines. We don’t want to require people to install software (even a free 30-day trial version) just to be a volunteer test participant, so anywhere we want to test with real users, we would need volunteer moderators with the software to be co-located.
• Con: Non-realistic experience when using a moderator-provided laptop during in-person tests. Real-world things influence how a user interacts in the browser with any web app — saved passwords, form auto-fill, chat windows popping up, email notifications, torrent downloads of Doctor Who hogging your bandwidth, screaming kids in the background, a ringing phone, you name it. To get a more realistic picture of usage/behavior, the researcher always prefers the setup that is as close to normal as possible. Using someone else’s laptop just doesn’t cut it.* And just as Morae limited us to PCs, Silverback is limited to Macs, but we want to test with users of both platforms. (And Linux! Don’t forget Linux!**)

So while guerilla testing with Silverback is cheaper/and more flexible than hiring a lab, and provides you with deeper observational information than an online service like usertesting.com, you’re still limited to doing testing in a place where you can have both a moderator and participants, which makes the participant pool quite limited, and not representative of the WordPress user base as a whole.

With the advent of Skype screensharing and Quicktime screen recording, we may finally be getting to a point where we can bridge that gap and include participants who are not local to the moderators — where we can get good recordings and can let participants use their own machines without significant cost or requiring unfamiliar software downloads. The drawback to this is that if you’re using skype to screenshare, you can’t also continue the video chat talking head.*** Still, observing real-time usage in a far-away participant’s own environment while being able to ask questions is a big step forward. So, I’m trying it out. We’ll see how it goes… Skype screensharing can get jumpy or laggy if you’re not on a reliably blazing internet connection.

If you are someone who uses WordPress multisite as a superadmin and would like to be a guinea pig to help me work out the kinks of this method (and get a look at some new UI stuff we’re considering at the same time), leave a note in the comments and I’ll get in touch over the next couple of days. Thanks!

When we get into Beta, assuming the trial run of this goes okay, I’m hoping to try and revive the distributed testing idea, so if you are a professional usability tester and would like to get involved with that in a couple of weeks, a note in the comments will get you added to the email list for when that’s ready to try again.

All that said, why do we do testing? It’s time-consuming and expensive, even when the software is cheap or free. Most agencies do it to show their clients that the design decisions they made were good ones. Most companies do it to find out what problems customers/users have with their products. With WordPress, we don’t so much have a “client” and our users tell us straight up in the forums what things cause them problems. So why should we do testing at all?

• Define benchmarks. One of the things we’re always trying to improve with WordPress is making it faster. How long it takes to complete various tasks in wp-admin is one benchmark of performance and usability that can be measured. Testing can provide a sample data pool with these stats.
• Test assumptions. With so many people weighing in on every design decision for WordPress, sometimes we have to forge ahead in what we think is the best direction despite siren songs from contributors who would prefer a different UI approach to something. Design by committee, camel, etc. That said, when’s there’s more than one UI idea or suggestion that seems reasonable for a given task, we don’t want to cling dogmatically to the status quo, either. Testing a few different design approaches allows us to see which designs people seem to respond to better.

So, could we live without testing? Sure. But do we want to? No. Seeing our work being used by regular people — not code contributors, not designers, not WordPress insiders with a vested interest in the choices made in the UI — keeps us humble. I challenge you to watch someone who’s not especially web-savvy figure out how to embed a video the first time. Change the tagline. Create a custem menu. I think the perpetual praise of WordPress as the most user-friendly platform for blogging and content management is justified, and we’ve worked hard to earn that reputation, but everyone on the core team is painfully aware of how much further we still need to go. Good testing can help us get there faster.

P.S. I haven’t even touched on testing with people of different abilities, different languages, etc., but that all needs to get more attention as well.

*And also means the researcher will have to re-arrange all her keys to be QWERTY for the sake of testing, then swap back to Dvorak afterward. Hmph.

** We usually forget Linux.

*** If we wanted to do another Mac-only thing, people with iPhones could use FaceTime for the talking head part while Skype was in screen share mode.

TripIt, You’re Killing Me.

So burned out on anything resembling a social network. That said, TripIt is pretty useful in a company like Automattic where most of us only see each other when we travel. When I first joined, I did the ‘find friends from your address book’ to fill out the people I see the most. However, for my work email, most people are not actually in my address book. My email client just autocompletes addresses that have been used before. I don’t, in fact, have ANY online address book that has a complete (or even close) roster. Anyway, I didn’t really care about the ux of adding connections on TripIt until just now. It seems that I can add from my email address book, or I can invite someone individually by entering their email. There does not seem to be a way to search for someone who’s already a member of TripIt. I could chalk that up to privacy, I guess, but someone just shared an itinerary with me, and there is no way to get connected from it. Her name isn’t linked, and there’s no place to search for her. Sure, I could go and enter her email and send her an “invite to TripIt” (vs. an invitation to connect on TripIt). But why can’t I just click her name, or search, or something?

In contrast, Dopplr was so much better. Making connections was as simple as clicking on their avatar when you saw them and realized you wanted to connect.

TripIt, I know we’ve all come over to your side of the playground, but this is one thing where Dopplr really just did it right.

Testing WordPress, PDX Edition

I’ll be here all week, folks.

Portland WordPress users: I’ll be in town tonight through next Tuesday morning to attend WordCamp, and would like to do some usability testing with you lovely people while I’m around. I have time slots all day tomorrow (Thursday), a few on Friday, a couple before/during/after WordCamp (though those will mostly be very short), and a couple on Tuesday morning. If you’d like to participate and can spare 20-40 mintues to help improve the next version of WordPress, send me an email (you can use the contact form on this site) or send me a message on Twitter (@janeforshort).

iPhone testing at WordCamp Portland

I’ll be doing some usability testing of WordPress and the WordPress iPhone app (new beta version!) while I’m in Portland for WordCamp (9/16-22). I’ve got five beta tester slots being held for participants (we can only have a total of 100 beta testers). If you use WordPress on your iPhone and would like to try the new beta version (and will be attending WordCamp), let me know.

**Update: Due to high demand, I’ll be raffling off the slots. Send me an email, or @janeforshort on twitter by 9pm tonight pacific time to let me know you’re interested, and I’ll pick 4 beta testers at random (one slot was already promised when I realized there would be more demand than supply). If you don’t get picked, you can still help with testing… I’ll have my iPhone at the WordCamp, and will let people try the beta app that way as well.

A Different Plugin Search Links Thing

When you search plugins from within the admin, it brings in the repo search results. I think that the plugin name should pop up the tabbed info from the repo page for the plugin, instead of linking to the repo page in a new window or the plugin site. I also think that the Install link should be on the left, not the far right, under the plugin name, so it’s in the same place as action links on other Plugins section screens, and that there should be a View Plugin Site link there, like there is on the other screens, too.

UI Gaffe

I have just noticed a UI gaffe for which I take full responsibility. In the plugins section of the admin tool, when you go to add new plugins, there is a utility for searching the plugin repository (awesome!) with a button labeled “Search Plugins” to submit the search request.

But! There’s also, in the section for managing plugins, that dorky search box that appears here and there in the admin and allows you to search whatever section you’re in at the time. It, too, has a button labeled “Search Plugins” to submit the search request (not awesome!). Bad UX girl!

In 2.9, I solemnly swear to do something about this. Also the overall clumsiness of the plugins section labeling and navigation. The dorky search box I might not touch, since there’s a GSoC project around extending search that seems cool, and I’ll want to see how that turns out before making changes to the existing search. Ultimately, I’d like to get rid of the section-by-section search box that takes up all that precious h2 real estate and get a blog-wide search going that we can stick in the nav area to search everything at once with the ability to filter results (so you could search posts, comments, tags, etc all at once but choose to see only a subset of results if you only want to see plugins, for example).