Firefox Plans Built-In Usability Platform

Though still in the concept stage, Mozilla is planning an interesting new platform called Test Pilot that will allow every Mozilla Labs project to push out usability tests directly to participating users. The platform will begin life as a Firefox extension with the goal of gathering a 1% representative sample of Firefox users — which is a tall order given the Firefox has many millions of them.

Mozilla Labs’ Head of User Experience Aza Raskin talked about Test Pilot this week in a blog post. “Its not just Firefox that needs a usability lab. Thunderbird needs one. Seamonkey needs one. Every Mozilla Labs project needs one. Test Pilot is a platform — starting as a Firefox extension — on top of which anything can be put through usability-testing boot-camp,” he wrote. That’s necessary, says Raskin, because currently Mozilla only has access to scattershot usability research collected via surveys, anecdotes, user feedback, etc.

Test Pilot would allow Mozilla to collect feedback from real users, in actual use scenarios and Raskin sees it as part of a massively scalable, open usability lab being built at Mozilla Labs. Test Pilot won’t constantly record data, says Raskin, but only just enough information to answer the question posed by the researcher. Raskin outlined how the software will work from the user end:

The first time the Test Pilot add-on is run, it will ask a few simple non-personally-identifiable questions to put you into a demographic bucket, e.g. technical level, locale, etc., and to let you opt in to additional anonymous instrumentation.

We’ll only collect aggregate anonymized data, publish all results under open-content licenses, and review every test to make sure your privacy is held sacred. Once in a while you may be asked to participate in a short survey based on your demographic. If you’ve opted into allowing additional anonymous instrumentation, an experiment may request some of that information for aggregated study.

Though it will begin with Firefox, eventually the plan is to have Test Pilot baked into every Mozilla Labs project. Mozilla plans to offer open access to test results, and will allow anyone to create research tests and run them once they pass review.

Open source blogger Matt Asay thins that Test Pilot will offer a major competitive advantage to Mozilla. “If Mozilla can successfully gather input from nontechnical, average users, it will have secured the holy grail of computing: deep customer feedback,” he writes. “What will Mozilla do with that feedback? Why, conquer the world, of course.”

But we have a few of important questions about Test Pilot that Raskin’s blog post didn’t answer. First: when is it coming? Test pilot was actually announced back in March 2008, and at the time the first version was promised to ship within a “few weeks.” Clearly, that didn’t happen if it is still in the concept stage 10 months later. But the fact that Raskin is talking about it again seems to indicate that a release of Test Pilot is either near, or it is at the very least still under active development.

Second: How many people will actually use it? That’s a big if in terms of the usefulness of Test Pilot. Even if the platform is well built and succeeds at capturing meaningful feedback data, it won’t be worth anything if no one uses it. Can Mozilla really convince 1% of its massive install base to join their user feedback panel?

Third: Will Test Pilot be made available to non-Mozilla developers? Because of the open source nature of Mozilla’s software, it seems like a safe bet that the Test Pilot platform will be open to other open source developers. But nothing specifically was mentioned about that in Raskin’s post.

Article source: Sitepoint