This content originally appeared on NN/g latest articles and announcements and was authored by Raluca Budiu
Summary: All the details of your benchmark study need to be stored in your research repository for further reference. They include the study methodology, the tasks, the screener, the definitions of the collected metrics, as well as the raw data obtained from the study.
As an organization refines the user experience of a product, a common question is whether the newer version of its product is better than what it had before. The answer to that question is usually provided by UX benchmarking , a practice that aims to measure and compare the UX of different versions of the product.
UX benchmarking involves running a set of iterations of the same summative study (called a benchmark study ) on various product versions, with each iteration aiming to capture the UX of the current version of the product. To gauge UX improvements from one version to the next, researchers would have to compare metrics such as success rate , task time, and user satisfaction for some important tasks. They will also have to go beyond the observed values of the metrics obtained in the two studies and run a statistical analysis to judge whether the observed differences are statistically significant or due to chance.
Imagine that the first iteration of the benchmark study, which evaluated the original version of the design, was conducted in person, whereas the second iteration was remote unmoderated . Often, people tend to stay more on task in in-person studies than in remote unmoderated ones, and, as a result, times from remote unmoderated studies tend to be longer. Thus, if we notice a difference in task time or success rates between the two design versions, it could be due not to a better design, but to the way in which the study was conducted.
Read Full Article
This content originally appeared on NN/g latest articles and announcements and was authored by Raluca Budiu
Raluca Budiu | Sciencx (2023-05-07T16:00:00+00:00) Documenting a UX-Benchmarking Study. Retrieved from https://www.scien.cx/2023/05/07/documenting-a-ux-benchmarking-study/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.