Basic Demographics For Your HCI User Study
February 21, 2017
Many HCI studies, especially student studies, involve a small section on participant demographics. In this post, I summarise the guidance I tend to give to my own students. There are many aspects of designing demographics questionnaires that this post won’t cover, but for a relatively simple basic user study, this should do the trick.
Principle 1: Don’t ask too many questions.
Make sure that what you are asking is relevant to your study, and focus on the most parsimonious way of getting that information. Fewer questions with fewer tick boxes are less daunting and generate more answers.
For example, when you ask about people’s age, don’t use age groups that cover ages 18-60 in increments of 5, unless you really need that level of granularity.
Principle 2: Respect people’s privacy.
Don’t ask detailed questions which may enable others to identify your participants, given what they know about when you conducted the study, and where you recruited the participants, unless they are relevant.
For example, questions about people’s country of origin are most often used to distinguish between native and non-native speakers of English – which matters if you test a system that extensively uses language. Yet, knowing somebody’s country of origin can easily identify participants who might be the only student from their country on your particular Masters programme.
Principle 3: Make demographics optional
Demographic data can be used to potentially identify people. Also, some people may not feel comfortable sharing that information with you, especially if all they are asked to do is to evaluate or use an app or a product that you have designed.
Principle 4: Put demographic questions last
If you make somebody reflect on an aspect of themselves that is associated with social stereotypes, they are more likely to conform to or enact those stereotypes later. This is an instance of a phenomenon called Stereotype Threat, and Wikipedia has very useful resources about this topic. If you want to amplify effects of stereotype threat in your data, put demographics first, otherwise, put them last.
Principle 5: Cover the basics.
The basics differ by discipline and research lab. I always like to include the following:
- age (18-24, 25-34, 35-44, 45-54, 55-64, 60+), with additional categories above 60 if I am working with older participants
- gender (male, female, prefer not to say, other), which makes space for gender fluid people
- occupation (student, employed full-time, employed part-time, self-employed, retired, home maker, unemployed, other), which acknowledges the important work that men and women who stay at home do (otherwise, they’d have to refer to themselves as unemployed). I also recommend making this a checkbox category, because people can be students while employed full time
- highest educational qualification (high school/secondary school; vocational qualification; university graduate; postgraduate qualification). This is the most country-specific one, and there are no hard and fast rules. I mainly like to include it because level of education is a potential indicator of socioeconomic status, and may also affect people’s performance on task
Principle 6: Check whether your participants have exposure to the technology / products that you are testing
There are several formal and informal sets of questions floating around that assess digital literacy, exposure and attitudes to technology, and the like. I prefer to stick with the minimum, which looks at what technology people own and use.
Below is a very basic example of a table that requires people to tick what technologies they own, and how frequently they use them.
|Don’t own one||Daily||Weekly||Rarely|
I hope that you find these hints useful. If you want to cite them, feel free to do so; if you have any comments, or would like me to expand on some of the points I made, please leave a comment. Comments are moderated, as I get a lot of spam, but I check regularly.