references/research_methods_canon.md
Product Research Methods Canon
Reference for method selection. Pairs with study_designer.py.
The two-axis map
UX/product research methods sort along two axes (Rohrer, NN/g): attitudinal vs behavioral (what people say vs what they do) and qualitative vs quantitative (why/how vs how-many). The single most important pre-method decision is the goal:
- Generative (discovery) — you don't yet know the problem. Methods: semi-structured interviews, contextual inquiry, diary studies. Output: themes, unmet needs, jobs-to-be-done.
- Evaluative — you have a solution and want to know if it works. Methods: moderated/unmoderated usability tests, concept tests. Output: task-success, severity-rated problems.
- Validation — you want to confirm demand/desirability before building. Methods: surveys, preference tests, fake-door tests, and (when live) A/B experiments.
Picking an evaluative method for a generative goal — "let's usability-test our way to product strategy" — is the most common and most expensive error.
Interviewing discipline
Steve Portigal's Interviewing Users is the operative craft reference: ask about recent, specific, real behavior ("tell me about the last time you…"), not hypotheticals or opinions ("would you use…"). People are unreliable narrators of their future selves but good storytellers of their past.
Jobs-to-be-Done
Christensen's and Ulwick's JTBD reframes research around the progress a person is trying to make in a circumstance, not their demographics or feature preferences. Outcome-Driven Innovation (Ulwick) operationalizes this into measurable desired outcomes — a bridge between qualitative discovery and quantitative validation.
Mixed methods
Strong research triangulates: qualitative discovery surfaces hypotheses; quantitative validation sizes them. Sauro & Lewis (Quantifying the User Experience) provides the statistical backbone for turning usability observations into defensible metrics (task time, completion, SUS) without over-claiming from small samples.
Sources
- Portigal, S., Interviewing Users, 2nd ed. (Rosenfeld, 2023).
- Christensen, Hall, Dillon & Duncan, Competing Against Luck (2016) — Jobs-to-be-Done.
- Ulwick, A., Jobs to Be Done: Theory to Practice (2016) — Outcome-Driven Innovation.
- Rohrer, C., When to Use Which User-Experience Research Methods — Nielsen Norman Group.
- Sauro, J., & Lewis, J., Quantifying the User Experience, 2nd ed. (Morgan Kaufmann, 2016).
- Goodman, Kuniavsky & Moed, Observing the User Experience, 2nd ed. (2012).