Tuesday, July 18, 2006

retrospective protocols

I just experienced a great example of why retrospective protocols are fraught with error. I was walking down the hall and asked a colleague how he was doing. I thought he said "hot" but I wasn't sure. I asked him what he said. He had no idea. He guessed that he said "I'm fine thank you." Clearly, this could not have been it - it sounds nothing like "hot."

What he did, according to retrospective protocol studies, is search his schema for a logical or typical way to answer the question, rather than his memory of what he really said. This happens a lot in retrospective protocols. Instead of reported what they were thinking at the time, users develop rational explanations that often do not correspond with reality. Often, what they were really thinking is not logical or easy to articulate. And also hard to remember, even soon after.