85-411/711 Cognitive Processes and Problem Solving Homework #2-Solutions The goal of this assignment (and the accompanying readings) is to offer you some practice in the skill of collecting and coding verbal protocols. The readings offer some exposure to the empirical work that supports the theory and assumptions underlying this methodology. A.2 List all the different terms and descriptors that are included in the first twenty statements of the [thermodynamics] encoding. Term Descriptors goal (suppl.) status, priority choose what, instance write what, where, content, ref assume what, for, reason evaluate what, criteria, result, certainty read what, sentence identify what, instance, ref equate what, with, units, ref simplify what, result, ref calculate what lookup what substitute what, units, for know what, how What's the ref descriptor for? It is simply used to indicate when the subject is referring back to a previous statement. B.6. Collect and Segment the protocols Here are a few segmented protocols that students collected. This will help you to better evaluate the generality and specificity of statements. Subject JB Sixty-seven time fifteen. Start with sixty-seven times ten which is six hundred and seventy. Add five times sixty-seven which is three hundred and thirty-five, so you get an answer of one thousand and five. Subject TH First, I just multiply the seven and the five. Uh. Bring the five down. Oh. That's not right. Bring over the three, and carry that over, and add that to six times five so I get 33. I get 335 for the first one. And the 1. I just multiply 67 by 1 right away, and then add a zero to the end. That's easy. And then I ad, uh, 335 to 670... and I end up with 10,... one thousand five. Great. Subject VM Sixty-seven times fifteen I have to calculate it now? Sixty-seven times fifteen. Ok. em.... I think about how I going to ... to do it the best. I take ten times sixty-seven. It's six ...six hundred and sixty-seven. And then five times. It is thirty ehm... thirty...two. ooh shit. thirty-three point five. Now add it up five. The point and three O seven. Seven hundred three point five. IT'S WRONG, BUT IT'S ALL RIGHT. I mean, shit what did I do, no it's not possib... Wait. It's wrong?? ...How did I make a mistake? Now its hundred and seventy, .... it's right! Ok. Go ahead. D.1. Encode the protocol Generally, everyone did a good job. However, there are a few methodological pitfalls that consistently appeared in the encodings. First, try to pick general (but meaningful) terms for your descriptors. Try to use one word for all the synonymous words used by the subject. For example, use "add" for "add", "plus", "and", etc. Second, in order to adequately represent the flow of action in the encoded protocol, it's often necessary to break down the components of what was said into multiple descriptors. For example, some students encoded "four plus three is seven" as: what result calculate four plus three seven Note that "what" isn't a particularly useful descriptor. Consider breaking this down into: addend1 addend2 result add four three seven Third, try to avoid inferring steps. The goal of encoding the protocol is to provide an objective coding of what the subjects said and did. If you add inferred steps at this stage, you won't know if your data reflect actual actions taken by the subjects or actions that are in your theory of performance. Fourth, try to encode as much of the protocol as possible. While it is difficult to know what to do with coughs and laughs, meta-comments about the difficulty of the task and what strategy is being selected are very interesting and quite easily coded. D.4&5 How does the protocol from your subject compare to others? What percentage of the comments were "general?" What about in other tasks? The protocols collected were all quite similar to one another. Students generally learn the standard way of solving multi-column multiplication, and that is what they do when given such a task. This is reflected by the fact that most people obtained 80-90% of "general" comments in the protocols they collected. You would probably obtain as high, if not higher, agreement between subjects adding two digit numbers, since that task is quite straight forward. Someone writing a history paper, on the other hand, has a number of possible approaches (i.e., the problem space is large, and the goal is somewhat ill-defined), and so their strategy for that task is likely to be more specific. D.6. What information about the subject's behavior do you think was revealed from the verbal protocol that could not have been gathered from simply observing the subject's non-verbal behavior? Verbal protocols are often used to gather information about subgoal orderand the time-course of the solution process (i.e., what information is going through STM, and when). Even in a simple task like 2-digit multiplication, protocols can shed light on strategic idiosyncracies and make it possible to performa detailed difficulty factors assessment (determine which portions of the solution process are the most difficult). Verbal protocols become even more useful for tasks that involve a higher ratio of internal processing to external actions, and that have a larger number of possible operators at each step, as you can get a sense for how subjects work through the decision process on their way to completion of the problem.