Nov 29 2006

Analyzing audience feedback

Although Nancy White’s posting on “Feedback from Sydney LearnScope Event” was written at the end of her marathon in Australia, I’ve thought about it quite a few times since then, perhaps because of the strong feelings I had after our workshop at the Prato Conference during the same month. Capturing feedback in a wiki is an interesting example of how technology inherently changes our experience of face-to-face meetings just as it changes what’s possible. Reading what the more than 64 people said about the session with Nancy was a bit overwhelming so I made a note to myself to come back and take another look. One easy way to do it was to put the comments into a tool like (a tool that happened to hear of from Nancy). Here’s the output:

<!– #htmltagcloud{ font-family:’lucida grande’,trebuchet,’trebuchet ms’,verdana,arial,helvetica,sans-serif; line-height:2.4em; word-spacing:normal; letter-spacing:normal; text-decoration:none; text-transform:none; text-align:justify; text-indent:0ex; background-color:#fff; margin:0 0 2em 0; border:2px dotted #ddd; padding:2em}#htmltagcloud a:link{text-decoration:none}#htmltagcloud a:visited{text-decoration:none}#htmltagcloud a:hover{text-decoration:none;color:white;background-color:#05f}#htmltagcloud a:active{text-decoration:none;color:white;background-color:#03d}span.tagcloud0{font-size:1.0em;padding:0em;color:#ACC1F3;z-index:10;position:relative}span.tagcloud0 a{text-decoration:none; color:#ACC1F3}span.tagcloud1{font-size:1.4em;padding:0em;color:#ACC1F3;z-index:9;position:relative}span.tagcloud1 a{text-decoration:none;color:#ACC1F3}span.tagcloud2{font-size:1.8em;padding:0em;color:#86A0DC;z-index:8;position:relative}span.tagcloud2 a{text-decoration:none;color:#86A0DC}span.tagcloud3{font-size:2.2em;padding:0em;color:#86A0DC;z-index:7;position:relative}span.tagcloud3 a{text-decoration:none;color:#86A0DC}span.tagcloud4{font-size:2.6em;padding:0em;color:#607EC5;z-index:6;position:relative}span.tagcloud4 a{text-decoration:none;color:#607EC5}span.tagcloud5{font-size:3.0em;padding:0em;color:#607EC5;z-index:5;position:relative}span.tagcloud5 a{text-decoration:none;color:#607EC5}span.tagcloud6{font-size:3.3em;padding:0em;color:#4C6DB9;z-index:4;position:relative}span.tagcloud6 a{text-decoration:none;color:#4C6DB9}span.tagcloud7{font-size:3.6em;padding:0em;color:#395CAE;z-index:3;position:relative}span.tagcloud7 a{text-decoration:none;color:#395CAE}span.tagcloud8{font-size:3.9em;padding:0em;color:#264CA2;z-index:2;position:relative}span.tagcloud8 a{text-decoration:none;color:#264CA2}span.tagcloud9{font-size:4.2em;padding:0em;color:#133B97;z-index:1;position:relative}span.tagcloud9 a{text-decoration:none;color:#133B97}span.tagcloud10{font-size:4.5em;padding:0em;color:#002A8B;z-index:0;position:relative}span.tagcloud10 a{text-decoration:none;color:#002A8B}span.freq{font-size:10pt !important;color:#bbb}// –>

audience bit boring clearly concepts delivery delved didn e-learners eb email engaging enjoyed entertaining esp excellent experiences feel felt flow food globally informative inspiring interaction interesting kept key learnscope loved messages mobile nancy ok parts practical presented presenter provoking really relevant resources sms speaker split stimulating techno tensions think via

The comments have an inherent linear structure that tagcrowd misses and which seems important. There’s an implicit “good to bad” Likert scale behind much of what people wrote. I thought that I could make up a kind of 5-number summary that systematically throws away some of the noise. This approach comes from John Tukey’s stem-and-leaf plots (I got interested in computing because of a long-standing interest in exploratory data analysis and statistics). So I did a rough sort of all the comments from the most positive to the most negative, and picked out the middle, the extremes, the quartiles, and so on.

(The text editor in WordPress seems to have trunkated some of the text, so I’m going to post the second part separately.)

3 responses so far

3 Responses to “Analyzing audience feedback”

  1. joitske says:

    Hi, I liked the tagcloud! I’ve been thinking on online feedback too. There seem to be so many more channels and posibilities to get feedback online, feedback which may be hidden otherwise. On the other hand, what to do with it? If there is so much information (and so much possible in terms of technological options), we may have to get smarter at when and how to ask feedback (and what to do with it).

    We just asked people to rate an online event, and now it seems hard to interprete the ratings…

  2. John.Smith says:

    Joitske, I think that you might have different questions at different points in time, depending on what decisions you have to make at that point. In a “paper world” asking questions and getting responses was very expensive, so that shapes what questions are asked.

  3. […] it occurs to me that one of the issues in the feedback people were giving to Nancy White was that they were assuming that there was no future relationship […]