Hi James, (sorry, for some reason this was posted as "anonymous" below")
It is possible, but would require a considerable effort. The question of whether the effort is worth it depends on the purpose of the experiment.
If it is meant to be a "valid" experiment (by which I mean, one that meets acceptable standards of scientific rigor, I have already done something like what you are proposing in my dissertation (see http://phdtree.org/pdf/25409619-model-ii-behavior-and-team-p
). I've also done some other studies that used various versions of the prototype software - see for example a few of them at:
Ricardo Romero, Richard Savage, Paul Figueroa & Ray Luechtefeld, "A Quantitative Study Of The Impact Of Automated Facilitation On Engineering Student Dyadic Task Completion," Presented at the 2011 Frontiers in Education Conference, October 2011 Rapid City, SD (refereed). (Available online at http://fie-conference.org/fie2011/papers/1276.pdf ).
Paul Figueroa, Richard Savage, Ricardo Romero & Ray Luechtefeld, "A Qualitative Analysis For The Facilitation Of Innovative Problem Completion Amongst Dyads Of Engineering Students," Presented at the 2011 Frontiers in Education Conference, October 2011 Rapid City, SD (refereed). (Available online at http://fie- conference.org/fie2011/papers/1278.pdf ).
Ray Luechtefeld, Steve E. Watkins, & Raj Kanwar Singh, "Expert System for Team Facilitation using Observational Learning," paper presented at the Frontiers In Education Conference, 10-13 October 2007 Milwaukee, Wisconsin (refereed) (Available online at http://fie- conference.org/fie2007/papers/1530.pdf ).
These three were done with the web interventions in a discussion board. I still have to complete write-ups on studies with the prototype that uses spoken interventions and speech recognition via mobile devices, and which is capable of crowd-sourced interventions.
That alpha prototype does not capture my latest thinking about the architecture of the "dialogic web", which is roughly similar in scope to an architecture of the "semantic web", though of course based on some completely different assumptions. While the semantic web is about working with clearly defined meanings, the dialogic web is about working with the multiplicity of meanings that compose everyday life.
If the purpose of doing the experiment is to prompt thought and build exposure, I might be interested in doing the human run-through, though it might make more sense to just do the programming. Since most of the conversational interventions that the dialogic web will use will be crowd-sourced (like Wikipedia) a human actor run-through will not necessarily reflect the final system. My goal in participating in this essay contest was to connect with others who would be interested in furthering the development of the dialogic web, so I'd certainly like to engage in conversation about that. You can best reach me at the IEEE email address on the papers linked above if you want to talk about it.
Does this respond adequately to your post? Let me know if I can provide more information.