In 2005, I analyzed the transcripts of players at a public exhibition of my IF Whom the Telling Changed. This analysis helped shape my thoughts on IF interfaces as I was starting work on my next major project, Blue Lacuna, designed to reduce the number of commands rejected by the parser. This fall, Lacuna was exhibited in a similar venue: I decided to record a new set of transcripts and see if I'd managed to improve things.
Takeaways for the impatient: Rejected commands do seem to be a significant factor in a player's decision to stop playing. Lacuna is twice as good at recognizing player commands than Telling. Players spent about 50% longer with Lacuna than with Telling. And the number of total player inputs rejected as typos or for trivial syntax reasons has been greatly reduced, thanks to keywords and extensions.
Find the full report, in all its pie-chart-filled glory, here.