If no product instrumentation is feasible, one common way to obtain usage information is via user tests, which are recorded on video. Using video analysis software, such as Noldus’ Observer XT, these videos can then be annotated semi-automatically and logs representing the actions of participants in the user test can be generated.

Process mining can provide richer analysis results based on these action logs than most usability measures, since it preserves the temporal aspect of the data. For example, the behavior of a group of participants in a user test can be visualized graphically by a process model.

One software tool that supports the annotation of videos is The Observer XT from Noldus. The Figure below depicts a screenshot of the Noldus software in action. In the top-left area, the actual video recording of the user test is shown. On the right side, a number of states, and shortcut keys for the start and end of each state, can be defined. This information needs to be added by the analyst who performed the user test. It relates to specific phases of the usage process during the test. For example, `Select channel’ would be a possible state during a television usability test.

Screenshot of the Noldus Observer XT software in action

Once these states are defined, the analyst can watch the video recording and hit the corresponding keys as she observes the participants’ behavior (for the start and the stop of the `Select Channel’ activity, respectively). Each event is then automatically extended with a time stamp that refers to the actual user test time (cf. bottom-left area in Figure above).

The resulting log can be exported as a text file as follows:

1. Choose File → Export → Observational Data….
2. Then select the observations which need to be exported. Press Next.
3. Then choose the directory in which the log files have to be saved. Select Save as and choose ASCII (*txt). This is the format that can be loaded directly in ProMimport (see next step). The file name of the exported log file can eventually be changed. Press Export. Done!

ProMimport is a tool that facilitates log conversion tasks in a plug-able architecture. It provides conversion filter for many real-life information systems and can be downloaded from the following web page http://promimport.sf.net.

Screenshot of the ProMimport tool converting Noldus’ Observer XT logs

There is a ProMimport plug-in available, which supports the conversion of Noldus logs to the log format that can be analyzed by ProM, the process mining tool (see section below).

How to obtain: Download zipped JAR file (7.4 KB)

How to install: Choose Filter –> Load filter plugin... from the ProMimport menu and locate the unzipped JAR file. Press OK. Done!

ProM is the leading process mining toolkit. It is developed and maintained by the process mining research group in Eindhoven and can be obtained from the following website http://prom.sf.net/. The latest release ProM 5.0 contains more than 230 plug-ins and offers a wide range of analysis tools.

Screenshot of the ProM framework

Logs recorded via Noldus’ Observer XT can be converted using ProMimport and then analyzed in ProM in many ways. Example analysis results that can be obtained with this approach are available in this Master's thesis.