Analyzing Experimental Data

Events, Event Stream, and Event File

The basic unit of MWorks experimental data is the event. An MWorks event consists of an event code, a time stamp, and a value (i.e. the payload).

Most events announce a change in the value of a variable. The codec (or variable codec) provides the mapping between event codes and their associated variables. Additionally, some system events use codes that are not associated with a variable.

The sequence of events generated by a running experiment is called the event stream. The on-disk record of an experiment’s event stream is known as the event file.

Accessing the Event File

Event files are stored on the machine running the MWorks server application (MWServer on macOS, the MWorks app on iOS).

macOS

On macOS, event files are stored in the directory $HOME/Documents/MWorks/Data, where $HOME is the path to the user’s home directory (e.g. /Users/chris).

iOS

To access event files stored on iOS, open the Files app and navigate to On My iPad ➝ MWorks ➝ MWorks ➝ Data. From here, you can share one or more event files with a Mac using AirDrop or copy them to a cloud storage service like iCloud Drive or Dropbox. You can also duplicate, move, and delete files.

Alternatively, you can share your event files with a Mac using iTunes. However, this is less convenient than using the Files app and requires you to copy the entire “MWorks” subfolder to your computer.

Analyzing the Event File

To analyze the results of a completed run of an experiment, you must extract the relevant events from the run’s event file. MWorks provides tools for loading event data in to Python or MATLAB.

Using Python

Setup

To use MWorks’ data analysis tools for Python, you must first make the mworks package available to your Python code. You can do this on a per-script basis by adding the directory /Library/Application Support/MWorks/Scripting/Python to the module search path:

import sys
sys.path.insert(0, '/Library/Application Support/MWorks/Scripting/Python')

If you want the mworks package to be available to all Python code you run, you can create an mworks.pth file in your per-user site-packages directory. For example, for Python 3.7, execute the following commands in a Terminal window:

mkdir -p ~/Library/Python/3.7/lib/python/site-packages
echo '/Library/Application Support/MWorks/Scripting/Python' > ~/Library/Python/3.7/lib/python/site-packages/mworks.pth

Opening and Closing

To open an event file, use the MWKFile class from the mworks.data module:

from mworks.data import MWKFile
f = MWKFile('my_data.mwk2')
f.open()

When you are done using the file, you should close it:

f.close()

Alternatively, you can use the with statement to handle opening and closing of the file in an implicit, exception-safe fashion:

with MWKFile('my_data.mwk2') as f:
    # Analysis code goes here
    ...

Selecting Events

Once the file is open, extract events using the get_events method. If called with no arguments, this method returns all events in the file:

all_events = f.get_events()

You can also use keyword arguments to filter events by code or time. The codes argument takes a list of event codes or variable names and selects only events with matching codes. For example, we can extract all events for the variables red_selected, green_selected, and blue_selected (which appear in the “Find the Circle” example in Running an Experiment) like this:

events = f.get_events(codes=['red_selected', 'green_selected', 'blue_selected'])

The time_range argument takes a list containing a start time and stop time and selects only events whose time stamps fall within that range:

events = f.get_events(time_range=[t1, t2])

You can specify both codes and time_range to filter on code and time simultaneously.

When analyzing large event files, you may want to extract events one at a time, in order to avoid filling up system memory. To do this, use the get_events_iter method. This method takes the same arguments as get_events. However, instead of loading all matching events in to a single, list-like object, get_events_iter returns an iterator, each iteration of which loads a single event from the data file. For example, here is how you would use get_events_iter to process every event:

for event in f.get_events_iter():
    # Use data from event
    ...

Accessing Event Data

Once you have extracted an event, you can access its event code, time stamp, and value via the code, time, and data properties of the event object:

>>> evt = events[0]
>>> evt.code
36
>>> evt.time
30343102
>>> evt.data
0

The code and time properties always have integer values. However, data can be a boolean, integer, float, string, list, or dictionary:

>>> events = f.get_events(codes=['#announceMessage'])
>>> events[15].data
{'origin': 1, 'message': '4 of 10 trials were correct (68217117)', 'type': 0, 'domain': 0}

Using the Codec

To convert an event code to a variable name, use the codec property of the MWKFile object:

>>> f.codec[36]
'red_selected'

To convert a variable name to an event code, use the reverse_codec property:

>>> f.reverse_codec['red_selected']
36

Using MATLAB

Setup

To use MWorks’ data analysis tools for MATLAB, you must first add the directory /Library/Application Support/MWorks/Scripting/MATLAB to the search path:

addpath('/Library/Application Support/MWorks/Scripting/MATLAB')

Selecting Events

To extract events from an event file, use the getEvents function. If called with just the filename, this function returns all events in the file:

all_events = getEvents('my_data.mwk2')

To extract only events with specific event codes, provide an array containing the desired codes as the second argument:

events = getEvents('my_data.mwk2', [36, 37, 38])

To further restrict the retrieved events to a particular time range, specify a minimum and maximum time as the third and fourth arguments, respectively:

events = getEvents('my_data.mwk2', [36, 37, 38], t1, t2)

To extract all events in a given time range, pass an empty array as the second argument:

events = getEvents('my_data.mwk2', [], t1, t2)

Accessing Event Data

The getEvents function returns a structure array containing all selected events. The event code, time stamp, and value of each event are accessible via the event_code, time_us, and data fields of each array element:

>> evt = events(1);
>> evt.event_code

ans =

  int32

   36

>> evt.time_us

ans =

  int64

   30343102

>> evt.data

ans =

  int64

   0

The event_code and time_us fields always have integer values. However, the type of data can be logical, integer, floating point, string, cell, struct, or Map:

>> events = getEvents('my_data.mwk2', [6]);
>> events(16).data

ans =

  struct with fields:

    message: '4 of 10 trials were correct (68217117)'
     origin: 1
       type: 0
     domain: 0

Using the Codec

To convert an event code to a variable name, use the Map returned by the getCodec function:

>> codec = getCodec('my_data.mwk2');
>> codec(36)

ans =

    'red_selected'

To convert a variable name to an event code, use the Map returned by the getReverseCodec function:

>> reverse_codec = getReverseCodec('my_data.mwk2');
>> reverse_codec('red_selected')

ans =

  int64

   36