Current Event File Format (MWK2)¶
Description¶
MWorks’ current event file format uses the file extension .mwk2
.
Each event file is a standalone SQLite database. The database contains a single table (events
) with three columns: code
, time
, and data
, corresponding to the three elements of an MWorks event.
In a given row of the events
table, the code
and time
values are always integers, while the type of the data
value can be any of SQLite’s supported datatypes. A data
value of type NULL, INTEGER, REAL, or TEXT represents the data for a single event. A data
value of type BLOB contains MessagePack-encoded data for one or more events (all with the same code and time), in one of the following forms:
an uncompressed stream of MessagePack-encoded values, with each value of MessagePack type nil, boolean, integer, float, string, binary, array, or map
a single MessagePack extension type value with type code 1, whose data is a zlib-compressed, UTF-8-encoded string
a single MessagePack extension type value with type code 2, whose data is a zlib-compressed stream of MessagePack-encoded values (i.e. like (1) after decompression)
Example Code¶
The following Python code demonstrates the MWK2 format in detail by a implementing a simple reader for .mwk2
files. Apart from the msgpack package, it requires only the Python standard library:
from __future__ import division, print_function, unicode_literals
import sqlite3
import zlib
import msgpack
try:
buffer
except NameError:
# Python 3
buffer = bytes
class MWK2Reader(object):
_compressed_text_type_code = 1
_compressed_msgpack_stream_type_code = 2
def __init__(self, filename):
self._conn = sqlite3.connect(filename)
self._unpacker = msgpack.Unpacker(raw=False)
def close(self):
self._conn.close()
def __enter__(self):
return self
def __exit__(self, type, value, tb):
self.close()
@staticmethod
def _decompress(data):
return zlib.decompress(data, -15)
def __iter__(self):
for code, time, data in self._conn.execute('SELECT * FROM events'):
if not isinstance(data, buffer):
yield (code, time, data)
else:
try:
obj = msgpack.unpackb(data, raw=False)
except msgpack.ExtraData:
# Multiple values, so not valid compressed data
pass
else:
if isinstance(obj, msgpack.ExtType):
if obj.code == self._compressed_text_type_code:
yield (code,
time,
self._decompress(obj.data).decode('utf-8'))
continue
elif (obj.code ==
self._compressed_msgpack_stream_type_code):
data = self._decompress(obj.data)
self._unpacker.feed(data)
try:
while True:
yield (code, time, self._unpacker.unpack())
except msgpack.OutOfData:
pass
The MWK2Reader
class defined above can be used as follows:
with MWK2Reader('my_data.mwk2') as event_file:
for code, time, data in event_file:
# Process the current event
...