Object Detector¶

Description¶

An object detector captures images from a camera and uses a Core ML model to locate and classify objects in them. For information on obtaining or creating a suitable model, see Getting a Core ML Model.

On macOS, an object detector can use any camera (internal or external) that is supported by the operating system. On iOS, it always uses the built-in, front-facing, wide-angle camera.

Signature¶

iodevice/object_detector

Required Parameters¶

model_path¶

Filesystem path to a Core ML object detection model (i.e. a .mlmodel file).

Note: The experiment must also declare the model file as a File Resource, e.g.:

resource ('my_model.mlmodel')

iodevice/object_detector my_detector (
    model_path = 'my_model.mlmodel'
    ...
    )

results¶

Variable in which to store the most recent detection results (as a list), subject to minimum_confidence. Each element in the list is itself a list with the following three elements:

The classification label assigned to the object (as a string).
The bounding box of the object, specified as a four-element list of the form [x_min, y_min, x_max, y_max], where (x_min, y_min) and (x_max, y_max) are the coordinates of the lower-left and upper-right corners, respectively, of the bounding box. Each coordinate is normalized to the dimensions of the image and lies between 0.0 and 1.0 (inclusive).
The confidence for the detection result. Typically, the value will be between 0.0 and 1.0 (inclusive), where 1 is most confident.

Note: The exact meaning of the confidence value depends on the model. If the model does not support confidence estimation, the value will always be 1.

If detection fails to produce any valid results, the variable is set to an empty list.

image_capture_interval¶

Example:: 250ms

Time interval between image captures. Must be greater than zero.

Optional Parameters¶

region_of_interest¶

Default:: [0.0, 0.0, 1.0, 1.0]

Region of the image on which to perform analysis.

The value of this parameter should be a four-element list of the form [x_min, y_min, x_max, y_max], where (x_min, y_min) and (x_max, y_max) are the coordinates of the lower-left and upper-right corners, respectively, of the region of interest. Each coordinate must be normalized to the dimensions of the image and lie between 0.0 and 1.0 (inclusive).

If this parameter is set to the name of a variable, the variable’s value can be changed at any time, even while the device is running. This enables run-time refinement of analysis performance via changes to the region of interest.

If this parameter is omitted, its value is not a list, or its value is an empty list, the entire image is analyzed.

minimum_confidence¶

Default:: 0.0

Minimum confidence for a result to be considered valid.

camera_unique_id¶

If this parameter is omitted, and multiple cameras are detected, the device will fail to initialize. The error message reporting the failure will include the names and unique ID’s of all available cameras. To allow initialization to succeed, set the value of this parameter to the unique ID of the desired camera.

Note: This parameter is ignored on iOS, where the built-in, front-facing, wide-angle camera is always used.

image_logging_interval¶

Default:: 0
Example:: 60s

If greater than zero, determines the rate at which captured images are stored in the variable specified by image_data.

By enabling image logging, you can capture additional images that can later be used to improve your model. In addition, MWClient’s Image Viewer window can display the logged images and provides a convenient interface for viewing and altering the current value of region_of_interest.

image_data¶

Variable in which to store logged images, as controlled by image_logging_interval. Each image is stored as a string containing JPEG-encoded data.

autostart¶

Default:: NO

If YES, the device will start I/O automatically (as if by an implicit Start IO Device action) when the experiment starts running.

Note: Device I/O will always stop automatically when the experiment stops running, irrespective of this setting.

alt¶

Alternative device to use if this one fails to initialize (e.g. because the required hardware is not available). If #allowAltFailover is set to 0 or false, this parameter is ignored.

Placement¶

Allowed at top level:	Yes
Allowed parent:	Folder