This document describes the communication protocol of The Eye Tribe Tracker known as the Tracker API.
Using the Tracker API requires a connected and correctly positioned Tracker Device and a running Tracker Server as explained in the Getting Started guide.
A connection must be established between a client implementation and the Tracker Server before messages can be exchanged using the Tracker API. Establishing such a connection involves opening a TCP Socket in your programming language of choice and connecting to localhost on port 6555. For a full source code example see Tutorials section.
The Tracker API protocol consists in the exchange of JSON formatted messages over TCP sockets. The Tracker API protocol is explained in the following.
The request message is a generic structure, which has three mandatory attributes:
category: The query category. The different types of requests are grouped into different categoriesrequest: The actual request of the message. What constitutes a valid request depends on the categoryvalues: an JSON array of parameters or a JSON object depending on the request typeConceptually, the category, request and values can be thought of as class.method.value. From this, the the general JSON structure for the client request message is defined as:
{
"category": string,
"request" : string,
"values" : [ … ] or { … }
}
| Category Type | Values |
tracker |
for SDK Tracker state and information related requests |
calibration |
for calibration related requests |
heartbeat |
for signalling heartbeats to the server |
The SDK Tracker reply message serves a dual purpose; firstly, for each request sent by the client, the tracker will provide the result of a request with reply message. Secondly, it serves as message publishing container for continuously published data, i.e. gaze data.
category: the category of the incoming message. This allows the client to differentiate between replies for a request or published datarequest: type of the requeststatuscode: HTTP status code compliant integral value denoting the overall result of the requestvalues: a JSON object of actual return values, if anyThe valid categories for replies are:
| Category Type | Values |
tracker |
for SDK Tracker state and information related requests |
calibration |
for calibration related requests |
heartbeat |
for signalling heartbeats to the server |
On error, the reply will hold suited statuscode and values object may contain:
statusmessage: string, an error description of why the request failed[get/set value]: string, an error description of why get/set request failedAll requests related to the Tracker Server are specified with the category tracker. This includes initial configuration requests to the tracker, once a client has connected to it, which consists of specifying desired frame rate, whether to push or pull and protocol version.
The Tracker category support two generic requests:
get: read specified tracker valuesset: write specified tracker values (if writable).{
"push": bool,
"heartbeatinterval": int,
"version": int,
"trackerstate": integer,
"framerate": integer,
"iscalibrated": bool,
"iscalibrating": bool,
"calibresult": object
"frame": string,
"screenindex": int,
"screenresw" : int,
"screenresh" : int,
"screenpsyw" : float,
"screenpsyh" : float
}
Below is the meaning of each state item value dependent on whether it is get or set.
| Attributes | Get | Set |
push:boolean |
Tracker server is running in push, or pull mode. | Start tracker server in push mode |
heartbeatinterval:integer |
The expected interval in millisecond heartbeats are sent from client | N/A - immutable, defined by Tracker Server |
version:integer |
The protocol version currently used by the client Defaults to latest version |
integer: The protocol version requested by the client |
trackerstate:integer |
The state of the physical tracker device, see Tracker State |
N/A - immutable, defined by SDK Tracker |
framerate:integer |
The frame rate that the tracker is running at | N/A - immutable, configured by user (config file) |
iscalibrated:boolean |
Indicates if we have a calibrated tracker. | N/A - immutable, state maintained by SDK |
iscalibrating:boolean |
Indicates if we are in calibration state. | N/A - immutable, state maintained by SDK |
calibresult:object |
Latest valid calibration result object | N/A - immutable, use for pull mode |
frame:object |
Latest frame data available, see Frame Object | N/A - immutable, use for pull mode |
screenindex:integer |
Index of screen in multi screen setup | Index of current screen |
screenresw:integer |
Screen resolution width in pixels | Screen resolution width in pixels |
screenresh:integer |
Screen resolution height in pixels | Screen resolution height in pixels |
screenpsyw:float |
Screen physical width in meters | Screen physical width in meters |
screenpsyh:float |
Screen physical height in meters | Screen physical height in meters |
getClient needs to know if the server is running and the system is calibrated. This is done by submitting the following request:
{
"category": "tracker",
"request" : "get",
"values": [ "push", "iscalibrated" ]
}
On success, the server replies:
{
"category": "tracker",
"request" : "get",
"statuscode" : 200,
"values": {
"push" : true,
"iscalibrated": true
}
}
Note that a user calibration must be performed before any on-screen gaze coordinates are transmitted.
setClient wants to start tracker. This is done submitting the following request:
{
"category": "tracker",
"request": "set",
"values": {
"push": true,
"version": 1
}
}
On successful request the server replies:
{
"category": "tracker",
"request": "set",
"statuscode" : 200
}
If a request fails the server reply with a status code 500 and a message providing further information.
{
"category": "tracker",
"request": "set",
"statuscode" : 500,
"values": {
"statusmessage" : "set failed, internal error"
}
}
If a request fails the server reply with a status code 500 and a message providing further information.
{
"category": "tracker",
"request": "set",
"values": {
"push" : false,
"version" : 1
}
}
Request fails as push is misspelled and provided version value is provided as string:
{
"category": "tracker",
"request": "set",
"statuscode": 400,
"values": {
"statusmessage": "puss, not such value exists",
"version": "only integers are supported"
}
}
When requesting the tracker device connectivity state value trackerstate it returns an integer/enum with a value as described below:
| State | Description | Value |
TRACKER_CONNECTED |
Tracker device is detected and working | 0 |
TRACKER_NOT_CONNECTED |
Tracker device is not detected | 1 |
TRACKER_CONNECTED_BADFW |
Tracker device is detected but not working due to wrong/unsupported firmware | 2 |
TRACKER_CONNECTED_NOUSB3 |
Tracker device is detected but not working due to unsupported USB host | 3 |
TRACKER_CONNECTED_NOSTREAM |
Tracker device is detected but not working due to no stream could be received | 4 |
When client connection to the SDK tracker has been established, the client will start receiving gaze data. The frame data is specified as:
{
"category": "tracker",
"statuscode": 200,
"values": {
"frame" : {
"timestamp": string, //string time representation e.g. '2014-04-15 15:28:46.628'
"time": int, //timestamp in milliseconds
"fix": bool, //is fixated?
"state": int, //32bit masked state integer
"raw": { //raw gaze coordinates in pixels
"x": int,
"y": int
},
"avg": { //smoothed gaze coordinates in pix
"x": int,
"y": int
},
"lefteye": {
"raw": { //raw coordinates in pixels
"x": int,
"y": int
},
"avg": { //smoothed coordinates in pix
"x": int,
"y": int
},
"psize": float, //pupil size
"pcenter": { //pupil coordinates normalized
"x": float,
"y": float
}
},
"righteye": {
"raw": { //raw coordinates in pixels
"x": int,
"y": int
},
"avg": { //smoothed coordinates in pix
"x": int,
"y": int
},
"psize": float, //pupil size
"pcenter": { //pupil coordinates normalized
"x": float,
"y": float
}
}
}
}
}
The state of the SDK Tracker is embedded in the "frame" object as the "state" value. The value is a masked integer and can hold several states at the same time. The state is extracted according to below table:
| State | Description | Values |
STATE_TRACKING_GAZE |
Tracker is calibrated and producing on-screen gaze coordinates. Eye control is enabled. | Mask: (0x1) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_EYES |
Tracker possibly calibrated and is tracking both eyes, including pupil and glint. | Mask: (0x2) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_PRESENCE |
Tracker possibly calibrated and is tracking presence of user. Presence defined as face or single eye. | Mask: (0x4) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_FAIL |
Tracker failed to track anything in this frame. | Mask: (0x8) true: ((state & mask) != 0) false: ((state & mask) == 0) |
STATE_TRACKING_LOST |
Tracker has failed to detect anything and tracking is now lost | Mask: (0x10) true: ((state & mask) != 0) false: ((state & mask) == 0) |
If tracker value "push" has been set to false, the client is expected to pull frame data. This is done by issuing the tracker-category request get with a value of frame:
{
"category": "tracker",
"request" : "get",
"values": [ "frame" ]
}
Note that requesting a frame can be done at any time, even when running in push mode.
To calibrate the tracker a client perform a calibration process. The API requests involved in a calibration process are encapsulated in the calibration category and are described below.
startThe start request prepares the tracker for a new calibration.
{
"category": "calibration",
"request" : "start",
"values": { "pointcount": integer }
}
pointcount: The number of calibration points the tracker needs to sample at to produce a valid calibration. The minimum number of calibration points allowed is 7, but it is recommended to use 9 or more.
{
"category": "calibration",
"request" : "start",
"statuscode" : 200
}
pointstartMark the beginning of a new calibration point for the tracker to process.
{
"category": "calibration",
"request" : "pointstart",
"values": { "x": integer, "y": integer }
}
x: x-coordinate of point y: y-coordinate of point
{
"category": "calibration",
"request" : "pointstart",
"statuscode" : 200
}
pointendMark the end of processing a calibration point.
{
"category": "calibration",
"request" : "pointend"
}
After each endpoint-request the tracker will reply with:
{
"category": "calibration",
"request" : "pointend",
"statuscode" : 200
}
For the final successful endpoint request the tracker will send back the calibration result. The data format of the calibresult is described further below.
{
"category": "calibration",
"request" : "pointend",
"statuscode" : 200,
"values": { "calibresult" : [ { … } ], … }
}
abortAt any point during the calibration an abort request may be sent to cancel an ongoing sequence. If the tracker have had a previous working calibration it will be reinstated.
{
"category": "calibration",
"request" : "abort"
}
{
"category": "calibration",
"request" : "abort",
"statuscode": 200
}
clearRemoves the current calibration from the tracker.
{
"category": "calibration",
"request" : "clear"
}
{
"category": "calibration",
"request" : "clear",
"statuscode" : 200
}
When the calibration process is completed, the last pointend request returns a calibresult object. This object is defined as:
{
"result": bool, // was the calibration successful?
"deg": float, // average error in degrees
"degl": float, // average error in degs, left eye
"degr": float, // average error in degs, right eye
"calibpoints": [
{
"state": int, //state of calibration point
"cp": { //coordinates in pixels
"x": float,
"y": float
},
"mecp": { //mean estimated coords in pixels
"x": float,
"y": float
},
"acd": { //accuracy in degrees
"ad": float,
"adl": float, //left eye
"adr": float //right eye
},
"mepix": { //mean error in pixels
"mep": float,
"mepl": float, //left eye
"mepr": float //right eye
},
"asdp": { //average std deviation in pixels
"asd": float,
"asdl": float, //left eye
"asdr": float //right eye
}
},
...
]
}
The calibpoints array contains a complete list of the results from each calibration point. The state describes the status of each point. The values are defined as:
| Value | Description | State |
0 |
No useful tracking data was gathered for the calibration point. The calibration point should be resampled. | RS_NODATA |
1 |
The data gathered for the calibration point is of questionable quality. A resample can be beneficial. | RS_RESAMPLE |
2 |
The calibration point is okay. No resample is required. | RS_VALID |
Once a client is connected to the tracker, it must send heartbeat requests to make itself known and prove that it is still alive. These heartbeats must be sent at a rate matching the heartbeatinterval value of the tracker category. The tracker will reply to each heartbeat allowing the client to recognize if connection have been lost.
heartbeatSince no operation is performed within the heartbeat category no special requests are currently available.
{
"category": "heartbeat"
}
{
"category": "heartbeat",
"statuscode": 200
}
Each reply from the tracker holds a status code. The status codes are compliant to standard HTTP status codes but The Eye Tribe SDK also define a few status codes of their own.
Tracker State ChangeThe state of the connected tracker device has changed. Connected clients should update themselves.802
| Title | Description | Value |
| Calibration Change | Calibration state has changed. Connected clients should update themselves. | 800 |
| Display Change | Active display index has changed in multi screen setup. Connected clients should update themselves. | 801 |
| Tracker State Change | The state of the connected tracker device has changed. Connected clients should update themselves. | 802 |