TET Java SDK & processing.org

[hallo]

Hi everybody!

I was wondering, if anybody successfully imported the TET Java Library in Processing?
Any ideas how this could work?

Thanks!
Philipp

[m.w.a.wijntjes]

Not yet, I'm also trying it out. But feel free to help. What I got now is:
1) make a .jar file of the java sdk (I did it in Eclipse, if you want I can send it), and put it into a folder "Code", in you sketch folder
2) Get the gson package from somewhere and also put it in the folder
3) open the EyeTribeUI, and perform calibration (server will now also start)
4) Run the code I paste below
5) It does actually connect, but it outputs weird hexadecimal stuff.

Note that I'm not so experienced, so if anyone can help, please do!

Processing code:

Code: Select all

import com.google.gson.Gson;
import com.theeyetribe.client.*;
void setup() {
  GazeManager gm=GazeManager.getInstance();
  boolean success = gm.activate(GazeManager.ApiVersion.VERSION_1_0, GazeManager.ClientMode.PUSH);
  GazeListener gazeListener = new GazeListener();
  gm.addGazeListener(gazeListener);
  size(640, 360);
}

void draw() {
}

private static class GazeListener implements IGazeListener
{
  @Override
    public void onGazeUpdate(GazeData gazeData)
  {
    System.out.println("weird hexadecimal stuf: "+gazeData.toString());
  }
}


[Anders]

For the record, the TET Java SDK is open source and available from the release section of the GitHub repo. Package includes Gson, so no need for additional files.

Forthermore, reading the basic introduction to the Java SDK on the developer website may help you out.

BR,
Anders

[m.w.a.wijntjes]

Thanks, Anders, I hadn't seen that one. I tried it out and got an error due to Processing (not your fault): for some reason it should be compiled for Java 1.6, and this one is later. So compiling it yourself (e.g. with Eclipse) will solve this problem.

[Anders]

... it should be compiled for Java 1.6, and this one is later

That must be an error from our side. The Java SDK should be 1.6 compliant to ensure support for Android. I'll look into that and make sure it is fixed for the next release.

[hallo]

Thanks for your replies!
I had the same problem with the error because of the Java 1.6 .jar file, but compiled it myself in eclipse, as m.w.a.wijntjes said.
To the gson thing: You have to put a gson.jar into the code folder, but you don't have to import it in the code.
I followed the steps and managed to receive the various eye tracking data in processing and added some lines to the class, so they get converted into datatypes, that can be used by processing functions.

Main

Code: Select all

//IMPORT LIBRARIES
import com.theeyetribe.client.*;

//SETUP
void setup() {
  GazeManager gm=GazeManager.getInstance();
  boolean success = gm.activate(GazeManager.ApiVersion.VERSION_1_0, GazeManager.ClientMode.PUSH);
  GazeListener gazeListener = new GazeListener();
  gm.addGazeListener(gazeListener);
  size(800, 800);
}

//DRAW
void draw() {
ellipse(gazeRaw.x, gazeRaw.y, 10, 10);
}

Class

Code: Select all

boolean fixation;
int msTimeStamp, state;
String timeStamp;
float lPupilSize, rPupilSize;
PVector
gazeRaw, gazeSmooth,
lPupilPos, rPupilPos,
lGazeRaw, rGazeRaw,
lGazeSmooth, rGazeSmooth;     
     

private class GazeListener implements IGazeListener
{
  @Override
    public void onGazeUpdate(GazeData gazeData)
  {   

      //GENERAL//
     
      //integer: state
      state = gazeData.state;
     
      //integer: time stamp in ms
      msTimeStamp = int(gazeData.timeStamp);
     
      //string: time stamp
      timeStamp = gazeData.timeStampString;

     
      //BOTH EYES//
     
      //boolean: is fixation?
      fixation = gazeData.isFixated;
     
      //from double to PVector: raw gaze coordinates of both eyes
      gazeRaw = new PVector((float)gazeData.rawCoordinates.x, (float)gazeData.rawCoordinates.y);
     
      //from double to PVector: smooth gaze x and y coordinates of both eyes
      gazeSmooth = new PVector((float)gazeData.smoothedCoordinates.x, (float)gazeData.smoothedCoordinates.y);

     
      //LEFT AND RIGHT EYE//
     
      //from double to float: pupil size of left and right eye
      //lPupilSize = (float)gazeData.leftEye.pupilSize;
      //rPupilSize = (float)gazeData.rightEye.pupilSize;
      ////ERROR: CANNOT CAST FROM DOUBLE TO FLOAT     

      //from double to PVector: pupil position of left eye
      lPupilPos = new PVector((float)gazeData.leftEye.pupilCenterCoordinates.x, (float)gazeData.leftEye.pupilCenterCoordinates.y);
     
      //from double to PVector: pupil position of right eye
      rPupilPos = new PVector((float)gazeData.rightEye.pupilCenterCoordinates.x, (float)gazeData.rightEye.pupilCenterCoordinates.y);
     
      //from double to PVector: raw gaze coordinates of left eye
      lGazeRaw = new PVector((float)gazeData.leftEye.rawCoordinates.x, (float)gazeData.leftEye.rawCoordinates.y);
     
      //from double to PVector: raw gaze coordinates of right eye
      rGazeRaw = new PVector((float)gazeData.rightEye.rawCoordinates.x, (float)gazeData.rightEye.rawCoordinates.y);
     
      //from double to PVector: smooth gaze of left eye
      rGazeSmooth = new PVector((float)gazeData.leftEye.smoothedCoordinates.x, (float)gazeData.leftEye.smoothedCoordinates.y);
     
      //from double to float: smooth gaze coordinates of right eye
      rGazeSmooth = new PVector((float)gazeData.rightEye.smoothedCoordinates.x, (float)gazeData.rightEye.smoothedCoordinates.y);
     
  }
}

Problems:
1. Pupil size can't be converted from Double to float. Error: "Cannot cast from Double to float"
2. Processing uses the gaze coordinates from the whole screen in the applet window.
3. As state, I get the numbers 4 or 7. What does that mean?

[Anders]

1. Pupil size can't be converted from Double to float. Error: "Cannot cast from Double to float"

This is a general Java issue. You are trying to implicitly cast a Double object to float primitive. Try using Double.floatValue() instead.

2. Processing uses the gaze coordinates from the whole screen in the applet window.

Gaze coordinates are relative to screen size. If you want to map these into a smaller window, then you need to do that yourself using size difference between screen and desired window.

3. As state, I get the numbers 4 or 7. What does that mean?

I advice you to consult the API documentation if in doubt about a value. The 'state' is a bitmasked value. Check out GazeData.stateToString() and you'll see how to make sense out of it.

Hope that helps!

/Anders

[m.w.a.wijntjes]

Thanks both of you, it works great! Just use full screen presentation in Processing (better than simply using a large screen, because in that case there is still an offset from you dock and menu bar, at least for mac). I'll play around with it a bit more and keep you updated if I find something worth sharing.

[JorgeCardoso]

Hi,

I'm about to start developing a Processing library for the Eye Tribe.

Is there any recommendation/feature that you would like to see?

Thanks,
Jorge

[m.w.a.wijntjes]

Hi Jorge,

Very good idea, please keep us (at least me) updated about this. For now, what I have works fine for me. I only have to program an experimental flow, with good data handling, and a script to show a simple heat map and maybe some other analysis.

Ideally, I would like to have calibration within Processing. Now I use the eyeTribeGUI, but ideally you would like to do some intermediate calibrations, or checks that you are still recording a reasonably accurate signal. I think that is a central thing program if you want to make a good library.