The Eye Tribe

Re: bkb is a program to control keyboard/mouse with eyes

2016-11-23T07:53:18+02:00

Hello! I am very interested to know how this keyboard function works. I would like to map only a few keys at a time. For example:
If the person is fixes the gaze to the right corner of the screen then the right mouse or key button takes action. The same happens if the person gazes to the left then the left arrow key will be clicked. Could this be done with the program?

Statistics: Posted by skatajuanvic5185@hotmail.com — 23 Nov 2016, 07:53

Re: bkb is a program to control keyboard/mouse with eyes

2015-07-14T23:22:15+02:00

Video in Russian about the person with the ALS who uses the bkb program: https://youtu.be/mMw2o6O0jRk

Statistics: Posted by skataMastaLomaster — 14 Jul 2015, 23:22

Re: bkb is a program to control keyboard/mouse with eyes

2015-07-14T22:56:46+02:00

To Julius:

OptiKey_Julius wrote:
If it helps I use the following method for point selection (mouse action) within OptiKey...

Points are tested to check for a sequence of adjacent points which all fall within a "lock-on" radius where the centre point is calculated...

The idea behind your method seems to be similar to what the other programmers do.
In particular, your "fixation radius" is called gDISPERSION_LIMIT in my program;
the "fixation period" is called FIXATION_LIMIT.

Thank you for sharing this, but I consider the fixation detection as solved in my program.
What I really like in your program is that it's more user friendly. And this is something that lays behind the algorithms and coding.

Thank you.

Statistics: Posted by skataMastaLomaster — 14 Jul 2015, 22:56

Re: bkb is a program to control keyboard/mouse with eyes

2015-07-14T22:41:45+02:00

To Jeff:

JeffKang wrote:
While on the topic of dwell progress, a thought for the future could be some sort of dwell accumulation, where you can return to keep building after you leave.

I thought: why do you need this?
The only reason I see is: the device cannot precisely detect the point you stare at.

The same happens with me.

So I use the following approaches:

1. Make bigger buttons.
Of course, if these buttons belong to your program, if you can manage them. In the latest build I've made the toolbar buttons customizable (in terms of their count/size).

2. Magnify the area with the button you need
Yes, if the button is small, and you cannot change its size, magnify the part of the screen.
(Note that the keyboard by default magnifies its parts now).

3. Smooth the cursor movements
The part that is the most math-related.
If your dwell point jumps from one button to another and back - you can filter and smooth its movements. This however might introduce some delays. So it's always a compromise between smothless and responsibility.
As for now I use the dumb "moving average" method which is however smart enough for allowing fast movements from one part of the screen to another.

Shortly, I think that these 3 methods is enough now. The dwell accumulation will require making changes in the "basement" of the program, and would not give much advantages. Sorry, I have to say "no".

Statistics: Posted by skataMastaLomaster — 14 Jul 2015, 22:41

Re: bkb is a program to control keyboard/mouse with eyes

2015-07-01T13:04:43+02:00

If it helps I use the following method for point selection (mouse action) within OptiKey...

Points are tested to check for a sequence of adjacent points which all fall within a "lock-on" radius where the centre point is calculated from the sequence of points, and that they cover a minimum amount of time (the "lock-on period"). If those criteria are met then the user has kept their focus within an acceptable area of the mid point of a circle and a fixation can begin. The fixation radius then changes to the "fixation radius", which is bigger and I then test to see if there is a sequence of adjacent points that are all within the "fixation radius" and cover a minimum amount of time (the "fixation period"). If that is true then a selection can occur at the centre point. If ANY points stray outside of either area (during lock-on or selection) then I discard the whole thing and the user can start again. The tricky bit was coming up with radii where the "lock-on" area is small enough to make the centre point accurate, but large enough to make it possible to keep all points within the circle for X ms. After that I widen the circle to make it easier to complete the fixation.

I can take you through the code if that would help you - it's here and uses a bit of Rx: https://github.com/JuliusSweetland/Opti ... nSource.cs

Statistics: Posted by skataOptiKey_Julius — 01 Jul 2015, 13:04

Re: bkb is a program to control keyboard/mouse with eyes

2015-06-14T00:45:51+02:00

Nice.
While on the topic of dwell progress, a thought for the future could be some sort of dwell accumulation, where you can return to keep building after you leave.
I'm mentioned in a thread for the GazeSpeaker program, and dwell accumulation is now in there.
viewtopic.php?f=22&t=254&p=1124&hilit=accumulate&sid=ceb818ba18d0602c668d1a8a571d9871#p1124

That's for buttons though.
Dwell accumulation seems more difficult for general magnification and clicking, where the target is not easily known yet.

How I picture it:
There's an invisible large circle around the fixation point, and it's like the spray-paint / airbrush tool of Paint.

Let's say there's a button in the top left corner.
You look in the top left in the general area of the button, and the progress builds.
The invisible circle "spray paints" the area, and the invisible paint gets thicker the longer it stays in 1 spot.

If you look completely away from that top left area, you start building from the beginning at the new location (top left area still has memory and paint though).

If you look away, and come back to the exact same area, the invisible circle overlaps with some of the painted area.

Where the progress bar resumes depends on how much previous paint is within the invisible circle.

Once an area within the invisible circle has achieved a sufficient amount of paint, it will magnify or click at the current fixation point.

If you come back to almost the exact top left spot, there's already a lot of paint, and thus, the progress bar will almost be done.

If you come back to a little lower right of the previous top left area, and 50% of the area hasn't been painted yet, the progress bar will resume at a lower point than directly looking at and painting the previous top left area.
Though, it will still be further ahead than had you started fixating and spray painting in a completely new area.

Yea.. I don't know how it would work lol.

Statistics: Posted by skataJeffKang — 14 Jun 2015, 00:45

Re: bkb is a program to control keyboard/mouse with eyes

2015-06-13T11:24:51+02:00

JeffKang wrote:
Thanks for the work, MastaLomaster.

My pleasure, Jeff.
Please check the latest build, it now contains the fixation progress bar in the cursor window:

So you know whether your eyes are considered to be fixating or not when you stare at some point.

Also there is a "Sleep" button. You need to fixate your eyes 3 times on it for the program to switch into the black screen mode. This is for the persons who can't switch off the monitor, but need it to have rest.
Return to the main operation mode - fixate your eyes on the right-bottom corner of the screen (with 3 small points). Also 3 times.

Statistics: Posted by skataMastaLomaster — 13 Jun 2015, 11:24

Re: bkb is a program to control keyboard/mouse with eyes

2015-06-10T22:42:39+02:00

Thanks for the work, MastaLomaster.

Statistics: Posted by skataJeffKang — 10 Jun 2015, 22:42

Re: bkb is a program to control keyboard/mouse with eyes

2015-06-04T09:29:12+02:00

shehan.desilva@gmail.com wrote:
i have been having problems with the pointer getting stuck on top of a key. the computer remains fully responsive with no CPU spikes etc. when i have the eye tribe GUI running, these "freeze moments" can be associated with blinking or when the tracker loses the eyes. given this, it might be an idea to gracefully handle this situation by graying out or completely dissapearing the pointer, reappearing when eye tribe finds the eyes.

You were right. It is necessary to monitor the quality of eye tracking. So far I've added the following metrics window:

This [running] chart shows the speed of the eye movements.
1 - the movements are above threshold, no fixation
2 - the movements are below threshold, waiting for the eyes to stay still for a predefined time to click or perform other action. The fixation time is defined in the "Settings" dialogue, as well as a movement threshold.
3 - grey area: the tracker has lost your eyes. (Blinking, head movement, bad distance, lighting conditions, etc.)

The window can be switched on/off in the settings dialogue.

Statistics: Posted by skataMastaLomaster — 04 Jun 2015, 09:29

Re: bkb is a program to control keyboard/mouse with eyes

2015-06-01T09:18:50+02:00

New build (revision D) includes larger toolbar buttons, keyboard magnifying, progress bar on toolbar, more settings.

Statistics: Posted by skataMastaLomaster — 01 Jun 2015, 09:18

Re: bkb is a program to control keyboard/mouse with eyes

2015-04-07T10:16:34+02:00

I'm happy the program is useful for you.

Unfortunately I'm not able to make significant changes right now, will return to this later.
To be honest, the issue with losing the eyes - is an issue with the device, not with the high-level software.
If your engine has oil leaks - it is good to have the oil pressure lamp in your dashboard. But you'd better repair or replace the engine.

Regards,
Mikhail.

Statistics: Posted by skataMastaLomaster — 07 Apr 2015, 10:16

Re: bkb is a program to control keyboard/mouse with eyes

2015-03-29T09:01:47+02:00

This is an excellent piece of software! thank you!.

i am using it with the eye tribe tracker.

i have been having problems with the pointer getting stuck on top of a key. the computer remains fully responsive with no CPU spikes etc. when i have the eye tribe GUI running, these "freeze moments" can be associated with blinking or when the tracker loses the eyes. given this, it might be an idea to gracefully handle this situation by graying out or completely dissapearing the pointer, reappearing when eye tribe finds the eyes.

thanks again, this is a wonderful tool

Statistics: Posted by skatashehan.desilva@gmail.com — 29 Mar 2015, 09:01

Re: bkb is a program to control keyboard/mouse with eyes

2015-02-09T12:47:43+02:00

f.decillis@unicampus.it wrote:
Hi, I'm a biomedical engineer and I'm working with the EyeTribe device, I did some tests with your program and I've to say you have done an excellent job
Now I would like to change some things with Visual Studio 2013, such as the choice of the device at the start of the program, but in the "source" folder of the repository in addition to .cpp and .h files I don't see the .sln file of the project to create a new .exe file
how can I do?
thank you

Hi.

These configuration files (.sln,vcxproj, etc.) depend on the directory structure I use.
You'd better create a new empty C++ project, add all the .cpp and .h and .rc files there.
Then define YOUR include directory (mine is, for example, "D:\TobiiGazeSdk402\include").
Don't forget other settings from the "Compiling the source codes" part.

Statistics: Posted by skataMastaLomaster — 09 Feb 2015, 12:47

Re: bkb is a program to control keyboard/mouse with eyes

2015-01-23T19:08:27+02:00

MastaLomaster wrote:
bkb is a program to control keyboard/mouse with eyes
It supports The Eye Trybe eye tracker, TobiiREX eye tracker, and any device that can move a mouse cursor (e.g. an airmouse).

Program and the source code are available for download here:
https://github.com/MastaLomaster/bkb

Installation
Just unzip the bkb32c-English.zip to any folder. Make sure that this folder remains the working directory of the program. Otherwise the program won't load messages.bkb and keyboard.bkb files, and you'll get Russian interface instead of English one.

(updated)
To run the program you also need Microsoft Visual C++ Redistributable for Visual Studio 2012 Update 4. It can be downloaded here:

http://www.microsoft.com/en-us/download ... x?id=30679

If it is not installed, you'll get the error message complaining that the file "msvcrt110.dll" cannot be found.

Using the program with The Eye Tribe tracker

The "Eye Tribe Server" program must be running. Also you need to calibrate the device with the "Eye Tribe UI" before running the bkb32c.exe

Using the program with the Tobii REX eye tracker

(updated)
You need the "TobiiGazeCore32.dll" file from the Tobii Gaze SDK 4.0 to be copied to the working directory of the program. By the way, I found it in my "C:\Program Files (x86)\Tobii\Tobii EyeX" directory after installing the "Tobii Eye Experience". Alternatively, Gaze SDK 4.0 can be downloaded from: http://developer.tobii.com/downloads/ (registration required). Look for the "TobiiGazeSdk-CApi-4.0.X.XXX-Win32" file, where X-current release numbers.

Before starting the program, visit Windows Control Panel, run the "Tobii EyeX Settings (32 bit)" program, and calibrate the device.

Keyboard click sounds
There is a click sound when you press the keyboard buttons. If you don't like the sound, place a WAV-file with the desired sound into the working directory of the program and name it "click.wav".

Basic work principles
After program started and a supported device is selected, you may see the toolbar on the right side. If you ise an eye tracker, the transparent window with the cursor will be shown, it will follow your eyes movements. When using an [air]mouse, the regular cursor is used. To select a tool fixate your eyes on the tool button.

Take a look at these videos to understand the modes of operation:

http://youtu.be/O68C4d2SNC8

**IMPORTANT**: choose Swahili language to watch English subtitles. Sorry, I don't know other ways to switch off the subtitles by default.

http://youtu.be/rqcN9IZ39_4

Known issues
- no easy way to exit the program. One have to close windows in the task bar
- doesn't work with fullscreen applacations so far
- doesn't work with the Metro-style interface of Windows 8/8.1, you have to use good old desktop
- drag-and-drop doesn't work in some cases, for example you cannot move desktop icons on some PCs
- windows moved and doesn't work properly after logout/switch user
- you cannot define timings (keyboard press, fixations, etc.) [fixed]
- impossible to click with a mouse holding the keyboard button pressed (e.g. Ctrl + click) [fixed]
- and many more small things....

Translate to other languages:
One can easily translate the User Interface and modify a keyboard. Just edit the "messages.bkb" and "keyboard.bkb" files. These are text unicode files. But (!) the file format and contents can be changed in future!!!

Compiling the source codes
(updated)
As for now, you have to use Microsoft Visual Studio 2012 (latest update preferred). This is due to the fact that the libraries used (from the Tobii Gaze SDK 4.0) are compiled the same way.

In the project properties enable unicode support

Include directories must contain the "include" one from the Tobii Gaze SDK 4.0. Gaze SDK 4.0 can be downloaded from: http://developer.tobii.com/downloads/ (registration required). Look for the "TobiiGazeSdk-CApi-4.0.X.XXX-Win32" file, where X-current release numbers.

No Tobii Gaze SDK libraries needed during the compilation/build.

You need just standard Windows libraries: Ws2_32.lib, winmm.lib,Msimg32.lib

The project must be linked dynamically to the MSVCRT110.dll, if you link statically, this will be in conflict with the Tobii Gaze SDK libraries used!

Hi, I'm a biomedical engineer and I'm working with the EyeTribe device, I did some tests with your program and I've to say you have done an excellent job
Now I would like to change some things with Visual Studio 2013, such as the choice of the device at the start of the program, but in the "source" folder of the repository in addition to .cpp and .h files I don't see the .sln file of the project to create a new .exe file
how can I do?
thank you

Statistics: Posted by skataf.decillis@unicampus.it — 23 Jan 2015, 19:08

Re: bkb is a program to control keyboard/mouse with eyes

2014-08-11T22:03:13+02:00

No, the repeating click is in the main toolbar. The tool is called "Left,.."

Statistics: Posted by skataMastaLomaster — 11 Aug 2014, 22:03