Etron's Eagle Eyes

I have tested the light structure depth camera by using Intel Perceptual Computing SDK.

I have a kinect II which comes from with MS XBOX ONE and waiting for the USB power cable release, the kinect II is build by TOF depth camera.

I contacted with Etron and try to get a stereo vision depth camera for the OFmT. How lucky, Etron support an engineering sample for the development. Unfortunately, this engineer sample has some hardware issues and for engineer test only.

I did not complete this stereo vision project and waiting for a new stereo vision sample from Etron if there is any.

After test the stereo vision camera, I found it will work on OFmT. The camera is not accurate as light structure depth camera but enough to work.

As you see the video, it can segment the hand from the background and also segment fingers by depth.

The noise in this video comes from the hardware issues.

Optical Floating multi-Touch -- Intel® Perceptual Computing SDK 2013

Issues of  Intel® Perceptual Computing SDK 2013 :

1. The angle of view is too wide, user's hand will be exhausted when willing to move the finger to edges.
2. The resolution is too low, this make the finger movement become too sensitive.
3. The gesture module for fingertips is not accurate and generate noise.
4. The runtime library is not work, user must download full version of SDK. What's the meaning of runtime library?
5. The color camera is not working properly with other applications.
6. The depth camera world coordinator center is between color and depth camera, and the depth is base on the center of depth camera. Programer must re-caculate the world coordinator  base on camera coordinator, what's meaning of the world coordinator?

Original depth map finger tips catcher

Touch tester demo

Switch between APPs demo

Picture viwer demo

Game demo

Optical Floating multi-Touch -- OFmT without Depth map

OFmT without depth map 示意圖

OFmT without depth map sensor

OFmT without depth map  原理影片

OFmT without depth map win8 touch test

OFmT without depth map win8 touch game demo -- Cube Cooker

Optical Floating multi-Touch – OFmT with depth map

OFmT with depth map 基本示意圖

基本上, depth map 會提供三個值(x,y,d), 至於 depth map 的產生方式不在此贅述

x : camera 的 x 值

y : camera 的 y 值

d : 物件距離 camera 的距離

以 OFmT 的觀點來看, 從 display surface 到物件的座標假設為(X,Y,D)

因此 camera 角度不同轉換到 display surface, D 的值也會不同

例一 : camera 90 度向上, d=D, 以 OFmT 的觀點來看, 實際上不需要用到 depth map

例二 : camera  0 度, 這種用法可以用在像 Google Glass 特殊的應用, 一般應用此法並不實際

例三 : camera 有角度, 角度不討論, D 的算法與例二類似, 在此不討論. Camera 置於 display surface 上方或下方也不討論

以例三而言是最實際的用法, 問題是 Camera 的角度如何固定.

應用一 : 外接螢幕或 AIO PC. 在這個領域 Camera 角度可以容易的固定

應用二 : Notebook, Camera 要放在 keyboard 上方, Camera 角度也可以固定

應用三 : Tablet, 基本上 tablet 與 mobile phone 的螢幕較小, 直接用 touch screen 就可以了. Floating touch 的意義不大.

應用一與應用二的 Camera 角度有可能因為機構的不同造成不同的機型 Camera 角度不同, 因此 D 的算法也會不同, 有可能需要客製化, 我相信這是為何大部分現有國外的解決方案會往 gesture 發展而非 OFmT. 對台灣的軟體廠商而言這反而提供了商業機會.

OFmT 與 gesture 比較的最大優點為 OFmT 只需要知道距離因此程序簡單, performance 相對好. Gesture 因為需要比對手型, 程序當然相對複雜. OFmT 的缺點是 Camera 角度不可由使用者自行調整. 但是如果 Camera 角度被調整了, 根據手眼協調原則仍然是可以用的, 只是