SearchImagesVideoTranslateMail
Login
Follow us on

Press Center / Blog

Yandex Labs collaborated with Carnegie Mellon University to build a new personalized TV experience

9 April, 2013

At Yandex Labs, we had a chance to work on a 3-month practicum with students and Professor Ian Lane from Silicon Valley department of Carnegie Mellon University. The project was ambitious but also fun: we wanted to build a new TV experience - personalized and interactive. We developed an application for TV that shows personalized content on a TV screen and allows the users to easily manipulate and interact with the content using hand gestures. The app is still a prototype and is not available for download, but we made this video to share our ideas with you.

The app brings users’ social network streams to their TV screens and allows them to navigate over this information using hand gestures. It is built on Mac OS X platform and we used Microsoft Kinect for gesture recognition.

The application features videos, music, photos and news shared by the user’s friends on social networks in a silent ‘screen saver’ mode. As soon as the user notices something interesting on the TV screen, they can easily play, open or interact with the current media object using hand gestures. For example, they can swipe their hand horizontally to flip through featured content, push a “magnetic button” to play music or video, move hands apart to open a news story for reading and then swipe vertically to scroll through it.

To train gesture recognition, the Carnegie Mellon students together with Professor Ian Lane evaluated several machine learning techniques, which included Neural Networks, Hidden Markov Models and Support Vector Machines (SVM), with SVM showing 20% better accuracy. They put a lot of effort in building a real training set – they collected 1,500 gesture recordings, each gesture sequenced into 90 frames, and manually labeled from 4,500 to 5,600 examples of each gesture. By limiting the number of gestures to be recognized at any given moment and taking into account the current type of content, the students were able to significantly improve the gesture recognition rate.

We have been thinking of controlling a social application with gestures for quite a while. When we found a team of like-minded enthusiasts, we took this opportunity and did a nearly three-month research project. The results of this effort were quite impressive and now we are looking whether we can implement them in a real life application.

Yandex N.V.
Registered Office in Amsterdam
Address: Schiphol Boulevard 165
1118 BG Schiphol
The Netherlands
tel.: +31 (0) 20 206 6970
Yandex LLC
Headquarters in Russia
Office in 
Moscow
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
16, Leo Tolstoy St., Moscow 119021, Russia
Reception
tel. +7 495 739-70-00, fax +7 495 739-70-70
Marketing & Media Relations
pr@yandex-team.ru
Investor Relations
tel. +7 495 974 3538
Office in 
Saint Petersburg
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
Piskarevskiy prospekt, building 2, block 2, 4th floor, Benois Business Centre, 195027, Saint Petersburg, Russia
Reception
tel. +7 812 633-36-00, fax +7 812 633-36-99
Toll-free number for regional users 8 800 333-9639 (or 8 800 333-YNDX)
Office in 
Ekaterinburg
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
10 Hohryakova St., Ekaterinburg 620014, Russia
Reception
fax +7 343 385-01-99
Sales Office
tel. +7 343 385-01-00 uralsales@yandex-team.ru
Toll-free number for regional users 8 800 333-9639 (or 8 800 333-YNDX)
Office in 
Novosibirsk
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
35 Krasnoyarskaya St., Novosibirsk 630004, Russia
Reception
fax +7 343 385-01-99
Sales Office
tel. +7 383 230-43-06 sales-nsk@yandex-team.ru
Office in 
Kazan
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
6 Spartakovskaya Street, 11th floor, right wing, Kazan 420107, Russia
Reception
tel. +7 843 524-71-71
Sales Office
kzn@yandex-team.ru
Office in 
Rostov-on-Don
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
70D, Gvardeysky Business Centre, Dolomanovsky Lane, Rostov-on-Don, 344011, Russia
Tel. +7 (863) 2-688-300
Office in 
Nizhny Novgorod
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address:
Lobachevsky Plaza, ulitsa Alekseevskaya, 10/16, Nizhny Novgorod, 603000
Tel. +7 (831) 233 06 06
Office in 
Kiev
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
Suite 30, 19–21 Bohdana Khmelnytskoho Street, Kyiv, 01030 Ukraine
Reception
+38 044 586 41 48, fax. +38 044 586 41 48 ext. 6665
Marketing & Media Relations
pr@yandex-team.com.ua
For general inquiries
info@yandex-team.com.ua
Client Service Department
+38 044 586 41 48 ext. 2601
Skype
yandex.ukraine (reception)
Office in 
Odessa
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
Polsky Uzviz 11, Morsky-2 Business Center (8th floor), Odessa 65026, Ukraine
Reception
tel./fax: +38 0487 37 44 10, +38 0445 86 41 48
Marketing & Media Relations
pr@yandex-team.com.ua
For general inquiries
info@yandex-team.com.ua
Client Service Department
+38 044 586 41 48 ext. 2601
Skype
yandex.ukraine (reception)
Office in 
Simferopol
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
1a (4th Floor), Kazanskaya St., Simferopol
Reception
tel. +7 495 739 70 00, fax +7 495 739 70 70
Office in 
Minsk
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
Office 308, Rubin Plaza Business Center, 5 Dzerzhinskogo prospekt, Minsk, 220036, Belarus
Reception
+375 17 328-19-61, fax. +375 17 328-15-14
Office in 
San Jose
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
2001 Gateway place, Suite 400W, San Jose, CA, 95110
Tel. +1-617-398-7870
Office in 
Istanbul
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address
Mecidiyeköy Yolu Caddesi No: 12,
Trump Towers 2. Kule, 15. Kat, Ofis No: 1501-1502
34387 Mecidiyeköy, Şişli, İstanbul - Türkiye
Tel. +90 212 386 87 60 (pbx), fax. +90 212 284 46 48
Marketing & Media Relations
pr@yandex.com.tr
Contact Us
Office in 
Lucerne
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address:
Citybay Business Center, Werftestrasse 4, CH 6005 Lucerne, Switzerland
Tel. +41-41-248-08-60, fax. +41-41-248-08-63
Office in 
Zürich
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address:
Odeonhaus Limmatquai 2, 8001 Zürich, Switzerland
Tel. + 41 44 252 50 00
Office in 
Berlin
  • Moscow
  • Saint Petersburg
  • Ekaterinburg
  • Novosibirsk
  • Kazan
  • Rostov-on-Don
  • Nizhny Novgorod
  • Kiev
  • Odessa
  • Simferopol
  • Minsk
  • San Jose
  • Istanbul
  • Lucerne
  • Zürich
  • Berlin
Pic / Map
Address:
Karl-Liebknecht-Straße, 1, 10178 Berlin, Germany
enEn