Interactive 3D content conversion for TV

Currently 3D (auto-) stereoscopic displays suffer from the lack of 3D content. As a consequence unsupervised (fully automatic) algorithms are often used to produce 3D models from stereoscopic content. However, the resulting 3D geometry often contains large errors that result in objects being placed at the wrong depth and visible texture rendering artefacts.

A possible solution to this problem is the introduction of supervised computer vision algorithms that maximize the conversion quality for a given amount of operator input (fixed cost) or, minimize conversion effort (cost) for a given output quality (fixed quality). These algorithms are difficult to design since they require taking into account the ease with which the operator interacts with the content. However, potentially a high quality can be achieved in the construction of three-dimensional models both using stereo and for monoscopic visual data as input.

Research questions

From the above problem we can pose the following research questions:

- How can modern computer vision algorithms such as object recognition and detection, region segmentation and machine learning be used in an interactive 2D to 3D conversion system?

- Is it possible to predict the required number of user inputs (amount of work) given a known performance of the varies computer vision algorithms (e.g. detection rate)?

- Is it possible to construct a learning algorithm that improves each of the computer vision methods using the required user inputs such that the system becomes more efficient over time?

Assignment

The student will investigate the above questions and investigate algorithms to answer these questions. A proof of concept will be created using C++ and OpenCV. The operator/user efficiency of the conversion tool and its resulting 3D geometry models will be shown and tested on an autostereoscopic 3D-TV.

This project is done in close collaboration with Philips Consumer Lifestyle in Eindhoven. For part of the project time, the student will be co-located at the premises of Philips.