Free-viewpoint 3DTV : system architecture, rendering techniques and perceptual improvements

Research and development of 3DTV has been strongly fueled by the rapid market acceptance of stereo 3D movies and underlying technology so that 3DTV is entering our everyday lives. As a result of this, the R & D community is already working on the next upcoming stages, such as multi-view processing. Multi-view in 3D is very attractive because the objects can be inspected from various angles. It means that the user can choose the point from which he prefers to watch the 3D video or object. We expect that this feature is highly attractive for both professional and consumer type of applications. For multi-view processing, many questions still have to be solved in order to optimize and standardize a whole 3DTV chain based on this principle. In this chapter, we concentrate specifically on various aspects of the free-viewpoint 3DTV chain. This chapter commences with introducing the concept of free-viewpoint 3DTV and the challenges that are still present in the domain. Since the quality of the synthesized view is of vital importance for the viewer, we assess the obtained quality by step-by-step analysis of the state-of-the-art algorithm and we also consider the influence of compression on the generated view. Taking into account the current developments on free-viewpoint in the MPEG group, we compare our view synthesis algorithm with the algorithm currently used in MPEG. Because most of currently available 3DTVs employ the stereo imaging, we present solutions for converting texture and depth 3D video to stereo video, and we also consider the parameters for stereo image generation for 3DTV. The chapter is concluded with our vision concerning the near future developments in research on free-viewpoint 3DTV and possible emerging solutions.