Camera-to-model back-raycasting for extraction of RGB-D images from pointclouds

Conventional raycasting methods extract 2D-images from pointclouds in two main steps. The pointcloud is voxelized and then, rays are casted from a virtual-camera center towards the model. The value for each pixel in the resulting image is calculated based on the closest non-empty voxel intersected with the corresponding ray. Both voxelizing and such raycasting limit the quality (resolution) of the extracted image and impose high memory demands. In this paper, we propose an alternative backraycasting method, where rays are casted from the model towards the virtual-camera center and intersecting an image plane. This does not require any voxel grid to be generated. Moreover, this method allows to obtain images with any required resolution with all the points involved. Besides this, a neighbours-consistency technique is introduced to enhance the resulting image quality. The proposed method has been evaluated based on several criteria and for various resolutions. Evaluation results show that the proposed method compared to the conventional approach executes upto 49 times faster and improves PSNR and SSIM metrics for the resulting images by 26% and 12%; respectively. This improvement is beneficial for such domains as feature matching, edge detection, OCR and calibration. To enable researchers generating the same results and extend this work, the dataset and implementation codes are publicly available [1].