High angular resolution diffusion imaging (HARDI) is a MRI imaging technique that is able to better capture the intra-voxel diffusion pattern compared to its simpler predecessor diffusion tensor imaging (DTI). However, HARDI in general produces very noisy diffusion patterns due to the low SNR from the scanners at high b-values. Furthermore, it still exhibits limitations in areas where the diffusion pattern is asymmetrical (bifurcations, splaying fibers, etc.). To overcome these limitations, enhancement and denoising of the data based on context information is a crucial step. In order to achieve it, convolutions are performed in the coupled spatial and angular domain. Therefore the kernels applied become also HARDI data. However, these approaches have high computational complexity of an already complex HARDI data processing. In this work, we present an accelerated framework for HARDI data regularizaton and enhancement. The convolution operators are optimized by: pre-calculating the kernels, analysing kernels shape and utilizing look-up-tables. We provide an increase of speed, compared to previous brute force approaches of simpler kernels. These methods can be used as a preprocessing for tractography and lead to new ways for investigation of brain white matter.