Real-time auralization based on a convolutional neural network trained by low-frequency wave-based calculations

Acoustic Virtual Reality (AVR) has been increasingly used in building design, where sound fields are expected to be updated in real-time as the receiver and source move around the space. At low frequencies, wave-based methods can be used in the pre-calculation stage to obtain credible sound fields, that are stored for later real-time interpolation, which may lead to large storage requirements. This research proposed Neural Networks (NNs) trained by results from low-frequency room acoustic calculations such that they can provide the binaural room impulse responses (BRIRs) in real-time in an AVR framework. The room sound fields were calculated by solving the Helmholtz equation through the Finite Element Method and stored at spherical receiver arrays to build the training datasets. Convolutional Neural Networks are used to predict the spherical harmonics (SH) coefficients of the sound field distribution on spherical receiver arrays with the positions of the source and receiver as input. Combined with head-related transfer functions, these SH coefficients can be used to obtain BRIRs efficiently. At the cost of training NNs, this method is applicable to AVR scenarios with moving source and receiver and arbitrary head orientation, with the advantages of fast real-time calculation and distinct storage data reduction.