A Deep Neural Network Based on Two‐Stage Training for Estimating Heart Rate Variability From Camera Videos

Studies have shown that heart rate variability (HRV) is a predictor of the prognosis of cardiovascular diseases. Contact heartbeat monitoring equipment is widely used, especially in hospitals, and benefits from the rapidity and accuracy of the detection of physiological health indicators. However, long‐term contact with equipment has many adverse effects. The purpose of this study was to improve the accuracy of HRV detection via noncontact equipment, thus enabling HRV to be assessed in various scenarios.

A novel deep learning approach was proposed for measuring heartbeats through camera videos. First, we performed facial segmentation and divided the face into 16 grid cells with different light balance scores. After the trend is filtered by the Hamming window, a transformer‐based neural network is used to further filter the signal. Finally, heart rate (HR) and HRV are estimated.

We used 1 million synthetic data points for pretraining and a public dataset in combination with a dataset that we constructed for task training. The final results were obtained on a test dataset that we constructed. The accuracy for HR with a low light balance score (0.867–0.983) was greater than that with a high score (0.667–0.750). Our method had higher accuracy in estimating HR than traditional filtering methods (0.167–0.417) and state‐of‐the‐art neural network filtering methods (0.783–0.917) did. The root mean square error of the HRV from the time domain was the lowest, and the correlation index score was the highest for the HRV from the frequency domain estimated by our method compared with those estimated by two neural networks.

Light balance, large sample training, and two‐stage training can improve the accuracy of HRV estimation.