Lost data recovery for structural vibration data based on improved U-shaped encoder–decoder networks

The advantage of structural health monitoring (SHM) lies in its ability to provide continuous real-time information about structural behaviorperformanceand potential damage [1]. In SHM for large-scale civil structuresstructural vibration responses provide valuable information for assessing the dynamic performance of structures [2]. Thereforethe severe loss of vibration data significantly impacts the effectiveness of structural damage identification and safety assessment.

Howeverthe inevitable hardware malfunctionssuch as sensor faultabnormal data acquisition and disturbed wireless transmissionresult in data loss in varying degrees [3]. There might be randomperiodicor continuous data loss [4]. When monitoring data suffers from complete loss or when data loss ratios are highnot only does the accurate identification of structural behavior face significant challengesbut also the assessment of structural performance and safety becomes exceptionally difficult. Although repairing hardware systems is a feasible solutionidentifying and resolving these issues entail significant human and time costs. More criticallybecause embedded sensors are fixed within the structurereplacing them is not feasible [5].

To address the issue of lost data recovery in SHMsome researchers have proposed methods based on finite element models [6][7][8]. In these methodsestablishing reliable relationships between dynamic responses relies on accurately modeling the structures. Howeverdue to unavoidable geometric and material uncertaintiesas well as the continuously changing environmental and operational conditionsprecise modeling of complex civil structures in use is a challenging task [9]. Compared to strategies based on finite element modelsdata-driven methods offer significant advantages in data recovery for large-scale structures. This approach can directly recover structural responses from measured dataeffectively reducing modeling complexity and workload. Data-driven methods automatically learn the mapping between known and unknown responses. Training data are considered to be sampled from the unknown distribution P(x,y)where the input vectors are extracted from the input space X with marginal probability P(x)and corresponding output points are observed in the output space Y with conditional probability P(y|x). The learning problem can be seen as finding an appropriate estimation function f:X→Ywhich represents the process of generating outputs from input vectors. Subsequentlythis function can be used for generalizationi.e.for predicting outputs for new inputs [10]as shown in Fig. 1. Currentlyin the field of data loss recoverydata-driven methods can be divided into three categories: those based on statistical methodsthose based on machine learning (ML)and those based on deep learning (DL) [11].

In terms of input based on statistical methodsresearchers have been devoted to utilizing statistical properties such as data probability distributions and correlationsthrough reasonable inference and interpolation methods. Initiallymean imputation method [12] and multiple imputation method [13] were proposed for this purpose. Van Le et al. [14] reported the reconstruction of lost data on concrete bridges using maximum likelihood and least-square approximation with polynomial equations. Kullaa et al. [15] proposed a method based on minimum mean square errorutilizing the covariance between data from different sensors to reconstruct sensor data. Statistical methods have the advantage of simplicitybut their limited assumptions about data distribution lead to poor reconstruction results [16].

Compared to traditional statistical methodsML algorithms exhibit adaptabilityas they can learn patterns and regularities from data without requiring excessive assumptions about data distribution. This makes machine learning methods more flexiblecapable of accommodating different types of SHM dataand providing more robust and accurate recovery in scenarios involving lost data. Bao et al. [17] proposed an innovative method based on compressed sensing techniques for signal loss recovery in wireless sensor networks installed on bridge structures. The performance of this method largely depends on the sparsity of data in a specific feature space. Yang and Nagarajaiah [18] applied sparse representation with low-rank structure to recover acceleration data from actual bridge and tower structures. This method may overlook key information during sampling and transmissionthereby improving the efficiency of reconstruction at the expense of accuracy. Additionallysome scholars have employed support vector machines [19][20] or backpropagation neural networks [21] to construct models for recovering lost data. Howeverthese traditional ML methods have limited capability in recovering responses with large-scale and long-term continuous losses. MoreoverML often requires a significant amount of time for intricate feature engineering.

With the development of computer scienceDL has achieved tremendous success in the fields of speech recognitionnatural language processingand computer vision [22]. DL is suitable for handling larger and more complex data sets [23]. Fan et al. [24] first proposed a bottleneck convolutional neural network (CNN) for recovering randomly lost sampling points during wireless transmission processes. They found that the modal identification results using the recovered signals with different data loss ratios were highly consistent with those obtained from complete real data. Jeong et al. [25] considered the spatiotemporal correlation among sensor data and proposed a bidirectional recurrent neural network (BRNN) for bridge data reconstruction. Jiang et al. [11] proposed a deep fully CNN with an encoder-decoder architecture to capture the overall semantic features of vibration data. This allows accurate modeling of the behavior of complex conditional probability distributions. Additionallya novel perceptual loss function was introduced to enable the network to effectively integrate data loss patterns. Howeverthe loss function has not been analyzed in detail to verify its effectivenessand secondlythe designed model is more complex with a larger number of parameters. Ni et al. [26] proposed a novel data compression and reconstruction method using autoencodersin which measured multi-source data can be stored at a very low compression ratio and accurately recovered to normal signals in both the time and frequency domains.

In recent yearsgenerative adversarial networks (GANs) [27] have achieved tremendous success in fields such as image synthesisimage transformationand audio synthesis. GAN is a type of deep learning model whose core principle involves a game-theoretic process where the generator and discriminator balance each other outresulting in the generator producing realistic data samples. GAN models are now being applied to data reconstruction in structural health monitoring. Fan et al. [28] proposed a specially designed segmented conditional generative adversarial network for monitoring data recovery algorithm. Gao et al. [16] proposed a slim generative adversarial imputation network for bridge SHM systems to recover lost deflection data. This framework utilizes a thin neural network with a generator-discriminator architecture to capture valuable information from non-lost portions of both faulty and other normal sensors. Hou et al. [29] proposed a deep learning method based on GANs and data augmentation techniques for interpolation between the same sensors. HoweverGANs may encounter challenges in mode collapsenon-convergenceand instability during trainingespecially in complex data distributions [30]. The training of both the generator and discriminator requires mutual learning to reach a balance pointwhich in some cases may necessitate a high number of training samples and computational resources. Additionallymost research focuses on the generator generating samplesand once the overall model training is completedthe discriminator is rarely usedleading to a waste of computational resources.

In summarytraditional SHM methods typically rely on handcrafted features or shallow learning models that are unable to handle large-scale data lossand although existing deep learning-based methods have made some progress in data recoverythey currently have limited practical application in large or very large bridge projectsin addition to their frequent lack of adaptability to different data loss patternswhich limits their ability to capture the inherently their ability to capture the complex patterns inherent in vibration dataespecially under extreme conditions. Further in-depth researchanalysisand comparison are needed to evaluate the effectiveness of single-sensor-based recovery versus joint recovery using multiple sensors. Additionallya key factor hindering recovery efficiency and accuracy is the lack of compact and in-depth network structuresas suitable network architectures are crucial for ensuring quality and efficiency.

The innovative U-shaped encoder and decoder neural network proposed in this study aims to restore lost sensor data more precisely and efficiently. The network innovatively incorporates the following significant features: Firstit adopts an encoder-decoder network structure and efficiently connects shallow and deep features by introducing skip connections. The attention gate mechanism is introduced in skip connectionsenabling the network to automatically focus on key features in the input datawhich is crucial for handling complex structural data and tasks. Vibration data recovery is a complex and time-consuming task. During the upsampling in the expansive paththe recreated spatial information may not be accuratemaking it challenging to effectively focus on the parts of the vibration signal that contain important informationas these key features may only occupy a small portion of the entire signal. Incorporating an attention gate (AG) mechanism into the U-Net can improve the performance of the modelespecially in tasks like image segmentationwhere it can enhance accuracy and generalization capabilities [31]. Secondvibration data recovery is inherently complex and time-consumingespecially for long sequences with significant data loss. As the network deepens to learn more abstract and advanced feature representationsit risks increased spatial information losswhich can negatively impact data recovery accuracy. To address this issueresidual connection [32] modules are introduced in both the encoder and decoder paths. Residual connections not only improve gradient flow during model training by allowing gradients to propagate more directly to earlier layers but also enable the direct transmission and reuse of features from earlier layers. By mitigating information lossresidual connections help the network extract and retain more comprehensive features at each layer. Furthermorean imputation mask matrix layer is introducedmeticulously designed to process and reconstruct the regions of lost data without altering the network's original loss function. This layer optimizes the network's focus on the reconstruction tasksthereby minimizing the computational load and enhancing the precision of data recovery.

This enhancement enables the effective restoration of data in scenarios with high loss ratiosapplicable to both single-channel and multi-channel configurations. The efficacy of the model was substantiated through validation with real-world data derived from a large-span bridge. This validation process included a comprehensive analysis of recovery levels across varying loss rates and a thorough assessment of the model’s performance in scenarios involving both single and multi-channel data. It is worth noting that the imputation mask matrix layer allows users to freely define the positions of lost datafacilitating their practical usage in engineering applications. The integration of these innovative designs enhances the performance of the proposed neural network in addressing lost sensor data issues. Previous literature generally overlooked validating each proposed modulewhile this paper compares and validates each one. The architecture of this paper is structured as follows: Section 2 elaborates on the theoretical foundation of the methods used in the study. Section 3 provides information on the source and processing of actual engineering case data. Section 4 describes the training process of the DL models for single- and multi-channel datacompares the prediction resultsand discusses the soundness of the designed model architecture. Section 5 explores scenarios of continuous data lossand presents prospects for expandable applications. The final section concludes with the main conclusions of this paper.