Review

Tensor network decomposition for data recovery: Recent advancements and future prospects

With advancements in sensing technologiesinstrumentationand computational capabilitiesaccess to vast and complex datasets with multiple dimensions or features has expanded significantly. Examples of such high-dimensional data include color images/videosmultispectral/hyperspectral imageslight field imagestraffic datagene expression profilesand user behavior dataetc. Howeverdue to limitations in sensing technologiesenvironmental interference during data acquisitionand the inherent complexity of high-dimensional structuresthese datasets often suffer from noisecorruptionand missing values. High-dimensional data recovery aims to reconstruct the underlying clean and complete data from such imperfect observations by exploiting intrinsic priors embedded in the data (Cao et al.2016Cichocki et al.2015Comon2014DingFu et al.2021Fu et al.2020Goldfarb and Qin2014Kanatsoulis et al.2018Li et al.2019Meng et al.2015Pan et al.2023Qin et al.2022Qiu et al.2021Sidiropoulos et al.2017WangMeng et al.2018WangZeng et al.2023Xie et al.2023Xie et al.2020XuKe et al.2024XuPeng et al.2024Zhang et al.2021ZhaoZhouZhang et al.2016Zhou and Cichocki2012Zhou et al.2014). Common intrinsic priors include global low-rankness (global correlation)local continuity (local correlation)and non-local self-similarity (non-local correlation). A critical challenge in high-dimensional data recovery is how to effectively model and exploit these priors to achieve accurate and robust reconstruction.

A tensor is a multi-dimensional arraywhere the count of dimensions (referred to as modes) is termed the tensor order. As a natural extension of vectors and matricestensors offer an intuitive and effective way to represent high-dimensional data (Carroll and Chang1970Harshman1970Hitchcock1928Kolda and Bader2009Tucker1966Wang et al.2024). For instancea color image can be represented as a third-order tensorwith its three modes corresponding to heightwidthand color channels. In particularby decomposing a target tensor into a set of smaller-scale factorstensor decomposition exhibits a remarkable capability in capturing the global correlations and reducing redundancy concealed within the data (Kolda & Bader2009). Tensor decomposition have a rich historyspanning over a century. Fig. 1 illustrates the evolution of tensor decomposition. In the last few decadesCANDECOMP/PARAFAC (CP) decomposition (Carroll and Chang1970Harshman1970Hitchcock1927Kiers2000)Tucker decomposition (Tucker1966)and tensor singular value decomposition (t-SVD) (Kilmer et al.2013Kilmer and Martin2011Semerci et al.2014) as classical methodshave achieved great success in data recovery problemssuch as image/video restoration (Ahmadi-Asl et al.2023Cheng et al.2024Gandy et al.2011Hou et al.2022Liu et al.2013Lu et al.2020Tu et al.2024WangLi et al.2020WangPeng et al.2023Wang et al.2022Xie et al.2018Xu et al.2015Yokota et al.2016Zhang and Aeron2017ZhangYang et al.2023)multispectral/hyperspectral image denoising (He et al.2022Hong et al.2021Luo et al.2022WangHong et al.2023Zheng et al.2020b)and traffic data imputation (Chen et al.2019ChenLei et al.2022ChenYang et al.2020). There exist several excellent review articles that furnish comprehensive summaries of the aforementioned tensor decompositions and their applications (Cichocki et al.2015Comon2014Kolda and Bader2009Lu et al.2011Sidiropoulos et al.2017WangMeng et al.2018Zhou et al.2014). For instancethe tutorial by Kolda and Bader (2009) delivers a thorough overview encompassing fundamental conceptsmodelsand algorithms for CP decompositionTucker decompositionand their variants. Another notable contribution is the tutorial by Sidiropoulos et al. (2017)which places a significant emphasis on an intricate review of tensor rank theory and explores contemporary advanced tensor computation algorithms of its timeoffering insightful derivations of several foundational results. Additionallythe tutorial by Cichocki et al. (2015) offers a reader-friendly introductionenriched with numerous figures that facilitate a smooth comprehension of tensor decompositions and their applications within the realm of data processing.

Tensor network (TN) decompositionan emerging method in scientific computing and machine learninghas received extensive attention¹ (Cichocki2018Cichocki et al.2016Cichocki et al.2017Liu et al.2023Oseledets2011WangPan et al.2023WangSu et al.2020Wang et al.2019WangZhaoChen et al.2021ZhaoZhouXie et al.2016Zheng et al.2021). TN decomposition breaks down an $N$th-order tensor into a series of smaller matrices/tensors (called TN cores) and defines tensor contraction as the interaction operations among these cores. Visualizing the cores as nodes and the operations between them as edgesthe formed unweighted graph is referred to as the TN topology (Li & Sun2020). Around eight years agoa highly-cited and well-structured book provided a detailed introduction to the fundamental conceptsoperationsand application scenarios of TN decompositions (Cichocki et al.2016Cichocki et al.2017). Recentlya particular type of TN decomposition has seen extensive development. This type adopts a simple unweighted graph as the TN topology and represents an $N$th-order tensor $\mathcal{X}$ using $N$ TN coreseach corresponding to one mode of $\mathcal{X}$. Representative methods include tensor train (TT) decomposition with a chain topology (Oseledets2011)tensor ring (TR) decomposition with a ring topology (ZhaoZhouXie et al.2016)and fully-connected tensor network (FCTN) decomposition with a fully-connected topology (Zheng et al.2021). These methods have found wide applications in high-dimensional data recovery taskssuch as image/video inpaintingmedical image reconstructionand remote sensing image recovery (Ahmadi-Asl et al.2020Asante-Mensah et al.2021Han et al.2024Huang et al.2022Kisil et al.2022Le et al.2024LiLin et al.2023Phan et al.2020QiuZhouWang et al.2024Qiu et al.2022Sedighin et al.2020Xu et al.2023Yu et al.2022Zhang et al.2024ZhengZhaoZheng et al.2024). Despite their significancea comprehensive review of recent advancements and future prospects in this line of research is still lacking. To bridge this gapwe present a systematic review specifically dedicated to this type of TN decomposition. Our goal is to provide a valuable resource that helps researchers navigate the complexities of this fielddeepen their understandingand foster future innovations.

In this articlewe begin by unraveling foundational operationscomputational rulesand properties associated with TN decompositions. Following this groundworkwe delve into the progression and differences of various TN decompositionssuch as TTTRand FCTN decompositions. Each decomposition type possesses unique strengths and weaknessesinfluencing their adoption in specific applications and problems. Beyond their individual characteristicswe explore the relationships between different TN ranks and their corresponding unfolding matrix ranks. This understanding enriches our grasp of TN ranksfacilitating informed choices in practical applications. We also review representative algorithms for TN decomposition computation. Subsequentlywe focus on a fundamental component within the realm of TN decompositions: the TN structure search (TN-SS) problem. In this contextwe introduce a spectrum of TN-SS algorithmseach dissected to reveal its strengths and weaknesses. Following thiswe shed light on various TN decomposition methods within high-dimensional data recovery. Through a series of experimentswe offer a comparative outlook on the effectiveness of different methods in real-world tasks. Finallywe highlighting the challenging problems and discuss potential future solutions within the field of TN decompositions.