奇异值分解(SVD)的公式 $A = U\Sigma V^T$ 尽管看起来抽象,但它描述了一个基本的几何过程。理解这种几何性质有助于直观把握 SVD 为何在降维和理解线性变换等应用中如此有效。设想一个由 $m \times n$ 矩阵 $A$ 表示的线性变换。当我们对向量 $x$ 应用此变换得到 $y = Ax$ 时,SVD 告诉我们这个变换可以分解为三个不同的几何步骤:旋转/反射 ($V^T$): 矩阵 $V^T$($V$ 的转置)是一个正交矩阵。当应用于输入向量 $x$ 时,$V^T x$ 在输入空间 $\mathbb{R}^n$ 中执行旋转并可能进行反射。它不改变向量的长度或它们之间的夹角。可以将此步骤看作是让输入空间沿着一组特殊的正交方向对齐,这些方向由 $V$ 的列(右奇异向量)给出。这些向量 $v_1, v_2, \dots, v_n$ 构成输入空间的一个标准正交基。$V^T$ 本质上是旋转空间,使得这些主输入方向与标准坐标轴对齐。缩放 ($\Sigma$): 矩阵 $\Sigma$ 是一个 $m \times n$ 矩形对角矩阵。其对角线元素是奇异值 $\sigma_1, \sigma_2, \dots, \sigma_r$(其中 $r$ 是 $A$ 的秩),所有其他元素均为零。此矩阵缩放旋转后的向量 $V^T x$ 的坐标。具体来说,它将第 $i$ 个坐标(对应于 $v_i$ 的方向)按奇异值 $\sigma_i$ 缩放。对应于零奇异值的方向有效地压缩为零。此步骤沿着新对齐的轴线拉伸或收缩空间。旋转/反射 ($U$): 矩阵 $U$ 也是一个正交矩阵($m \times m$)。这最后一步将缩放后的向量 $\Sigma V^T x$ 在输出空间 $\mathbb{R}^m$ 中执行另一次旋转并可能进行反射。$U$ 的列 $u_1, u_2, \dots, u_m$(左奇异向量)构成输出空间的一个标准正交基。此步骤将缩放后的向量从轴对齐的方向(步骤 2 之后)旋转到它们在输出空间中的最终位置,使它们与由 $U$ 的列定义的主输出方向对齐。变换的可视化设想将变换 $A$ 应用于 2D 单位圆(或 3D 单位球)上的所有点。SVD 几何上分解变换过程如下:输入: 从以原点为中心的单位圆开始。应用 $V^T$: 这会旋转单位圆。由于是刚性变换,结果仍是单位圆,仅仅是方向可能不同。由右奇异向量 $v_1, v_2$ 定义的轴旋转以对齐标准轴 $e_1, e_2$。应用 $\Sigma$: 这会沿着标准轴线按比例 $\sigma_1$ 和 $\sigma_2$ 缩放旋转后的圆。圆变成一个椭圆,其长轴和短轴与标准坐标轴对齐。半轴的长度为 $\sigma_1$ 和 $\sigma_2$。应用 $U$: 这会旋转椭圆而不改变其形状。最终结果是一个椭圆,其轴与左奇异向量 $u_1, u_2$ 对齐。SVD 本质上告诉我们,任何线性变换 $A$ 将输入空间中的标准正交基向量($V$ 的列)映射到输出空间中的正交向量(通过奇异值 $\sigma_i$ 缩放的 $U$ 的列)。也就是说,$Av_i = \sigma_i u_i$,其中 $i = 1, \dots, r$。{"layout": {"title": {"text": "矩阵 A 对单位圆的几何作用"}, "xaxis": {"title": {"text": "X轴"}, "range": [-2.5, 2.5], "scaleanchor": "y", "scaleratio": 1}, "yaxis": {"title": {"text": "Y轴"}, "range": [-2.5, 2.5]}, "width": 500, "height": 500, "legend": {"yanchor": "top", "y": 0.99, "xanchor": "left", "x": 0.01}, "margin": {"l": 20, "r": 20, "t": 40, "b": 20}, "plot_bgcolor": "#e9ecef"}, "data": [{"x": [1.0, 0.9980267284282716, 0.9921147013144843, 0.9822872507282355, 0.9685831611522909, 0.9510565162951535, 0.9297764858895338, 0.9048270524660175, 0.8763066800438635, 0.8443279255020151, 0.8090169943749475, 0.7705132427757886, 0.7289686274214116, 0.6845471059286887, 0.6374239897486913, 0.5877852522924731, 0.5358267949790984, 0.4817536741017154, 0.4257792915650724, 0.3681245526846775, 0.30901699437494745, 0.2486898871648546, 0.1873813145857246, 0.12533323356430426, 0.06279051952931344, 0.0, -0.06279051952931335, -0.12533323356430412, -0.18738131458572448, -0.2486898871648545, -0.30901699437494734, -0.3681245526846774, -0.4257792915650723, -0.4817536741017153, -0.5358267949790983, -0.587785252292473, -0.6374239897486912, -0.6845471059286886, -0.7289686274214115, -0.7705132427757885, -0.8090169943749473, -0.844327925502015, -0.8763066800438634, -0.9048270524660174, -0.9297764858895337, -0.9510565162951535, -0.9685831611522908, -0.9822872507282355, -0.9921147013144843, -0.9980267284282716, -1.0, -0.9980267284282716, -0.9921147013144843, -0.9822872507282355, -0.9685831611522909, -0.9510565162951536, -0.9297764858895338, -0.9048270524660175, -0.8763066800438635, -0.8443279255020151, -0.8090169943749475, -0.7705132427757886, -0.7289686274214116, -0.6845471059286888, -0.6374239897486913, -0.5877852522924732, -0.5358267949790985, -0.48175367410171546, -0.4257792915650725, -0.3681245526846776, -0.30901699437494756, -0.2486898871648547, -0.1873813145857247, -0.12533323356430444, -0.06279051952931366, -1.2246467991473532e-16, 0.06279051952931327, 0.12533323356430406, 0.18738131458572437, 0.24868988716485442, 0.30901699437494723, 0.3681245526846773, 0.4257792915650722, 0.4817536741017152, 0.5358267949790982, 0.587785252292473, 0.637423989748691, 0.6845471059286885, 0.7289686274214114, 0.7705132427757885, 0.8090169943749473, 0.844327925502015, 0.8763066800438633, 0.9048270524660174, 0.9297764858895336, 0.9510565162951535, 0.9685831611522908, 0.9822872507282355, 0.9921147013144843, 0.9980267284282716, 1.0], "y": [0.0, 0.06279051952931335, 0.12533323356430412, 0.18738131458572448, 0.2486898871648545, 0.30901699437494734, 0.3681245526846774, 0.4257792915650723, 0.4817536741017153, 0.5358267949790983, 0.587785252292473, 0.6374239897486912, 0.6845471059286886, 0.7289686274214115, 0.7705132427757885, 0.8090169943749473, 0.844327925502015, 0.8763066800438634, 0.9048270524660174, 0.9297764858895337, 0.9510565162951535, 0.9685831611522908, 0.9822872507282355, 0.9921147013144843, 0.9980267284282716, 1.0, 0.9980267284282716, 0.9921147013144843, 0.9822872507282355, 0.9685831611522909, 0.9510565162951536, 0.9297764858895338, 0.9048270524660175, 0.8763066800438635, 0.8443279255020151, 0.8090169943749475, 0.7705132427757886, 0.7289686274214116, 0.6845471059286888, 0.6374239897486913, 0.5877852522924732, 0.5358267949790985, 0.48175367410171546, 0.4257792915650725, 0.3681245526846776, 0.30901699437494756, 0.2486898871648547, 0.1873813145857247, 0.12533323356430444, 0.06279051952931366, 1.2246467991473532e-16, -0.06279051952931327, -0.12533323356430406, -0.18738131458572437, -0.24868988716485442, -0.30901699437494723, -0.3681245526846773, -0.4257792915650722, -0.4817536741017152, -0.5358267949790982, -0.587785252292473, -0.637423989748691, -0.6845471059286885, -0.7289686274214114, -0.7705132427757885, -0.8090169943749473, -0.844327925502015, -0.8763066800438633, -0.9048270524660174, -0.9297764858895336, -0.9510565162951535, -0.9685831611522908, -0.9822872507282355, -0.9921147013144843, -0.9980267284282716, -1.0, -0.9980267284282716, -0.9921147013144843, -0.9822872507282355, -0.9685831611522908, -0.9510565162951535, -0.9297764858895336, -0.9048270524660174, -0.8763066800438633, -0.844327925502015, -0.8090169943749473, -0.7705132427757885, -0.7289686274214114, -0.6845471059286885, -0.637423989748691, -0.587785252292473, -0.5358267949790982, -0.4817536741017152, -0.4257792915650722, -0.3681245526846773, -0.30901699437494723, -0.24868988716485442, -0.18738131458572437, -0.12533323356430406, -0.06279051952931327], "mode": "lines", "line": {"color": "#adb5bd", "width": 2}, "name": "单位圆"}, {"x": [1.0, 1.0608172479575848, 1.1177479348787884, 1.169614301456471, 1.2152663223045817, 1.2538130325901008, 1.2845529717790676, 1.3070807265677328, 1.321037350087727, 1.3263607180522166, 1.3232039887498948, 1.311926485551577, 1.2929897333499999, 1.2670157333499998, 1.2346979794973825, 1.1968605045849463, 1.1543804691581966, 1.1081803482034307, 1.0591585831301447, 1.008152478211466, 0.9559415470596248, 0.903249774329709, 0.8506994460442583, 0.7988164671286084, 0.7480488390586268, 0.6988169943749474, 0.6514663094042606, 0.6062699671286082, 0.5634480301714488, 0.5231648070000001, 0.4855439887498947, 0.4506600258961434, 0.4185479189864838, 0.3891973482034306, 0.3625663258209016, 0.3385892410411642, 0.31717674204130876, 0.2982318165213771, 0.2816454901635526, 0.2672985668774998, 0.25507398874989465, 0.24485585100403006, 0.23652033613158677, 0.22994378103203476, 0.22500015766863198, 0.22156954888530704, 0.2195362823045816, 0.21878780341291252, 0.21922000262896854, 0.2207319856865432, 0.22321699437494745, 0.2265752131132448, 0.2306994460442582, 0.23548345414114795, 0.2408264823045817, 0.2466303106699949, 0.2528041435543911, 0.2592641035980522, 0.26593136011527485, 0.2727304745342302, 0.2795899887498948, 0.2864431613273771, 0.2932248937428231, 0.29987394185737725, 0.3063319794973825, 0.3125437568774146, 0.31845789413719385, 0.3240256999220597, 0.3292013105515563, 0.3339418525961446, 0.3382056938052608, 0.3419537018545637, 0.3451493806300028, 0.3477588803000278, 0.3497511609822471, 0.3510978831242609, 0.3517738512783297, 0.35175754740171526, 0.3510306823364838, 0.34957843276630064, 0.34738915470596245, 0.3444544756404744, 0.3407680356846773, 0.33632607654416954, 0.3311271001152748, 0.32517209100403, 0.3184644278209016, 0.3110097045849463, 0.3028161186368825, 0.29389401680000004, 0.2842552062871371, 0.27391338899858776, 0.26288403124858145, 0.2511841508808237, 0.23883267636311785, 0.22585049016355267, 0.2122599671286084, 0.19808503017144896, 0.1833505195293134, 0.16808199437494748], "y": [0.0, 0.06279051952931335, 0.12533323356430412, 0.18738131458572448, 0.2486898871648545, 0.30901699437494734, 0.3681245526846774, 0.4257792915650723, 0.4817536741017153, 0.5358267949790983, 0.587785252292473, 0.6374239897486912, 0.6845471059286886, 0.7289686274214115, 0.7705132427757885, 0.8090169943749473, 0.844327925502015, 0.8763066800438634, 0.9048270524660174, 0.9297764858895337, 0.9510565162951535, 0.9685831611522908, 0.9822872507282355, 0.9921147013144843, 0.9980267284282716, 1.0, 0.9980267284282716, 0.9921147013144843, 0.9822872507282355, 0.9685831611522909, 0.9510565162951536, 0.9297764858895338, 0.9048270524660175, 0.8763066800438635, 0.8443279255020151, 0.8090169943749475, 0.7705132427757886, 0.7289686274214116, 0.6845471059286888, 0.6374239897486913, 0.5877852522924732, 0.5358267949790985, 0.48175367410171546, 0.4257792915650725, 0.3681245526846776, 0.30901699437494756, 0.2486898871648547, 0.1873813145857247, 0.12533323356430444, 0.06279051952931366, 1.2246467991473532e-16, -0.06279051952931327, -0.12533323356430406, -0.18738131458572437, -0.24868988716485442, -0.30901699437494723, -0.3681245526846773, -0.4257792915650722, -0.4817536741017152, -0.5358267949790982, -0.587785252292473, -0.637423989748691, -0.6845471059286885, -0.7289686274214114, -0.7705132427757885, -0.8090169943749473, -0.844327925502015, -0.8763066800438633, -0.9048270524660174, -0.9297764858895336, -0.9510565162951535, -0.9685831611522908, -0.9822872507282355, -0.9921147013144843, -0.9980267284282716, -1.0, -0.9980267284282716, -0.9921147013144843, -0.9822872507282355, -0.9685831611522908, -0.9510565162951535, -0.9297764858895336, -0.9048270524660174, -0.8763066800438633, -0.844327925502015, -0.8090169943749473, -0.7705132427757885, -0.7289686274214114, -0.6845471059286885, -0.637423989748691, -0.587785252292473, -0.5358267949790982, -0.4817536741017152, -0.4257792915650722, -0.3681245526846773, -0.30901699437494723, -0.24868988716485442, -0.18738131458572437, -0.12533323356430406, -0.06279051952931327], "mode": "lines", "line": {"color": "#f03e3e", "width": 3}, "name": "经 A 变换后"}]}由矩阵 $A = \begin{pmatrix} 1 & 1 \ 0 & 1 \end{pmatrix}$ 定义的变换将单位圆(灰色)映射为椭圆(红色)。SVD 将这一整体变换分解为一系列的旋转 ($V^T$)、轴对齐的缩放 ($\Sigma$) 和另一次旋转 ($U$)。这种几何视角对于理解使用 SVD 进行降维尤其有帮助。奇异值 $\sigma_i$ 量化了每个主方向的重要性。较大的奇异值对应于数据(或变换)具有最大方差或“扩散”的方向。通过仅保留对应于最大奇异值的分量,我们保留了变换最重要的几何特征,同时可能丢弃与小奇异值相关的维度(这些维度可能代表噪声或不重要的变化)。