MSE损失梯度推导 离群点梯度爆炸完整示例重点关注梯度损失函数的导数 MSE的梯度dLdWe⋅x\frac{dL}{dW}e \cdot xdWdL​e⋅x相比于MAE多了e误差所以误差变小时梯度也会随之变小一、统一符号与基础公式和MAE保持完全一致1. 变量定义符号含义xxx模型输入特征WWW网络权重y^\hat{y}y^​模型预测值yyy真实标签eee误差ey^−ye\hat{y}-yey^​−yLLLMSE损失L12e2L\frac{1}{2}e^2L21​e2加1/2方便求导消系数η\etaη学习率2. 核心公式线性预测简化无偏置b0b0b0y^W⋅x\hat{y}W \cdot xy^​W⋅x链式梯度求导dLdWdLde⋅dedy^⋅dy^dW\frac{dL}{dW}\frac{dL}{de} \cdot \frac{de}{d\hat{y}} \cdot \frac{d\hat{y}}{dW}dWdL​dedL​⋅dy^​de​⋅dWdy^​​dLdee\displaystyle \frac{dL}{de}ededL​ededy^1\displaystyle \frac{de}{d\hat{y}}1dy^​de​1dy^dWx\displaystyle \frac{d\hat{y}}{dW}xdWdy^​​x合并梯度dLdWe⋅x\frac{dL}{dW}e \cdot xdWdL​e⋅x权重更新规则WnewWold−η⋅dLdWW_{new}W_{old} - \eta \cdot \frac{dL}{dW}Wnew​Wold​−η⋅dWdL​二、案例1正常样本误差很小收敛顺滑固定参数输入x1x1x1学习率η0.1\eta0.1η0.1真实标签y5y5y5初始权重Wold5.2W_{old}5.2Wold​5.2步骤1计算初始预测与误差y^oldWold⋅x5.2×15.2eold5.2−50.2 \begin{align} \hat{y}_{old} W_{old} \cdot x 5.2 \times 1 5.2 \\ e_{old} 5.2 - 5 0.2 \end{align}y^​old​eold​​Wold​⋅x5.2×15.25.2−50.2​​步骤2计算梯度dLdWe⋅x0.2×10.2 \frac{dL}{dW}e \cdot x 0.2 \times 1 0.2dWdL​e⋅x0.2×10.2步骤3更新权重Wnew5.2−0.1×0.25.18 W_{new}5.2 - 0.1 \times 0.2 5.18Wnew​5.2−0.1×0.25.18步骤4新误差y^new5.18,enew5.18−50.18 \hat{y}_{new}5.18,\quad e_{new}5.18-50.18y^​new​5.18,enew​5.18−50.18特点误差变小梯度同步变小更新幅度越来越轻平稳向0靠近无震荡。三、案例2存在离群点误差极大 → 梯度爆炸核心演示场景说明正常样本真实值集中在y5y5y5出现一个离群点真实标签y20y20y20输入x1x1x1学习率η0.1\eta0.1η0.1设当前权重Wold5W_{old}5Wold​5步骤1计算预测与超大误差y^old5×15eold5−20−15 \begin{align} \hat{y}_{old} 5 \times 1 5 \\ e_{old} 5 - 20 -15 \end{align}y^​old​eold​​5×155−20−15​​误差绝对值达到15属于极端离群误差。步骤2计算梯度dLdWe⋅x−15×1−15 \frac{dL}{dW}e \cdot x -15 \times 1 -15dWdL​e⋅x−15×1−15步骤3单次权重更新幅度WnewWold−η⋅dLdW5−0.1×(−15)51.56.5 W_{new}W_{old} - \eta \cdot \frac{dL}{dW} 5 - 0.1 \times (-15) 5 1.5 6.5Wnew​Wold​−η⋅dWdL​5−0.1×(−15)51.56.5现象1更新步长巨大正常误差0.2时仅调整0.02离群点一次直接调整1.5参数剧烈跳动。步骤4更新后误差依旧巨大梯度持续爆炸y^new6.5,enew6.5−20−13.5 \hat{y}_{new}6.5,\quad e_{new}6.5-20-13.5y^​new​6.5,enew​6.5−20−13.5新梯度dLdW−13.5×1−13.5 \frac{dL}{dW}-13.5 \times 1-13.5dWdL​−13.5×1−13.5下一轮继续大幅更新Wnext6.5−0.1×(−13.5)7.85 W_{next}6.5 - 0.1 \times (-13.5)7.85Wnext​6.5−0.1×(−13.5)7.85误差只会缓慢缩小每一步都超大步长模型剧烈震荡、难以收敛严重时直接发散。四、对比MAE同一离群场景凸显MAE对异常值鲁棒同样离群点e−15, x1e-15,\ x1e−15,x1MAE梯度dLdW−1×1−1 \frac{dL}{dW}-1 \times 1-1dWdL​−1×1−1单次权重更新Wnew5−0.1×(−1)5.1 W_{new}5 - 0.1 \times (-1)5.1Wnew​5−0.1×(−1)5.1无论误差是-15还是-0.2MAE梯度永远固定±1更新幅度恒定不会出现大步长爆炸。核心总结MSE梯度和误差eee成正比误差越大梯度线性放大数据里存在离群点时会产生巨大梯度单次更新权重幅度极大造成梯度爆炸、模型震荡发散MSE优点误差接近0时梯度极小微调精准光滑MAE优点梯度恒定不怕离群点缺点0点不可导小误差区间容易震荡Smooth L1取长补短误差小时用MSE光滑收敛误差大时切换MAE限制梯度规避两者缺陷。