词嵌入日志文件输出如下️GPT层 输入idx:tensor([[47,45,46,58],[59,57,1,39]])idx形状:torch.Size([2,4])位置编码pos:tensor([[0,1,2,3]])pos形状:torch.Size([1,4])​ 输入模型的idx是一个二维数组里面的数值是输入的2句文本内容每个字4对应在字典的位置索引。​ 我们输入了2条句子每条句子有4个字。词嵌入表如下wte权重:Parameter containing:tensor([[-0.0048,0.0045,-0.0129,-0.0266,-0.0135,0.0222],[0.0334,0.0103,-0.0260,0.0165,0.0114,0.0168],[0.0129,0.0107,0.0262,-0.0025,-0.0202,-0.0226],[0.0065,-0.0103,0.0056,0.0214,0.0046,0.0489],[-0.0023,0.0296,-0.0131,-0.0077,0.0216,0.0140],[-0.0090,-0.0010,-0.0194,0.0168,-0.0038,0.0011],[-0.0372,-0.0242,0.0102,-0.0105,0.0007,-0.0226],[0.0445,0.0148,-0.0158,-0.0097,-0.0390,-0.0094],[0.0279,0.0146,-0.0426,-0.0027,-0.0365,0.0522],[0.0139,0.0071,0.0078,0.0187,-0.0444,0.0054],[-0.0108,-0.0134,-0.0015,-0.0236,-0.0009,-0.0302],[0.0061,-0.0037,-0.0094,-0.0003,0.0119,0.0224],[-0.0077,0.0012,0.0186,0.0034,0.0082,0.0008],[0.0057,-0.0118,0.0056,0.0153,0.0139,-0.0155],[-0.0078,-0.0151,-0.0144,0.0065,0.0084,-0.0246],[0.0075,-0.0111,-0.0409,0.0212,-0.0203,0.0089],[-0.0225,0.0255,-0.0437,0.0231,0.0333,0.0131],[-0.0102,0.0355,0.0097,-0.0005,-0.0008,-0.0023],[0.0032,-0.0198,0.0053,-0.0133,-0.0021,0.0200],[-0.0055,0.0082,-0.0067,-0.0027,-0.0200,-0.0529],[-0.0232,0.0339,0.0176,0.0172,0.0387,-0.0381],[-0.0159,-0.0052,0.0031,0.0304,0.0094,-0.0262],[0.0127,0.0281,0.0310,0.0452,0.0053,-0.0126],[-0.0189,-0.0468,-0.0242,0.0020,0.0078,-0.0179],[-0.0247,0.0033,-0.0068,-0.0027,0.0004,0.0218],[0.0058,0.0126,-0.0063,0.0014,-0.0110,0.0544],[0.0175,-0.0194,0.0173,-0.0105,-0.0011,0.0048],[0.0030,0.0031,-0.0305,0.0106,0.0061,0.0009],[-0.0635,-0.0178,0.0186,-0.0068,-0.0124,0.0210],[0.0174,0.0164,-0.0026,0.0073,-0.0010,-0.0164],[0.0088,0.0267,-0.0242,-0.0190,0.0136,0.0292],[-0.0109,-0.0200,-0.0161,-0.0077,-0.0132,0.0072],[0.0348,-0.0203,0.0144,-0.0001,0.0166,0.0140],[-0.0040,-0.0256,0.0030,0.0069,0.0145,0.0082],[0.0235,0.0066,0.0100,-0.0055,0.0131,0.0346],[0.0423,0.0122,0.0066,0.0094,-0.0151,0.0172],[0.0421,-0.0216,0.0240,-0.0476,-0.0168,-0.0304],[-0.0325,-0.0160,-0.0047,-0.0154,0.0047,0.0138],[0.0028,-0.0111,0.0051,-0.0079,0.0380,-0.0416],[-0.0331,0.0482,-0.0087,0.0186,-0.0184,0.0072],[0.0286,-0.0301,0.0072,0.0018,0.0247,-0.0257],[-0.0183,-0.0036,0.0327,-0.0008,0.0283,-0.0064],[-0.0408,-0.0287,-0.0058,0.0290,0.0164,0.0129],[0.0104,-0.0098,0.0334,-0.0123,0.0244,-0.0233],[-0.0058,-0.0010,-0.0164,-0.0019,-0.0154,0.0015],[-0.0384,-0.0407,-0.0128,-0.0101,0.0059,0.0131],[-0.0098,-0.0195,-0.0235,-0.0097,-0.0067,-0.0150],[-0.0067,0.0038,0.0158,-0.0166,-0.0258,-0.0508],[-0.0072,0.0224,0.0164,-0.0142,-0.0472,-0.0141],[-0.0158,0.0127,-0.0079,-0.0024,0.0022,-0.0017],[0.0002,-0.0137,-0.0176,0.0333,-0.0344,-0.0000],[-0.0073,-0.0415,-0.0042,0.0165,0.0054,-0.0111],[-0.0283,0.0248,-0.0153,-0.0106,0.0204,0.0060],[-0.0019,-0.0446,-0.0168,-0.0219,-0.0152,0.0002],[-0.0150,-0.0032,0.0365,0.0064,-0.0271,0.0006],[-0.0010,-0.0109,0.0184,0.0104,-0.0193,0.0128],[0.0146,0.0035,-0.0068,-0.0162,0.0050,0.0310],[0.0460,0.0110,-0.0022,-0.0176,0.0225,0.0603],[0.0104,-0.0197,0.0008,-0.0417,0.0035,-0.0217],[-0.0077,0.0117,-0.0146,0.0083,-0.0188,-0.0226],[0.0039,-0.0006,-0.0009,0.0089,-0.0107,-0.0449],[0.0072,-0.0181,0.0040,0.0193,-0.0102,-0.0017],[0.0092,0.0163,0.0243,0.0131,0.0170,-0.0180],[-0.0345,0.0454,-0.0112,0.0235,-0.0337,-0.0387],[0.0198,-0.0341,0.0116,0.0199,-0.0414,-0.0140]],requires_gradTrue)wte权重形状:torch.Size([65,6])词嵌入后得到词嵌入tok_emb:tensor([[[-0.0067,0.0038,0.0158,-0.0166,-0.0258,-0.0508],[-0.0384,-0.0407,-0.0128,-0.0101,0.0059,0.0131],[-0.0098,-0.0195,-0.0235,-0.0097,-0.0067,-0.0150],[0.0104,-0.0197,0.0008,-0.0417,0.0035,-0.0217]],[[-0.0077,0.0117,-0.0146,0.0083,-0.0188,-0.0226],[0.0460,0.0110,-0.0022,-0.0176,0.0225,0.0603],[0.0334,0.0103,-0.0260,0.0165,0.0114,0.0168],[-0.0331,0.0482,-0.0087,0.0186,-0.0184,0.0072]]],grad_fnEmbeddingBackward0)tok_emb形状:torch.Size([2,4,6])词嵌入就是在wte权重里查表可以明显的看到第2条句子的第3个字的id是1所以取了[ 0.0334, 0.0103, -0.0260, 0.0165, 0.0114, 0.0168]作为这个字的意思表达。词嵌入层就是使用一推数值来表示一个Token的意思这些数值人类是看不懂的。——谙弆悕