问题为段言的AI辅助编程模型进行微调训练duan:基于中文深层认知特性的编程语言项目 - AtomGit将max-len的size 设为8k,结果爆显存了后来试了试就2048可以pass第 3 步配置 LoRA trainable params: 2,162,688 || all params: 496,195,456 || trainable%: 0.4359 第 4 步加载数据集 训练数据: 978 条 max_len: 8192 类别分布: 复合: 135 列表: 131 字符串: 124 异常: 99 类: 82 字典: 73 段落: 54 函数: 51 循环: 48 变量: 48 条件: 48 暗坑: 46 文件: 30 导入: 9 第 5 步训练GPU GPU: Tesla T4 预计总步数: ~184 预计时间: ~291 秒 (4.9 分钟) batch_size: 2 x grad_accum: 8 等效 batch 16 epochs: 3, lr: 0.0002, LoRA rank: 16 precision: bf16, QLoRA: False warmup_ratio is deprecated and will be removed in v5.2. Use warmup_steps instead. The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizers values. Updated tokens: {bos_token_id: None, pad_token_id: 151643}. 0%| | 0/93 [00:00?, ?it/s]Traceback (most recent call last): File /kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py, line 652, in module main() File /kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py, line 620, in main final_dir train( ^^^^^^ File /kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py, line 454, in train trainer.train() File /usr/local/lib/python3.12/dist-packages/transformers/trainer.py, line 2174, in train return inner_training_loop( ^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/trainer.py, line 2536, in _inner_training_loop tr_loss_step self.training_step(model, inputs, num_items_in_batch) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/trainer.py, line 3809, in training_step loss self.compute_loss(model, inputs, num_items_in_batchnum_items_in_batch) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/trainer.py, line 3880, in compute_loss outputs model(**inputs) ^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1776, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1787, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/parallel/data_parallel.py, line 197, in forward outputs self.parallel_apply(replicas, inputs, module_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/parallel/data_parallel.py, line 214, in parallel_apply return parallel_apply( ^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/parallel/parallel_apply.py, line 133, in parallel_apply output.reraise() File /usr/local/lib/python3.12/dist-packages/torch/_utils.py, line 775, in reraise raise exception torch.OutOfMemoryError: Caught OutOfMemoryError in replica 0 on device 0. Original Traceback (most recent call last): File /usr/local/lib/python3.12/dist-packages/torch/nn/parallel/parallel_apply.py, line 103, in _worker output module(*input, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1776, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1787, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/peft/peft_model.py, line 1993, in forward return self.base_model( ^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1776, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1787, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/peft/tuners/tuners_utils.py, line 330, in forward return self.model.forward(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/utils/generic.py, line 835, in wrapper output func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py, line 475, in forward outputs: BaseModelOutputWithPast self.model( ^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1776, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1787, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/utils/generic.py, line 1002, in wrapper outputs func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py, line 410, in forward hidden_states decoder_layer( ^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/modeling_layers.py, line 92, in __call__ return self._gradient_checkpointing_func(partial(super().__call__, **kwargs), *args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/_compile.py, line 54, in inner return disable_fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/_dynamo/eval_frame.py, line 1181, in _fn return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/utils/checkpoint.py, line 512, in checkpoint ret function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1776, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1787, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py, line 298, in forward hidden_states, _ self.self_attn( ^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1776, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/torch/nn/modules/module.py, line 1787, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/models/qwen2/modeling_qwen2.py, line 234, in forward attn_output, attn_weights attention_interface( ^^^^^^^^^^^^^^^^^^^^ File /usr/local/lib/python3.12/dist-packages/transformers/integrations/sdpa_attention.py, line 92, in sdpa_attention_forward attn_output torch.nn.functional.scaled_dot_product_attention( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 7.00 GiB. GPU 0 has a total capacity of 14.56 GiB of which 5.61 GiB is free. Including non-PyTorch memory, this process has 8.95 GiB memory in use. Of the allocated memory 8.69 GiB is allocated by PyTorch, and 72.39 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_ALLOC_CONFexpandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)建议PYTORCH_ALLOC_CONFexpandable还是不行设为2048解决问题[预设] qwen3.5-2b: Qwen3.5-2B (2B, 多模态架构, LoRA ~5GB / QLoRA ~3GB, 需 transformers5.0) model_path/kaggle/working/duan/tools/ai_copilot/model_cache/qwen3.5-2b output_dir/kaggle/working/duan/tools/ai_copilot/output/qwen3.5_2b_duan_gpu max_len1024, batch_size1, grad_accum4 lora_rank32, lora_alpha64, lr0.0001 [T4 检测] Tesla T4 (15.6GB, SM 7.5) [Unsloth] 自动处理精度和显存优化 当前配置: max_len1024, batch_size1, QLoRATrue [WARN] Qwen3.5 系列官方不建议使用 QLoRA量化差异可能高于正常水平 但显存不足时仍可使用如效果不佳请换用 bf16 LoRA 模式 继续使用 QLoRA 训练... 第 1 步环境检查GPU 模式 [OK] PyTorch 2.10.0cu128 [OK] 检测到 2 块 GPU: GPU 0: Tesla T4 (15.6 GB) GPU 1: Tesla T4 (15.6 GB) [OK] Kaggle 双 T4 环境检测到将启用 DDP 分布式训练 [INFO] 此 GPU 不支持原生 bf16需 SM 8.0将自动使用 fp16 [OK] transformers 5.14.1 Skipping import of cpp extensions due to incompatible torch version. Please upgrade to torch 2.11.0 (found 2.10.0cu128). [OK] peft 0.19.1 [OK] bitsandbytes 0.49.2 [OK] torchao 0.17.0 [OK] unsloth 已安装可用 --unsloth 启用 [OK] 数据集: /kaggle/working/duan/tools/ai_copilot/sft_dataset.jsonl (978 条) 第 2 步加载模型 模型路径: /kaggle/working/duan/tools/ai_copilot/model_cache/qwen3.5-2b QLoRA 4bit: 是 [多GPU] 检测到 2 块 GPU将启用 DDP 分布式训练 [Kaggle] 双 T4 环境已确认 [INFO] GPU 不支持 bf16Turing 架构如 T4使用 fp16 替代 [INFO] 使用 fp16 精度GPU 不支持 bf16 /kaggle/working/duan/tools/ai_copilot/train_gpu_lora.py:589: UserWarning: WARNING: Unsloth should be imported before transformers, peft to ensure all optimizations are applied. Your code may run slower or encounter memory issues without these optimizations. Please restructure your imports with import unsloth at the top of your file. from unsloth import FastLanguageModel Unsloth: Will patch your computer to enable 2x faster free finetuning. [WARN] Unsloth 导入失败: name auto_docstring is not defined [WARN] 自动降级到标准 HFPEFT 模式 Unsloth 可能与当前 Python/transformers 版本不兼容 可去掉 --unsloth 使用标准模式训练 [FIX] 已屏蔽 unsloth_zoo monkey-patch恢复标准 HF 路径 [transformers] The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d Loading weights: 100%|███████████████████████| 320/320 [00:0100:00, 242.46it/s] [QLoRA] 模型已 4bit 量化加载, SDPA attention [WARN] QLoRA 多 GPU: bitsandbytes 量化模型仅在 GPU 0 上运行 多 GPU 并行不生效建议去掉 --qlora 以使用双 T4 DDP 设备: cuda 模型参数量: 1195.5M [OK] gradient checkpointing 已启用 第 3 步配置 LoRA trainable params: 2,949,120 || all params: 1,884,774,208 || trainable%: 0.1565 第 4 步加载数据集 训练数据: 978 条 max_len: 1024 类别分布: 复合: 135 列表: 131 字符串: 124 异常: 99 类: 82 字典: 73 段落: 54 函数: 51 循环: 48 变量: 48 条件: 48 暗坑: 46 文件: 30 导入: 9 第 5 步训练GPU GPU: Tesla T4 (15.6 GB) 预计总步数: ~734 预计时间: ~703 秒 (11.7 分钟) batch_size: 1 x grad_accum: 4 等效 batch 4 epochs: 3, lr: 0.0001, LoRA rank: 32 precision: fp16 权重 fp32 LoRA (无 AMP), QLoRA: True [OPTIM] adamw_8bit (8-bit AdamW, 省 ~75% 优化器显存) [transformers] The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizers values. Updated tokens: {eos_token_id: 248046, pad_token_id: 248044}. {loss: 2.203, grad_norm: 10.14, learning_rate: 1.111e-05, epoch: 0.0409} {loss: 1.938, grad_norm: 11.06, learning_rate: 2.5e-05, epoch: 0.0818} {loss: 1.842, grad_norm: 8.35, learning_rate: 3.889e-05, epoch: 0.1227} {loss: 1.314, grad_norm: 4.42, learning_rate: 5.278e-05, epoch: 0.1636} {loss: 0.9932, grad_norm: 4.952, learning_rate: 6.667e-05, epoch: 0.2045} {loss: 0.9426, grad_norm: 4.323, learning_rate: 8.056e-05, epoch: 0.2454} {loss: 0.8306, grad_norm: 4.247, learning_rate: 9.444e-05, epoch: 0.2863} {loss: 0.6309, grad_norm: 4.116, learning_rate: 9.998e-05, epoch: 0.3272} {loss: 0.4392, grad_norm: 4.536, learning_rate: 9.986e-05, epoch: 0.3681} {loss: 0.5706, grad_norm: 5.684, learning_rate: 9.962e-05, epoch: 0.409} {loss: 0.4475, grad_norm: 3.037, learning_rate: 9.928e-05, epoch: 0.4499} {loss: 0.335, grad_norm: 3.829, learning_rate: 9.883e-05, epoch: 0.4908} {loss: 0.336, grad_norm: 4.08, learning_rate: 9.827e-05, epoch: 0.5317} {loss: 0.3219, grad_norm: 3.651, learning_rate: 9.76e-05, epoch: 0.5726} {loss: 0.3859, grad_norm: 5.835, learning_rate: 9.682e-05, epoch: 0.6135} {loss: 0.2622, grad_norm: 3.23, learning_rate: 9.594e-05, epoch: 0.6544} {loss: 0.453, grad_norm: 4.503, learning_rate: 9.496e-05, epoch: 0.6953} {loss: 0.3871, grad_norm: 4.261, learning_rate: 9.388e-05, epoch: 0.7362} {loss: 0.3932, grad_norm: 3.55, learning_rate: 9.27e-05, epoch: 0.7771} {loss: 0.2394, grad_norm: 2.763, learning_rate: 9.143e-05, epoch: 0.818} {loss: 0.3433, grad_norm: 2.859, learning_rate: 9.006e-05, epoch: 0.8589} {loss: 0.3537, grad_norm: 3.831, learning_rate: 8.86e-05, epoch: 0.8998} {loss: 0.2138, grad_norm: 3.705, learning_rate: 8.706e-05, epoch: 0.9407} {loss: 0.2411, grad_norm: 4.92, learning_rate: 8.544e-05, epoch: 0.9816} {loss: 0.1983, grad_norm: 3.793, learning_rate: 8.374e-05, epoch: 1.016} {loss: 0.2174, grad_norm: 7.729, learning_rate: 8.196e-05, epoch: 1.057} {loss: 0.1598, grad_norm: 3.7, learning_rate: 8.011e-05, epoch: 1.098} {loss: 0.2643, grad_norm: 4.253, learning_rate: 7.819e-05, epoch: 1.139} {loss: 0.2372, grad_norm: 3.132, learning_rate: 7.622e-05, epoch: 1.18} {loss: 0.2047, grad_norm: 1.599, learning_rate: 7.418e-05, epoch: 1.221} {loss: 0.1716, grad_norm: 2.777, learning_rate: 7.209e-05, epoch: 1.262} {loss: 0.162, grad_norm: 5.104, learning_rate: 6.995e-05, epoch: 1.303} {loss: 0.1407, grad_norm: 2.402, learning_rate: 6.776e-05, epoch: 1.344} {loss: 0.1522, grad_norm: 3.454, learning_rate: 6.554e-05, epoch: 1.384} {loss: 0.1693, grad_norm: 3.723, learning_rate: 6.328e-05, epoch: 1.425} {loss: 0.1325, grad_norm: 3.864, learning_rate: 6.099e-05, epoch: 1.466} {loss: 0.1998, grad_norm: 2.124, learning_rate: 5.868e-05, epoch: 1.507} {loss: 0.1384, grad_norm: 3.873, learning_rate: 5.635e-05, epoch: 1.548} {loss: 0.1281, grad_norm: 2.123, learning_rate: 5.401e-05, epoch: 1.589} {loss: 0.2184, grad_norm: 5.875, learning_rate: 5.165e-05, epoch: 1.63} {loss: 0.1393, grad_norm: 2.365, learning_rate: 4.929e-05, epoch: 1.671} {loss: 0.1659, grad_norm: 3.061, learning_rate: 4.694e-05, epoch: 1.712} {loss: 0.09851, grad_norm: 3.075, learning_rate: 4.459e-05, epoch: 1.753} {loss: 0.1217, grad_norm: 2.83, learning_rate: 4.225e-05, epoch: 1.793} {loss: 0.1022, grad_norm: 2.245, learning_rate: 3.993e-05, epoch: 1.834} {loss: 0.1441, grad_norm: 4.338, learning_rate: 3.763e-05, epoch: 1.875} {loss: 0.1768, grad_norm: 2.467, learning_rate: 3.536e-05, epoch: 1.916} {loss: 0.09541, grad_norm: 1.974, learning_rate: 3.312e-05, epoch: 1.957} {loss: 0.07662, grad_norm: 1.124, learning_rate: 3.092e-05, epoch: 1.998} {loss: 0.05175, grad_norm: 0.7626, learning_rate: 2.876e-05, epoch: 2.033} {loss: 0.07802, grad_norm: 0.5053, learning_rate: 2.665e-05, epoch: 2.074} {loss: 0.1381, grad_norm: 2.901, learning_rate: 2.459e-05, epoch: 2.115} {loss: 0.1125, grad_norm: 2.289, learning_rate: 2.259e-05, epoch: 2.155} {loss: 0.08472, grad_norm: 2.452, learning_rate: 2.065e-05, epoch: 2.196} {loss: 0.09751, grad_norm: 0.896, learning_rate: 1.877e-05, epoch: 2.237} {loss: 0.1345, grad_norm: 3.427, learning_rate: 1.697e-05, epoch: 2.278} {loss: 0.0417, grad_norm: 1.938, learning_rate: 1.523e-05, epoch: 2.319} {loss: 0.08852, grad_norm: 1.324, learning_rate: 1.358e-05, epoch: 2.36} {loss: 0.1007, grad_norm: 3.997, learning_rate: 1.2e-05, epoch: 2.401} {loss: 0.09326, grad_norm: 0.5659, learning_rate: 1.051e-05, epoch: 2.442} {loss: 0.09737, grad_norm: 1.744, learning_rate: 9.11e-06, epoch: 2.483} {loss: 0.05748, grad_norm: 3.018, learning_rate: 7.799e-06, epoch: 2.524} {loss: 0.08245, grad_norm: 2.699, learning_rate: 6.581e-06, epoch: 2.564} {loss: 0.08583, grad_norm: 2.702, learning_rate: 5.46e-06, epoch: 2.605} {loss: 0.05044, grad_norm: 1.507, learning_rate: 4.439e-06, epoch: 2.646} {loss: 0.052, grad_norm: 1.309, learning_rate: 3.518e-06, epoch: 2.687} {loss: 0.08038, grad_norm: 1.494, learning_rate: 2.701e-06, epoch: 2.728} {loss: 0.08601, grad_norm: 2.069, learning_rate: 1.989e-06, epoch: 2.769} {loss: 0.04808, grad_norm: 2.772, learning_rate: 1.384e-06, epoch: 2.81} {loss: 0.05216, grad_norm: 2.606, learning_rate: 8.874e-07, epoch: 2.851} {loss: 0.07321, grad_norm: 4.124, learning_rate: 4.998e-07, epoch: 2.892} {loss: 0.03868, grad_norm: 1.86, learning_rate: 2.223e-07, epoch: 2.933} {loss: 0.05519, grad_norm: 3.036, learning_rate: 5.562e-08, epoch: 2.973} {train_runtime: 8815, train_samples_per_second: 0.333, train_steps_per_second: 0.042, train_loss: 0.3028, epoch: 3} 100%|███████████████████████████████████████| 369/369 [2:26:5500:00, 23.89s/it] 训练完成耗时 8816 秒 (146.9 分钟) LoRA 权重保存到: /kaggle/working/duan/tools/ai_copilot/output/qwen3.5_2b_duan_gpu/final 训练信息保存到: /kaggle/working/duan/tools/ai_copilot/output/qwen3.5_2b_duan_gpu/training_info.json 全部完成 训练完毕合并模型Writing model shards: 100%|███████████████████████| 1/1 [00:0900:00, 9.72s/it] 合并后模型保存到: /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b Modelfile 保存到: /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/Modelfile 完成 合并后模型: /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b 下一步: 推理测试: python local_infer.py --fine-tuned模型合并后转为gguf格式!python llama.cpp/convert_hf_to_gguf.py \ /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b \ --outfile /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf \ --outtype f16输出INFO:gguf.gguf_writer:Writing the following files: INFO:gguf.gguf_writer:/kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf: n_tensors 320, total_size 3.8G Writing: 100%|███████████████████████████| 3.76G/3.76G [00:3200:00, 115Mbyte/s] INFO:hf-to-gguf:Model successfully exported to /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf量化再编译llama.cpp%cd /kaggle/working/llama.cpp !mkdir build %cd build !cmake .. !make -j4量化%cd /kaggle/working/llama.cpp/build/bin/ # 执行量化 (例如转为 Q4_K_M) !./llama-quantize \ /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator.gguf \ /kaggle/working/duan/tools/ai_copilot/output/duan_translator_merged_3.5_2b/duan_translator_q4_k_m.gguf \ Q4_K_M然后打包成tar包下载到本地文件大小skywalkfb98:~/Downloads $ ls -ls /home/skywalk/Downloads/kaggle/working/duan/tool total 1239105 1239101 -rw-r--r-- 1 skywalk skywalk 1274396096 7月 23 23:14 duan_translator_q4_k_m.gguf 5 -rw-r--r-- 1 skywalk skywalk 610 7月 23 22:40 Modelfile