SAM 2 视频分割实战从参考帧点提示到多目标跟踪这篇教程根据我复现 SAM 2 视频分割流程时整理重点演示如何加载视频 predictor把视频拆成帧在参考帧上给出目标点提示然后沿视频传播 mask。相比单图分割视频分割更关注目标在时间维度上的连续性。本文适合用来理解 SAM 2 如何从一帧提示扩展到整段视频跟踪。本文会重点跑通以下流程安装 SAM 2 视频推理环境下载 checkpoint 并加载视频 predictor准备视频并拆分为图像帧在参考帧上为多个目标添加点提示执行视频传播并导出带 mask 的结果视频如果你正在系统学习目标检测、实例分割、OCR、多目标跟踪或视觉大模型建议收藏本文配套 notebook、示例图片和运行环境说明后续会继续整理。如果环境配置卡住可以在评论区说明具体报错。 文章目录SAM 2 视频分割实战从参考帧点提示到多目标跟踪⚙️ 环境准备 加载 SAM 2 视频模型️ 预处理视频帧 初始化推理状态 在参考帧选择目标 执行视频分割推理 小结 同系列教程汇总⚙️ 环境准备先检查运行环境并安装依赖。建议在 Colab 或带 NVIDIA GPU 的环境中运行避免训练或视频推理阶段显存不足。!nvidia-smiimportos HOMEos.getcwd()print(HOME:,HOME)!git clone https://github.com/facebookresearch/segment-anything-2.git%cd{HOME}/segment-anything-2!pip install-e.-q !python setup.py build_ext--inplace!pip install-q supervision[assets]jupyter_bbox_widget!mkdir-p{HOME}/checkpoints !wget-q https://dl.fbaipublicfiles.com/segment_anything_2/072824/sam2_hiera_tiny.pt-P{HOME}/checkpoints !wget-q https://dl.fbaipublicfiles.com/segment_anything_2/072824/sam2_hiera_small.pt-P{HOME}/checkpoints !wget-q https://dl.fbaipublicfiles.com/segment_anything_2/072824/sam2_hiera_base_plus.pt-P{HOME}/checkpoints !wget-q https://dl.fbaipublicfiles.com/segment_anything_2/072824/sam2_hiera_large.pt-P{HOME}/checkpointsimportcv2importtorchimportbase64importnumpyasnpimportsupervisionassvfrompathlibimportPathfromsupervision.assetsimportdownload_assets,VideoAssetsfromsam2.build_samimportbuild_sam2_video_predictor IS_COLABTrueifIS_COLAB:fromgoogle.colabimportoutput output.enable_custom_widget_manager()fromjupyter_bbox_widgetimportBBoxWidgettorch.autocast(device_typecuda,dtypetorch.bfloat16).__enter__()iftorch.cuda.get_device_properties(0).major8:torch.backends.cuda.matmul.allow_tf32Truetorch.backends.cudnn.allow_tf32True 加载 SAM 2 视频模型视频任务需要加载 video predictor它会维护视频帧的推理状态。DEVICEtorch.device(cudaiftorch.cuda.is_available()elsecpu)CHECKPOINTf{HOME}/checkpoints/sam2_hiera_large.ptCONFIGsam2_hiera_l.yamlsam2_modelbuild_sam2_video_predictor(CONFIG,CHECKPOINT)️ 预处理视频帧为了控制显存占用先裁剪视频片段并按比例缩放再拆成连续帧。# 请从数据集后台下载示例视频上传到当前环境后修改 SOURCE_VIDEO。SOURCE_VIDEOf{HOME}/basketball.mp4sv.VideoInfo.from_video_path(SOURCE_VIDEO)SCALE_FACTOR0.5START_IDX100END_IDX300SOURCE_FRAMESPath(HOME)/Path(SOURCE_VIDEO).stem SOURCE_FRAMES.mkdir(parentsTrue,exist_okTrue)frames_generatorsv.get_video_frames_generator(SOURCE_VIDEO,startSTART_IDX,endEND_IDX)images_sinksv.ImageSink(target_dir_pathSOURCE_FRAMES.as_posix(),overwriteTrue,image_name_pattern{:05d}.jpeg)withimages_sink:forframeinframes_generator:framesv.scale_image(frame,SCALE_FACTOR)images_sink.save_image(frame)TARGET_VIDEOPath(HOME)/f{Path(SOURCE_VIDEO).stem}-result.mp4SOURCE_FRAME_PATHSsorted(sv.list_files_with_extensions(SOURCE_FRAMES.as_posix(),extensions[jpeg])) 初始化推理状态初始化状态后SAM 2 才能把参考帧提示传播到后续帧。inference_statesam2_model.init_state(video_pathSOURCE_FRAMES.as_posix())sam2_model.reset_state(inference_state) 在参考帧选择目标在第一帧或关键帧上为每个目标打点并给不同目标分配 object id。defencode_image(filepath):withopen(filepath,rb)asf:image_bytesf.read()encodedstr(base64.b64encode(image_bytes),utf-8)returndata:image/jpg;base64,encodedOBJECTS[ball,player-1,player-2]FRAME_IDX0FRAME_PATHPath(SOURCE_FRAMES)/f{FRAME_IDX:05d}.jpegwidgetBBoxWidget(classesOBJECTS)widget.imageencode_image(FRAME_PATH)widgetdefault_box[{x:705,y:302,width:0,height:0,label:ball},{x:587,y:300,width:0,height:0,label:player-1},{x:753,y:267,width:0,height:0,label:player-2}]boxeswidget.bboxesifwidget.bboxeselsedefault_boxforobject_id,labelinenumerate(OBJECTS,start1):boxes[boxforboxinwidget.bboxesifbox[label]label]iflen(boxes)0:continuepointsnp.array([[box[x],box[y]]forboxinboxes],dtypenp.float32)labelsnp.ones(len(points))_,object_ids,mask_logitssam2_model.add_new_points(inference_stateinference_state,frame_idxFRAME_IDX,obj_idobject_id,pointspoints,labelslabels,) 执行视频分割推理最后沿视频传播 mask并把标注后的帧写入目标视频。video_infosv.VideoInfo.from_video_path(SOURCE_VIDEO)video_info.widthint(video_info.width*SCALE_FACTOR)video_info.heightint(video_info.height*SCALE_FACTOR)COLORS[#FF1493,#00BFFF,#FF6347,#FFD700]mask_annotatorsv.MaskAnnotator(colorsv.ColorPalette.from_hex(COLORS),color_lookupsv.ColorLookup.CLASS)frame_sample[]withsv.VideoSink(TARGET_VIDEO.as_posix(),video_infovideo_info)assink:forframe_idx,object_ids,mask_logitsinsam2_model.propagate_in_video(inference_state):frame_pathSOURCE_FRAME_PATHS[frame_idx]framecv2.imread(frame_path)masks(mask_logits0.0).cpu().numpy()masksnp.squeeze(masks).astype(bool)detectionssv.Detections(xyxysv.mask_to_xyxy(masksmasks),maskmasks,class_idnp.array(object_ids))annotated_framemask_annotator.annotate(sceneframe.copy(),detectionsdetections)sink.write_frame(annotated_frame)ifframe_idx%video_info.fps0:frame_sample.append(annotated_frame)sv.plot_images_grid(imagesframe_sample[:4],grid_size(2,2)) 小结SAM 2 视频分割的核心是先建立视频帧状态再把参考帧提示传播到后续帧。要获得稳定结果参考帧选择、目标点位置和视频分辨率都需要结合显存一起调。这一类 notebook 建议按“先环境、再数据、再单样例、最后批量推理”的顺序复现。遇到报错时优先检查 GPU、依赖版本、数据集目录和模型权重路径。后续我会继续按源项目顺序整理同系列中的目标检测、实例分割、OCR、多目标跟踪和视觉大模型教程。 同系列教程汇总Google Gemini 3.5 Flash 零样本目标检测教程从提示词到可视化结果GLM-OCR 文档识别实战教程从验证码、公式到车牌 OCRRF-DETR ByteTrack 多目标跟踪实战教程从命令行到 Python 视频轨迹可视化SAM 3 图像分割实战教程文本、框和点提示的多种分割方式SAM 3 视频分割实战教程用文本提示分割并跟踪视频中的目标SAM 2 视频分割实战从参考帧点提示到多目标跟踪-本文