Cosmos-Transfer1-DiffusionRenderer部署指南从本地环境到生产系统的完整流程【免费下载链接】cosmos-transfer1-diffusion-rendererCosmos-Transfer1-DiffusionRenderer: High-quality video de-lighting and re-lighting based on Cosmos video diffusion framework项目地址: https://gitcode.com/gh_mirrors/co/cosmos-transfer1-diffusion-rendererCosmos-Transfer1-DiffusionRenderer是基于NVIDIA Cosmos视频扩散框架的高质量视频去光照和重新光照工具能够实现可控的视频光照操作、编辑和合成数据增强帮助AI系统提升对不同光照条件的鲁棒性。 系统要求与环境准备最低配置要求操作系统Linux已测试Ubuntu 20.04/22.04/24.04Python版本3.10.xGPU要求至少16GB VRAM推荐48GB以上如A100/A6000CUDA版本12.0或更高磁盘空间至少70GB空闲空间快速安装步骤首先克隆项目仓库git clone https://gitcode.com/gh_mirrors/co/cosmos-transfer1-diffusion-renderer cd cosmos-transfer1-diffusion-renderer方法1Conda环境安装推荐# 创建并激活conda环境 conda env create --file cosmos-predict1.yaml conda activate cosmos-predict1 # 安装依赖 pip install -r requirements.txt # 修复Transformer Engine链接问题 ln -sf $CONDA_PREFIX/lib/python3.10/site-packages/nvidia/*/include/* $CONDA_PREFIX/include/ ln -sf $CONDA_PREFIX/lib/python3.10/site-packages/nvidia/*/include/* $CONDA_PREFIX/include/python3.10 # 安装Transformer Engine pip install transformer-engine[pytorch]1.12.0方法2Docker容器安装# 构建Docker镜像 docker build -f Dockerfile . -t cosmos-predict1:latest⚠️ 环境测试安装完成后运行以下命令验证环境CUDA_HOME$CONDA_PREFIX PYTHONPATH$(pwd) python scripts/test_environment.py 模型权重下载模型权重约56GB需通过Hugging Face获取创建Hugging Face访问令牌获取教程登录Hugging Facehuggingface-cli login下载权重CUDA_HOME$CONDA_PREFIX PYTHONPATH$(pwd) python scripts/download_diffusion_renderer_checkpoints.py --checkpoint_dir checkpoints 快速上手图像推理示例1. 图像逆渲染提取G-buffer该步骤从输入图像中估计反照率、金属度、粗糙度、深度和法线等G-buffer数据CUDA_HOME$CONDA_PREFIX PYTHONPATH$(pwd) python cosmos_predict1/diffusion/inference/inference_inverse_renderer.py \ --checkpoint_dir checkpoints --diffusion_transformer_dir Diffusion_Renderer_Inverse_Cosmos_7B \ --dataset_pathasset/examples/image_examples/ --num_video_frames 1 --group_mode webdataset \ --video_save_folderasset/example_results/image_delighting/ --save_videoFalse图1Cosmos-Transfer1-DiffusionRenderer处理的示例图像展示了输入图像的细节质量2. 图像重新光照使用上一步生成的G-buffer数据和环境贴图进行重新光照CUDA_HOME$CONDA_PREFIX PYTHONPATH$(pwd) python cosmos_predict1/diffusion/inference/inference_forward_renderer.py \ --checkpoint_dir checkpoints --diffusion_transformer_dir Diffusion_Renderer_Forward_Cosmos_7B \ --dataset_pathasset/example_results/image_delighting/gbuffer_frames --num_video_frames 1 \ --envlight_ind 0 1 2 3 --use_custom_envmapTrue \ --video_save_folderasset/example_results/image_relighting/ 视频处理流程1. 视频帧提取首先从视频中提取帧python scripts/dataproc_extract_frames_from_video.py --input_folder asset/examples/video_examples/ --output_folder asset/examples/video_frames_examples/ --frame_rate 24 --resize 1280x704 --max_frames572. 视频逆渲染CUDA_HOME$CONDA_PREFIX PYTHONPATH$(pwd) python cosmos_predict1/diffusion/inference/inference_inverse_renderer.py \ --checkpoint_dir checkpoints --diffusion_transformer_dir Diffusion_Renderer_Inverse_Cosmos_7B \ --dataset_pathasset/examples/video_frames_examples/ --num_video_frames 57 --group_mode folder \ --video_save_folderasset/example_results/video_delighting/3. 视频重新光照CUDA_HOME$CONDA_PREFIX PYTHONPATH$(pwd) python cosmos_predict1/diffusion/inference/inference_forward_renderer.py \ --checkpoint_dir checkpoints --diffusion_transformer_dir Diffusion_Renderer_Forward_Cosmos_7B \ --dataset_pathasset/example_results/video_delighting/gbuffer_frames --num_video_frames 57 \ --envlight_ind 0 1 2 3 --use_custom_envmapTrue \ --video_save_folderasset/example_results/video_relighting/图2Cosmos-Transfer1-DiffusionRenderer的视频重新光照效果展示包含输入视频、估计的基础颜色、深度、法向量和多种光照变换结果⚙️ 生产环境优化建议内存优化对于显存不足的情况添加--offload_diffusion_transformer --offload_tokenizer参数降低输入分辨率如--resize 640x360减少同时处理的视频帧数性能提升使用多GPU并行处理需修改配置文件cosmos_predict1/diffusion/config/inference/启用混合精度推理添加--fp16参数预缓存环境贴图到内存 更多资源官方文档INSTALL.md推理脚本cosmos_predict1/diffusion/inference/配置文件cosmos_predict1/diffusion/config/示例数据asset/examples/通过以上步骤您可以快速部署Cosmos-Transfer1-DiffusionRenderer并实现高质量的图像和视频光照处理。无论是本地开发还是生产环境部署这套流程都能帮助您高效地利用这一强大的视频扩散渲染工具。【免费下载链接】cosmos-transfer1-diffusion-rendererCosmos-Transfer1-DiffusionRenderer: High-quality video de-lighting and re-lighting based on Cosmos video diffusion framework项目地址: https://gitcode.com/gh_mirrors/co/cosmos-transfer1-diffusion-renderer创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考