aclnnSparseFlashMlaGradMetadata【免费下载链接】ops-transformer本项目是CANN提供的transformer类大模型算子库实现网络在NPU上加速计算。项目地址: https://gitcode.com/cann/ops-transformer 查看源码产品支持情况产品是否支持Ascend 950PR/Ascend 950DT√Atlas A3 训练系列产品/Atlas A3 推理系列产品xAtlas A2 训练系列产品/Atlas A2 推理系列产品xAtlas 200I/500 A2 推理产品xAtlas 推理系列产品xAtlas 训练系列产品x功能说明接口功能该接口为AI CPU算子接口是aclnnSparseFlashMlaGrad算子的前置算子接口。根据aclnnSparseFlashMlaGrad算子接口的输入信息计算并输出负载均衡结果。输出结果可以作为aclnnSparseFlashMlaGrad算子接口的输入减少aclnnSparseFlashMlaGrad算子接口的执行耗时。该算子不建议单独使用建议与aclnnSparseFlashMlaGrad算子配合使用形成完整的工作流。接受aclnnSparseFlashMlaGrad算子接口输入数据shape信息包含batchSize、qSeqlen、kSeqlen、mask。通过对输入分块并模拟计算耗时均匀分配分块到可用核上以降低aclnnSparseFlashMlaGrad算子的整体计算耗时并提高硬件利用率。分配结果输出后后续作为输入供aclnnSparseFlashMlaGrad算子使用。分配结果包含每个AIC核基本块的起始点和终止点已经每个AIV核的FD任务信息。详细内容可以参考调用示例。函数原型每个算子分为两段式接口必须先调用aclnnSparseFlashMlaGradMetadataGetWorkspaceSize获取workspace大小在调用aclnnSparseFlashMlaGradMetadata执行计算。aclnnStatus aclnnSparseFlashMlaGradMetadataGetWorkspaceSize( const aclTensor *cuSeqlensQOptional, const aclTensor *cuSeqlensOriKvOptional, const aclTensor *cuSeqlensCmpKvOptional, const aclTensor *sequsedQOptional, const aclTensor *sequsedOriKvOptional, const aclTensor *sequsedCmpKvOptional, const aclTensor *cmpResidualKvOptional, const aclTensor *oriTopkLengthOptional, const aclTensor *cmpTopkLengthOptional, int64_t numHeadsQ, int64_t numHeadsKv, int64_t headDim, int64_t batchSize, int64_t maxSeqlenQ, int64_t maxSeqlenOriKv, int64_t maxSeqlenCmpKv, int64_t oriTopk, int64_t cmpTopk, int64_t cmpRatio, int64_t oriMaskMode, int64_t cmpMaskMode, int64_t oriWinLeft, int64_t oriWinRight, const char *layoutQOptional, const char *layoutKvOptional, bool hasOriKv, bool hasCmpKv, const aclTensor *metadata, uint64_t *workspaceSize, aclOpExecutor **executor)aclnnStatus aclnnSparseFlashMlaGradMetadata( void *workspace, uint64_t workspaceSize, aclOpExecutor *executor, aclrtStream stream)aclnnSparseFlashMlaGradMetadataGetWorkspaceSize参数说明参数名输入/输出描述使用说明数据类型数据格式维度(shape)非连续TensorcuSeqlensQOptionalaclTensor*输入表示不同Batch中Query的有效Sequence Length。支持空TensorlayoutQOptional为TND场景下必传。第一个值为额外值并固定为0。shape固定为(B1, )。INT32ND1维√cuSeqlensOriKvOptionalaclTensor*输入表示不同Batch中ori_kv的有效Sequence Length。支持空Tensor。layoutKvOptional为TND场景下必传。第一个值为额外值并固定为0。shape固定为(B1, )。INT32ND1维√cuSeqlensCmpKvOptionalaclTensor*输入表示不同Batch中cmp_kv的有效Sequence Length。支持空Tensor。layoutKvOptional为TND场景下必传。第一个值为额外值并固定为0。shape固定为(B1, )。INT32ND1维√sequsedQOptionalaclTensor*输入表示不同Batch中Query实际参与运算的Sequence Length。支持空Tensor。shape固定为(B, )。INT32ND1维√sequsedOriKvOptionalaclTensor*输入表示不同Batch中ori_kv实际参与运算的Sequence Length。支持空Tensor。shape固定为(B, )。INT32ND1维√sequsedCmpKvOptionalaclTensor*输入表示不同Batch中cmp_kv实际参与运算的Sequence Length。支持空Tensor。shape固定为(B, )。INT32ND1维√cmpResidualKvOptionalaclTensor*输入表示不同Batch中cmp_kv压缩后Sequence Length的余数配合cmpRatio实现cmp_kv部分的mask和负载计算。支持空Tensor。cmpRatio不为1且mask为3场景下必传。shape固定为(B, )。INT32ND1维√oriTopkLengthOptionalaclTensor*输入表示不同q token对应的ori_kv部分关键稀疏token的个数。支持空Tensor。shape为(B, S1, N2)或(T1, N2)。INT32ND2维、3维√cmpTopkLengthOptionalaclTensor*输入表示不同q token对应的cmp_kv部分关键稀疏token的个数。支持空Tensor。shape为(B, S1, N2)或(T1, N2)。INT32ND2维、3维√numHeadsQint64_t输入表示Query的head个数。当前支持[1, 128]。----numHeadsKvint64_t输入Key和Value对应的多头数。当前仅支持1。----headDimint64_t输入注意力头的维度。当前仅支持512。----batchSizeint64_t输入表示Batch数量。支持非负数。建议值为0。----maxSeqlenQint64_t输入表示Query的最长Sequence Length。支持非负数。建议值为0。----maxSeqlenOriKvint64_t输入表示ori_kv的最长Sequence Length。支持非负数。建议值为0。----maxSeqlenCmpKvint64_t输入表示cmp_kv的最长Sequence Length。支持非负数。建议值为0。----oriTopkint64_t输入表示ori_kv中筛选出的关键稀疏token的个数。0表示非稀疏场景。支持非负数。建议值为0。----cmpTopkint64_t输入表示cmp_kv中筛选出的关键稀疏token的个数。0表示非稀疏场景。支持非负数。建议值为0。----cmpRatioint64_t输入表示对cmp_kv的压缩率。当前支持[1, 128]。建议值1表示无压缩。----oriMaskModeint64_t输入表示q和ori_kv计算的mask模式。0表示No mask。3表示rightDownCausal模式。4表示sliding window模式对应由oriWinLeft和oriWinRight划分的窗口场景。建议值为0。----cmpMaskModeint64_t输入表示q和cmp_kv计算的mask模式。0表示No mask。3表示rightDownCausal模式对应以右顶点为划分的下三角场景。建议值为0。----oriWinLeftint64_t输入表示q和ori_kv计算中q对过去token计算的数量。取值范围≥-1-1表示无穷大。建议值为-1。----oriWinRightint64_t输入表示q和ori_kv计算中q对未来token计算的数量。取值范围≥-1-1表示无穷大。建议值为-1。----layoutQOptionalchar*输入表示Query的排列格式。支持 BSND、TND。建议值为BSND。----layoutKvOptionalchar*输入表示Key的排列格式。支持 BSND、TND。建议值为BSND。----hasOriKvbool输入用于标识是否含有ori_kv。true: 含有ori_kv。false: 不含有ori_kv。建议值为true。----hasCmpKvbool输入用于标识是否含有cmp_kv。true: 含有cmp_kv。false: 不含有cmp_kv。建议值为true。----metadataaclTensor*输出表示负载均衡结果输出。-INT32ND1维shape固定为(1024)×workspaceSizeuint64_t*输出返回需要在Device侧申请的workspace大小。-----executoraclOpExecutor**输出返回op执行器包含了算子计算流程。-----返回值返回aclnnStatus状态码具体参见aclnn返回码。第一段接口完成入参校验出现以下场景时报错返回值错误码描述ACLNN_ERR_INNER_CREATE_EXECUTOR561101创建aclOpExecutor失败。ACLNN_ERR_INNER_NULLPTR561103参数workspaceSize、executor是空指针或参数cuSeqlensQOptional、cuSeqlensOriKvOptional、cuSeqlensCmpKvOptional、sequsedQOptional、sequsedOriKvOptional、sequsedCmpKvOptional、cmpResidualKvOptional、oriTopkLengthOptional、cmpTopkLengthOptional进行Contiguous处理后为空指针。ACLNN_ERR_PARAM_INVALID161002参数cuSeqlensQOptional、cuSeqlensOriKvOptional、cuSeqlensCmpKvOptional、sequsedQOptional、sequsedOriKvOptional、sequsedCmpKvOptional、cmpResidualKvOptional、oriTopkLengthOptional、cmpTopkLengthOptional、numHeadsQ、numHeadsKv、headDim、batchSize、maxSeqlenQ、maxSeqlenOriKv、maxSeqlenCmpKv、oriTopk、cmpTopk、cmpRatio、oriMaskMode、cmpMaskMode、oriWinLeft、oriWinRight、layoutQOptional、layoutKvOptional、hasOriKv、hasCmpKv的规格不在支持范围内。aclnnSparseFlashMlaGradMetadata参数说明参数名输入/输出描述workspace输入在Device侧申请的workspace内存地址。workspaceSize输入在Device侧申请的workspace大小由第一段接口aclnnSparseFlashMlaGradMetadataGetWorkspaceSize获取。executor输入op执行器包含了算子计算流程。stream输入指定执行任务的Stream。返回值返回aclnnStatus状态码具体参见aclnn返回码。约束说明aclnnSparseFlashMlaGradMetadata默认确定性实现。BBatch表示输入样本批量大小。Batch取值规则优先获取sequsedQOptional中的Batch信息。如果未传入sequsedQOptional且layoutQOptional为TND和传入了cuSeqlensQOptional则获取cuSeqlensQOptional中的Batch信息。除上所述使用batchSize。Query Sequence Length取值规则优先获取sequsedQOptional中的Sequence Length信息。如果未传入sequsedQOptional且layoutQOptional为TND和传入了cuSeqlensQOptional则获取cuSeqlensQOptional中的Sequence Length信息。除上所述使用maxSeqlenQ。ori_kv、cmp_kv Sequence Length与Query的获取规则一致。BSND场景当传入的layoutQOptional为BSND时在未传入sequsedQOptional的情况下必传maxSeqlenQ参数。当传入的layoutKvOptional为BSND时若hasOriKv为true在未传入sequsedOriKvOptional的情况下必传maxSeqlenOriKv参数若hasCmpKv为true在未传入sequsedCmpKvOptional的情况下必传maxSeqlenCmpKv参数。TND场景当传入的layoutQOptional为TND时必传cuSeqlensQOptional参数。当传入的layoutKvOptional为TND时若hasOriKv为true必传cuSeqlensOriKvOptional若hasCmpKv为true必传cuSeqlensCmpKvOptional参数。调用示例示例代码如下仅供参考具体编译和执行过程请参考编译与运行样例。#include iostream #include vector #include cmath #include cstring #include limits #include functional #include utility #include acl/acl.h #include aclnnop/aclnn_sparse_flash_mla_grad_metadata.h #define CHECK_LOG_RET(cond, ret_val, fmt, ...) \ do { \ if (!(cond)) { \ printf(fmt \n, ##__VA_ARGS__); \ return (ret_val); \ } \ } while (0) // Constants constexpr uint32_t AIC_CORE_MAX_NUM 36; constexpr uint32_t AIV_CORE_MAX_NUM 72; constexpr uint32_t SMLAG_METADATA_TOTAL_SIZE 1024; using SMLAG_METADATA_T int32_t; constexpr uint32_t GRAD_METADATA_SIZE 7; // Grad Metadata Index Definitions constexpr uint32_t TOTAL_NUM 0; constexpr uint32_t FORMER_CORE_PROCESS_NUM 1; constexpr uint32_t REMAIN_CORE_PROCESS_NUM 2; constexpr uint32_t REMAIN_CORE_NUM 3; constexpr uint32_t USED_CORE_NUM 4; constexpr uint32_t MAX_ORI_KV_SIZE 5; constexpr uint32_t MAX_CMP_KV_SIZE 6; struct SmlagMetadata { uint32_t gradMetadata[GRAD_METADATA_SIZE]; }; struct ScopeGuard { explicit ScopeGuard(std::functionvoid() onExitScope) : m_exitFunc(std::move(onExitScope)), m_isDismissed(false) {} // 禁止拷贝 ScopeGuard(const ScopeGuard) delete; ScopeGuard operator(const ScopeGuard) delete; ~ScopeGuard() { if (!m_isDismissed) { m_exitFunc(); } } void Dismiss() { m_isDismissed true; } std::functionvoid() m_exitFunc; bool m_isDismissed; }; struct Tensor { void *hostAddr { nullptr }; void *deviceAddr { nullptr }; aclTensor *data { nullptr }; }; struct ArgScenario { bool hasCuSeq { false }; bool hasSeqused { false }; }; struct ArgContext { // required input int64_t numHeadsQ { 0 }; int64_t numHeadsKv { 0 }; int64_t headDim { 0 }; // optional input Tensor cuSeqlensQOptional {}; Tensor cuSeqlensOriKvOptional {}; Tensor cuSeqlensCmpKvOptional {}; Tensor sequsedQOptional {}; Tensor sequsedOriKvOptional {}; Tensor sequsedCmpKvOptional {}; Tensor cmpResidualKvOptional {}; Tensor oriTopkLengthOptional {}; Tensor cmpTopkLengthOptional {}; int64_t batchSize { 0 }; int64_t maxSeqlenQ { 0 }; int64_t maxSeqlenOriKv { 0 }; int64_t maxSeqlenCmpKv { 0 }; int64_t oriTopk { 0 }; int64_t cmpTopk { 0 }; int64_t cmpRatio { 0 }; int64_t oriMaskMode { 0 }; int64_t cmpMaskMode { 0 }; int64_t oriWinLeft { -1 }; int64_t oriWinRight { -1 }; char *layoutQOptional { nullptr }; char *layoutKvOptional { nullptr }; bool hasOriKv { true }; bool hasCmpKv { true }; // output Tensor metadata {}; }; int64_t GetShapeSize(const std::vectorint64_t shape) { int64_t shapeSize 1; for (auto i : shape) { shapeSize * i; } return shapeSize; } aclnnStatus Init(int32_t deviceId, aclrtStream* stream) { // 固定写法初始化 auto ret aclInit(nullptr); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclInit failed. ERROR: %d, ret); ret aclrtSetDevice(deviceId); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclrtSetDevice failed. ERROR: %d, ret); ret aclrtCreateStream(stream); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclrtCreateStream failed. ERROR: %d, ret); return ACL_SUCCESS; } void Finalize(int32_t deviceId, aclrtStream stream) { aclrtDestroyStream(stream); aclrtResetDevice(deviceId); aclFinalize(); } aclnnStatus CreateTensor(aclDataType dataType, const std::vectorint64_t shape, Tensor tensor) { auto size GetShapeSize(shape) * aclDataTypeSize(dataType); // 调用aclrtMallocHost申请host侧内存 auto ret aclrtMallocHost((tensor.hostAddr), size); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclrtMallocHost failed. ERROR: %d, ret); memset(tensor.hostAddr, 0, size); // 调用aclrtMalloc申请device侧内存 ret aclrtMalloc((tensor.deviceAddr), size, ACL_MEM_MALLOC_HUGE_FIRST); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclrtMalloc failed. ERROR: %d, ret); // 调用aclCreateTensor接口创建aclTensor tensor.data aclCreateTensor(shape.data(), shape.size(), dataType, nullptr, 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(), tensor.deviceAddr); CHECK_LOG_RET(tensor.data ! nullptr, ACL_ERROR_FAILURE, aclCreateTensor failed); // 调用aclrtMemcpy将host侧数据拷贝到device侧内存上 ret aclrtMemcpy(tensor.deviceAddr, size, tensor.hostAddr, size, ACL_MEMCPY_HOST_TO_DEVICE); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclrtMemcpy failed. ERROR: %d, ret); return ACL_SUCCESS; } void DestroyTensor(Tensor tensor) { if (tensor.data ! nullptr) { aclDestroyTensor(tensor.data); tensor.data nullptr; } if (tensor.deviceAddr ! nullptr) { aclrtFree(tensor.deviceAddr); tensor.deviceAddr nullptr; } if (tensor.hostAddr ! nullptr) { aclrtFreeHost(tensor.hostAddr); tensor.hostAddr nullptr; } } void DestroyArgs(ArgContext context) { DestroyTensor(context.metadata); DestroyTensor(context.cuSeqlensQOptional); DestroyTensor(context.cuSeqlensOriKvOptional); DestroyTensor(context.cuSeqlensCmpKvOptional); DestroyTensor(context.sequsedQOptional); DestroyTensor(context.sequsedOriKvOptional); DestroyTensor(context.sequsedCmpKvOptional); DestroyTensor(context.cmpResidualKvOptional); DestroyTensor(context.oriTopkLengthOptional); DestroyTensor(context.cmpTopkLengthOptional); if (context.layoutQOptional ! nullptr) { free(context.layoutQOptional); context.layoutQOptional nullptr; } if (context.layoutKvOptional ! nullptr) { free(context.layoutKvOptional); context.layoutKvOptional nullptr; } } aclnnStatus CreateArgs(const ArgScenario scenario, ArgContext context) { ScopeGuard argsGuard([] { DestroyArgs(context); }); aclnnStatus ret; context.numHeadsQ 64; context.numHeadsKv 1; context.headDim 512; ret CreateTensor(aclDataType::ACL_INT32, { SMLAG_METADATA_TOTAL_SIZE }, context.metadata); // 1024: Fix size CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create metadata failed. Error: %d, ret); context.oriTopk 0; context.cmpTopk 0; context.cmpRatio 128; context.oriMaskMode 4; context.cmpMaskMode 3; context.oriWinLeft 127; context.oriWinRight 0; context.layoutQOptional (char *)malloc(sizeof(char) * 16); context.layoutKvOptional (char *)malloc(sizeof(char) * 16); CHECK_LOG_RET(context.layoutQOptional ! nullptr, ACL_ERROR_FAILURE, Create layoutQOptional failed); CHECK_LOG_RET(context.layoutKvOptional ! nullptr, ACL_ERROR_FAILURE, Create layoutKvOptional failed); strcpy(context.layoutQOptional, BSND); // BSND,TND strcpy(context.layoutKvOptional, BSND); // BSND,TND context.hasOriKv true; context.hasCmpKv true; context.batchSize 4; context.maxSeqlenOriKv 1024; context.maxSeqlenCmpKv 1024; context.maxSeqlenQ 1024; if (scenario.hasCuSeq) { // (B1,), first element is always 0 ret CreateTensor(aclDataType::ACL_INT32, { context.batchSize 1 }, context.cuSeqlensQOptional); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create cuSeqlensQOptional failed. Error: %d, ret); ret CreateTensor(aclDataType::ACL_INT32, { context.batchSize 1 }, context.cuSeqlensOriKvOptional); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create cuSeqlensOriKvOptional failed. Error: %d, ret); ret CreateTensor(aclDataType::ACL_INT32, { context.batchSize 1 }, context.cuSeqlensCmpKvOptional); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create cuSeqlensCmpKvOptional failed. Error: %d, ret); } if (scenario.hasSeqused) { // (B,) ret CreateTensor(aclDataType::ACL_INT32, { context.batchSize }, context.sequsedQOptional); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create sequsedQOptional failed. Error: %d, ret); ret CreateTensor(aclDataType::ACL_INT32, { context.batchSize }, context.sequsedOriKvOptional); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create sequsedOriKvOptional failed. Error: %d, ret); ret CreateTensor(aclDataType::ACL_INT32, { context.batchSize }, context.sequsedCmpKvOptional); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create sequsedCmpKvOptional failed. Error: %d, ret); } if (context.hasCmpKv context.cmpRatio ! 1 context.cmpMaskMode 3) { // (B,) ret CreateTensor(aclDataType::ACL_INT32, { context.batchSize }, context.cmpResidualKvOptional); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create cmpResidualKvOptional failed. Error: %d, ret); } argsGuard.Dismiss(); return ACL_SUCCESS; } int main() { // 1. 固定写法device/stream初始化参考对外接口列表 // 根据自己的实际device填写deviceId int32_t deviceId 0; aclrtStream stream; auto ret Init(deviceId, stream); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Init acl failed. ERROR: %d, ret); ScopeGuard sysGuard([] { Finalize(deviceId, stream); }); // 2. 构造输入与输出需要根据API的接口定义构造 ArgScenario scenario {}; scenario.hasCuSeq false; scenario.hasSeqused false; ArgContext context {}; ret CreateArgs(scenario, context); CHECK_LOG_RET(ret ACL_SUCCESS, ret, Create input arguments failed. ERROR: %d, ret); ScopeGuard argsGuard([] { DestroyArgs(context); }); // 3. 调用CANN算子库API需要修改为具体的API // 调用aclnnSparseFlashMlaGradMetadata第一段接口 uint64_t workspaceSize 0; aclOpExecutor *executor nullptr; void *workspaceAddr nullptr; ret aclnnSparseFlashMlaGradMetadataGetWorkspaceSize( context.cuSeqlensQOptional.data, context.cuSeqlensOriKvOptional.data, context.cuSeqlensCmpKvOptional.data, context.sequsedQOptional.data, context.sequsedOriKvOptional.data, context.sequsedCmpKvOptional.data, context.cmpResidualKvOptional.data, context.oriTopkLengthOptional.data, context.cmpTopkLengthOptional.data, context.numHeadsQ, context.numHeadsKv, context.headDim, context.batchSize, context.maxSeqlenQ, context.maxSeqlenOriKv, context.maxSeqlenCmpKv, context.oriTopk, context.cmpTopk, context.cmpRatio, context.oriMaskMode, context.cmpMaskMode, context.oriWinLeft, context.oriWinRight, context.layoutQOptional, context.layoutKvOptional, context.hasOriKv, context.hasCmpKv, context.metadata.data, workspaceSize, executor); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclnnSparseFlashMlaGradMetadataGetWorkspaceSize failed. ERROR: %d\n, ret); if (workspaceSize static_castuint64_t(0)) { ret aclrtMalloc(workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST); CHECK_LOG_RET(ret ACL_SUCCESS, ret, allocate workspace failed. ERROR: %d\n, ret); } ScopeGuard workspaceGuard([] { if (workspaceAddr ! nullptr) { aclrtFree(workspaceAddr); workspaceAddr nullptr; } }); // 调用aclnnSparseFlashMlaGradMetadata第二段接口 ret aclnnSparseFlashMlaGradMetadata(workspaceAddr, workspaceSize, executor, stream); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclnnSparseFlashMlaGradMetadata failed. ERROR: %d\n, ret); // 4. 固定写法同步等待任务执行结束 ret aclrtSynchronizeStream(stream); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclrtSynchronizeStream failed. ERROR: %d\n, ret); // 5. 打印输出 SmlagMetadata result {}; ret aclrtMemcpy(result, sizeof(result), context.metadata.deviceAddr, sizeof(result), ACL_MEMCPY_DEVICE_TO_HOST); CHECK_LOG_RET(ret ACL_SUCCESS, ret, aclrtMemcpy failed. ERROR: %d\n, ret); for (uint32_t i 0; i GRAD_METADATA_SIZE; i) { printf(AIC Core%u\n, i); printf( Total Num : %u\n, result.gradMetadata[TOTAL_NUM]); printf( Former Core Process Num : %u\n, result.gradMetadata[FORMER_CORE_PROCESS_NUM]); printf( Remain Core Process Num : %u\n, result.gradMetadata[REMAIN_CORE_PROCESS_NUM]); printf( Remain Core Num : %u\n, result.gradMetadata[REMAIN_CORE_NUM]); printf( Used Core Num : %u\n, result.gradMetadata[USED_CORE_NUM]); printf( Max Ori Kv Size : %u\n, result.gradMetadata[MAX_ORI_KV_SIZE]); printf( Max Cmp Kv Size : %u\n, result.gradMetadata[MAX_CMP_KV_SIZE]); } return 0; }【免费下载链接】ops-transformer本项目是CANN提供的transformer类大模型算子库实现网络在NPU上加速计算。项目地址: https://gitcode.com/cann/ops-transformer创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考