标签RAG、Embedding、Milvus、向量数据库、SpringAI Alibaba、相似度检索一、前言前面三篇完成了模型接入、提示词、记忆和智能体。从本篇开始进入 RAG检索增强生成核心部分先讲原理、Embedding 和 Milvus 向量数据库部署。二、AI 技术困局为什么传统技术力不从心2.1 传统技术栈的中年危机MySQL擅长结构化查询无法做语义搜索Redis擅长缓存和键值存储无法存储高维向量Elasticsearch文本检索强但语义理解能力有限2.2 向量让 AI 理解像什么将文本、图片等离散数据转换为连续的数值向量向量之间的距离就代表语义相似度。三、什么是 RAGRAGRetrieval-Augmented Generation检索增强生成工作流程文档加载读取 PDF、Markdown、Word、Excel 等文档文本分割将长文档切分为语义片段向量化用 Embedding 模型将片段转换为向量向量存储存入 Milvus 等向量数据库语义检索根据用户问题检索最相关的片段答案生成将检索结果作为上下文交给大模型生成回答核心优势解决大模型知识截止问题减少幻觉答案基于真实文档让企业私有知识被 AI 理解四、Embedding 与 Qwen-embedding4.1 什么是 EmbeddingEmbedding嵌入/向量化是将离散文本转换为连续数值向量的数学过程。4.2 向量维度常见维度384、512、768、1024、1536 等。维度越高表达能力越强但存储和计算成本也越高需要在精度和性能之间权衡4.3 相似度度量度量方式说明余弦相似度衡量向量方向差异欧氏距离衡量向量绝对距离内积衡量向量投影关系4.4 SpringAI Alibaba 测试 Embeddingapplication.ymlspring:ai:dashscope:api-key:${dashscope.api.key}chat:model:qwen-plusembedding:options:model:text-embedding-v4控制器代码packagenet.xdclass.controller;importorg.springframework.ai.embedding.EmbeddingModel;importorg.springframework.beans.factory.annotation.Autowired;importorg.springframework.web.bind.annotation.*;importjava.util.Arrays;RestControllerRequestMapping(/embedding)publicclassEmbeddingController{AutowiredprivateEmbeddingModelembeddingModel;GetMapping(/single)publicStringembedSingle(RequestParam(defaultValue你好世界)Stringtext){float[]embeddingembeddingModel.embed(text);returnString.format(向量维度: %d, 样本值: %s,embedding.length,Arrays.toString(Arrays.copyOf(embedding,10)));}GetMapping(/similarity)publicStringcalculateSimilarity(RequestParam(defaultValue你好世界)Stringtext1,RequestParam(defaultValue您好)Stringtext2){float[]vec1embeddingModel.embed(text1);float[]vec2embeddingModel.embed(text2);doublesimilaritycalculateCosineSimilarity(vec1,vec2);returnString.format(%s 与 %s 的相似度: %.4f,text1,text2,similarity);}privatedoublecalculateCosineSimilarity(float[]v1,float[]v2){doubledot0,norm10,norm20;for(inti0;iv1.length;i){dotv1[i]*v2[i];norm1Math.pow(v1[i],2);norm2Math.pow(v2[i],2);}returndot/(Math.sqrt(norm1)*Math.sqrt(norm2));}}五、Milvus 向量数据库部署5.1 为什么选 Milvus存储计算分离架构支持十亿级向量提供完整 Java SDK与 Spring Boot 集成良好支持 Docker 一键部署可视化工具 Attu 管理方便5.2 Docker 安装 Milvus 单机版创建 docker-compose.ymlversion:3.5services: etcd: image: quay.io/coreos/etcd:v3.5.18 container_name: milvus-etcd environment: -ETCD_AUTO_COMPACTION_MODErevision -ETCD_AUTO_COMPACTION_RETENTION1000volumes: - ./etcd-data:/etcd command: etcd -advertise-client-urlshttp://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd minio: image: minio/minio:RELEASE.2023-03-20T20-16-18Z container_name: milvus-minio environment: MINIO_ACCESS_KEY: minioadmin MINIO_SECRET_KEY: minioadmin volumes: - ./minio-data:/minio_data command: minio server /minio_data --console-address:9001healthcheck: test:[CMD,curl,-f,http://localhost:9000/minio/health/live]interval: 30s timeout: 20s retries:3standalone: image: milvusdb/milvus:v2.4.0-rc.1 container_name: milvus-standalone command:[milvus,run,standalone]environment: ETCD_ENDPOINTS: etcd:2379 MINIO_ADDRESS: minio:9000 volumes: - ./standalone-data:/var/lib/milvus ports: -19530:19530-9091:9091depends_on: - etcd - minio attu: image: zilliz/attu:latest container_name: milvus-attu environment: MILVUS_URL: standalone:19530 ports: -8000:3000depends_on: - standalone启动docker-compose up -ddocker-compose ps5.3 关键端口端口用途19530Milvus 服务端口Java 程序连接9091监控指标端口2379etcd 端口内部使用9000MinIO 端口内部使用8000Attu 可视化界面5.4 Attu 可视化工具浏览器访问http://localhost:8000连接地址localhost:19530单机版默认无密码Attu 功能Collection 管理数据上传与搜索监控面板类 SQL 查询控制台六、SpringAI Alibaba 一键接入 Milvus6.1 添加依赖dependencygroupIdorg.springframework.ai/groupIdartifactIdspring-ai-milvus-store/artifactId/dependencydependencygroupIdorg.springframework.ai/groupIdartifactIdspring-ai-autoconfigure-vector-store-milvus/artifactId/dependencydependencygroupIdorg.springframework.ai/groupIdartifactIdspring-ai-advisors-vector-store/artifactId/dependency6.2 application.yml 配置spring:ai:vectorstore:milvus:enabled:trueinitialize-schema:trueclient:host:localhostport:19530username:minioadminpassword:minioadmindatabaseName:defaultcollectionName:vector_storeembeddingDimension:1024metricType:COSINE6.3 初始化 Milvus 数据packagenet.xdclass.config;importorg.slf4j.Logger;importorg.slf4j.LoggerFactory;importorg.springframework.ai.document.Document;importorg.springframework.ai.vectorstore.VectorStore;importorg.springframework.beans.factory.annotation.Value;importorg.springframework.boot.ApplicationRunner;importorg.springframework.context.annotation.Bean;importorg.springframework.context.annotation.Configuration;importjava.util.List;ConfigurationpublicclassMilvusInitializer{privatestaticfinalLoggerloggerLoggerFactory.getLogger(MilvusInitializer.class);Bean(milvusDataInitializer)publicApplicationRunnermilvusInitializer(VectorStorevectorStore,Value(${spring.ai.vectorstore.milvus.initialize-demo-data:true})booleaninitializeDemoData){returnargs-{if(initializeDemoData){logger.info(开始初始化 Milvus 数据...);ListDocumentdemoDocsList.of(newDocument(苹果是一种常见的水果富含维生素C和纤维素),newDocument(香蕉是热带水果含有丰富的钾元素),newDocument(橙子味道酸甜富含维生素C),newDocument(草莓外观鲜红口感甜美富含抗氧化物质),newDocument(葡萄可以制作葡萄酒含有多种有益成分));intbatchSize10;for(inti0;idemoDocs.size();ibatchSize){intendIndexMath.min(ibatchSize,demoDocs.size());vectorStore.add(demoDocs.subList(i,endIndex));logger.info(已添加批次文档到 Milvus: {}-{},i,endIndex-1);}}};}}七、ChatClient 集成 QuestionAnswerAdvisor 实现 RAG7.1 配置专用 RAG ChatClientpackagenet.xdclass.config;importcom.alibaba.cloud.ai.dashscope.chat.DashScopeChatOptions;importcom.alibaba.cloud.ai.memory.redis.RedisChatMemoryRepository;importorg.springframework.ai.chat.client.ChatClient;importorg.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;importorg.springframework.ai.chat.client.advisor.SimpleLoggerAdvisor;importorg.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;importorg.springframework.ai.chat.memory.MessageWindowChatMemory;importorg.springframework.ai.chat.model.ChatModel;importorg.springframework.ai.vectorstore.SearchRequest;importorg.springframework.ai.vectorstore.VectorStore;importorg.springframework.beans.factory.annotation.Qualifier;importorg.springframework.context.annotation.Bean;importorg.springframework.context.annotation.Configuration;ConfigurationpublicclassChatConfig{BeanpublicChatClientmilvusRagChatClient(Qualifier(dashscopeChatModel)ChatModeldashscopeChatModel,RedisChatMemoryRepositoryredisChatMemoryRepository,VectorStorevectorStore){returnChatClient.builder(dashscopeChatModel).defaultSystem( 你是一个专业的知识库问答助手只能基于从 Milvus 向量数据库中检索到的信息回答问题。 1. 只能使用检索到的信息回答不得凭空创造信息。 2. 如果信息不足明确告知用户根据现有知识库信息无法回答该问题。 3. 回答简洁明了重点突出结构清晰。 ).defaultOptions(DashScopeChatOptions.builder().withModel(qwen-plus).withTemperature(0.7).build()).defaultAdvisors(newSimpleLoggerAdvisor(),MessageChatMemoryAdvisor.builder(MessageWindowChatMemory.builder().chatMemoryRepository(redisChatMemoryRepository).maxMessages(Integer.MAX_VALUE).build()).build(),QuestionAnswerAdvisor.builder(vectorStore).searchRequest(SearchRequest.builder().topK(5).similarityThreshold(0.7).build()).build()).build();}}7.2 RAG 查询接口packagenet.xdclass.controller;importorg.springframework.ai.chat.client.ChatClient;importorg.springframework.ai.chat.client.advisor.vectorstore.QuestionAnswerAdvisor;importorg.springframework.ai.document.Document;importorg.springframework.ai.vectorstore.SearchRequest;importorg.springframework.ai.vectorstore.VectorStore;importorg.springframework.beans.factory.annotation.Autowired;importorg.springframework.beans.factory.annotation.Qualifier;importorg.springframework.web.bind.annotation.*;importreactor.core.publisher.Flux;importjava.util.Arrays;importjava.util.List;RestControllerRequestMapping(/milvus)publicclassMilvusRagController{AutowiredprivateVectorStorevectorStore;AutowiredQualifier(milvusRagChatClient)privateChatClientchatClient;PostMapping(value/add-documents,producestext/html;charsetutf-8)publicStringaddDocuments(String[]text){ListDocumentdocumentsArrays.stream(text).map(Document::new).toList();intbatchSize10;for(inti0;idocuments.size();ibatchSize){vectorStore.add(documents.subList(i,Math.min(ibatchSize,documents.size())));}return成功添加 documents.size() 个文档到 Milvus;}GetMapping(value/rag-query,producestext/html;charsetutf-8)publicFluxStringragQuery(RequestParamStringquery){returnchatClient.prompt().user(query).advisors(QuestionAnswerAdvisor.builder(vectorStore).searchRequest(SearchRequest.builder().query(query).build()).build()).stream().content();}}八、小结本篇完成了RAG 核心概念与工作流程Embedding 向量化与相似度计算Milvus 向量数据库 Docker 部署SpringAI Alibaba 接入 Milvus基于 QuestionAnswerAdvisor 的基础 RAG 实现下一篇将介绍 RAG 进阶让企业知识库支持 PDF、Markdown 等文档上传与检索并完成项目实战。