操作指南本节中的每个指南都针对您作为有经验的用户在使用 Ragas 时可能遇到的实际问题提供了专注的解决方案。这些指南设计得简洁直接为您的问题提供快速解决方案。我们假设您对 Ragas 的概念有基本了解且能够熟练使用。如果不是请先浏览 快速入门 Get Started部分。LangChain 集成本教程演示如何使用 Ragas 评估基于 LangChain 构建的 RAG 问答应用。此外我们还将探讨 Ragas App 如何帮助分析和提升应用性能。构建简单的问答应用要构建问答系统我们首先创建一个小型数据集并使用其嵌入向量在向量数据库中建立索引。importosfromdotenvimportload_dotenvfromlangchain_core.documentsimportDocument load_dotenv()content_list[Andrew Ng is the CEO of Landing AI and is known for his pioneering work in deep learning. He is also widely recognized for democratizing AI education through platforms like Coursera.,Sam Altman is the CEO of OpenAI and has played a key role in advancing AI research and development. He is a strong advocate for creating safe and beneficial AI technologies.,Demis Hassabis is the CEO of DeepMind and is celebrated for his innovative approach to artificial intelligence. He gained prominence for developing systems that can master complex games like AlphaGo.,Sundar Pichai is the CEO of Google and Alphabet Inc., and he is praised for leading innovation across Googles vast product ecosystem. His leadership has significantly enhanced user experiences on a global scale.,Arvind Krishna is the CEO of IBM and is recognized for transforming the company towards cloud computing and AI solutions. He focuses on providing cutting-edge technologies to address modern business challenges.,]langchain_documents[]forcontentincontent_list:langchain_documents.append(Document(page_contentcontent,))fromragas.embeddingsimportOpenAIEmbeddingsfromlangchain_core.vectorstoresimportInMemoryVectorStoreimportopenai openai_clientopenai.OpenAI()embeddingsOpenAIEmbeddings(clientopenai_client,modeltext-embedding-3-small)vector_storeInMemoryVectorStore(embeddings)_vector_store.add_documents(langchain_documents)现在我们将构建一个基于 RAG 的系统将检索器、LLM 和提示词集成到一个 Retrieval QA Chain检索问答链中。检索器从知识库中获取相关文档。LLM 将基于检索到的文档生成响应提示词将引导模型的响应帮助它理解上下文并生成相关且连贯的语言输出。在 LangChain 中我们可以通过向量存储的 .as_retriever 方法创建检索器。有关详细信息请参阅 LangChain 关于向量存储检索器的文档。retrievervector_store.as_retriever(search_kwargs{k:1})我们将定义一个 Chain链它处理用户查询并检索相关数据将其通过结构化提示词传递给模型。然后解析模型的输出生成最终的字符串响应。fromlangchain_core.promptsimportChatPromptTemplatefromlangchain_core.output_parsersimportStrOutputParser templateAnswer the question based only on the following context: {context} Question: {query} promptChatPromptTemplate.from_template(template)qa_chainprompt|llm|StrOutputParser()defformat_docs(relevant_docs):return\n.join(doc.page_contentfordocinrelevant_docs)queryWho is the CEO of OpenAI?relevant_docsretriever.invoke(query)qa_chain.invoke({context:format_docs(relevant_docs),query:query})输出The CEO of OpenAI is Sam Altman.评估sample_queries[Which CEO is widely recognized for democratizing AI education through platforms like Coursera?,Who is Sam Altman?,Who is Demis Hassabis and how did he gained prominence?,Who is the CEO of Google and Alphabet Inc., praised for leading innovation across Googles product ecosystem?,How did Arvind Krishna transformed IBM?,]expected_responses[Andrew Ng is the CEO of Landing AI and is widely recognized for democratizing AI education through platforms like Coursera.,Sam Altman is the CEO of OpenAI and has played a key role in advancing AI research and development. He strongly advocates for creating safe and beneficial AI technologies.,Demis Hassabis is the CEO of DeepMind and is celebrated for his innovative approach to artificial intelligence. He gained prominence for developing systems like AlphaGo that can master complex games.,Sundar Pichai is the CEO of Google and Alphabet Inc., praised for leading innovation across Googles vast product ecosystem. His leadership has significantly enhanced user experiences globally.,Arvind Krishna is the CEO of IBM and has transformed the company towards cloud computing and AI solutions. He focuses on delivering cutting-edge technologies to address modern business challenges.,]要评估问答系统我们需要将查询、预期响应和其他指标特定的需求组织到 EvaluationDataset 评估数据集中。fromragasimportEvaluationDataset dataset[]forquery,referenceinzip(sample_queries,expected_responses):relevant_docsretriever.invoke(query)responseqa_chain.invoke({context:format_docs(relevant_docs),query:query})dataset.append({user_input:query,retrieved_contexts:[rdoc.page_contentforrdocinrelevant_docs],response:response,reference:reference,})evaluation_datasetEvaluationDataset.from_list(dataset)我们将使用以下指标评估问答应用LLMContextRecallLLM上下文召回率 评估检索到的上下文与参考答案中的主张的契合程度无需人工参考上下文标注即可估算召回率。Faithfulness忠实度 评估生成答案中的所有主张是否可以直接从提供的上下文中推断出来。Factual Correctness事实正确性 通过与参考答案比较使用基于主张的评估和自然语言推理来检查生成响应的事实准确性。有关这些指标以及它们如何应用于评估 RAG 系统的详细信息请访问 Ragas 指标文档。fromragasimportevaluatefromragas.llmsimportLangchainLLMWrapperfromragas.metricsimportLLMContextRecall,Faithfulness,FactualCorrectness evaluator_llmLangchainLLMWrapper(llm)resultevaluate(datasetevaluation_dataset,metrics[LLMContextRecall(),Faithfulness(),FactualCorrectness()],llmevaluator_llm,)result输出{context_recall:1.0000,faithfulness:0.9000,factual_correctness:0.9260}