操作指南本节中的每个指南都针对您作为有经验的用户在使用 Ragas 时可能遇到的实际问题提供了专注的解决方案。这些指南设计得简洁直接为您的问题提供快速解决方案。我们假设您对 Ragas 的概念有基本了解且能够熟练使用。如果不是请先浏览 快速入门 Get Started部分。R2R 集成R2R 是一款用于 AI 检索增强生成RAG的一体化解决方案具备生产级功能包括多模态内容导入、混合搜索功能、用户/文档管理等。概述在本教程中我们将利用 R2R 的 /rag 端点对小型数据集执行检索增强生成RAG。评估生成的响应。分析评估的追踪信息。R2R 设置安装依赖首先安装必要的包%pipinstallr2r-q设置本地环境配置 R2R_API_KEY 、 OPENAI_API_KEY 和 RAGAS_APP_TOKEN 可选。%pip install r2r-q获取数据dataset[OpenAI is one of the most recognized names in the large language model space, known for its GPT series of models. These models excel at generating human-like text and performing tasks like creative writing, answering questions, and summarizing content. GPT-4, their latest release, has set benchmarks in understanding context and delivering detailed responses.,Anthropic is well-known for its Claude series of language models, designed with a strong focus on safety and ethical AI behavior. Claude is particularly praised for its ability to follow complex instructions and generate text that aligns closely with user intent.,DeepMind, a division of Google, is recognized for its cutting-edge Gemini models, which are integrated into various Google products like Bard and Workspace tools. These models are renowned for their conversational abilities and their capacity to handle complex, multi-turn dialogues.,Meta AI is best known for its LLaMA (Large Language Model Meta AI) series, which has been made open-source for researchers and developers. LLaMA models are praised for their ability to support innovation and experimentation due to their accessibility and strong performance.,Meta AI with its LLaMA models aims to democratize AI development by making high-quality models available for free, fostering collaboration across industries. Their open-source approach has been a game-changer for researchers without access to expensive resources.,Microsoft’s Azure AI platform is famous for integrating OpenAI’s GPT models, enabling businesses to use these advanced models in a scalable and secure cloud environment. Azure AI powers applications like Copilot in Office 365, helping users draft emails, generate summaries, and more.,Amazon’s Bedrock platform is recognized for providing access to various language models, including its own models and third-party ones like Anthropic’s Claude and AI21’s Jurassic. Bedrock is especially valued for its flexibility, allowing users to choose models based on their specific needs.,Cohere is well-known for its language models tailored for business use, excelling in tasks like search, summarization, and customer support. Their models are recognized for being efficient, cost-effective, and easy to integrate into workflows.,AI21 Labs is famous for its Jurassic series of language models, which are highly versatile and capable of handling tasks like content creation and code generation. The Jurassic models stand out for their natural language understanding and ability to generate detailed and coherent responses.,In the rapidly advancing field of artificial intelligence, several companies have made significant contributions with their large language models. Notable players include OpenAI, known for its GPT Series (including GPT-4); Anthropic, which offers the Claude Series; Google DeepMind with its Gemini Models; Meta AI, recognized for its LLaMA Series; Microsoft Azure AI, which integrates OpenAI’s GPT Models; Amazon AWS (Bedrock), providing access to various models including Claude (Anthropic) and Jurassic (AI21 Labs); Cohere, which offers its own models tailored for business use; and AI21 Labs, known for its Jurassic Series. These companies are shaping the landscape of AI by providing powerful models with diverse capabilities.,]设置 R2R 客户端fromr2rimportR2RClient clientR2RClient()导入数据ingest_responseclient.documents.create(chunksdataset,)使用 /rag 端点/rag 端点通过将搜索结果与语言模型输出相结合来实现检索增强生成。生成过程可以使用 rag_generation_config 参数进行自定义而检索过程可以使用 search_settings 进行配置。queryWhat makes Meta AI’s LLaMA models stand out?search_settings{limit:2,graph_settings:{enabled:False,limit:2},}responseclient.retrieval.rag(queryquery,search_settingssearch_settings)print(response.results.generated_answer)输出Meta AI’s LLaMA models stand out due to theiropen-source nature,which supports innovationandexperimentation by making high-quality models accessible to researchersanddevelopers[1].This approach democratizes AI development,fostering collaboration across industriesandenabling researchers without access to expensive resources to workwithadvanced AI models[2].评估使用 Ragas 评估 R2R 客户端有了 R2R 客户端后我们可以使用 Ragas 的 R2R 集成进行评估。此过程涉及以下关键组件R2R 客户端和配置 指定 RAG 设置的 R2RClient 和 /rag 配置。评估数据集 您需要一个包含 Ragas 指标所需所有必要输入的 Ragas EvaluationDataset 。Ragas 指标 Ragas 提供了多种评估指标来评估 RAG 的不同方面如忠实度faithfulness、答案相关性answer relevance和上下文召回率context recall。您可以在 Ragas 文档中查看所有可用指标的完整列表。构建 Ragas 评估数据集EvaluationDataset 是 Ragas 中用于表示评估样本的数据类型。您可以在核心概念部分找到关于其结构和用法的更多详细信息。我们将使用 Ragas 中的 transform_to_ragas_dataset 函数为我们的数据获取 EvaluationDataset 。questions[Who are the major players in the large language model space?,What is Microsoft’s Azure AI platform known for?,What kind of models does Cohere provide?,]references[The major players include OpenAI (GPT Series), Anthropic (Claude Series), Google DeepMind (Gemini Models), Meta AI (LLaMA Series), Microsoft Azure AI (integrating GPT Models), Amazon AWS (Bedrock with Claude and Jurassic), Cohere (business-focused models), and AI21 Labs (Jurassic Series).,Microsoft’s Azure AI platform is known for integrating OpenAI’s GPT models, enabling businesses to use these models in a scalable and secure cloud environment.,Cohere provides language models tailored for business use, excelling in tasks like search, summarization, and customer support.,]r2r_responses[]search_settings{limit:2,graph_settings:{enabled:False,limit:2},}forqueinquestions:responseclient.retrieval.rag(queryque,search_settingssearch_settings)r2r_responses.append(response)fromragas.integrations.r2rimporttransform_to_ragas_dataset ragas_eval_datasettransform_to_ragas_dataset(user_inputsquestions,r2r_responsesr2r_responses,referencesreferences)输出EvaluationDataset(features[user_input,retrieved_contexts,response,reference],len3)选择指标为了评估我们的 RAG 端点我们将使用以下指标答案相关性Response Relevancy 衡量响应对用户输入查询的相关程度。上下文精确度Context Precision 衡量成功检索到的相关文档或信息片段的数量。忠实度Faithfulness 衡量响应对检索到的上下文在事实层面的一致性。fromragas.metricsimportAnswerRelevancy,ContextPrecision,Faithfulnessfromragasimportevaluatefromlangchain_openaiimportChatOpenAIfromragas.llmsimportLangchainLLMWrapper llmChatOpenAI(modelgpt-4o-mini)evaluator_llmLangchainLLMWrapper(llm)ragas_metrics[AnswerRelevancy(llmevaluator_llm),ContextPrecision(llmevaluator_llm),Faithfulness(llmevaluator_llm)]resultsevaluate(datasetragas_eval_dataset,metricsragas_metrics)输出Querying Client:100%|██████████|3/3[00:00?,?it/s]Evaluating:100%|██████████|9/9[00:00?,?it/s]user_inputretrieved_contextsresponsereferenceanswer_relevancycontext_precisionfaithfulness0Who are the major players in the large languag…[In the rapidly advancing field of artificial …The major players in the large language model …The major players include OpenAI (GPT Series),…1.0000001.01.0000001What is Microsoft’s Azure AI platform known for?[Microsoft’s Azure AI platform is famous for i…Microsoft’s Azure AI platform is known for int…Microsoft’s Azure AI platform is known for int…0.9489081.00.8333332What kind of models does Cohere provide?[Cohere is well-known for its language models …Cohere provides language models tailored for b…Cohere provides language models tailored for b…0.9037651.01.000000追踪评估为了更好地理解评估得分我们可以使用以下代码获取评估结果的追踪信息和判定理由。results.upload()快乐编码