From RAG to material: Classes realized from constructing real-world RAGs at GenAIIC – Half 1

The AWS Generative AI Innovation Center (GenAIIC) is a crew of AWS science and technique specialists who’ve deep data of generative AI. They assist AWS clients jumpstart their generative AI journey by constructing proofs of idea that use generative AI to deliver enterprise worth. For the reason that inception of AWS GenAIIC in Could 2023, we have now witnessed excessive buyer demand for chatbots that may extract data and generate insights from large and sometimes heterogeneous data bases. Such use instances, which increase a big language mannequin’s (LLM) data with exterior knowledge sources, are referred to as Retrieval-Augmented Generation (RAG).

This two-part collection shares the insights gained by AWS GenAIIC from direct expertise constructing RAG options throughout a variety of industries. You should utilize this as a sensible information to constructing higher RAG options.

On this first publish, we give attention to the fundamentals of RAG structure and how you can optimize text-only RAG. The second publish outlines how you can work with a number of knowledge codecs resembling structured knowledge (tables, databases) and pictures.

Anatomy of RAG

RAG is an environment friendly strategy to present an FM with extra data by utilizing exterior knowledge sources and is depicted within the following diagram:

• Retrieval: Primarily based on a consumer’s query (1), related data is retrieved from a data base (2) (for instance, an OpenSearch index).
• Augmentation: The retrieved data is added to the FM immediate (3.a) to reinforce its data, together with the consumer question (3.b).
• Era: The FM generates a solution (4) by utilizing the data offered within the immediate.

The next is a common diagram of a RAG workflow. From left to proper are the retrieval, the augmentation, and the technology. In observe, the data base is usually a vector retailer.

A deeper dive within the retriever

In a RAG structure, the FM will base its reply on the data offered by the retriever. Subsequently, a RAG is barely pretty much as good as its retriever, and lots of the suggestions that we share in our sensible information are about how you can optimize the retriever. However what’s a retriever precisely? Broadly talking, a retriever is a module that takes a question as enter and outputs related paperwork from a number of data sources related to that question.

Doc ingestion

In a RAG structure, paperwork are sometimes saved in a vector retailer. As proven within the following diagram, vector shops are populated by chunking the paperwork into manageable items (1) (if a doc is brief sufficient, chunking won’t be required) and remodeling every chunk of the doc right into a high-dimensional vector utilizing a vector embedding (2), such because the Amazon Titan embeddings mannequin. These embeddings have the attribute that two chunks of texts which can be semantically shut have vector representations which can be additionally shut in that embedding (within the sense of the cosine or Euclidean distance).

The next diagram illustrates the ingestion of textual content paperwork within the vector retailer utilizing an embedding mannequin. Word that the vectors are saved alongside the corresponding textual content chunk (3), in order that at retrieval time, whenever you establish the chunks closest to the question, you may return the textual content chunk to be handed to the FM immediate.

Semantic search

Vector shops permit for environment friendly semantic search: as proven within the following diagram, given a consumer question (1), we vectorize it (2) (utilizing the identical embedding because the one which was used to construct the vector retailer) after which search for the closest vectors within the vector retailer (3), which is able to correspond to the doc chunks which can be semantically closest to the preliminary question (4). Though vector shops and semantic search have grow to be the default in RAG architectures, extra conventional keyword-based search remains to be beneficial, particularly when trying to find domain-specific phrases (resembling technical jargon) or names. Hybrid search is a approach to make use of each semantic search and key phrases to rank a doc, and we are going to give extra particulars on this system within the part on superior RAG strategies.

The next diagram illustrates the retrieval of textual content paperwork which can be semantically near the consumer question. You have to use the identical embedding mannequin at ingestion time and at search time.

Implementation on AWS

A RAG chatbot might be arrange in a matter of minutes utilizing Amazon Bedrock Knowledge Bases. The data base might be linked to an Amazon Simple Storage Service (Amazon S3) bucket and can routinely chunk and index the paperwork it comprises in an OpenSearch index, which is able to act because the vector retailer. The retrieve_and_generate API does each the retrieval and a name to an FM (Amazon Titan or Anthropic’s Claude household of fashions on Amazon Bedrock), for a totally managed resolution. The retrieve API solely implements the retrieval element and permits for a extra customized method downstream, resembling doc publish processing earlier than calling the FM individually.

On this weblog publish, we are going to present suggestions and code to optimize a totally customized RAG resolution with the next parts:

• An OpenSearch Serverless vector search assortment because the vector retailer
• Customized chunking and ingestion features to ingest the paperwork within the OpenSearch index
• A customized retrieval operate that takes a consumer question as an enter and outputs the related paperwork from the OpenSearch index
• FM calls to your mannequin of alternative on Amazon Bedrock to generate the ultimate reply.

On this publish, we give attention to a customized resolution to assist readers perceive the internal workings of RAG. Many of the suggestions we offer might be tailored to work with Amazon Bedrock Knowledge Bases, and we are going to level this out within the related sections.

Overview of RAG use instances

Whereas working with clients on their generative AI journey, we encountered quite a lot of use instances that match throughout the RAG paradigm. In conventional RAG use instances, the chatbot depends on a database of textual content paperwork (.doc, .pdf, or .txt). Partly 2 of this publish, we are going to focus on how you can lengthen this functionality to photographs and structured knowledge. For now, we’ll give attention to a typical RAG workflow: the enter is a consumer query, and the output is the reply to that query, derived from the related textual content chunks or paperwork retrieved from the database. Use instances embrace the next:

• Customer support– This could embrace the next:
  • Inner– Stay brokers use an inside chatbot to assist them reply buyer questions.
  • Exterior– Prospects straight chat with a generative AI chatbot.
  • Hybrid– The mannequin generates good replies for stay brokers that they will edit earlier than sending to clients.
• Worker coaching and sources– On this use case, chatbots can use worker coaching manuals, HR sources, and IT service paperwork to assist staff onboard sooner or discover the data they should troubleshoot inside points.
• Industrial upkeep– Upkeep manuals for complicated machines can have a number of hundred pages. Constructing a RAG resolution round these manuals helps upkeep technicians discover related data sooner. Word that upkeep manuals typically have photographs and schemas, which might put them in a multimodal bucket.
• Product data search– Area specialists have to establish related merchandise for a given use case, or conversely discover the correct technical details about a given product.
• Retrieving and summarizing monetary information– Analysts want essentially the most up-to-date data on markets and the economic system and depend on massive databases of stories or commentary articles. A RAG resolution is a strategy to effectively retrieve and summarize the related data on a given matter.

Within the following sections, we are going to give suggestions that you should use to optimize every facet of the RAG pipeline (ingestion, retrieval, and reply technology) relying on the underlying use case and knowledge format. To confirm that the modifications enhance the answer, you first want to have the ability to assess the efficiency of the RAG resolution.

Evaluating a RAG resolution

Opposite to conventional machine studying (ML) fashions, for which analysis metrics are properly outlined and easy to compute, evaluating a RAG framework remains to be an open downside. First, accumulating floor reality (data recognized to be right) for the retrieval element and the technology element is time consuming and requires human intervention. Secondly, even with a number of question-and-answer pairs out there, it’s troublesome to routinely consider if the RAG reply is shut sufficient to the human reply.

In our expertise, when a RAG system performs poorly, we discovered the retrieval half to nearly at all times be the perpetrator. Massive pre-trained fashions resembling Anthropic’s Claude mannequin will generate high-quality solutions if supplied with the correct data, and we discover two important failure modes:

• The related data isn’t current within the retrieved paperwork: On this case, the FM can attempt to make up a solution or use its personal data to reply. Including guardrails towards such conduct is crucial.
• Related data is buried inside an extreme quantity of irrelevant knowledge: When the scope of the retriever is just too broad, the FM can get confused and begin mixing up a number of knowledge sources, leading to a incorrect reply. Extra superior fashions resembling Anthropic’s Claude Sonnet 3.5 and Opus are reported to be extra strong towards such conduct, however that is nonetheless a danger to pay attention to.

To guage the standard of the retriever, you should use the next conventional retrieval metrics:

• Prime-k accuracy: Measures whether or not at the very least one related doc is discovered throughout the prime okay retrieved paperwork.
• Imply Reciprocal Rank (MRR)– This metric considers the rating of the retrieved paperwork. It’s calculated as the common of the reciprocal ranks (RR) for every question. The RR is the inverse of the rank place of the primary related doc. For instance, if the primary related doc is in third place, the RR is 1/3. A better MRR signifies that the retriever can rank essentially the most related paperwork larger.
• Recall– This metric measures the power of the retriever to retrieve related paperwork from the corpus. It’s calculated because the variety of related paperwork which can be efficiently retrieved over the full variety of related paperwork. Greater recall signifies that the retriever can discover a lot of the related data.
• Precision– This metric measures the power of the retriever to retrieve solely related paperwork and keep away from irrelevant ones. It’s calculated by the variety of related paperwork efficiently retrieved over the full variety of paperwork retrieved. Greater precision signifies that the retriever isn’t retrieving too many irrelevant paperwork.

Word that if the paperwork are chunked, the metrics should be computed on the chunk stage. This implies the bottom reality to judge a retriever is pairs of query and listing of related doc chunks. In lots of instances, there is just one chunk that comprises the reply to the query, so the bottom reality turns into query and related doc chunk.

To guage the standard of the generated response, two important choices are:

• Analysis by material specialists: this offers the very best reliability when it comes to analysis however can’t scale to a lot of questions and slows down iterations on the RAG resolution.
• Analysis by FM (additionally known as LLM-as-a-judge):
  • With a human-created place to begin: Present the FM with a set of floor reality question-and-answer pairs and ask the FM to judge the standard of the generated reply by evaluating it to the bottom reality one.
  • With an FM-generated floor reality: Use an FM to generate question-and-answer pairs for given chunks, after which use this as a floor reality, earlier than resorting to an FM to match RAG solutions to that floor reality.

We advocate that you simply use an FM for evaluations to iterate sooner on enhancing the RAG resolution, however to make use of subject-matter specialists (or at the very least human analysis) to offer a ultimate evaluation of the generated solutions earlier than deploying the answer.

A rising variety of libraries supply automated analysis frameworks that depend on extra FMs to create a floor reality and consider the relevance of the retrieved paperwork in addition to the standard of the response:

• Ragas– This framework provides FM-based metrics beforehand described, resembling context recall, context precision, reply faithfulness, and reply relevancy. It must be tailored to Anthropic’s Claude fashions due to its heavy dependence on particular prompts.
• LlamaIndex– This framework offers a number of modules to independently consider the retrieval and technology parts of a RAG system. It additionally integrates with different instruments resembling Ragas and DeepEval. It comprises modules to create floor reality (query-and-context pairs and question-and-answer pairs) utilizing an FM, which alleviates the usage of time-consuming human assortment of floor reality.
• RefChecker– That is an Amazon Science library targeted on fine-grained hallucination detection.

Troubleshooting RAG

Analysis metrics give an general image of the efficiency of retrieval and technology, however they don’t assist diagnose points. Diving deeper into poor responses can assist you perceive what’s inflicting them and what you are able to do to alleviate the difficulty. You’ll be able to diagnose the difficulty by analysis metrics and in addition by having a human evaluator take a better take a look at each the LLM reply and the retrieved paperwork.

The next is a quick overview of points and potential fixes. We are going to describe every of the strategies in additional element, together with real-world use instances and code examples, within the subsequent part.

• The related chunk wasn’t retrieved (retriever has low prime okay accuracy and low recall or noticed by human analysis):
  • Strive growing the variety of paperwork retrieved by the closest neighbor search and re-ranking the outcomes to chop again on the variety of chunks after retrieval.
  • Strive hybrid search. Utilizing key phrases together with semantic search (referred to as hybrid search) would possibly assist, particularly if the queries include names or domain-specific jargon.
  • Strive question rewriting. Having an FM detect the intent or rewrite the question can assist create a question that’s higher suited to the retriever. For example, a consumer question resembling “What data do you’ve within the data base in regards to the financial outlook in China?” comprises quite a lot of context that isn’t related to the search and can be extra environment friendly if rewritten as “financial outlook in China” for search functions.

• Too many chunks had been retrieved (retriever has low precision or noticed by human analysis):
  • Strive utilizing key phrase matching to limit the search outcomes. For instance, when you’re searching for details about a selected entity or property in your data base, solely retrieve paperwork that explicitly point out them.
  • Strive metadata filtering in your OpenSearch index. For instance, when you’re searching for data in information articles, attempt utilizing the date discipline to filter solely the latest outcomes.
  • Strive utilizing question rewriting to get the correct metadata filtering. This superior approach makes use of the FM to rewrite the consumer question as a extra structured question, permitting you to take advantage of OpenSearch filters. For instance, when you’re searching for the specs of a selected product in your database, the FM can extract the product identify from the question, and you’ll then use the product identify discipline to filter out the product identify.
  • Strive utilizing reranking to chop down on the variety of chunks handed to the FM.
• A related chunk was retrieved, nevertheless it’s lacking some context (can solely be assessed by human analysis):
  • Strive altering the chunking technique. Needless to say small chunks are good for exact questions, whereas massive chunks are higher for questions that require a broad context:
    • Strive growing the chunk measurement and overlap as a primary step.
    • Strive utilizing section-based chunking. When you have structured paperwork, use sections delimiters to chop your paperwork into chunks to have extra coherent chunks. Remember that you simply would possibly lose a number of the extra fine-grained context in case your chunks are bigger.
  • Strive small-to-large retrievers. If you wish to hold the fine-grained particulars of small chunks however be sure you retrieve all of the related context, small-to-large retrievers will retrieve your chunk together with the earlier and subsequent ones.
• If not one of the above assist:
  • Think about coaching a customized embedding.
• The retriever isn’t at fault, the issue is with FM technology (evaluated by a human or LLM):
  • Strive immediate engineering to mitigate hallucinations.
  • Strive prompting the FM to make use of quotes in its solutions, to permit for handbook truth checking.
  • Strive utilizing one other FM to judge or right the reply.

A sensible information to enhancing the retriever

Word that not all of the strategies that comply with should be applied collectively to optimize your retriever—some would possibly even have reverse results. Use the previous troubleshooting information to get a shortlist of what would possibly work, then take a look at the examples within the corresponding sections that comply with to evaluate if the strategy might be useful to your retriever.

Hybrid search

Instance use case: A big producer constructed a RAG chatbot to retrieve product specs. These paperwork include technical phrases and product names. Think about the next instance queries:

query_1 = "What's the viscosity of product XYZ?"
query_2 = "How viscous is XYZ?"

The queries are equal and should be answered with the identical doc. The key phrase element will just remember to’re boosting paperwork mentioning the identify of the product, XYZ whereas the semantic element will ensure that paperwork containing viscosity get a excessive rating, even when the question comprises the phrase viscous.

Combining vector search with key phrase search can successfully deal with domain-specific phrases, abbreviations, and product names that embedding fashions would possibly battle with. Virtually, this may be achieved in OpenSearch by combining a k-nearest neighbors (k-NN) question with key phrase matching. The weights for the semantic search in comparison with key phrase search might be adjusted. See the next instance code:

vector_embedding = compute_embedding(question)
measurement = 10
semantic_weight = 10
keyword_weight = 1
search_query = {"measurement":measurement, "question": { "bool": { "ought to":[] , "should":[] } } }
    # semantic search
    search_query['query']['bool']['should'].append(
            {"function_score": 
             { "question": 
              {"knn": 
               {"vector_field": 
                {"vector": vector_embedding, 
                "okay": 10 # The variety of nearest neighbors to retrieve
                }}}, 
              "weight": semantic_weight } })
              
    # key phrase search
    search_query['query']['bool']['should'].append({
             "function_score": 
            { "question": 
             {"match": 
             # It will improve the rating of chunks that match the phrases within the question
              {"chunk_text":  question} 
              },
             "weight": keyword_weight } })

Amazon Bedrock Information Bases additionally helps hybrid search, however you may’t regulate the weights for semantic in comparison with key phrase search.

Including metadata data to textual content chunks

Instance use case: Utilizing the identical instance of a RAG chatbot for product specs, think about product specs which can be a number of pages lengthy and the place the product identify is barely current within the header of the doc. When ingesting the doc into the data base, it’s chunked into smaller items for the embedding mannequin, and the product identify solely seems within the first chunk, which comprises the header. See the next instance:

# Word: the next doc was generated by Anthropic’s Claude Sonnet 
# and doesn't include details about an actual product

document_name = "Chemical Properties for Product XYZ"

chunk_1 = """
Product Description:
XYZ is a multi-purpose cleansing resolution designed for industrial and business use. 
It's a concentrated liquid formulation containing anionic and non-ionic surfactants, 
solvents, and alkaline builders.

Chemical Composition:
- Water (CAS No. 7732-18-5): 60-80%
- 2-Butoxyethanol (CAS No. 111-76-2): 5-10%
- Sodium Hydroxide (CAS No. 1310-73-2): 2-5%
- Ethoxylated Alcohols (CAS No. 68439-46-3): 1-3%
- Sodium Metasilicate (CAS No. 6834-92-0): 1-3%
- Perfume (Proprietary Combination): <1%
"""

# chunk 2 under would not include any point out of "XYZ"
chunk_2 = """
Bodily Properties:
- Look: Clear, yellow liquid
- Odor: Delicate, citrus perfume
- pH (focus): 12.5 - 13.5
- Particular Gravity: 1.05 - 1.10
- Solubility in Water: Full
- VOC Content material: <10%

Shelf-life:
When saved in its authentic, unopened container at temperatures between 15°C and 25°C,
 the product has a shelf lifetime of 24 months from the date of manufacture.
As soon as opened, the shelf life is lowered as a result of potential contamination and publicity to
 air. It's endorsed to make use of the product inside 6 months after opening the container.
"""

# append the doc identify to the chunks to enhance context,
# now chunk 2 will include the product identify

chunk_1 = document_name + chunk_1
chunk_2 = document_name + chunk_2

document_title = "Resolving community points"

chunk_1 = """
[....]

# Troubleshooting web entry:

1. Examine your bodily connections:
   - Make sure that the Ethernet cable (if utilizing a wired connection) is securely 
   plugged into each your laptop and the modem/router.
   - If utilizing a wi-fi connection, examine that your system's Wi-Fi is turned 
   on and related to the proper community.

2. Restart your units:
   - Reboot your laptop, laptop computer, or cell system.
   - Energy cycle your modem and router by unplugging them from the facility supply, 
   ready for a minute, after which plugging them again in.

"""

chunk_2 = """
3. Examine for community outages:
   - Contact your web service supplier (ISP) to inquire about any recognized 
   outages or service disruptions in your space.
   - Go to your ISP's web site or examine their social media channels for updates on 
   service standing.
  
4. Examine for interference:
   - If utilizing a wi-fi connection, attempt shifting your system nearer to the router or entry level.
   - Establish and get rid of potential sources of interference, resembling microwaves, cordless telephones, or different wi-fi units working on the identical frequency.

# Router configuration

[....]
"""

document_name = doc['document_name'] 
current_chunk = doc['current_chunk']

question = {
    "question": {
        "bool": {
            "should": [
                {
                    "match": {
                        "document_name": document_name
                    }
                }
            ],
            "ought to": [
                {"term": {"chunk_number": current_chunk - 1}},
                {"term": {"chunk_number": current_chunk + 1}}
            ],
            "minimum_should_match": 1
        }
    }
}

question = """" 
Are you able to listing all of the substances within the nuts and seeds granola?
Put the allergens in all caps. 
"""

import json

query_rewriting_prompt = """
Rewrite the question as a json with the next keys:
- rewritten_query: a greater model of the consumer's question that shall be used to compute 
an embedding and do semantic search
- key phrases: an inventory of key phrases that correspond to the question, for use in a 
search engine, it mustn't include the product identify.
- product_name: if the question is a a couple of particular product, give the identify right here,
 in any other case say None.

<instance>
H: what are the ingedients within the savory path combine?
A: {{
  "rewritten_query": "substances savory path combine",
  "key phrases": ["ingredients"],
  "product_name": "savory path combine"
}}
</instance>

<question>
{question}
</question>

Solely output the json, nothing else.
"""

def rewrite_query(question):
    response = call_FM(query_rewriting_prompt.format(question=question))
    print(response)
    json_query = json.hundreds(response)
    return json_query
    
rewrite_query(question)

{ 
"rewritten_query":"substances nuts and seeds granola allergens",
"key phrases": ["ingredients", "allergens"], 
"product_name": "nuts and seeds granola" 
}

oss = get_opensearch_serverless_client()
request = {
"product_info": product_info, # full textual content for the product data
"vector_field_product":embed_query_titan(product_info), # embedding for product data
"product_name": product_name,
"date": date, # elective discipline, can permit to kind by most up-to-date
"_op_type": "index",
"supply": file_key # that is the s3 location, you may change this with a URL
}
oss.index(index = index_name, physique = request)

question = """
Are you able to listing all of the substances within the nuts and seeds granola?
Put the allergens in daring and all caps.
"""
# utilizing the rewrite_query operate from the earlier part
json_query = rewrite_query(question) 

# get the product identify and key phrases from the json question
product_name = json_query["product_name"] 
key phrases = json_query["keywords"]

# compute the vector embedding of the rewritten question
vector_embedding = compute_embedding(json_query["rewritten_query"])

#initialize search question dictionary
search_query = {"measurement":10, "question": { "bool": { "ought to":[] , "should":[] } } }
# add should with match_phrase clause to filter on product identify
search_query['query']['bool']['should'].append(
    {"match_phrase": {
            "product_name": product_name # Extracted product identify should match product identify discipline 
        }
        }

# semantic search
search_query['query']['bool']['should'].append(
        {"function_score": 
            { "question": 
            {"knn": 
            {"vector_field_product": 
            {"vector": vector_embedding, 
            "okay": 10 # The variety of nearest neighbors to retrieve
            }}}, 
            "weight": semantic_weight } })
            
# key phrase search
search_query['query']['bool']['should'].append(
{"function_score": 
        { "question": 
            {"match": 
            # It will improve the rating of chunks that match the phrases within the question
            {"product_info":  question} 
            },
            "weight": keyword_weight } })

# Word: The scouting stories for Participant A and Participant B used within the instance under
# had been generated by Anthropic’s Claude Sonnet 3.
quotes_tags_prompt = """
You might be an Ai assistant chatbot designed to reply questions on your crew's baseballe scouting stories.

Listed below are some stories about gamers that you should use to reply the query:

<doc_0>
{doc_0}
</doc_0>

<doc_1>
{doc_1}
</doc_1>

<directions>
- In <scratchpad> tags, it's best to put the doc names and quotes from these paperwork 
 that assist you reply this query. 
 You have to format this as a number of jsons format with 'document_name' and 'quote' as keys. 
- Then put your reply in <reply> tags. 
 For those who seek advice from paperwork in your reply, be sure you are utilizing the corresponding tag 
 e.g. in doc_0, in doc_3 and so forth.
- If you cannot reply the query as a result of there's not sufficient data within the stories say 
 "I am unable to reply this primarily based on the offered stories." 
</directions>

<query>
{query}
</query>
"""

"""
<scratchpad>
[
{
"document_name": "doc_0",
"quote": "The main concern with Player A is his approach at the plate. He can be overly aggressive at times and will chase pitches out of the zone. Improving his plate discipline and pitch recognition will be key to him reaching his full offensive potential."
},
{
"document_name": "doc_1",
"quote": "The main area of focus for Player B's development will be improving his plate discipline and pitch recognition. He can be overly aggressive at times, leading to a high strikeout rate and a lower on-base percentage than his hit tool would suggest."
}
]
</scratchpad>
<reply>
The principle weaknesses of the gamers are:
For Participant A:
- Overly aggressive method on the plate, chasing pitches out of the zone
- Wants to enhance plate self-discipline and pitch recognition to succeed in full offensive potential
For Participant B:
- Could be overly aggressive on the plate, resulting in excessive strikeout charge and decrease on-base proportion
- Wants to enhance plate self-discipline and pitch recognition
</reply>
"""

# Word: The scouting stories for Participant A used within the instance under
# had been generated by Anthropic’s Claude Sonnet 3.

quotes_in_text_prompt = """
You might be an Ai assistant chatbot designed to reply questions on your crew's baseballe scouting stories.

Listed below are some stories about gamers that you should use to reply the query:

<doc_0>
{doc_0}
</doc_0>

...

<doc_10>
{doc_10}
</doc_10>

<directions>
- Put your reply in <reply> tags.
- Use as a lot data from totally different stories as doable.
- It is best to solely use data within the paperwork to reply. If you do not have sufficient data within the stories to reply, say you may't reply primarily based on the stories.
- It is best to floor your reply by quoting the related paperwork by utilizing citation marks.
- After the quotes, put an inline quotation <instance>Participant A is "very athletic" (doc_2)</instance>
</directions>

<query>
{query}
</query>
"""

Under is the FM response to the query "What are the strengths of participant A?":

"""
<scratchpad>
[
{
"document_name": "doc_0",
"quote": "The main concern with Player A is his approach at the plate. He can be overly aggressive at times and will chase pitches out of the zone. Improving his plate discipline and pitch recognition will be key to him reaching his full offensive potential."
},
{
"document_name": "doc_1",
"quote": "The main area of focus for Player B's development will be improving his plate discipline and pitch recognition. He can be overly aggressive at times, leading to a high strikeout rate and a lower on-base percentage than his hit tool would suggest."
}
]
</scratchpad>
<reply>
The principle weaknesses of the gamers are:
For Participant A:
- Overly aggressive method on the plate, chasing pitches out of the zone
- Wants to enhance plate self-discipline and pitch recognition to succeed in full offensive potential
For Participant B:
- Could be overly aggressive on the plate, resulting in excessive strikeout charge and decrease on-base proportion
- Wants to enhance plate self-discipline and pitch recognition
</reply>
"""