Implement Retrieval and Evaluation for a Simple RAG System

You are provided with a basic "toy RAG" (Retrieval Augmented Generation) setup, including helper utilities and two pandas DataFrames:

• docs_df: Contains a corpus of text passages with columns doc_id (string/int) and text (string).
• queries_df: Contains query information with columns query_id, query_text, and relevant_doc_ids (a list of doc_ids considered relevant for that query).

A utility function embed_texts(texts: List[str]) -> np.ndarray is available, which takes a list of strings and returns a 2D NumPy array of embeddings with shape (n, d).

Part A: Build Document Embeddings

Compute embeddings for every text passage in docs_df and store them efficiently for subsequent retrieval operations.

Part B: Retrieve Top-K by Cosine Similarity

For each query in queries_df:

1. Compute the embedding for the query_text.
2. Calculate the cosine similarity between the query embedding and all stored document embeddings.
3. Identify and return the top 10 documents, ranked by their similarity scores.

The output for each query should include:

• The query_id.
• A ranked list of the 10 retrieved doc_ids.
• (Optional) The corresponding similarity scores for the top 10 documents.

Part C: Evaluate Retrieval

Implement an evaluation mechanism across all queries using the following metrics:

• Recall@10: The fraction of queries where at least one relevant document (from relevant_doc_ids) appears within the top 10 retrieved documents.
• MRR@10 (Mean Reciprocal Rank): The mean reciprocal rank of the first relevant document found within the top 10 retrieved documents. If no relevant document is found in the top 10, its reciprocal rank is 0.

Constraints and Notes

• The docs_df can be substantial; prioritize efficient, vectorized operations and avoid explicit Python loops over documents where possible.
• Embeddings generated by embed_texts may or may not be pre-normalized. Your cosine similarity calculation must correctly handle both scenarios.
• You are restricted to using only NumPy and Pandas libraries. External vector databases or other specialized libraries are not permitted.

Describe your proposed approach and implement the necessary code components as if filling in a notebook.