# ms-marco

## Overview

MSMARCO (Microsoft Machine Reading Comprehension) is a large scale dataset focused on machine reading comprehension, question answering, and passage ranking. It contains questions, passages and answers. The passages are top-k results Bing engin searched based on question. And when human editor created answers, they refer to these passages and selected the passages.

MSMARCO V1.1 was a original question answering dataset dataset with 100,000 real Bing questions and human-generated answers. Since then, several other datasets have been released, including a 1,000,000 question dataset, a natural language generation dataset, a passage ranking dataset, a keyphrase extraction dataset, a web crawling dataset, and an interactive search dataset. More information about MSMARCO dataset, refer to below link!\
[msmarco official github](https://github.com/microsoft/MSMARCO-Question-Answering)

MSMARCOEvaluator also support rank aware metrics like NDCG, AP, CG, IDCG, RR, etc.

## Example Use

Note: MSMARCO dataset version is optional(v1.1 or 2.1). Default is v1.1. We use validation set in v2.1 because v2.1 passage data `is_selected` values are all -1.\
Ingest size must be same or larger than evaluate size.

```Python
from RAGchain.benchmark.dataset import MSMARCOEvaluator

pipeline = <your pipeline>
retrievals = [<your retrieval>]
db = <your db>

evaluator = MSMARCOEvaluator(pipeline, evaluate_size=20)
evaluator.ingest(retrievals, db) # ingest dataset to db and retrievals
result = evaluator.evaluate()

# print result summary (mean values)
print(result.results)
# print result DataFrame
print(result.each_results)
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://nomadamas.gitbook.io/ragchain-docs/ragchain-structure/benchmark/dataset-evaluator/ms-marco.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.