MKQA is a multilingual open-domain question answering benchmark dataset released by Apple in 2020, with the related paper titled "MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering", aimed at evaluating the performance of cross-lingual knowledge question answering models.

The dataset contains over 37,000 natural language question-answer pairs, covering 24 languages. The data is sourced from the Natural Questions dataset, translated and annotated through crowdsourcing, providing queries and their answers in a multilingual context, including various types such as entities, long answers, and short phrases.

Dataset Composition

• example_id: Unique identifier for the sample, string type.
• queries: Query texts in 24 languages, with each language corresponding to a string field (e.g., ar, da, de, en, es, etc.).
• query: Original English query text, string type.
• answers: Answer lists in 24 languages, with each language corresponding to a structure containing the following subfields:
• type: Answer type, including entity, long_answer, unanswerable, date, number, number_with_unit, short_phrase, and binary.
• entity: Entity name, string type.
• text: Answer text, string type.
• aliases: Alias list, string array type.

Citation

@misc{mkqa,
    title = {MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering},
    author = {Shayne Longpre and Yi Lu and Joachim Daiber},
    year = {2020},
    URL = {https://arxiv.org/pdf/2007.15207.pdf}
}