Trec¶
Qrels¶
Functions and classes for dealing with English qrel files. The specification can be viewed at: http://trec.nist.gov/data/qrels_eng/
Usage:
>>> sample_qrels = '1 0 AP880212-0161 0\n1 0 AP880216-0139 1\n1 0 AP880216-0169 0'
>>> qrels = loads(sample_qrels)
>>> qrels.topic
['1', '1', '1']
>>> [x.topic for x in qrels['1']]
['1', '1', '1']
Harry Scells Mar 2017
-
class
irkit.trec.qrels.
Qrel
(topic: str, iteration: int, document_num: str, relevancy: int)¶ A line in a qrels file conforming to the specification at: http://trec.nist.gov/data/qrels_eng/
-
class
irkit.trec.qrels.
Qrels
(qrels: typing.List[irkit.trec.qrels.Qrel])¶ A python representation of a qrels file. It is a list of Qrel objects.
-
dump
(fp: _io.TextIOWrapper) → None¶ Dump the qrels to a file
Parameters: fp – A File pointer
-
dumps
() → str¶ Dump the qrels to a string.
Returns: Formatted qrels
-
-
irkit.trec.qrels.
load
(qrels: _io.TextIOWrapper) → irkit.trec.qrels.Qrels¶ Load qrels from a file.
Parameters: qrels – File pointer Returns: Qrels object
-
irkit.trec.qrels.
loads
(qrels: str) → irkit.trec.qrels.Qrels¶ Load qrels from a string.
Parameters: qrels – Some string representation of qrels Returns: Qrels object
Runs¶
Functions and classes for dealing with trec_eval run files. For a description on how run files look, see: http://faculty.washington.edu/levow/courses/ling573_SPR2011/hw/trec_eval_desc.htm
Usage:
>>> sample_run = '''351 0 DOC1 1 100 run-name\n351 0 DOC2 2 50 run-name'''
>>> runs = loads(sample_run)
>>> runs.dumps()
'351 Q0 DOC1 1 100 run-name
351 Q0 DOC2 2 50 run-name'
>>> [str(run) for run in runs.runs]
['351 Q0 DOC1 1 100 run-name', '351 Q0 DOC2 2 50 run-name']
>>> [str(run) for run in runs['351']]
['351 Q0 DOC1 1 100 run-name', '351 Q0 DOC2 2 50 run-name']
>>> runs.rank
['1', '2']
>>> TrecEvalRuns(runs['351']).rank
['1', '2']
Harry Scells May 2017
-
class
irkit.trec.run.
TrecEvalRun
(topic: str, q: int, doc_id: str, rank: int, score: float, run_id: str)¶ TrecEvalRun is a container class for a line in a trec_eval run file.
-
class
irkit.trec.run.
TrecEvalRuns
(runs: typing.List[irkit.trec.run.TrecEvalRun])¶ TrecEvalRuns is a wrapper around a TrecEvalRun which is just a container class for a line in a trec_eval run file. This class contains some convenience functions for dealing with runs, such as getting a list of the runs but topic id or slicing by column.
-
dump
(fp: _io.TextIOWrapper) → None¶ Dump the qrels to a file
Parameters: fp – A File pointer
-
dumps
() → str¶ Dump the qrels to a string.
Returns: Formatted qrels
-
-
irkit.trec.run.
load
(runs: _io.TextIOWrapper) → irkit.trec.run.TrecEvalRuns¶ Load a trec_eval run file.
Parameters: runs – A file pointer containing runs. Returns: TrecEvalRuns
-
irkit.trec.run.
loads
(runs: str) → irkit.trec.run.TrecEvalRuns¶ Load a trec_eval run file from a string.
Parameters: runs – A string containing runs. Returns: TrecEvalRuns
Results¶
Functions and classes for dealing with trec_eval results files.
Harry Scells Mar 2017
-
class
irkit.trec.results.
TrecEvalResults
(run_id: str, results: typing.Dict, queries: typing.Dict)¶ An object that stores the results output by trec_eval.
-
irkit.trec.results.
load
(trec_result_file: _io.TextIOWrapper) → irkit.trec.results.TrecEvalResults¶ Load trec_eval results from a file.
Parameters: trec_result_file – File pointer Returns: Qrels object
-
irkit.trec.results.
loads
(trec_results: str) → irkit.trec.results.TrecEvalResults¶ Load trec_eval results from a string.
Parameters: trec_results – Some string representation of trec results Returns: TrecEvalResults object