自然语言代写 | INFS7410 Project – Part 1

这个作业是实施多种信息检索方法，评估它们并在实际用例的上下文中进行比较
INFS7410 Project – Part 1

Project aim: The aim of this project is to implement a number of information retrieval methods,
evaluate them and compare them in the context of a real use-case.
Project Part 1 aim
The aim of Part 1 is to:
Setup your infrastructure to index the collection and evaluate queries.
Implement common information retrieval baselines.
Tune your retrieval implementations to improve their effectiveness.
Implement rank fusion methods.
The Information Retrieval Task: Web Passage Ranking
In this project we will consider the problem of open-domain passage ranking in answer to web
queries. In this context, users pose queries to the search engine and expect answers in the form
of a ranked list of passages (maximum 1000 passages to be retrieved).
The provided queries are real queries submitted to the Microsoft Bing search engine. In the
collection, there are approximately 8.8 million passages and the goal is to rank them based on
their relevance to the queries.
What we provide you with
A collection of 8.8 million text passages extracted from web pages
A list of queries to be used for training. Training is divided further into a small batch
( train_set.small.txt ) and a large batch ( train_set.txt ). You should use the small bath
of training for developing your initial implementations and the large batch for tuning your
implementations. If you have problems running the larger set because it takes long time
(unlikely for part 1), then use the small training set for tuning too.
A list of queries to be used for development ( dev_set.txt ). You should perform your
evaluation and analysis of your implementations on this dataset.
Relevance assessments for both the training ( train_set.qrels , train_set.small.qrels )
and development portions ( dev_set.qrels ) of the queries (qrels).
A Java Maven project that contains the Terrier dependencies and a skeleton code to give you
a start.
A project report that, following the provided template, details: an explanation of the retrieval
methods used, include the formulas that represent the models you implemented and a
snapshot of the code that implements that formula, an explanation of the evaluation
settings followed, the evaluation of results (as described above), inclusive of analysis, a
discussion of the findings.
Required methods to implement
In Part 1 of the project you are required to implement the following retrieval methods. All
implementations should be based on your own code.
1. TF-IDF: create your own implementation using the Terrier API to extract index statistics. See
the videos in Week 4 for background information.
2. BM25: create your own implementation using the Terrier API to extract index statistics. See
the videos in Week 4 for background information.
3. Pseudo-relevance feedback using BM25 (by using the full version of the RSJ weight): create
your own implementation using the Terrier API to extract index statistics. See the videos in
Week 4 for background information.
4. Jelinek-Mercer Language Model: create your own implementation using the Terrier API to
extract index statistics. See the videos in Week 4 for background information.
5. Dirichlet Language Model: create your own implementation using the Terrier API to extract
index statistics. See the videos in Week 4 for background information.
6. The rank fusion method Borda; you need to create your own implementation of this. See the
videos in Week 5 for background information.
7. The rank fusion method CombSUM; you need to create your own implementation of this.
See the videos in Week 5 for background information.
8. The rank fusion method CombMNZ; you need to create your own implementation of this.
See the videos in Week 5 for background information.
For each of the BM25, Jelinek-Mercer Language Model, and Dirichlet Language Model
implementations, you are also required to tune the parameters of these methods. You must
perform a parameter search over 5 sensibly chosen parameter values depending on the method
(10 when the method has two parameters). For Pseudo-relevance feedback using BM25, set the
best parameter values obtained with BM25.
For the rank fusion methods, consider fusing the highest performing tuned run from each of the
TF-IDF, BM25, Pseudo-relevance feedback using BM25, Jelinek-Mercer Language Model, and
Dirichlet Language Model implementations.
We strongly recommend you use the provided Maven project to implement your project. You
should have already attempted many of the implementations above as part of the computer
pracs exercises.