learning to rank

File:MLR-search-engine-example.svg

Ranking is a central part of many information retrieval problems, such as document retrieval, collaborative filtering, sentiment analysis, and online advertising.

A possible architecture of a machine-learned search engine is shown in the accompanying figure.

Training data consists of queries and documents matching them together with the relevance degree of each match. It may be prepared manually by human assessors (or raters, as Google calls them), who check results for some queries and determine relevance of each result. It is not feasible to check the relevance of all documents, and so typically a technique called pooling is used — only the top few documents, retrieved by some existing ranking models are checked. This technique may introduce selection bias. Alternatively, training data may be derived automatically by analyzing clickthrough logs (i.e. search results which got clicks from users),{{citation

| author=Joachims, T.

| journal=Proceedings of the ACM Conference on Knowledge Discovery and Data Mining

| url=http://www.cs.cornell.edu/people/tj/publications/joachims_02c.pdf

| title=Optimizing Search Engines using Clickthrough Data

| year=2002

| access-date=2009-11-11

| archive-date=2009-12-29

| archive-url=https://web.archive.org/web/20091229194717/http://www.cs.cornell.edu/people/tj/publications/joachims_02c.pdf

| url-status=live

}} query chains,{{citation |author1=Joachims T. |author2=Radlinski F. |title=Query Chains: Learning to Rank from Implicit Feedback |url=http://radlinski.org/papers/Radlinski05QueryChains.pdf |year=2005 |journal=Proceedings of the ACM Conference on Knowledge Discovery and Data Mining |arxiv=cs/0605035 |bibcode=2006cs........5035R |access-date=2009-12-19 |archive-date=2011-07-27 |archive-url=https://web.archive.org/web/20110727195024/http://radlinski.org/papers/Radlinski05QueryChains.pdf |url-status=live }} or such search engines' features as Google's (since-replaced) SearchWiki. Clickthrough logs can be biased by the tendency of users to click on the top search results on the assumption that they are already well-ranked.

Training data is used by a learning algorithm to produce a ranking model which computes the relevance of documents for actual queries.

Typically, users expect a search query to complete in a short time (such as a few hundred milliseconds for web search), which makes it impossible to evaluate a complex ranking model on each document in the corpus, and so a two-phase scheme is used.{{citation |author1=B. Cambazoglu |author2=H. Zaragoza |author3=O. Chapelle |author4=J. Chen |author5=C. Liao |author6=Z. Zheng |author7=J. Degenhardt. |title=Early exit optimizations for additive machine learned ranking systems |journal=WSDM '10: Proceedings of the Third ACM International Conference on Web Search and Data Mining, 2010. |url=http://olivier.chapelle.cc/pub/wsdm2010.pdf |access-date=2009-12-23 |archive-date=2019-08-28 |archive-url=https://web.archive.org/web/20190828063315/http://olivier.chapelle.cc/pub/wsdm2010.pdf |url-status=dead }} First, a small number of potentially relevant documents are identified using simpler retrieval models which permit fast query evaluation, such as the vector space model, Boolean model, weighted AND,{{citation

| author1=Broder A.|author1-link=Andrei Broder

| author2=Carmel D.

| author3=Herscovici M.

| author4=Soffer A.|author4-link=Aya Soffer

| author5=Zien J.

|title=Proceedings of the twelfth international conference on Information and knowledge management

|chapter=Efficient query evaluation using a two-level retrieval process

| year=2003

| pages=426–434

| doi=10.1145/956863.956944

| isbn=978-1-58113-723-1

| s2cid=2432701

| url=http://cis.poly.edu/westlab/papers/cntdstrb/p426-broder.pdf

| access-date=2009-12-15

| archive-url=https://web.archive.org/web/20090521102038/http://cis.poly.edu/westlab/papers/cntdstrb/p426-broder.pdf

| archive-date=2009-05-21

| url-status=dead

}} or BM25. This phase is called top-$k$ document retrieval and many heuristics were proposed in the literature to accelerate it, such as using a document's static quality score and tiered indexes.{{citation

|author1=Manning C. |author2=Raghavan P. |author3=Schütze H. | title=Introduction to Information Retrieval

| publisher=Cambridge University Press

| year=2008}}. Section [http://nlp.stanford.edu/IR-book/html/htmledition/efficient-scoring-and-ranking-1.html 7.1] {{Webarchive|url=https://web.archive.org/web/20090719140459/http://nlp.stanford.edu/IR-book/html/htmledition/efficient-scoring-and-ranking-1.html |date=2009-07-19 }} In the second phase, a more accurate but computationally expensive machine-learned model is used to re-rank these documents.

Learning to rank algorithms have been applied in areas other than information retrieval:

• In machine translation for ranking a set of hypothesized translations;{{citation

| author=Kevin K. Duh

| title=Learning to Rank with {{sic|hide=y|Partially|-}}Labeled Data

| year=2009

| url=http://ssli.ee.washington.edu/people/duh/thesis/uwthesis.pdf

| access-date=2009-12-27

| archive-date=2011-07-20

| archive-url=https://web.archive.org/web/20110720112832/http://ssli.ee.washington.edu/people/duh/thesis/uwthesis.pdf

| url-status=live

}}

• In computational biology for ranking candidate 3-D structures in protein structure prediction problems;
• In recommender systems for identifying a ranked list of related news articles to recommend to a user after he or she has read a current news article.Yuanhua Lv, Taesup Moon, Pranam Kolari, Zhaohui Zheng, Xuanhui Wang, and Yi Chang, [http://sifaka.cs.uiuc.edu/~ylv2/pub/www11-relatedness.pdf Learning to Model Relatedness for News Recommendation] {{Webarchive|url=https://web.archive.org/web/20110827065356/http://sifaka.cs.uiuc.edu/~ylv2/pub/www11-relatedness.pdf |date=2011-08-27 }}, in International Conference on World Wide Web (WWW), 2011.

For the convenience of MLR algorithms, query-document pairs are usually represented by numerical vectors, which are called feature vectors. Such an approach is sometimes called bag of features and is analogous to the bag of words model and vector space model used in information retrieval for representation of documents.

Components of such vectors are called features, factors or ranking signals. They may be divided into three groups (features from document retrieval are shown as examples):

• Query-independent or static features — those features, which depend only on the document, but not on the query. For example, PageRank or document's length. Such features can be precomputed in off-line mode during indexing. They may be used to compute document's static quality score (or static rank), which is often used to speed up search query evaluation.{{cite conference

| first=M.

| last=Richardson

| author2=Prakash, A.

| author3=Brill, E.

| title=Beyond PageRank: Machine Learning for Static Ranking

| book-title=Proceedings of the 15th International World Wide Web Conference

| pages=707–715

| year=2006

| url=http://research.microsoft.com/en-us/um/people/mattri/papers/www2006/staticrank.pdf

| conference=

| access-date=2009-11-18

| archive-date=2009-08-15

| archive-url=https://web.archive.org/web/20090815173829/http://research.microsoft.com/en-us/um/people/mattri/papers/www2006/staticrank.pdf

| url-status=live

}}

• Query-dependent or dynamic features — those features, which depend both on the contents of the document and the query, such as TF-IDF score or other non-machine-learned ranking functions.
• Query-level features or query features, which depend only on the query. For example, the number of words in a query.

Some examples of features, which were used in the well-known [https://www.microsoft.com/en-us/research/project/letor-learning-rank-information-retrieval/ LETOR] dataset:

• TF, TF-IDF, BM25, and language modeling scores of document's zones (title, body, anchors text, URL) for a given query;
• Lengths and IDF sums of document's zones;
• Document's PageRank, HITS ranks and their variants.

Selecting and designing good features is an important area in machine learning, which is called feature engineering.

{{main|Evaluation measures (information_retrieval)#Offline_metrics}}

There are several measures (metrics) which are commonly used to judge how well an algorithm is doing on training data and to compare the performance of different MLR algorithms. Often a learning-to-rank problem is reformulated as an optimization problem with respect to one of these metrics.

Examples of ranking quality measures:

• Mean average precision (MAP);
• DCG and NDCG;
• Precision@n, NDCG@n, where "@n" denotes that the metrics are evaluated only on top n documents;
• Mean reciprocal rank;
• Kendall's tau;
• Spearman's rho.

DCG and its normalized variant NDCG are usually preferred in academic research when multiple levels of relevance are used.{{Cite web |url=http://www.stanford.edu/class/cs276/handouts/lecture15-learning-ranking.ppt |title=Archived copy |access-date=2009-12-14 |archive-date=2011-01-04 |archive-url=https://web.archive.org/web/20110104111500/http://www.stanford.edu/class/cs276/handouts/lecture15-learning-ranking.ppt |url-status=live }} Other metrics such as MAP, MRR and precision, are defined only for binary judgments.

Recently, there have been proposed several new evaluation metrics which claim to model user's satisfaction with search results better than the DCG metric:

• Expected reciprocal rank (ERR);{{citation

|author1=Olivier Chapelle |author2=Donald Metzler |author3=Ya Zhang |author4=Pierre Grinspan |title=Expected Reciprocal Rank for Graded Relevance

|url=http://research.yahoo.com/files/err.pdf |archive-url=https://web.archive.org/web/20120224053008/http://research.yahoo.com/files/err.pdf |url-status=dead |archive-date=2012-02-24 |journal=CIKM

|year=2009

}}

• Yandex's pfound.{{citation

|author1=Gulin A.

|author2=Karpovich P.

|author3=Raskovalov D.

|author4=Segalovich I.

|title=Yandex at ROMIP'2009: optimization of ranking algorithms by machine learning methods

|url=http://romip.ru/romip2009/15_yandex.pdf

|journal=Proceedings of ROMIP'2009

|year=2009

|pages=163–168

|access-date=2009-11-13

|archive-date=2009-11-22

|archive-url=https://web.archive.org/web/20091122074302/http://romip.ru/romip2009/15_yandex.pdf

|url-status=live

}} (in Russian)

Both of these metrics are based on the assumption that the user is more likely to stop looking at search results after examining a more relevant document, than after a less relevant document.

Learning to Rank approaches are often categorized using one of three approaches: pointwise (where individual documents are ranked), pairwise (where pairs of documents are ranked into a relative order), and listwise (where an entire list of documents are ordered).

Tie-Yan Liu of Microsoft Research Asia has analyzed existing algorithms for learning to rank problems in his book Learning to Rank for Information Retrieval. He categorized them into three groups by their input spaces, output spaces, hypothesis spaces (the core function of the model) and loss functions: the pointwise, pairwise, and listwise approach. In practice, listwise approaches often outperform pairwise approaches and pointwise approaches. This statement was further supported by a large scale experiment on the performance of different learning-to-rank methods on a large collection of benchmark data sets.{{citation |author1=Tax, Niek |author2=Bockting, Sander |author3=Hiemstra, Djoerd |journal=Information Processing & Management |volume=51 |issue=6 |title=A cross-benchmark comparison of 87 learning to rank methods |pages=757–772 |year=2015 |url=http://wwwhome.cs.utwente.nl/~hiemstra/papers/ipm2015.pdf |doi=10.1016/j.ipm.2015.07.002 |s2cid=22782599 |access-date=2017-10-15 |archive-url=https://web.archive.org/web/20170809115827/http://wwwhome.cs.utwente.nl/~hiemstra/papers/ipm2015.pdf |archive-date=2017-08-09 |url-status=dead }}

In this section, without further notice, $x$ denotes an object to be evaluated, for example, a document or an image, $f(x)$ denotes a single-value hypothesis, $h(\backslash cdot)$ denotes a bi-variate or multi-variate function and $L(\backslash cdot)$ denotes the loss function.

In this case, it is assumed that each query-document pair in the training data has a numerical or ordinal score. Then the learning-to-rank problem can be approximated by a regression problem — given a single query-document pair, predict its score. Formally speaking, the pointwise approach aims at learning a function $f(x)$ predicting the real-value or ordinal score of a document $x$ using the loss function $L(f;\; x\_j,\; y\_j)$.

A number of existing supervised machine learning algorithms can be readily used for this purpose. Ordinal regression and classification algorithms can also be used in pointwise approach when they are used to predict the score of a single query-document pair, and it takes a small, finite number of values.

In this case, the learning-to-rank problem is approximated by a classification problem — learning a binary classifier $h(x\_u,\; x\_v)$ that can tell which document is better in a given pair of documents. The classifier shall take two documents as its input and the goal is to minimize a loss function $L(h;\; x\_u,\; x\_v,\; y\_\{u,v\})$. The loss function typically reflects the number and magnitude of inversions in the induced ranking.

In many cases, the binary classifier $h(x\_u,\; x\_v)$ is implemented with a scoring function $f(x)$. As an example, RankNet {{cite journal |last1=Burges |first1=Chris J. C. |last2=Shaked |first2=Tal |last3=Renshaw |first3=Erin |last4=Lazier |first4=Ari |last5=Deeds |first5=Matt |last6=Hamilton |first6=Nicole |last7=Hullender |first7=Greg |title=Learning to Rank using Gradient Descent |url=https://www.microsoft.com/en-us/research/publication/learning-to-rank-using-gradient-descent |date=1 August 2005 |journal= |access-date=31 March 2021 |archive-date=26 February 2021 |archive-url=https://web.archive.org/web/20210226045904/https://www.microsoft.com/en-us/research/publication/learning-to-rank-using-gradient-descent/ |url-status=live }} adapts a probability model and defines $h(x\_u,\; x\_v)$ as the estimated probability of the document $x\_u$ has higher quality than $x\_v$:

:$P\_\{u,\; v\}(f)\; =$

\text{CDF} (f(x_u) - f(x_v)),

where $\backslash text\{CDF\}(\backslash cdot)$ is a cumulative distribution function, for example, the standard logistic CDF, i.e.

:$$

\text{CDF}(x)

=

\frac{

1

}{

1 + \exp\left[ - x \right]

}.

These algorithms try to directly optimize the value of one of the above evaluation measures, averaged over all queries in the training data. This is often difficult in practice because most evaluation measures are not continuous functions with respect to ranking model's parameters, and so continuous approximations or bounds on evaluation measures have to be used. For example the SoftRank algorithm.Taylor, M.J., Guiver, J., Robertson, S.E., & Minka, T.P. (2008). SoftRank: optimizing non-smooth rank metrics. Web Search and Data Mining. LambdaMART is a pairwise algorithm which has been empirically shown to approximate listwise objective functions.{{Cite journal |last=Burges |first=Chris J. C. |date=2010-06-23 |title=From RankNet to LambdaRank to LambdaMART: An Overview |url=https://www.microsoft.com/en-us/research/publication/from-ranknet-to-lambdarank-to-lambdamart-an-overview/ |language=en-US}}

A partial list of published learning-to-rank algorithms is shown below with years of first publication of each method:

:

Note: as most supervised learning-to-rank algorithms can be applied to pointwise, pairwise and listwise case, only those methods which are specifically designed with ranking in mind are shown above.

Norbert Fuhr introduced the general idea of MLR in 1992, describing learning approaches in information retrieval as a generalization of parameter estimation;{{citation

| last=Fuhr

| first=Norbert

| journal=Computer Journal

| title=Probabilistic Models in Information Retrieval

| volume=35

| number=3

| pages=243–255

| year=1992

| doi=10.1093/comjnl/35.3.243

| doi-access=free

}} a specific variant of this approach (using polynomial regression) had been published by him three years earlier. Bill Cooper proposed logistic regression for the same purpose in 1992 and used it with his Berkeley research group to train a successful ranking function for TREC. Manning et al.{{citation |author1=Manning C. |author2=Raghavan P. |author3=Schütze H. |title=Introduction to Information Retrieval |publisher=Cambridge University Press |year=2008}}. Sections [http://nlp.stanford.edu/IR-book/html/htmledition/references-and-further-reading-7.html 7.4] {{Webarchive|url=https://web.archive.org/web/20090721115530/http://nlp.stanford.edu/IR-book/html/htmledition/references-and-further-reading-7.html |date=2009-07-21 }} and [http://nlp.stanford.edu/IR-book/html/htmledition/references-and-further-reading-15.html 15.5] {{Webarchive|url=https://web.archive.org/web/20100509011556/http://nlp.stanford.edu/IR-book/html/htmledition/references-and-further-reading-15.html |date=2010-05-09 }} suggest that these early works achieved limited results in their time due to little available training data and poor machine learning techniques.

Several conferences, such as NeurIPS, SIGIR and ICML have had workshops devoted to the learning-to-rank problem since the mid-2000s (decade).

Commercial web search engines began using machine-learned ranking systems since the 2000s (decade). One of the first search engines to start using it was AltaVista (later its technology was acquired by Overture, and then Yahoo), which launched a gradient boosting-trained ranking function in April 2003.Jan O. Pedersen. [http://jopedersen.com/Presentations/The_MLR_Story.pdf The MLR Story] {{Webarchive|url=https://web.archive.org/web/20110713120113/http://jopedersen.com/Presentations/The_MLR_Story.pdf |date=2011-07-13 }}{{US Patent|7197497}}

Bing's search is said to be powered by [https://www.microsoft.com/en-us/research/wp-content/uploads/2005/08/icml_ranking.pdf RankNet] algorithm,{{Cite web |url=http://www.bing.com/community/blogs/search/archive/2009/06/01/user-needs-features-and-the-science-behind-bing.aspx?PageIndex=4 |title=Bing Search Blog: User Needs, Features and the Science behind Bing |access-date=2009-11-19 |archive-date=2009-11-25 |archive-url=https://web.archive.org/web/20091125205204/http://www.bing.com/community/blogs/search/archive/2009/06/01/user-needs-features-and-the-science-behind-bing.aspx?PageIndex=4 |url-status=live }}{{when|date=February 2014}} which was invented at Microsoft Research in 2005.

In November 2009 a Russian search engine Yandex announced[http://webmaster.ya.ru/replies.xml?item_no=5707&ncrnd=5118 Yandex corporate blog entry about new ranking model "Snezhinsk"] {{Webarchive|url=https://web.archive.org/web/20120301165959/http://webmaster.ya.ru/replies.xml?item_no=5707&ncrnd=5118 |date=2012-03-01 }} (in Russian) that it had significantly increased its search quality due to deployment of a new proprietary MatrixNet algorithm, a variant of gradient boosting method which uses oblivious decision trees.The algorithm wasn't disclosed, but a few details were made public in [http://download.yandex.ru/company/experience/GDD/Zadnie_algoritmy_Karpovich.pdf] {{Webarchive|url=https://web.archive.org/web/20100601171627/http://download.yandex.ru/company/experience/GDD/Zadnie_algoritmy_Karpovich.pdf|date=2010-06-01}} and [http://download.yandex.ru/company/experience/searchconf/Searchconf_Algoritm_MatrixNet_Gulin.pdf] {{Webarchive|url=https://web.archive.org/web/20100601171629/http://download.yandex.ru/company/experience/searchconf/Searchconf_Algoritm_MatrixNet_Gulin.pdf|date=2010-06-01}}. Recently they have also sponsored a machine-learned ranking competition "Internet Mathematics 2009"{{Cite web |url=http://imat2009.yandex.ru/academic/mathematic/2009/en/ |title=Yandex's Internet Mathematics 2009 competition page |access-date=2009-11-11 |archive-url=https://web.archive.org/web/20150317144535/http://imat2009.yandex.ru/academic/mathematic/2009/en/ |archive-date=2015-03-17 |url-status=dead }} based on their own search engine's production data. Yahoo has announced a similar competition in 2010.{{Cite web |url=http://learningtorankchallenge.yahoo.com/ |title=Yahoo Learning to Rank Challenge |access-date=2010-02-26 |archive-url=https://web.archive.org/web/20100301011649/http://learningtorankchallenge.yahoo.com/ |archive-date=2010-03-01 |url-status=dead }}

As of 2008, Google's Peter Norvig denied that their search engine exclusively relies on machine-learned ranking.{{cite web

|url = http://anand.typepad.com/datawocky/2008/05/are-human-experts-less-prone-to-catastrophic-errors-than-machine-learned-models.html

|archive-url = https://archive.today/20120701122042/http://anand.typepad.com/datawocky/2008/05/are-human-experts-less-prone-to-catastrophic-errors-than-machine-learned-models.html

|archive-date = 2012-07-01

|title = Are Machine-Learned Models Prone to Catastrophic Errors?

|date = 2008-05-24

|last = Rajaraman

|first = Anand

|author-link = Anand Rajaraman

|access-date = 2009-11-11

|url-status = live

}} Cuil's CEO, Tom Costello, suggests that they prefer hand-built models because they can outperform machine-learned models when measured against metrics like click-through rate or time on landing page, which is because machine-learned models "learn what people say they like, not what people actually like".{{cite web

| url = http://www.cuil.com/info/blog/2009/06/26/so-how-is-bing-doing

| archive-url = https://archive.today/20090627213358/http://www.cuil.com/info/blog/2009/06/26/so-how-is-bing-doing

| archive-date = 2009-06-27

| title = Cuil Blog: So how is Bing doing?

| date = 2009-06-26

| last = Costello

| first = Tom}}

In January 2017, the technology was included in the open source search engine Apache Solr.{{Cite news|url=https://www.techatbloomberg.com/blog/bloomberg-integrated-learning-rank-apache-solr/|title=How Bloomberg Integrated Learning-to-Rank into Apache Solr {{!}} Tech at Bloomberg|date=2017-01-23|work=Tech at Bloomberg|access-date=2017-02-28|language=en-US|archive-date=2017-03-01|archive-url=https://web.archive.org/web/20170301092711/https://www.techatbloomberg.com/blog/bloomberg-integrated-learning-rank-apache-solr/|url-status=live}} It is also available in the open source OpenSearch and Elasticsearch.{{Cite web |title=Learning to Rank for Amazon OpenSearch Service - Amazon OpenSearch Service |url=https://docs.aws.amazon.com/opensearch-service/latest/developerguide/learning-to-rank.html |access-date=2023-09-22 |website=docs.aws.amazon.com}}{{Cite web |title=Elasticsearch Learning to Rank: the documentation — Elasticsearch Learning to Rank documentation |url=https://elasticsearch-learning-to-rank.readthedocs.io/en/latest/ |access-date=2023-09-22 |website=elasticsearch-learning-to-rank.readthedocs.io}} These implementations make learning to rank widely accessible for enterprise search.

Similar to recognition applications in computer vision, recent neural network based ranking algorithms are also found to be susceptible to covert adversarial attacks, both on the candidates and the queries.{{cite arXiv | last1=Zhou | first1=Mo | last2=Niu | first2=Zhenxing | last3=Wang | first3=Le | last4=Zhang | first4=Qilin | last5=Hua | first5=Gang | title=Adversarial Ranking Attack and Defense | year=2020 | class=cs.CV | eprint=2002.11293v2 }} With small perturbations imperceptible to human beings, ranking order could be arbitrarily altered. In addition, model-agnostic transferable adversarial examples are found to be possible, which enables black-box adversarial attacks on deep ranking systems without requiring access to their underlying implementations.{{cite journal | last1=Li | first1=Jie | last2=Ji | first2=Rongrong | last3=Liu | first3=Hong | last4=Hong | first4=Xiaopeng | last5=Gao | first5=Yue | last6=Tian | first6=Qi | title=Universal Perturbation Attack Against Image Retrieval | website=International Conference on Computer Vision (ICCV 2019) | year=2019 | url=https://openaccess.thecvf.com/content_ICCV_2019/html/Li_Universal_Perturbation_Attack_Against_Image_Retrieval_ICCV_2019_paper.html | pages=4899–4908 | arxiv=1812.00552 | access-date=2020-07-04 | archive-date=2020-07-06 | archive-url=https://web.archive.org/web/20200706051031/https://openaccess.thecvf.com/content_ICCV_2019/html/Li_Universal_Perturbation_Attack_Against_Image_Retrieval_ICCV_2019_paper.html | url-status=live }}

Conversely, the robustness of such ranking systems can be improved via adversarial defenses such as the Madry defense.{{cite arXiv | last1=Madry | first1=Aleksander | last2=Makelov | first2=Aleksandar | last3=Schmidt | first3=Ludwig | last4=Tsipras | first4=Dimitris | last5=Vladu | first5=Adrian | title=Towards Deep Learning Models Resistant to Adversarial Attacks | date=2017-06-19 | class=stat.ML | eprint=1706.06083v4 }}

• Content-based image retrieval
• Multimedia information retrieval
• Image retrieval
• Triplet loss

{{reflist}}

; Competitions and public datasets

• [https://www.microsoft.com/en-us/research/project/letor-learning-rank-information-retrieval/ LETOR: A Benchmark Collection for Research on Learning to Rank for Information Retrieval]
• [https://web.archive.org/web/20150912134134/http://imat2009.yandex.ru/en Yandex's Internet Mathematics 2009]
• [https://web.archive.org/web/20100301011649/http://learningtorankchallenge.yahoo.com/ Yahoo! Learning to Rank Challenge]
• [http://research.microsoft.com/en-us/projects/mslr/default.aspx Microsoft Learning to Rank Datasets]

Category:Information retrieval techniques

Category:Machine learning

Category:Ranking functions