Quantification (machine learning)

The main variants of quantification, according to the characteristics of the set of classes used, are:

• Binary quantification, corresponding to the case in which there are only $n=2$ classes and each data item belongs to exactly one of them;
• Single-label multiclass quantification, corresponding to the case in which there are $n>2$ classes and each data item belongs to exactly one of them;
• Multi-label multiclass quantification, corresponding to the case in which there are $n\backslash geq2$ classes and each data item can belong to zero, one, or several classes at the same time;
• Ordinal quantification, corresponding to the single-label multiclass case in which a total order is defined on the set of classes.
• Regression quantification, a task which stands to 'standard' quantification as regression stands to classification. Strictly speaking, this task is not a quantification task as defined above (since the individual items do not have class labels but are labelled by real values), but has enough commonalities with other quantification tasks to be considered one of them.

Most known quantification methods address the binary case or the single-label multiclass case, and only few of them address the multi-label, ordinal, and regression cases.

Binary-only methods include the Mixture Model (MM) method, the HDy method,{{cite journal

|author1=Víctor González-Castro

|author2=Rocío Alaiz-Rodríguez

|author3=Enrique Alegre

|title=Class distribution estimation based on the {H}ellinger distance

|journal=Information Sciences

|volume=218

|pages=146–164

|year=2013

|doi=10.1016/j.ins.2012.05.028

}} SVM(KLD), and SVM(Q).

Methods that can deal with both the binary case and the single-label multiclass case include probabilistic classify and count (PCC), adjusted classify and count (ACC), probabilistic adjusted classify and count (PACC), and the Saerens-Latinne-Decaestecker EM-based method (SLD).

Methods for multi-label quantification include regression-based quantification (RQ) and label powerset-based quantification (LPQ).{{cite journal

|author1=Alejandro Moreo

|author2=Manuel Francisco

|author3=Fabrizio Sebastiani

|title=Multi-label quantification

|journal=ACM Transactions on Knowledge Discovery from Data

|year=2024

|volume=18

|issue=1

|doi=10.1145/3606264

|arxiv=2211.08063

}}

Methods for the ordinal case include Ordinal Quantification Tree (OQT),{{cite book

|author1=Giovanni Da San Martino

|author2=Wei Gao

|author3=Fabrizio Sebastiani

|title=Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

|chapter=Ordinal Text Quantification

|date=2016

|pages=937–940

|year=2016

|doi=10.1145/2911451.2914749

|isbn=9781450340694

|s2cid=8102324

|chapter-url=https://ink.library.smu.edu.sg/sis_research/4569

}} and ordinal versions of the above-mentioned ACC, PACC, and SLD methods.{{cite journal

|author1=Mirko Bunse

|author2=Alejandro Moreo

|author3=Fabrizio Sebastiani

|author4=Martin Senz

|title=Ordinal quantification through regularization

|journal=Data Mining and Knowledge Discovery

|year=2024

|volume=38

|issue=6

|pages=4076-4121

}}

Methods for the regression case include Regress and splice and Adjusted regress and sum.{{cite journal

|author1=Antonio Bella

|author2=Cèsar Ferri

|author3=José Hernández-Orallo

|author4=María José Ramírez-Quintana

|title=Aggregative quantification for regression

|journal=Data Mining and Knowledge Discovery

|year=2014

|volume=28

|issue=2

|pages=475–518

|doi=10.1007/s10618-013-0308-z

|hdl=10251/49300

|hdl-access=free

}}

Several evaluation measures can be used for evaluating the error of a quantification method.

Since quantification consists of generating a predicted probability distribution that estimates a true probability distribution, these evaluation measures are ones that compare two probability distributions. Most evaluation measures for quantification belong to the class of divergences. Evaluation measures for binary quantification and single-label multiclass quantification are{{cite journal

|title= Evaluation measures for quantification: An axiomatic approach

|author1=Fabrizio Sebastiani

|journal=Information Retrieval Journal

|volume=23

|number=3

|pages=255–288

|year=2020

|doi=10.1007/s10791-019-09363-y

|arxiv=1809.01991

|s2cid=52170301

}}

• Absolute Error
• Squared Error
• Relative Absolute Error
• Kullback-Leibler divergence
• Pearson Divergence

Evaluation measures for ordinal quantification are

• Normalized Match Distance (a particular case of the Earth Mover's Distance)
• Root Normalized Order-Aware Distance

Quantification is of special interest in fields such as the social sciences,{{cite journal

|title= A method of automated nonparametric content analysis for social science

|author1= Daniel J. Hopkins

|author2= Gary King

|journal=American Journal of Political Science

|volume=54

|number=1

|pages=229–247

|year=2010

|doi=10.1111/j.1540-5907.2009.00428.x

|s2cid= 1177676

}}

epidemiology,{{cite journal

|title= Verbal autopsy methods with multiple causes of death

|author1=Gary King

|author2=Ying Lu

|journal=Statistical Science

|volume=23

|number=1

|pages=78–91

|year=2008

|doi=10.1214/07-sts247

|arxiv=0808.0645

|s2cid=4084198

}}

market research, and ecological modelling,{{cite journal

|title= Validation methods for plankton image classification systems

|author1=Pablo González

|author2=Eva Álvarez

|author3=Jorge Díez

|author4=Ángel López-Urrutia

|author5=Juan J. del Coz

|journal=Limnology and Oceanography: Methods

|volume=15

|pages=221–237

|year=2017

|issue=3

|doi=10.1002/lom3.10151

|bibcode=2017LimOM..15..221G

|s2cid=59438870

|url=https://epic.awi.de/id/eprint/47173/1/Gonzalez_et_al_2017_LandOMeths.pdf

}}

since these fields are inherently concerned with aggregate data. However, quantification is also useful as a building block for solving other downstream tasks, such as improving the accuracy of classifiers on out-of-distribution data,{{cite journal

|title=Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure

|author1=Marco Saerens

|author2=Patrice Latinne

|author3=Christine Decaestecker

|journal=Neural Computation

|volume=14

|number=1

|pages=21–41

|year=2002

|doi=10.1162/089976602753284446

|pmid=11747533

|s2cid=18254013

|url=https://dipot.ulb.ac.be/dspace/bitstream/2013/68391/1/Decaestecker_NeuralComp02.pdf

}}

{{cite journal

|title= Word sense disambiguation with distribution estimation

|author1=Yee Seng Chan

|author2=Hwee Tou Ng

|journal=Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005)

|location=Edinburgh, UK

|pages=1010–1015

|year=2005

|url=https://dl.acm.org/doi/10.5555/1642293.1642455

}}

allocating resources,

measuring classifier bias,{{cite journal

|title= Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach

|author1=Alessandro Fabris

|author2=Andrea Esuli

|author3=Alejandro Moreo

|author4=Fabrizio Sebastiani

|journal=Journal of Artificial Intelligence Research

|year=2023

|volume=76

|url=https://www.jair.org/index.php/jair/article/view/14033/26912.pdf

|pages=1117–1180

|doi=10.1613/jair.1.14033

|arxiv=2109.08549

|s2cid=247315416

}}

and estimating the accuracy of classifiers on out-of-distribution data.{{cite journal

|title= A simple method for classifier accuracy prediction under prior probability shift

|author1=Lorenzo Volpi

|author2=Alejandro Moreo

|author3=Fabrizio Sebastiani

|journal=Proceedings of the 27th International Conference on Discovery Science (DS 2024)

|location=Pisa, IT

|pages=267-283

|year=2024

|url=https://link.springer.com/chapter/10.1007/978-3-031-78980-9_17

}}

• LQ 2021: the 1st International Workshop on Learning to Quantify{{cite web

|url=https://cikmlq2021.github.io/

|title=LQ 2021: the 1st International Workshop on Learning to Quantify

}}

• LQ 2022: the 2nd International Workshop on Learning to Quantify{{cite web

|url=https://lq-2022.github.io/

|title=LQ 2022: the 2nd International Workshop on Learning to Quantify

}}

• LQ 2023: the 3rd International Workshop on Learning to Quantify{{cite web

|url=https://lq-2023.github.io/

|title=LQ 2023: the 3rd International Workshop on Learning to Quantify

}}

• LQ 2024: the 4th International Workshop on Learning to Quantify{{cite web

|url=https://lq-2024.github.io/

|title=LQ 2024: the 4th International Workshop on Learning to Quantify

}}

• LeQua 2022: the 1st Data Challenge on Learning to Quantify {{cite web

|url=

https://lequa2022.github.io/

|title=LeQua 2022: A Data Challenge on Learning to Quantify

}}

• LeQua 2024: the 2nd Data Challenge on Learning to Quantify {{cite web

|url=

https://lequa2042.github.io/

|title=LeQua 2024: A Data Challenge on Learning to Quantify

}}

• QuaPy: An open-source Python-based software library for quantification{{cite web

|url=https://github.com/HLT-ISTI/QuaPy

|title=QuaPy: A Python-Based Framework for Quantification

|website=GitHub

|date=23 November 2021

}}

• QuantificationLib: A Python library for quantification and prevalence estimation{{cite web

|url=https://aicgijon.github.io/quantificationlib/

|title=QuantificationLib: A Python library for quantification and prevalence estimation

|website=GitHub

|date=8 April 2024

}}

{{reflist}}

Category:Machine learning