multi-task learning

The key challenge in multi-task learning, is how to combine learning signals from multiple tasks into a single model. This may strongly depend on how well different task agree with each other, or contradict each other. There are several ways to address this challenge:

Within the MTL paradigm, information can be shared across some or all of the tasks. Depending on the structure of task relatedness, one may want to share information selectively across the tasks. For example, tasks may be grouped or exist in a hierarchy, or be related according to some general metric. Suppose, as developed more formally below, that the parameter vector modeling each task is a linear combination of some underlying basis. Similarity in terms of this basis can indicate the relatedness of the tasks. For example, with sparsity, overlap of nonzero coefficients across tasks indicates commonality. A task grouping then corresponds to those tasks lying in a subspace generated by some subset of basis elements, where tasks in different groups may be disjoint or overlap arbitrarily in terms of their bases.Kumar, A., & Daume III, H., (2012) Learning Task Grouping and Overlap in Multi-Task Learning. http://icml.cc/2012/papers/690.pdf Task relatedness can be imposed a priori or learned from the data.Jawanpuria, P., & Saketha Nath, J., (2012) A Convex Feature Learning Formulation for Latent Task Structure Discovery. http://icml.cc/2012/papers/90.pdf Hierarchical task relatedness can also be exploited implicitly without assuming a priori knowledge or learning relations explicitly.Hajiramezanali, E. & Dadaneh, S. Z. & Karbalayghareh, A. & Zhou, Z. & Qian, X. Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data. 32nd Conference on Neural Information Processing Systems (NIPS 2018), Montréal, Canada. {{ArXiv|1810.09433}}Zweig, A. & Weinshall, D. Hierarchical Regularization Cascade for Joint Learning. Proceedings: of 30th International Conference on Machine Learning (ICML), Atlanta GA, June 2013. http://www.cs.huji.ac.il/~daphna/papers/Zweig_ICML2013.pdf For example, the explicit learning of sample relevance across tasks can be done to guarantee the effectiveness of joint learning across multiple domains.

One can attempt learning a group of principal tasks using a group of auxiliary tasks, unrelated to the principal ones. In many applications, joint learning of unrelated tasks which use the same input data can be beneficial. The reason is that prior knowledge about task relatedness can lead to sparser and more informative representations for each task grouping, essentially by screening out idiosyncrasies of the data distribution. Novel methods which builds on a prior multitask methodology by favoring a shared low-dimensional representation within each task grouping have been proposed. The programmer can impose a penalty on tasks from different groups which encourages the two representations to be orthogonal. Experiments on synthetic and real data have indicated that incorporating unrelated tasks can result in significant improvements over standard multi-task learning methods.

Related to multi-task learning is the concept of knowledge transfer. Whereas traditional multi-task learning implies that a shared representation is developed concurrently across tasks, transfer of knowledge implies a sequentially shared representation. Large scale machine learning projects such as the deep convolutional neural network GoogLeNet,{{Cite book|arxiv = 1409.4842 |doi = 10.1109/CVPR.2015.7298594 |isbn = 978-1-4673-6964-0|chapter = Going deeper with convolutions |title = 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) |pages = 1–9 |year = 2015 |last1 = Szegedy |first1 = Christian |last2 = Wei Liu |first2 = Youssef |last3 = Yangqing Jia |first3 = Tomaso |last4 = Sermanet |first4 = Pierre |last5 = Reed |first5 = Scott |last6 = Anguelov |first6 = Dragomir |last7 = Erhan |first7 = Dumitru |last8 = Vanhoucke |first8 = Vincent |last9 = Rabinovich |first9 = Andrew |s2cid = 206592484 }} an image-based object classifier, can develop robust representations which may be useful to further algorithms learning related tasks. For example, the pre-trained model can be used as a feature extractor to perform pre-processing for another learning algorithm. Or the pre-trained model can be used to initialize a model with similar architecture which is then fine-tuned to learn a different classification task.{{Cite web|url = https://www.mit.edu/~9.520/fall15/slides/class24/deep_learning_overview.pdf|title = Deep Learning Overview|last = Roig|first = Gemma|access-date = 2019-08-26|archive-date = 2016-03-06|archive-url = https://web.archive.org/web/20160306020712/http://www.mit.edu/~9.520/fall15/slides/class24/deep_learning_overview.pdf|url-status = dead}}

Traditionally Multi-task learning and transfer of knowledge are applied to stationary learning settings. Their extension to non-stationary environments is termed Group online adaptive learning (GOAL).Zweig, A. & Chechik, G. Group online adaptive learning. Machine Learning, DOI 10.1007/s10994-017- 5661-5, August 2017. http://rdcu.be/uFSv Sharing information could be particularly useful if learners operate in continuously changing environments, because a learner could benefit from previous experience of another learner to quickly adapt to their new environment. Such group-adaptive learning has numerous applications, from predicting financial time-series, through content recommendation systems, to visual understanding for adaptive autonomous agents.

Multi-task optimization focuses on solving optimizing the whole process.{{cite journal | doi=10.1109/TETCI.2017.2769104 | title=Insights on Transfer Optimization: Because Experience is the Best Teacher | year=2018 | last1=Gupta | first1=Abhishek | last2=Ong | first2=Yew-Soon | last3=Feng | first3=Liang | journal=IEEE Transactions on Emerging Topics in Computational Intelligence | volume=2 | pages=51–64 | hdl=10356/147980 | s2cid=11510470 | hdl-access=free }}{{cite journal | doi=10.1109/TEVC.2015.2458037 | title=Multifactorial Evolution: Toward Evolutionary Multitasking | year=2016 | last1=Gupta | first1=Abhishek | last2=Ong | first2=Yew-Soon | last3=Feng | first3=Liang | journal=IEEE Transactions on Evolutionary Computation | volume=20 | issue=3 | pages=343–357 | hdl=10356/148174 | s2cid=13767012 | hdl-access=free }} The paradigm has been inspired by the well-established concepts of transfer learning{{cite journal | doi=10.1109/TKDE.2009.191 | title=A Survey on Transfer Learning | year=2010 | last1=Pan | first1=Sinno Jialin | last2=Yang | first2=Qiang | journal=IEEE Transactions on Knowledge and Data Engineering | volume=22 | issue=10 | pages=1345–1359 | s2cid=740063 }} and multi-task learning in predictive analytics.Caruana, R., "Multitask Learning", pp. 95-134 in Sebastian Thrun, Lorien Pratt (eds.) Learning to Learn, (1998) Springer {{ISBN|9780792380474}}

The key motivation behind multi-task optimization is that if optimization tasks are related to each other in terms of their optimal solutions or the general characteristics of their function landscapes,{{cite journal | doi=10.1016/j.engappai.2017.05.008 | title=Coevolutionary multitasking for concurrent global optimization: With case studies in complex engineering design | year=2017 | last1=Cheng | first1=Mei-Ying | last2=Gupta | first2=Abhishek | last3=Ong | first3=Yew-Soon | last4=Ni | first4=Zhi-Wei | journal=Engineering Applications of Artificial Intelligence | volume=64 | pages=13–24 | s2cid=13767210 | doi-access=free }} the search progress can be transferred to substantially accelerate the search on the other.

The success of the paradigm is not necessarily limited to one-way knowledge transfers from simpler to more complex tasks. In practice an attempt is to intentionally solve a more difficult task that may unintentionally solve several smaller problems.{{cite arXiv | eprint=1707.03300 | last1=Cabi | first1=Serkan | author2=Sergio Gómez Colmenarejo | last3=Hoffman | first3=Matthew W. | last4=Denil | first4=Misha | last5=Wang | first5=Ziyu | author6=Nando de Freitas | title=The Intentional Unintentional Agent: Learning to Solve Many Continuous Control Tasks Simultaneously | year=2017 | class=cs.AI }}

There is a direct relationship between multitask optimization and multi-objective optimization.J. -Y. Li, Z. -H. Zhan, Y. Li and J. Zhang, "Multiple Tasks for Multiple Objectives: A New Multiobjective Optimization Method via Multitask Optimization," in IEEE Transactions on Evolutionary Computation, {{doi|10.1109/TEVC.2023.3294307}}

In some cases, the simultaneous training of seemingly related tasks may hinder performance compared to single-task models.{{Cite journal |last1=Standley |first1=Trevor |last2=Zamir |first2=Amir R. |last3=Chen |first3=Dawn |last4=Guibas |first4=Leonidas |last5=Malik |first5=Jitendra |last6=Savarese |first6=Silvio |date=2020-07-13 |title=Learning the Pareto Front with Hypernetworks |url=https://proceedings.mlr.press/v119/standley20a.html |journal=International Conference on Machine Learning (ICML)|pages=9120–9132 |arxiv=1905.07553 }} Commonly, MTL models employ task-specific modules on top of a joint feature representation obtained using a shared module. Since this joint representation must capture useful features across all tasks, MTL may hinder individual task performance if the different tasks seek conflicting representation, i.e., the gradients of different tasks point to opposing directions or differ significantly in magnitude. This phenomenon is commonly referred to as negative transfer. To mitigate this issue, various MTL optimization methods have been proposed. Commonly, the per-task gradients are combined into a joint update direction through various aggregation algorithms or heuristics.

There are several common approaches for multi-task optimization: Bayesian optimization, evolutionary computation, and approaches based on Game theory.

Multi-task Bayesian optimization is a modern model-based approach that leverages the concept of knowledge transfer to speed up the automatic hyperparameter optimization process of machine learning algorithms.Swersky, K., Snoek, J., & Adams, R. P. (2013). [http://papers.nips.cc/paper/5086-multi-task-bayesian-optimization.pdf Multi-task bayesian optimization]. Advances in neural information processing systems (pp. 2004-2012). The method builds a multi-task Gaussian process model on the data originating from different searches progressing in tandem.Bonilla, E. V., Chai, K. M., & Williams, C. (2008). [http://papers.nips.cc/paper/3189-multi-task-gaussian-process-prediction.pdf Multi-task Gaussian process prediction]. Advances in neural information processing systems (pp. 153-160). The captured inter-task dependencies are thereafter utilized to better inform the subsequent sampling of candidate solutions in respective search spaces.

Evolutionary multi-tasking has been explored as a means of exploiting the implicit parallelism of population-based search algorithms to simultaneously progress multiple distinct optimization tasks. By mapping all tasks to a unified search space, the evolving population of candidate solutions can harness the hidden relationships between them through continuous genetic transfer. This is induced when solutions associated with different tasks crossover.Ong, Y. S., & Gupta, A. (2016). [http://www.cil.ntu.edu.sg/mfo/downloads/MultitaskOptimization_manuscript.pdf Evolutionary multitasking: a computer science view of cognitive multitasking]. Cognitive Computation, 8(2), 125-142. Recently, modes of knowledge transfer that are different from direct solution crossover have been explored.{{cite journal | doi=10.1109/TCYB.2018.2845361 | title=Evolutionary Multitasking via Explicit Autoencoding | year=2019 | last1=Feng | first1=Liang | last2=Zhou | first2=Lei | last3=Zhong | first3=Jinghui | last4=Gupta | first4=Abhishek | last5=Ong | first5=Yew-Soon | last6=Tan | first6=Kay-Chen | last7=Qin | first7=A. K. | journal=IEEE Transactions on Cybernetics | volume=49 | issue=9 | pages=3457–3470 | pmid=29994415 | s2cid=51613697 }}{{Cite journal |last1=Jiang |first1=Yi |last2=Zhan |first2=Zhi-Hui |last3=Tan |first3=Kay Chen |last4=Zhang |first4=Jun |date=January 2024 |title=Block-Level Knowledge Transfer for Evolutionary Multitask Optimization |url=https://ieeexplore.ieee.org/document/10130298 |journal=IEEE Transactions on Cybernetics |volume=54 |issue=1 |pages=558–571 |doi=10.1109/TCYB.2023.3273625 |pmid=37216256 |issn=2168-2267}}

Game-theoretic approaches to multi-task optimization propose to view the optimization problem as a game, where each task is a player. All players compete through the reward matrix of the game, and try to reach a solution that satisfies all players (all tasks). This view provide insight about how to build efficient algorithms based on gradient descent optimization (GD), which is particularly important for training deep neural networks.{{Cite book |last1=Goodfellow |first1=Ian |title=Deep Learning |last2=Bengio |first2=Yoshua |last3=Courville |first3=Aaron |publisher=MIT Press |year=2016 |isbn=978-0-262-03561-3}} In GD for MTL, the problem is that each task provides its own loss, and it is not clear how to combine all losses and create a single unified gradient, leading to several different aggregation strategies.{{Cite web |last1=Liu |first1=L. |last2=Li |first2=Y. |last3=Kuang |first3=Z. |last4=Xue |first4=J. |last5=Chen |first5=Y. |last6=Yang |first6=W. |last7=Liao |first7=Q. |last8=Zhang |first8=W. |date=2021-05-04 |title=Towards Impartial Multi-task Learning |url=https://iclr.cc/ |access-date=2022-11-20 |website=In: Proceedings of the International Conference on Learning Representations (ICLR 2021). ICLR: Virtual event. (2021)}}{{Cite journal |last1=Tianhe |first1=Yu |last2=Saurabh |first2=Kumar |last3=Abhishek |first3=Gupta |last4=Sergey |first4=Levine |last5=Karol |first5=Hausman |last6=Chelsea |first6=Finn |date=2020 |title=Gradient Surgery for Multi-Task Learning |url=https://proceedings.neurips.cc/paper/2020/hash/3fe78a8acf5fda99de95303940a2420c-Abstract.html |journal=Advances in Neural Information Processing Systems |language=en |volume=33|arxiv=2001.06782 }}{{Cite arXiv |last1=Liu |first1=Bo |last2=Liu |first2=Xingchao |last3=Jin |first3=Xiaojie |last4=Stone |first4=Peter |last5=Liu |first5=Qiang |date=2021-10-26 |title=Conflict-Averse Gradient Descent for Multi-task Learning |class=cs.LG |eprint=2110.14048}} This aggregation problem can be solved by defining a game matrix where the reward of each player is the agreement of its own gradient with the common gradient, and then setting the common gradient to be the Nash Cooperative bargainingAviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, Ethan Fetaya, (2022). [https://proceedings.mlr.press/v162/navon22a.html Multi-Task Learning as a Bargaining Game]. International conference on machine learning. of that system.

Algorithms for multi-task optimization span a wide array of real-world applications. Recent studies highlight the potential for speed-ups in the optimization of engineering design parameters by conducting related designs jointly in a multi-task manner. In machine learning, the transfer of optimized features across related data sets can enhance the efficiency of the training process as well as improve the generalization capability of learned models.Chandra, R., Gupta, A., Ong, Y. S., & Goh, C. K. (2016, October). [http://www.cil.ntu.edu.sg/mfo/downloads/cvmultask.pdf Evolutionary multi-task learning for modular training of feedforward neural networks]. In International Conference on Neural Information Processing (pp. 37-46). Springer, Cham.Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014). [http://papers.nips.cc/paper/5347-how-transferable-are-features-in-deep-n%E2%80%A6 How transferable are features in deep neural networks?] In Advances in neural information processing systems (pp. 3320-3328). In addition, the concept of multi-tasking has led to advances in automatic hyperparameter optimization of machine learning models and ensemble learning.{{cite book | doi=10.1109/CEC.2016.7748363 | chapter=Learning ensemble of decision trees through multifactorial genetic programming | title=2016 IEEE Congress on Evolutionary Computation (CEC) | year=2016 | last1=Wen | first1=Yu-Wei | last2=Ting | first2=Chuan-Kang | pages=5293–5300 | isbn=978-1-5090-0623-6 | s2cid=2617811 }}{{cite book | doi=10.1145/3205455.3205638 | chapter=Evolutionary feature subspaces generation for ensemble classification | title=Proceedings of the Genetic and Evolutionary Computation Conference | year=2018 | last1=Zhang | first1=Boyu | last2=Qin | first2=A. K. | last3=Sellis | first3=Timos | pages=577–584 | isbn=978-1-4503-5618-3 | s2cid=49564862 }}

Applications have also been reported in cloud computing,{{cite book | doi=10.1007/978-3-319-94472-2_10 | chapter=An Evolutionary Multitasking Algorithm for Cloud Computing Service Composition | title=Services – SERVICES 2018 | series=Lecture Notes in Computer Science | year=2018 | last1=Bao | first1=Liang | last2=Qi | first2=Yutao | last3=Shen | first3=Mengqing | last4=Bu | first4=Xiaoxuan | last5=Yu | first5=Jusheng | last6=Li | first6=Qian | last7=Chen | first7=Ping | volume=10975 | pages=130–144 | isbn=978-3-319-94471-5 }} with future developments geared towards cloud-based on-demand optimization services that can cater to multiple customers simultaneously.Tang, J., Chen, Y., Deng, Z., Xiang, Y., & Joy, C. P. (2018). [https://www.ijcai.org/proceedings/2018/0538.pdf A Group-based Approach to Improve Multifactorial Evolutionary Algorithm]. In IJCAI (pp. 3870-3876). Recent work has additionally shown applications in chemistry.{{citation |mode=cs1 |doi=10.26434/chemrxiv.13250216.v2 |title=Multi-task Bayesian Optimization of Chemical Reactions |work=chemRxiv |date=2021 |last1=Felton |first1=Kobi |last2=Wigh |first2=Daniel |last3=Lapkin |first3=Alexei}} In addition, some recent works have applied multi-task optimization algorithms in industrial manufacturing.{{Cite journal |last1=Jiang |first1=Yi |last2=Zhan |first2=Zhi-Hui |last3=Tan |first3=Kay Chen |last4=Zhang |first4=Jun |date=October 2023 |title=A Bi-Objective Knowledge Transfer Framework for Evolutionary Many-Task Optimization |journal=IEEE Transactions on Evolutionary Computation |volume=27 |issue=5 |pages=1514–1528 |doi=10.1109/TEVC.2022.3210783 |issn=1089-778X|doi-access=free }}{{Cite journal |last1=Jiang |first1=Yi |last2=Zhan |first2=Zhi-Hui |last3=Tan |first3=Kay Chen |last4=Kwong |first4=Sam |last5=Zhang |first5=Jun |date=2024 |title=Knowledge Structure Preserving-Based Evolutionary Many-Task Optimization |journal=IEEE Transactions on Evolutionary Computation |volume=29 |issue=2 |pages=287–301 |doi=10.1109/TEVC.2024.3355781 |issn=1089-778X|doi-access=free }}

The MTL problem can be cast within the context of RKHSvv (a complete inner product space of vector-valued functions equipped with a reproducing kernel). In particular, recent focus has been on cases where task structure can be identified via a separable kernel, described below. The presentation here derives from Ciliberto et al., 2015.

Suppose the training data set is $\backslash mathcal\{S\}\_t\; =\backslash \{(x\_i^t,y\_i^t)\backslash \}\_\{i=1\}^\{n\_t\}$, with $x\_i^t\backslash in\backslash mathcal\{X\}$, $y\_i^t\backslash in\backslash mathcal\{Y\}$, where {{mvar|t}} indexes task, and $t\; \backslash in\; 1,...,T$. Let $n=\backslash sum\_\{t=1\}^Tn\_t$. In this setting there is a consistent input and output space and the same loss function $\backslash mathcal\{L\}:\backslash mathbb\{R\}\backslash times\backslash mathbb\{R\}\backslash rightarrow\; \backslash mathbb\{R\}\_+$ for each task: . This results in the regularized machine learning problem:

{{NumBlk|:|$$\backslash min\_\{f\; \backslash in\; \backslash mathcal\{H\}\}\backslash sum\; \_\{t=1\}\; ^T\; \backslash frac\{1\}\{n\_t\}\; \backslash sum\; \_\{i=1\}\; ^\{n\_t\}\; \backslash mathcal\{L\}(y\_i^t,\; f\_t(x\_i^t))+\backslash lambda\; ||f||\_\backslash mathcal\{H\}\; ^2$$|{{EquationRef|1}}}}

where $\backslash mathcal\{H\}$ is a vector valued reproducing kernel Hilbert space with functions $f:\backslash mathcal\; X\; \backslash rightarrow\; \backslash mathcal\{Y\}^T$ having components $f\_t:\backslash mathcal\{X\}\backslash rightarrow\; \backslash mathcal\; \{Y\}$.

The reproducing kernel for the space $\backslash mathcal\{H\}$ of functions $f:\backslash mathcal\; X\; \backslash rightarrow\; \backslash mathbb\{R\}^T$ is a symmetric matrix-valued function $\backslash Gamma\; :\backslash mathcal\; X\backslash times\; \backslash mathcal\; X\; \backslash rightarrow\; \backslash mathbb\{R\}^\{T\; \backslash times\; T\}$ , such that $\backslash Gamma\; (\backslash cdot\; ,x)c\backslash in\; \backslash mathcal\{H\}$ and the following reproducing property holds:

{{NumBlk|:|$$\backslash langle\; f(x),c\; \backslash rangle\; \_\; \{\backslash mathbb\{R\}^T\}\; =\; \backslash langle\; f,\backslash Gamma\; (x,\backslash cdot\; )\; c\; \backslash rangle\; \_\; \{\backslash mathcal\; \{H\}\}$$|{{EquationRef|2}}}} The reproducing kernel gives rise to a representer theorem showing that any solution to equation {{EquationNote|1}} has the form:

{{NumBlk|:|$$f(x)=\backslash sum\; \_\{t=1\}^T\; \backslash sum\; \_\{i=1\}^\{n\_t\}\; \backslash Gamma(x,x\_i^t)c\_i^t$$|{{EquationRef|3}}}}

The form of the kernel {{math|Γ}} induces both the representation of the feature space and structures the output across tasks. A natural simplification is to choose a separable kernel, which factors into separate kernels on the input space {{mathcal|X}} and on the tasks $\backslash \{1,...,T\backslash \}$. In this case the kernel relating scalar components $f\_t$ and $f\_s$ is given by $\backslash gamma((x\_i,t),(x\_j,s\; ))\; =\; k(x\_i,x\_j)k\_T(s,t)=k(x\_i,x\_j)A\_\{s,t\}$. For vector valued functions $f\backslash in\; \backslash mathcal\; H$ we can write $\backslash Gamma(x\_i,x\_j)=k(x\_i,x\_j)A$, where {{mvar|k}} is a scalar reproducing kernel, and {{mvar|A}} is a symmetric positive semi-definite $T\backslash times\; T$ matrix. Henceforth denote $S\_+^T=\backslash \{\backslash text\{PSD\; matrices\}\; \backslash \}\; \backslash subset\; \backslash mathbb\; R^\{T\; \backslash times\; T\}$ .

This factorization property, separability, implies the input feature space representation does not vary by task. That is, there is no interaction between the input kernel and the task kernel. The structure on tasks is represented solely by {{mvar|A}}. Methods for non-separable kernels {{math|Γ}} is a current field of research.

For the separable case, the representation theorem is reduced to $f(x)=\backslash sum\; \_\{i=1\}\; ^N\; k(x,x\_i)Ac\_i$. The model output on the training data is then {{mvar|KCA}} , where {{mvar|K}} is the $n\; \backslash times\; n$ empirical kernel matrix with entries $K\_\{i,j\}=k(x\_i,x\_j)$, and {{mvar|C}} is the $n\; \backslash times\; T$ matrix of rows $c\_i$.

With the separable kernel, equation {{EquationNote|1}} can be rewritten as

{{NumBlk|:|$$\backslash min\; \_\{C\backslash in\; \backslash mathbb\{R\}^\{n\backslash times\; T\}\}\; V(Y,KCA)\; +\; \backslash lambda\; tr(KCAC^\{\backslash top\})$$|{{EquationRef|P}}}}

where {{mvar|V}} is a (weighted) average of {{mathcal|L}} applied entry-wise to {{mvar|Y}} and {{mvar|KCA}}. (The weight is zero if $Y\_i^t$ is a missing observation).

Note the second term in {{EquationNote|P}} can be derived as follows:

:$\backslash begin\{align\}$

\|f\|^2_\mathcal{H} &= \left\langle \sum _{i=1} ^n k(\cdot,x_i)Ac_i, \sum _{j=1} ^n k(\cdot ,x_j)Ac_j \right\rangle_{\mathcal H }

\\

&= \sum _{i,j=1} ^n \langle k(\cdot,x_i)A c_i, k(\cdot ,x_j)Ac_j\rangle_{\mathcal H } & \text{(bilinearity)}

\\

&= \sum _{i,j=1} ^n \langle k(x_i,x_j)A c_i, c_j\rangle_{\mathbb R^T } & \text{(reproducing property)}

\\

&= \sum _{i,j=1} ^n k(x_i,x_j) c_i^\top A c_j=tr(KCAC^\top )

\end{align}

There are three largely equivalent ways to represent task structure: through a regularizer; through an output metric, and through an output mapping.

{{math_theorem|name=Regularizer|1=With the separable kernel, it can be shown (below) that $||f||^2\_\backslash mathcal\{H\}\; =\; \backslash sum\_\{s,t=1\}^T\; A^\backslash dagger\; \_\{t,s\}\; \backslash langle\; f\_s,\; f\_t\; \backslash rangle\; \_\{\backslash mathcal\; H\_k\}$, where $A^\backslash dagger\; \_\{t,s\}$ is the $t,s$ element of the pseudoinverse of $A$, and $\backslash mathcal\; H\_k$ is the RKHS based on the scalar kernel $k$, and $f\_t(x)=\backslash sum\; \_\{i=1\}\; ^n\; k(x,x\_i)A\_t^\backslash top\; c\_i$. This formulation shows that $A^\backslash dagger\; \_\{t,s\}$ controls the weight of the penalty associated with $\backslash langle\; f\_s,\; f\_t\; \backslash rangle\; \_\{\backslash mathcal\; H\_k\}$. (Note that $\backslash langle\; f\_s,\; f\_t\; \backslash rangle\; \_\{\backslash mathcal\; H\_k\}$ arises from $||f\_t||\_\{\backslash mathcal\; H\_k\}\; =\; \backslash langle\; f\_t,\; f\_t\; \backslash rangle\; \_\{\backslash mathcal\; H\_k\}$.)

{{Proof|

$\backslash begin\{align\}$

\|f\|^2_\mathcal{H} &= \left\langle \sum _{i=1} ^n \gamma ((x_i,t_i),\cdot )c_i^{t_i}, \sum _{j=1} ^n \gamma ((x_j,t_j), \cdot )c_j^{t_j}\right\rangle_{\mathcal H } \\

&=\sum _{i,j=1} ^n c_i^{t_i} c_j^{t_j} \gamma ((x_i,t_i),(x_j,t_j)) \\

&=\sum _{i,j=1} ^n \sum _{s,t=1} ^T c_i^{t} c_j^{s} k(x_i,x_j)A_{s,t} \\

&=\sum _{i,j=1} ^n k(x_i,x_j) \langle c_i, A c_j\rangle_{\mathbb R^T} \\

&=\sum _{i,j=1} ^n k(x_i,x_j) \langle c_i, A A^\dagger A c_j\rangle_{\mathbb R^T} \\

&=\sum _{i,j=1} ^n k(x_i,x_j) \langle Ac_i, A^\dagger A c_j\rangle_{\mathbb R^T} \\

&=\sum _{i,j=1} ^n \sum _{s,t=1} ^T (Ac_i)^t (A c_j)^s k(x_i,x_j) A^\dagger_{s,t} \\

&= \sum _{s,t=1} ^T A^\dagger_{s,t} \langle \sum _{i=1} ^n k(x_i,\cdot )(Ac_i)^t, \sum _{j=1} ^n k(x_j,\cdot )(A c_j)^s \rangle _{\mathcal H_k} \\

&= \sum _{s,t=1} ^T A^\dagger_{s,t} \langle f_t, f_s \rangle _{\mathcal H_k}

\end{align}

}}}}

{{math_theorem|name=Output metric|an alternative output metric on $\backslash mathcal\; Y^T$ can be induced by the inner product $\backslash langle\; y\_1,y\_2\; \backslash rangle\; \_\backslash Theta=\backslash langle\; y\_1,\backslash Theta\; y\_2\; \backslash rangle\_\{\backslash mathbb\; R^T\}$. With the squared loss there is an equivalence between the separable kernels $k(\backslash cdot,\backslash cdot)I\_T$ under the alternative metric, and $k(\backslash cdot,\backslash cdot)\backslash Theta$, under the canonical metric.}}

{{math_theorem|name=Output mapping|Outputs can be mapped as $L:\backslash mathcal\; Y^T\; \backslash rightarrow\; \backslash mathcal\; \backslash tilde\; Y$ to a higher dimensional space to encode complex structures such as trees, graphs and strings. For linear maps {{mvar|L}}, with appropriate choice of separable kernel, it can be shown that $A=L^\backslash top\; L$.}}

Via the regularizer formulation, one can represent a variety of task structures easily.

• Letting $A^\backslash dagger\; =\; \backslash gamma\; I\_T\; +\; (\; \backslash gamma\; -\; \backslash lambda)\backslash frac\; \{1\}\; T\; \backslash mathbf\{1\}\backslash mathbf\{1\}^\backslash top$ (where $I\_T$ is the TxT identity matrix, and $\backslash mathbf\{1\}\backslash mathbf\{1\}^\backslash top$ is the TxT matrix of ones) is equivalent to letting {{math|Γ}} control the variance $\backslash sum\_t\; ||\; f\_t\; -\; \backslash bar\; f||\; \_\{\backslash mathcal\; H\_k\}$ of tasks from their mean $\backslash frac\; 1\; T\; \backslash sum\_t\; f\_t$. For example, blood levels of some biomarker may be taken on {{mvar|T}} patients at $n\_t$ time points during the course of a day and interest may lie in regularizing the variance of the predictions across patients.
• Letting $A^\backslash dagger\; =\; \backslash alpha\; I\_T\; +(\backslash alpha\; -\; \backslash lambda\; )M$ , where $M\_\{t,s\}\; =\; \backslash frac\; 1$

  G_r

  \mathbb I(t,s\in G_r) is equivalent to letting $\backslash alpha$ control the variance measured with respect to a group mean: $\backslash sum\; \_\{r\}\; \backslash sum\; \_\{t\; \backslash in\; G\_r\; \}\; ||f\_t\; -\; \backslash frac\; 1$

  G_r

  \sum _{s\in G_r)} f_s|| . (Here $|G\_r|$ the cardinality of group r, and $\backslash mathbb\; I$ is the indicator function). For example, people in different political parties (groups) might be regularized together with respect to predicting the favorability rating of a politician. Note that this penalty reduces to the first when all tasks are in the same group.
• Letting $A^\backslash dagger\; =\; \backslash delta\; I\_T\; +\; (\backslash delta\; -\backslash lambda)L$, where $L=D-M$ is the Laplacian for the graph with adjacency matrix M giving pairwise similarities of tasks. This is equivalent to giving a larger penalty to the distance separating tasks t and s when they are more similar (according to the weight $M\_\{t,s\}$,) i.e. $\backslash delta$ regularizes $\backslash sum\; \_\{t,s\}||f\_t\; -\; f\_s\; ||\_\{\backslash mathcal\; H\; \_k\; \}^2\; M\_\{t,s\}$.
• All of the above choices of A also induce the additional regularization term $\backslash lambda\; \backslash sum\_t\; ||f||\; \_\{\backslash mathcal\; H\_k\}\; ^2$ which penalizes complexity in f more broadly.

Learning problem {{EquationNote|P}} can be generalized to admit learning task matrix A as follows:

{{NumBlk|:|$$\backslash min\; \_\{C\; \backslash in\; \backslash mathbb\{R\}^\{n\backslash times\; T\},A\; \backslash in\; S\_+^T\}\; V(Y,KCA)\; +\; \backslash lambda\; tr(KCAC^\{\backslash top\})+F(A)$$|{{EquationRef|Q}}}}

Choice of $F:S\_+^T\backslash rightarrow\; \backslash mathbb\; R\_+$ must be designed to learn matrices A of a given type. See "Special cases" below.

Restricting to the case of convex losses and coercive penalties Ciliberto et al. have shown that although {{EquationNote|Q}} is not convex jointly in C and A, a related problem is jointly convex.

Specifically on the convex set $\backslash mathcal\; C=\backslash \{(C,A)\backslash in\; \backslash mathbb\; R^\{n\; \backslash times\; T\}\backslash times\; S\_+^T\; |\; Range(C^\backslash top\; KC)\backslash subseteq\; Range(A)\backslash \}$, the equivalent problem

{{NumBlk|:|$$\backslash min\; \_\{C\; ,A\; \backslash in\; \backslash mathcal\; C\; \}\; V(Y,KC)\; +\; \backslash lambda\; tr(A^\backslash dagger\; C^\{\backslash top\}KC)+F(A)$$|{{EquationRef|R}}}}

is convex with the same minimum value. And if $(C\_R,\; A\_R)$ is a minimizer for {{EquationNote|R}} then $(C\_R\; A^\backslash dagger\; \_R,\; A\_R)$ is a minimizer for {{EquationNote|Q}}.

{{EquationNote|R}} may be solved by a barrier method on a closed set by introducing the following perturbation:

{{NumBlk|:|$$\backslash min\; \_\{C\; \backslash in\; \backslash mathbb\{R\}^\{n\backslash times\; T\},A\; \backslash in\; S\_+^T\}\; V(Y,KC)\; +\; \backslash lambda\; tr(A^\backslash dagger\; (C^\{\backslash top\}KC+\backslash delta^2I\_T))+F(A)$$|{{EquationRef|S}}}}

The perturbation via the barrier $\backslash delta\; ^2\; tr(A^\backslash dagger)$ forces the objective functions to be equal to $+\backslash infty$ on the boundary of $R^\{n\; \backslash times\; T\}\backslash times\; S\_+^T$ .

{{EquationNote|S}} can be solved with a block coordinate descent method, alternating in C and A. This results in a sequence of minimizers $(C\_m,A\_m)$ in {{EquationNote|S}} that converges to the solution in {{EquationNote|R}} as $\backslash delta\_m\; \backslash rightarrow\; 0$, and hence gives the solution to {{EquationNote|Q}}.

Spectral penalties - Dinnuzo et al{{Cite journal|url = http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Dinuzzo_54.pdf|title = Learning output kernels with block coordinate descent.|last = Dinuzzo|first = Francesco|date = 2011|journal = Proceedings of the 28th International Conference on Machine Learning (ICML-11)|archive-url = https://web.archive.org/web/20170808223410/http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Dinuzzo_54.pdf|archive-date = 2017-08-08|url-status = dead}} suggested setting F as the Frobenius norm $\backslash sqrt\{tr(A^\backslash top\; A)\}$. They optimized {{EquationNote|Q}} directly using block coordinate descent, not accounting for difficulties at the boundary of $\backslash mathbb\; R^\{n\backslash times\; T\}\; \backslash times\; S\_+^T$.

Clustered tasks learning - Jacob et al{{Cite journal|title = Clustered multi-task learning: A convex formulation|last = Jacob|first = Laurent|date = 2009|journal = Advances in Neural Information Processing Systems|bibcode = 2008arXiv0809.2085J|arxiv = 0809.2085}} suggested to learn A in the setting where T tasks are organized in R disjoint clusters. In this case let $E\backslash in\; \backslash \{0,1\backslash \}^\{T\backslash times\; R\}$ be the matrix with $E\_\{t,r\}=\backslash mathbb\; I\; (\backslash text\{task\; \}t\backslash in\; \backslash text\{group\; \}r)$. Setting $M\; =\; I\; -\; E^\backslash dagger\; E^T$, and $U\; =\; \backslash frac\; 1\; T\; \backslash mathbf\{11\}^\backslash top$, the task matrix $A^\backslash dagger$ can be parameterized as a function of $M$: $A^\backslash dagger(M)\; =\; \backslash epsilon\; \_M\; U+\backslash epsilon\_B\; (M-U)+\backslash epsilon\; (I-M)$ , with terms that penalize the average, between clusters variance and within clusters variance respectively of the task predictions. M is not convex, but there is a convex relaxation $\backslash mathcal\; S\_c\; =\; \backslash \{M\backslash in\; S\_+^T:I-M\backslash in\; S\_+^T\; \backslash land\; tr(M)\; =\; r\; \backslash \}$. In this formulation, $F(A)=\backslash mathbb\; I(A(M)\backslash in\; \backslash \{A:M\backslash in\; \backslash mathcal\; S\_C\backslash \})$.

Non-convex penalties - Penalties can be constructed such that A is constrained to be a graph Laplacian, or that A has low rank factorization. However these penalties are not convex, and the analysis of the barrier method proposed by Ciliberto et al. does not go through in these cases.

Non-separable kernels - Separable kernels are limited, in particular they do not account for structures in the interaction space between the input and output domains jointly. Future work is needed to develop models for these kernels.

A Matlab package called Multi-Task Learning via StructurAl Regularization (MALSAR) Zhou, J., Chen, J. and Ye, J. MALSAR: Multi-tAsk Learning via StructurAl Regularization. Arizona State University, 2012. http://www.public.asu.edu/~jye02/Software/MALSAR. [http://www.public.asu.edu/~jye02/Software/MALSAR/Manual.pdf On-line manual] implements the following multi-task learning algorithms: Mean-Regularized Multi-Task Learning,Evgeniou, T., & Pontil, M. (2004). [https://web.archive.org/web/20171212193041/https://pdfs.semanticscholar.org/1ea1/91c70559d21be93a4d128f95943e80e1b4ff.pdf Regularized multi–task learning]. Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 109–117).{{cite journal | last1 = Evgeniou | first1 = T. | last2 = Micchelli | first2 = C. | last3 = Pontil | first3 = M. | year = 2005 | title = Learning multiple tasks with kernel methods | url = http://jmlr.org/papers/volume6/evgeniou05a/evgeniou05a.pdf | journal = Journal of Machine Learning Research | volume = 6 | page = 615 }} Multi-Task Learning with Joint Feature Selection,{{cite journal | last1 = Argyriou | first1 = A. | last2 = Evgeniou | first2 = T. | last3 = Pontil | first3 = M. | year = 2008a | title = Convex multi-task feature learning | journal = Machine Learning | volume = 73 | issue = 3| pages = 243–272 | doi=10.1007/s10994-007-5040-8| doi-access = free }} Robust Multi-Task Feature Learning,Chen, J., Zhou, J., & Ye, J. (2011). [https://www.academia.edu/download/44101186/Integrating_low-rank_and_group-sparse_st20160325-15067-1mftmbg.pdf Integrating low-rank and group-sparse structures for robust multi-task learning]{{dead link|date=July 2022|bot=medic}}{{cbignore|bot=medic}}. Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. Trace-Norm Regularized Multi-Task Learning,Ji, S., & Ye, J. (2009). [http://www.machinelearning.org/archive/icml2009/papers/151.pdf An accelerated gradient method for trace norm minimization]. Proceedings of the 26th Annual International Conference on Machine Learning (pp. 457–464). Alternating Structural Optimization,{{cite journal | last1 = Ando | first1 = R. | last2 = Zhang | first2 = T. | year = 2005 | title = A framework for learning predictive structures from multiple tasks and unlabeled data | url = http://www.jmlr.org/papers/volume6/ando05a/ando05a.pdf | journal = The Journal of Machine Learning Research | volume = 6 | pages = 1817–1853 }}Chen, J., Tang, L., Liu, J., & Ye, J. (2009). [http://leitang.net/papers/ICML09_CASO.pdf A convex formulation for learning shared structures from multiple tasks]. Proceedings of the 26th Annual International Conference on Machine Learning (pp. 137–144). Incoherent Low-Rank and Sparse Learning,Chen, J., Liu, J., & Ye, J. (2010). [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3783291/ Learning incoherent sparse and low-rank patterns from multiple tasks]. Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 1179–1188). Robust Low-Rank Multi-Task Learning, Clustered Multi-Task Learning,Jacob, L., Bach, F., & Vert, J. (2008). [https://hal-ensmp.archives-ouvertes.fr/docs/00/32/05/73/PDF/cmultitask.pdf Clustered multi-task learning: A convex formulation]. Advances in Neural Information Processing Systems， 2008Zhou, J., Chen, J., & Ye, J. (2011). [http://papers.nips.cc/paper/4292-clustered-multi-task-learning-via-alternating-structure-optimization.pdf Clustered multi-task learning via alternating structure optimization]. Advances in Neural Information Processing Systems. Multi-Task Learning with Graph Structures.

• Multi-Target Prediction: A Unifying View on Problems and Methods Willem Waegeman, Krzysztof Dembczynski, Eyke Huellermeier https://arxiv.org/abs/1809.02352v1

{{div col}}

• Artificial intelligence
• Artificial neural network
• Automated machine learning (AutoML)
• Evolutionary computation
• Foundation model
• General game playing
• Human-based genetic algorithm
• Kernel methods for vector output
• Multiple-criteria decision analysis
• Multi-objective optimization
• Multicriteria classification
• Robot learning
• Transfer learning
• James–Stein estimator

{{div col end}}

{{Reflist}}

• [https://web.archive.org/web/20041118134329/http://big.cs.uiuc.edu/webpage/cumulativeLearning/cumulativeLearning.html The Biosignals Intelligence Group at UIUC]
• [http://www.cse.wustl.edu/~kilian/research/multitasklearning/multitasklearning.html Washington University in St. Louis Department of Computer Science]

• [http://www.public.asu.edu/~jye02/Software/MALSAR/index.html The Multi-Task Learning via Structural Regularization Package]
• [https://web.archive.org/web/20131224113826/http://klcl.pku.edu.cn/member/sunxu/code.htm Online Multi-Task Learning Toolkit (OMT)] A general-purpose online multi-task learning toolkit based on conditional random field models and stochastic gradient descent training (C#, .NET)

{{Optimization algorithms}}

Category:Machine learning