Copulas in signal processing

{{About|the use of mathematical copulas in signal processing|other uses|Copula (disambiguation)}}

A copula is a mathematical function that provides a relationship between marginal distributions of random variables and their joint distributions. Copulas are important because it represents a dependence structure without using marginal distributions. Copulas have been widely used in the field of finance, but their use in signal processing is relatively new. Copulas have been employed in the field of wireless communication for classifying radar signals, change detection in remote sensing applications, and EEG signal processing in medicine. In this article, a short introduction to copulas is presented, followed by a mathematical derivation to obtain copula density functions, and then a section with a list of copula density functions with applications in signal processing.

Introduction

Using Sklar's theorem, a copula can be described as a cumulative distribution function (CDF) on a unit-space with uniform marginal distributions on the interval (0, 1). The CDF of a random variable X is the probability that X will take a value less than or equal to x when evaluated at x itself. A copula can represent a dependence structure without using marginal distributions. Therefore, it is simple to transform the uniformly distributed variables of copula (u, v, and so on) into the marginal variables (x, y, and so on) by the inverse marginal cumulative distribution function.{{Cite journal|last=Bellmann|first=K.|date=1978|title=BATSCHELET, E.: Introduction to Mathematics for Life Scientists. 2nd Ed. Springer-Verlag, Berlin-Heidelberg-New York 1975. 643 S., 227 Abb., DM 38,-.|url=http://dx.doi.org/10.1002/bimj.4710200510|journal=Biometrical Journal|volume=20|issue=5|pages=531|doi=10.1002/bimj.4710200510|issn=0323-3847|url-access=subscription}} Using the chain rule, copula distribution function can be partially differentiated with respect to the uniformly distributed variables of copula, and it is possible to express the multivariate probability density function (PDF) as a product of a multivariate copula density function and marginal PDF''s.{{Cite book|last1=Cherubini|first1=Umberto|last2=Luciano|first2=Elisa|last3=Vecchiato|first3=Walter|date=2004|title=Copula Methods in Finance|url=http://dx.doi.org/10.1002/9781118673331|doi=10.1002/9781118673331|isbn=9781118673331}} The mathematics for converting a copula distribution function into a copula density function is shown for a bivariate case, and a family of copulas used in signal processing are listed in a TABLE 1.

Mathematical derivation

For any two random variables X and Y, the continuous joint probability distribution function can be written as

: $F_{XY}(x,y) = \Pr \begin{Bmatrix} X \leq{x},Y\leq{y} \end{Bmatrix},$

where $F_X(x) = \Pr \begin{Bmatrix} X \leq{x} \end{Bmatrix}$ and

$F_Y(y) = \Pr \begin{Bmatrix} Y \leq{y} \end{Bmatrix}$ are the marginal cumulative distribution functions of the random variables X and Y, respectively.

then the copula distribution function $C(u, v)$ can be defined using Sklar's theorem{{Cite book|last1=Appell|first1=Paul|url=http://dx.doi.org/10.5962/bhl.title.18731|title=Théorie des fonctions algébriques et de leurs intégrales étude des fonctions analytiques sur une surface de Riemann / par Paul Appell, Édouard Goursat.|last2=Goursat|first2=Edouard|date=1895|publisher=Gauthier-Villars|location=Paris|doi=10.5962/bhl.title.18731}}{{Cite journal|last1=Durante|first1=Fabrizio|last2=Fernández-Sánchez|first2=Juan|last3=Sempi|first3=Carlo|date=2013|title=A topological proof of Sklar's theorem|journal=Applied Mathematics Letters|volume=26|issue=9|pages=945–948|doi=10.1016/j.aml.2013.04.005|issn=0893-9659|doi-access=free}} as:

$F_{XY}(x,y) = C( F_X (x) , F_Y (y) ) \triangleq C( u, v )$

where $u = F_X(x)$ and $v = F_Y(y)$ are marginal distribution functions, $F_{XY}(x,y)$ joint and $u, v \in (0,1)$ .

We start by using the relationship between joint probability density function (PDF) and joint cumulative distribution function (CDF) and its partial derivatives.

: $\begin{alignat}{6}$

f_{XY}(x,y) = {} & {\partial^2 F_{XY}(x,y) \over\partial x\,\partial y } \\

\vdots \\

f_{XY}(x,y) = {} & {\partial^2 C(F_X(x),F_Y(y)) \over\partial x\,\partial y} \\

\vdots \\

f_{XY}(x,y) = {} & {\partial^2 C(u,v) \over\partial u\,\partial v} \cdot {\partial F_X(x) \over\partial x} \cdot {\partial F_Y(y) \over\partial y} \\

\vdots \\

f_{XY}(x,y) = {} & c(u,v) f_X(x) f_Y(y) \\

\vdots \\

\frac{f_{XY}(x,y)}{f_X(x) f_Y(y) } = {} & c(u,v)

\end{alignat}

:(Equation 1)

where $c(u,v)$ is the copula density function, $f_X(x)$ and $f_Y(y)$ are the marginal probability density functions of X and Y, respectively. It is important understand that there are four elements in the equation 1, and if three of the four are know, the fourth element can me calculated. For example, equation 1 may be used

when joint probability density function between two random variables is known, the copula density function is known, and one of the two marginal functions are known, then, the other marginal function can be calculated, or
when the two marginal functions and the copula density function are known, then the joint probability density function between the two random variables can be calculated, or
when the two marginal functions and the joint probability density function between the two random variables are known, then the copula density function can be calculated.

Summary table

The use of copula in signal processing is fairly new compared to finance. Here, a family of new bivariate copula density functions are listed with importance in the area of signal processing. Here, $u=F_X(x)$

and $v=F_Y(y)$

are marginal distributions functions and $f_X(x)$

and $f_Y(y)$

are marginal density functions

class="wikitable"

!scope="col" style="width: 510px;" | Coupla density: c(u, v)

!Use

Gaussian

| $\begin{align}$

= {} & \frac{1}{\sqrt{1-\rho^2}} \exp\left (-\frac{(a^2+b^2)\rho^2-2 ab\rho}{ 2(1-\rho^2) } \right ) \\

& \text{where } \rho\in (-1,1)\\

& \text{where } a=\sqrt{2} \operatorname{erf}^{-1}({2u-1}) \\

& \text{where } b =\sqrt{2}\operatorname{erf}^{-1}({2v-1}) \\

& \text{where } \operatorname{erf}(z) = \frac{2}{\sqrt{\pi}} \int\limits_0^z \exp (-t^2) \, dt

\end{align}

|supervised classification of synthetic aperture radar (SAR) images,{{Cite journal|last1=Storvik|first1=B.|last2=Storvik|first2=G.|last3=Fjortoft|first3=R.|date=2009|title=On the Combination of Multisensor Data Using Meta-Gaussian Distributions|url=http://dx.doi.org/10.1109/tgrs.2009.2012699|journal=IEEE Transactions on Geoscience and Remote Sensing|volume=47|issue=7|pages=2372–2379|doi=10.1109/tgrs.2009.2012699|bibcode=2009ITGRS..47.2372S|s2cid=371395|issn=0196-2892|url-access=subscription}}

validating biometric authentication,{{Cite journal|last=Dass|first=S.C.|author2=Yongfang Zhu|last3=Jain|first3=A.K.|date=2006|title=Validating a Biometric Authentication System: Sample Size Requirements|url=http://dx.doi.org/10.1109/tpami.2006.255|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|volume=28|issue=12|pages=1902–1319|doi=10.1109/tpami.2006.255|pmid=17108366|s2cid=1272268|issn=0162-8828|url-access=subscription}} modeling stochastic dependence in large-scale integration of wind power,{{Cite journal|last1=Papaefthymiou|first1=G.|last2=Kurowicka|first2=D.|date=2009|title=Using Copulas for Modeling Stochastic Dependence in Power System Uncertainty Analysis|url=http://dx.doi.org/10.1109/tpwrs.2008.2004728|journal=IEEE Transactions on Power Systems|volume=24|issue=1|pages=40–49|doi=10.1109/tpwrs.2008.2004728|bibcode=2009ITPSy..24...40P|issn=0885-8950}} unsupervised classification of radar signals{{Cite journal|last1=Brunel|first1=N.J.-B.|last2=Lapuyade-Lahorgue|first2=J.|last3=Pieczynski|first3=W.|date=2010|title=Modeling and Unsupervised Classification of Multivariate Hidden Markov Chains With Copulas|url=http://dx.doi.org/10.1109/tac.2009.2034929|journal=IEEE Transactions on Automatic Control|volume=55|issue=2|pages=338–349|doi=10.1109/tac.2009.2034929|s2cid=941655|issn=0018-9286}}

Exponential

| $\begin{align}$

= {} & \frac{1}{1-\rho} \exp\left ( \frac{\rho(\ln(1-u)+\ln(1-v))}{1-\rho} \right )

\cdot I_0\left ( \frac{2\sqrt{\rho \ln(1-u)\ln(1-v)}}{1-\rho} \right )\\

& \text{where } x=F_X^{-1}(u)=-\ln(1-u)/\lambda \\

& \text{where } y=F_Y^{-1}(v)=-\ln(1-v)/\mu

\end{align}

|queuing system with infinitely servers{{Cite book|last1=Lai|first1=Chin Diew|last2=Balakrishnan|first2=N.|date=2009|title=Continuous Bivariate Distributions|url=http://dx.doi.org/10.1007/b101765|doi=10.1007/b101765|isbn=978-0-387-09613-1}}

Rayleigh

|bivariate exponential, Rayleigh, and Weibull copulas have been proved to be equivalent{{Cite journal|last1=Durrani|first1=T.S.|last2=Zeng|first2=X.|date=2007|title=Copulas for bivariate probability distributions|url=http://dx.doi.org/10.1049/el:20073737|journal=Electronics Letters|volume=43|issue=4|pages=248|doi=10.1049/el:20073737|bibcode=2007ElL....43..248D|issn=0013-5194|url-access=subscription}}{{Cite journal|last=Liu|first=X.|date=2010|title=Copulas of bivariate Rayleigh and log-normal distributions|url=http://dx.doi.org/10.1049/el.2010.2777|journal=Electronics Letters|volume=46|issue=25|pages=1669–1671|doi=10.1049/el.2010.2777|bibcode=2010ElL....46.1669L|issn=0013-5194|url-access=subscription}}{{Cite journal|last1=Zeng|first1=Xuexing|last2=Ren|first2=Jinchang|last3=Wang|first3=Zheng|last4=Marshall|first4=Stephen|last5=Durrani|first5=Tariq|date=2014|title=Copulas for statistical signal processing (Part I): Extensions and generalization|url=http://dx.doi.org/10.1016/j.sigpro.2013.07.009|journal=Signal Processing|volume=94|pages=691–702|doi=10.1016/j.sigpro.2013.07.009|bibcode=2014SigPr..94..691Z |issn=0165-1684}}

|change detection from SAR images{{Cite journal|last1=Hachicha|first1=S.|last2=Chaabene|first2=F.|editor1-first=Robert J|editor1-last=Frouin|editor2-first=Hong Rhyong|editor2-last=Yoo|editor3-first=Joong-Sun|editor3-last=Won|editor4-first=Aiping|editor4-last=Feng|date=2010|title=SAR change detection using Rayleigh copula|url=http://dx.doi.org/10.1117/12.870023|journal=Remote Sensing of the Coastal Ocean, Land, and Atmosphere Environment|volume=7858|pages=78581F|publisher=SPIE|doi=10.1117/12.870023|bibcode=2010SPIE.7858E..1FH|s2cid=129437866|url-access=subscription}}

Weibull

|bivariate exponential, Rayleigh, and Weibull copulas have been proved to be equivalent

|digital communication over fading channels{{Citation|title=Coded Communication over Fading Channels|date=2005|url=http://dx.doi.org/10.1002/0471715220.ch13|work=Digital Communication over Fading Channels|pages=758–795|publisher=John Wiley & Sons, Inc.|doi=10.1002/0471715220.ch13|isbn=978-0-471-71522-1|access-date=2020-04-06|url-access=subscription}}

Log-normal

|bivariate log-normal copula and Gaussian copula are equivalent

|shadow fading along with multipath effect in wireless channel{{Cite journal|last1=Das|first1=Saikat|last2=Bhattacharya|first2=Amitabha|date=2020|title=Application of the Mixture of Lognormal Distribution to Represent the First-Order Statistics of Wireless Channels|url=https://ieeexplore.ieee.org/document/8993782|journal=IEEE Systems Journal|volume=14|issue=3|pages=4394–4401|doi=10.1109/JSYST.2020.2968409|bibcode=2020ISysJ..14.4394D|s2cid=213729677|issn=1932-8184|url-access=subscription}}{{Cite journal|last1=Alouini|first1=M.-S.|last2=Simon|first2=M.K.|date=2002|title=Dual diversity over correlated log-normal fading channels|url=https://ieeexplore.ieee.org/document/1175472|journal=IEEE Transactions on Communications|language=en|volume=50|issue=12|pages=1946–1959|doi=10.1109/TCOMM.2002.806552|issn=0090-6778|url-access=subscription}}

Farlie–Gumbel–Morgenstern (FGM)

| $\begin{align}$

= {} & 1+\theta(1-2u)(1-2v) \\

& \text{where } \theta \in[-1,1]

\end{align}

|information processing of uncertainty in knowledge-based systems{{Citation|last1=Kolesárová|first1=Anna|title=Generalized Farlie-Gumbel-Morgenstern Copulas|date=2018|url=http://link.springer.com/10.1007/978-3-319-91473-2_21|work=Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations|volume=853|pages=244–252|editor-last=Medina|editor-first=Jesús|publisher=Springer International Publishing|language=en|doi=10.1007/978-3-319-91473-2_21|isbn=978-3-319-91472-5|access-date=2020-04-06|last2=Mesiar|first2=Radko|last3=Saminger-Platz|first3=Susanne|series=Communications in Computer and Information Science |editor2-last=Ojeda-Aciego|editor2-first=Manuel|editor3-last=Verdegay|editor3-first=José Luis|editor4-last=Pelta|editor4-first=David A.|url-access=subscription}}

Clayton

| $\begin{align}$

= {} & (1+\theta)u^{(-1-\theta)}(-1 +u^{-\theta} + v^{-\theta})^{(-2-1/\theta)} \\

& \text{where } \theta \in(-1,\infty), \theta\neq0

\end{align}

|location estimation of random signal source and hypothesis testing using heterogeneous data{{Cite journal|last1=Sundaresan|first1=Ashok|last2=Varshney|first2=Pramod K. |author-link2= Pramod Varshney|date=2011|title=Location Estimation of a Random Signal Source Based on Correlated Sensor Observations|url=http://dx.doi.org/10.1109/tsp.2010.2084084|journal=IEEE Transactions on Signal Processing|volume=59|issue=2|pages=787–799|doi=10.1109/tsp.2010.2084084|bibcode=2011ITSP...59..787S|s2cid=5725233|issn=1053-587X|url-access=subscription}}{{Cite journal|last1=Iyengar|first1=Satish G.|last2=Varshney|first2=Pramod K.|author-link2= Pramod Varshney|last3=Damarla|first3=Thyagaraju|date=2011|title=A Parametric Copula-Based Framework for Hypothesis Testing Using Heterogeneous Data|url=http://dx.doi.org/10.1109/tsp.2011.2105483|journal=IEEE Transactions on Signal Processing|volume=59|issue=5|pages=2308–2319|doi=10.1109/tsp.2011.2105483|bibcode=2011ITSP...59.2308I|s2cid=5549193|issn=1053-587X|url-access=subscription}}

Frank

| $\begin{align}$

= {} & \frac

{\theta e^{\theta(1+u+v)(e^{\theta}-1)}}

{(e^\theta-e^{\theta(1+u)}-e^{\theta(1+v)}+e^{\theta(u+v)})^2}\\

& \text{where } \theta \in(-\infty,+\infty), \theta\neq0

\end{align}

|change detection in remote sensing applications{{Cite journal|last1=Mercier|first1=G.|last2=Moser|first2=G.|last3=Serpico|first3=S.B.|date=2008|title=Conditional Copulas for Change Detection in Heterogeneous Remote Sensing Images|url=http://dx.doi.org/10.1109/tgrs.2008.916476|journal=IEEE Transactions on Geoscience and Remote Sensing|volume=46|issue=5|pages=1428–1441|doi=10.1109/tgrs.2008.916476|bibcode=2008ITGRS..46.1428M|s2cid=12208493|issn=0196-2892|url-access=subscription}}

Student's t

| $\begin{align}$

= {} & \frac{\Gamma(0.5v)\Gamma(0.5v+1)( 1+(t_v^{-2}(u)+t_v^{-2}(v) -2 \rho t_v^{-1}(u) t_v^{-1}(v))/(v(1-\rho^2)))^{-0.5(v+2)} )}

{\sqrt{1-\rho^2} \cdot \Gamma(0.5(v+1))^2 (1+ t_v^{-2}(u)/v)^{-0.5(v+1)} (1+ t_v^{-2}(v)/v)^{-0.5(v+1)} } \\

& \text{where } \rho\in (-1,1)\\

& \text{where } \phi(z)= \frac{1}{\sqrt{2\pi}} \int\limits_{-\infty}^z \exp \left(\frac{-t^2}{2}\right) \, dt \\

& \text{where } t_v(x\mid v)= \int\limits_{-\infty}^x \frac{\Gamma{(0.5(v+1))}}{\sqrt{v\pi}(\Gamma{0.5v})(1+v^{-1}t^2)^{0.5(v+1)}} dt\\

& \text{where } v=\text{degrees of freedom} \\

& \text{where } \Gamma \text{ is the Gamma function}

\end{align}

|supervised SAR image classification,

fusion of correlated sensor decisions{{Cite journal|last1=Sundaresan|first1=Ashok|last2=Varshney|first2=Pramod K.|author-link2= Pramod Varshney|last3=Rao|first3=Nageswara S. V.|date=2011|title=Copula-Based Fusion of Correlated Decisions|url=http://dx.doi.org/10.1109/taes.2011.5705686|journal=IEEE Transactions on Aerospace and Electronic Systems|volume=47|issue=1|pages=454–471|doi=10.1109/taes.2011.5705686|bibcode=2011ITAES..47..454S|s2cid=22562771|issn=0018-9251|url-access=subscription}}

Nakagami-m

Rician

TABLE 1: Copula density function of a family of copulas used in signal processing.

References

Category:Signal processing