Tsallis entropy

{{Short description|Generalization of the standard Boltzmann–Gibbs entropy}}

In physics, the Tsallis entropy is a generalization of the standard Boltzmann–Gibbs entropy.

It is proportional to the expectation of the q-logarithm of a distribution.

History

The concept was introduced in 1988 by Constantino Tsallis{{Cite journal | last1 = Tsallis | first1 = C. | doi = 10.1007/BF01016429 | title = Possible generalization of Boltzmann-Gibbs statistics | journal = Journal of Statistical Physics | volume = 52 | issue = 1–2 | pages = 479–487 | year = 1988 |bibcode = 1988JSP....52..479T | hdl = 10338.dmlcz/142811 | s2cid = 16385640 | hdl-access = free }} as a basis for generalizing the standard statistical mechanics and is identical in form to Havrda–Charvát structural α-entropy,{{Cite journal | last1 = Havrda | first1 = J. | last2 = Charvát | first2 = F. | title = Quantification method of classification processes. Concept of structural α-entropy | journal = Kybernetika | volume = 3 | issue = 1 | pages = 30–35 | year = 1967 | url = http://dml.cz/bitstream/handle/10338.dmlcz/125526/Kybernetika_03-1967-1_3.pdf}} introduced in 1967 within information theory.

Definition

Given a discrete set of probabilities $\{p_i\}$ with the condition $\sum_i p_i=1$ , and $q$ any real number, the Tsallis entropy is defined as

: $S_q({p_i}) = k \cdot \frac{1}{q-1} \left( 1 - \sum_i p_i^q \right),$

where $q$ is a real parameter sometimes called entropic-index and $k$ a positive constant.

In the limit as $q \to 1$ , the usual Boltzmann–Gibbs entropy is recovered, namely

: $S_\text{BG} = S_1(p) = -k \sum_i p_i \ln p_i ,$

where one identifies $k$ with the Boltzmann constant $k_B$ .

For continuous probability distributions, we define the entropy as

: $S_q[p] = {1 \over q - 1} \left( 1 - \int (p(x))^q\, dx \right),$

where $p(x)$ is a probability density function.

= Cross-entropy =

The cross-entropy pendant is the expectation of the negative q-logarithm with respect to a second distribution, $r$ . So $\tfrac{1}{q-1}(1 - {\textstyle \sum_i} p_i^q\cdot \tfrac{r_i}{p_i})$ .

Using $t = q - 1$ , this may be written $(1 - E_r[p^t])/t$ . For smaller $t$ , values $p_i^t$ all tend towards $1$ .

The limit $q\to 1$ computes the negative of the slope of $E_r[p^t]$ at $t=0$ and one recovers $-{\textstyle \sum_i} r_i \ln p_i$ . So for fixed small $t$ , raising this expectation relates to log-likelihood maximalization.

Properties

= Identities =

A logarithm can be expressed in terms of a slope through $\tfrac{d}{dx} p^x = p^{x} \ln p$ resulting in the following formula for the standard entropy:

: $S = -\lim_{x\rightarrow 1}\tfrac{d}{dx} \sum_i p_i^x = -{\textstyle \sum_i} p_i \ln p_i$

Likewise, the discrete Tsallis entropy satisfies

: $S_q = -\lim_{x\rightarrow 1}D_q \sum_i p_i^x$

where D_q is the q-derivative with respect to x.

= Non-additivity =

Given two independent systems A and B, for which the joint probability density satisfies

: $p(A, B) = p(A) p(B),\,$

the Tsallis entropy of this system satisfies

: $S_q(A,B) = S_q(A) + S_q(B) + (1-q)S_q(A) S_q(B).\,$

From this result, it is evident that the parameter $|1-q|$ is a measure of the departure from additivity. In the limit when q = 1,

: $S(A,B) = S(A) + S(B),\,$

which is what is expected for an additive system. This property is sometimes referred to as "pseudo-additivity".

= Exponential families =

Many common distributions like the normal distribution belongs to the statistical exponential families.

Tsallis entropy for an exponential family can be written {{Cite journal | doi = 10.1088/1751-8113/45/3/032003| title = A closed-form expression for the Sharma–Mittal entropy of exponential families| journal = Journal of Physics A: Mathematical and Theoretical| volume = 45| issue = 3| pages = 032003| year = 2012| last1 = Nielsen | first1 = F. | last2 = Nock | first2 = R. |arxiv = 1112.4221 |bibcode = 2012JPhA...45c2003N | s2cid = 8653096}} as

: $H^T_q(p_F(x;\theta)) = \frac{1}{1-q} \left((e^{F(q\theta)-q F(\theta)}) E_p[e^{(q-1)k(x)}]-1 \right)$

where F is log-normalizer and k the term indicating the carrier measure.

For multivariate normal, term k is zero, and therefore the Tsallis entropy is in closed-form.

Applications

The Tsallis Entropy has been used along with the Principle of maximum entropy to derive the Tsallis distribution.

In scientific literature, the physical relevance of the Tsallis entropy has been debated.{{Cite journal | last1 = Cho | first1 = A. | title = A Fresh Take on Disorder, Or Disorderly Science? | journal = Science | volume = 297 | issue = 5585 | pages = 1268–1269 | year = 2002 | doi = 10.1126/science.297.5585.1268 | pmid = 12193769 | s2cid = 5441957 }}{{Cite journal | last1 = Abe | first1 = S. | last2 = Rajagopal | first2 = A.K. | title = Revisiting Disorder and Tsallis Statistics | journal = Science | volume = 300 | issue = 5617 | pages = 249–251 | year = 2003 | doi = 10.1126/science.300.5617.249d | pmid = 12690173 | s2cid = 39719500 }}{{Cite journal | last1 = Pressé | first1 = S. | last2 = Ghosh | first2 = K. | last3=Lee | first3 = J. | last4 = Dill | first4 = K. | title = Nonadditive Entropies Yield Probability Distributions with Biases not Warranted by the Data | journal = Phys. Rev. Lett. | volume = 111 | issue = 18 | pages = 180604 | year = 2013 | doi = 10.1103/PhysRevLett.111.180604 | bibcode=2013PhRvL.111r0604P | pmid=24237501| arxiv = 1312.1186 | s2cid = 2577710 }} However, from the years 2000 on, an increasingly wide spectrum of natural, artificial and social complex systems have been identified which confirm the predictions and consequences that are derived from this nonadditive entropy, such as nonextensive statistical mechanics,{{cite book|last=Tsallis|first=Constantino|title=Introduction to nonextensive statistical mechanics : approaching a complex world|year=2009|publisher=Springer|location=New York|isbn=978-0-387-85358-1|edition=Online-Ausg.}} which generalizes the Boltzmann–Gibbs theory.

Among the various experimental verifications and applications presently available in the literature, the following ones deserve a special mention:

The distribution characterizing the motion of cold atoms in dissipative optical lattices predicted in 2003{{Cite journal | last1 = Lutz | first1 = E. | doi = 10.1103/PhysRevA.67.051402 | title = Anomalous diffusion and Tsallis statistics in an optical lattice | journal = Physical Review A | volume = 67 | issue = 5 | pages = 051402 | year = 2003 |arxiv = cond-mat/0210022 |bibcode = 2003PhRvA..67e1402L | s2cid = 119403353 }} and observed in 2006.{{Cite journal | last1 = Douglas | first1 = P. | last2 = Bergamini | first2 = S. | last3 = Renzoni | first3 = F. | title = Tunable Tsallis Distributions in Dissipative Optical Lattices | doi = 10.1103/PhysRevLett.96.110601 | journal = Physical Review Letters | volume = 96 | issue = 11 | year = 2006 | pmid = 16605807|bibcode = 2006PhRvL..96k0601D | page=110601| url = http://discovery.ucl.ac.uk/142750/1/142750.pdf }}
The fluctuations of the magnetic field in the solar wind enabled the calculation of the q-triplet (or Tsallis triplet).{{Cite journal | last1 = Burlaga | first1 = L. F. | last2 = - Viñas | first2 = A. F. | doi = 10.1016/j.physa.2005.06.065 | title = Triangle for the entropic index q of non-extensive statistical mechanics observed by Voyager 1 in the distant heliosphere | journal = Physica A: Statistical Mechanics and Its Applications | volume = 356 | issue = 2–4 | pages = 375 | year = 2005 |arxiv = physics/0507212 |bibcode = 2005PhyA..356..375B | s2cid = 18823047 }}
The velocity distributions in a driven dissipative dusty plasma.{{Cite journal | last1 = Liu | first1 = B. | last2 = Goree | first2 = J. | doi = 10.1103/PhysRevLett.100.055003 | title = Superdiffusion and Non-Gaussian Statistics in a Driven-Dissipative 2D Dusty Plasma | journal = Physical Review Letters | volume = 100 | issue = 5 | year = 2008 | pmid = 18352381|arxiv = 0801.3991 |bibcode = 2008PhRvL.100e5003L | page=055003| s2cid = 2022402 }}
Spin glass relaxation.{{Cite journal | last1 = Pickup | first1 = R. | last2 = Cywinski | first2 = R. | last3 = Pappas | first3 = C. | last4 = Farago | first4 = B. | last5 = Fouquet | first5 = P. | title = Generalized Spin-Glass Relaxation | doi = 10.1103/PhysRevLett.102.097202 | journal = Physical Review Letters | volume = 102 | issue = 9 | year = 2009 | pmid = 19392558|bibcode = 2009PhRvL.102i7202P | page=097202| arxiv = 0902.4183 | s2cid = 6454082 }}
Trapped ion interacting with a classical buffer gas.{{Cite journal | last1 = Devoe | first1 = R. | doi = 10.1103/PhysRevLett.102.063001 | title = Power-Law Distributions for a Trapped Ion Interacting with a Classical Buffer Gas | journal = Physical Review Letters | volume = 102 | issue = 6 | year = 2009 | pmid = 19257583|arxiv = 0903.0637 |bibcode = 2009PhRvL.102f3001D | page=063001| s2cid = 15945382 }}
High energy collisional experiments at LHC/CERN (CMS, ATLAS and ALICE detectors){{Cite journal | last1 = Khachatryan | first1 = V. | last2 = Sirunyan | first2 = A. | last3 = Tumasyan | first3 = A. | last4 = Adam | first4 = W. | last5 = Bergauer | first5 = T. | last6 = Dragicevic | first6 = M. | last7 = Erö | first7 = J. | last8 = Fabjan | first8 = C. | last9 = Friedl | first9 = M. | last10 = Frühwirth | first10 = R. | last11 = Ghete | first11 = V. M. | last12 = Hammer | first12 = J. | last13 = Hänsel | first13 = S. | last14 = Hoch | first14 = M. | last15 = Hörmann | first15 = N. | last16 = Hrubec | first16 = J. | last17 = Jeitler | first17 = M. | last18 = Kasieczka | first18 = G. | last19 = Kiesenhofer | first19 = W. | last20 = Krammer | first20 = M. | last21 = Liko | first21 = D. | last22 = Mikulec | first22 = I. | last23 = Pernicka | first23 = M. | last24 = Rohringer | first24 = H. | last25 = Schöfbeck | first25 = R. | last26 = Strauss | first26 = J. | last27 = Taurok | first27 = A. | last28 = Teischinger | first28 = F. | last29 = Waltenberger | first29 = W. | last30 = Walzel | first30 = G. | title = Transverse-Momentum and Pseudorapidity Distributions of Charged Hadrons in pp Collisions at {{sqrt|s}}=7 TeV | doi = 10.1103/PhysRevLett.105.022002 | journal = Physical Review Letters | volume = 105 | issue = 2 | pages = 022002 | year = 2010 | pmid = 20867699|arxiv = 1005.3299 |bibcode = 2010PhRvL.105b2002K | s2cid = 119196941 | display-authors = 29 }}{{Cite journal | last1 = Chatrchyan | first1 = S. | last2 = Khachatryan | first2 = V. | last3 = Sirunyan | first3 = A. M. | last4 = Tumasyan | first4 = A. | last5 = Adam | first5 = W. | last6 = Bergauer | first6 = T. | last7 = Dragicevic | first7 = M. | last8 = Erö | first8 = J. | last9 = Fabjan | first9 = C. | last10 = Friedl | display-authors = 29 | first10 = M. | last11 = Frühwirth | first11 = R. | last12 = Ghete | first12 = V. M. | last13 = Hammer | first13 = J. | last14 = Hänsel | first14 = S. | last15 = Hoch | first15 = M. | last16 = Hörmann | first16 = N. | last17 = Hrubec | first17 = J. | last18 = Jeitler | first18 = M. | last19 = Kiesenhofer | first19 = W. | last20 = Krammer | first20 = M. | last21 = Liko | first21 = D. | last22 = Mikulec | first22 = I. | last23 = Pernicka | first23 = M. | last24 = Rohringer | first24 = H. | last25 = Schöfbeck | first25 = R. | last26 = Strauss | first26 = J. | last27 = Taurok | first27 = A. | last28 = Teischinger | first28 = F. | last29 = Wagner | first29 = P. | last30 = Waltenberger | first30 = W.| title = Charged particle transverse momentum spectra in pp collisions at $ {{sqrt|s}}= 0.9 and 7 TeV | doi = 10.1007/JHEP08(2011)086 | journal = Journal of High Energy Physics | volume = 2011 | issue = 8 | pages = 86 | year = 2011 |arxiv = 1104.3547 |bibcode = 2011JHEP...08..086C | s2cid = 122835798 }} and RHIC/Brookhaven (STAR and PHENIX detectors).{{Cite journal | last1 = Adare | first1 = A. | last2 = Afanasiev | first2 = S. | last3 = Aidala | first3 = C. | last4 = Ajitanand | first4 = N. | last5 = Akiba | first5 = Y. | last6 = Al-Bataineh | first6 = H. | last7 = Alexander | first7 = J. | last8 = Aoki | first8 = K. | last9 = Aphecetche | first9 = L. | last10 = Armendariz | first10 = R. | last11 = Aronson | first11 = S. H. | last12 = Asai | first12 = J. | last13 = Atomssa | first13 = E. T. | last14 = Averbeck | first14 = R. | last15 = Awes | first15 = T. C. | last16 = Azmoun | first16 = B. | last17 = Babintsev | first17 = V. | last18 = Bai | first18 = M. | last19 = Baksay | first19 = G. | last20 = Baksay | first20 = L. | last21 = Baldisseri | first21 = A. | last22 = Barish | first22 = K. N. | last23 = Barnes | first23 = P. D. | last24 = Bassalleck | first24 = B. | last25 = Basye | first25 = A. T. | last26 = Bathe | first26 = S. | last27 = Batsouli | first27 = S. | last28 = Baublis | first28 = V. | last29 = Baumann | first29 = C. | last30 = Bazilevsky | first30 = A. | title = Measurement of neutral mesons in p+p collisions at {{sqrt|s}}=200 GeV and scaling properties of hadron production | doi = 10.1103/PhysRevD.83.052004 | journal = Physical Review D | volume = 83 | issue = 5 | pages = 052004 | year = 2011 |arxiv = 1005.3674 |bibcode = 2011PhRvD..83e2004A | s2cid = 85560021 | display-authors = 29 }}

Among the various available theoretical results which clarify the physical conditions under which Tsallis entropy and associated statistics apply, the following ones can be selected:

Anomalous diffusion.{{Cite journal | last1 = Plastino | first1 = A. R. | last2 = Plastino | first2 = A. | doi = 10.1016/0378-4371(95)00211-1 | title = Non-extensive statistical mechanics and generalized Fokker-Planck equation | journal = Physica A: Statistical Mechanics and Its Applications | volume = 222 | issue = 1–4 | pages = 347–354 | year = 1995 |bibcode = 1995PhyA..222..347P }}{{Cite journal | last1 = Tsallis | first1 = C. | last2 = Bukman | first2 = D. | doi = 10.1103/PhysRevE.54.R2197 | title = Anomalous diffusion in the presence of external forces: Exact time-dependent solutions and their thermostatistical basis | journal = Physical Review E | volume = 54 | issue = 3 | pages = R2197–R2200 | year = 1996 | pmid = 9965440|arxiv = cond-mat/9511007 |bibcode = 1996PhRvE..54.2197T | s2cid = 16272548 }}
Uniqueness theorem.{{Cite journal | last1 = Abe | first1 = S. | doi = 10.1016/S0375-9601(00)00337-6 | title = Axioms and uniqueness theorem for Tsallis entropy | journal = Physics Letters A | volume = 271 | issue = 1–2 | pages = 74–79 | year = 2000 |arxiv = cond-mat/0005538 |bibcode = 2000PhLA..271...74A | s2cid = 119513564 }}
Sensitivity to initial conditions and entropy production at the edge of chaos.{{Cite journal | last1 = Lyra | first1 = M. | last2 = Tsallis | first2 = C. | doi = 10.1103/PhysRevLett.80.53 | title = Nonextensivity and Multifractality in Low-Dimensional Dissipative Systems | journal = Physical Review Letters | volume = 80 | issue = 1 | pages = 53–56 | year = 1998 |arxiv = cond-mat/9709226 |bibcode = 1998PhRvL..80...53L | s2cid = 15039078 }}{{Cite journal | last1 = Baldovin | first1 = F. | last2 = Robledo | first2 = A. | doi = 10.1103/PhysRevE.69.045202 | title = Nonextensive Pesin identity: Exact renormalization group analytical results for the dynamics at the edge of chaos of the logistic map | journal = Physical Review E | volume = 69 | issue = 4 | pages = 045202 | year = 2004 | pmid = 15169059|arxiv = cond-mat/0304410 |bibcode = 2004PhRvE..69d5202B | s2cid = 30277614 }}
Probability sets that make the nonadditive Tsallis entropy to be extensive in the thermodynamical sense.{{Cite journal | last1 = Tsallis | first1 = C. | last2 = Gell-Mann | first2 = M. | last3 = Sato | first3 = Y. | doi = 10.1073/pnas.0503807102 | title = Asymptotically scale-invariant occupancy of phase space makes the entropy Sq extensive | journal = Proceedings of the National Academy of Sciences | volume = 102 | issue = 43 | pages = 15377–82 | year = 2005 | pmid = 16230624| pmc = 1266086|arxiv = cond-mat/0502274 |bibcode = 2005PNAS..10215377T | doi-access = free }}
Strongly quantum entangled systems and thermodynamics.{{Cite journal | last1 = Caruso | first1 = F. | last2 = Tsallis | first2 = C. | doi = 10.1103/PhysRevE.78.021102 | title = Nonadditive entropy reconciles the area law in quantum systems with classical thermodynamics | journal = Physical Review E | volume = 78 | issue = 2 | pages = 021102 | year = 2008 | pmid = 18850781|bibcode = 2008PhRvE..78b1102C | arxiv = cond-mat/0612032 | s2cid = 18006627 }}
Thermostatistics of overdamped motion of interacting particles.{{Cite journal | last1 = Andrade | first1 = J. | last2 = Da Silva | first2 = G. | last3 = Moreira | first3 = A. | last4 = Nobre | first4 = F. | last5 = Curado | first5 = E. | title = Thermostatistics of Overdamped Motion of Interacting Particles | doi = 10.1103/PhysRevLett.105.260601 | journal = Physical Review Letters | volume = 105 | issue = 26 | pages = 260601 | year = 2010 | pmid = 21231636|arxiv = 1008.1421 |bibcode = 2010PhRvL.105z0601A | s2cid = 14831948 }}{{Cite journal | last1 = Ribeiro | first1 = M. | last2 = Nobre | first2 = F. | last3 = Curado | first3 = E. M. | title = Time evolution of interacting vortices under overdamped motion | doi = 10.1103/PhysRevE.85.021146 | journal = Physical Review E | volume = 85 | issue = 2 | pages = 021146 | year = 2012 | pmid = 22463191|bibcode = 2012PhRvE..85b1146R | s2cid = 25200027 | url = https://hal.archives-ouvertes.fr/hal-02881463/file/PRE2012.pdf }}
Nonlinear generalizations of the Schrödinger, Klein–Gordon and Dirac equations.{{Cite journal | last1 = Nobre | first1 = F. | last2 = Rego-Monteiro | first2 = M. | last3 = Tsallis | first3 = C. | title = Nonlinear Relativistic and Quantum Equations with a Common Type of Solution | doi = 10.1103/PhysRevLett.106.140601 | journal = Physical Review Letters | volume = 106 | issue = 14 | year = 2011 | pmid = 21561176|arxiv = 1104.5461 |bibcode = 2011PhRvL.106n0601N | page=140601| s2cid = 12679518 }}
Blackhole entropy calculation.{{Cite journal | last1 = Majhi | first1 = Abhishek | doi = 10.1016/j.physletb.2017.10.043 | title = Non-extensive statistical mechanics and black hole entropy from quantum geometry | journal = Physics Letters B | volume = 775 | pages = 32–36 | year = 2017 |bibcode =2017PhLB..775...32M | arxiv = 1703.09355| s2cid = 119397503 }}

For further details a bibliography is available at http://tsallis.cat.cbpf.br/biblio.htm

Generalized entropies

Several interesting physical systems{{cite journal |author-last2=Krischer |author-last1=García-Morales |author-first1=V. |author-first2=K. |title=Superstatistics in nanoscale electrochemical systems |journal=Proceedings of the National Academy of Sciences |date=2011 |volume=108 |issue=49 |pages=19535–19539 |doi=10.1073/pnas.1109844108 |bibcode=2011PNAS..10819535G |pmid=22106266 |pmc=3241754|doi-access=free }} abide by entropic functionals that are more general than the standard Tsallis entropy. Therefore, several physically meaningful generalizations have been introduced. The two most general of these are notably: Superstatistics, introduced by C. Beck and E. G. D. Cohen in 2003{{cite journal |author-last1=Beck |author-first1=C. |author-last2=Cohen |author-first2=E. G. D. |journal=Physica A: Statistical Mechanics and Its Applications |date=2003 |volume=322 |pages=267–275 |title=Superstatistics |doi=10.1016/S0378-4371(03)00019-0 |arxiv=cond-mat/0205097 |bibcode=2003PhyA..322..267B|s2cid=261331784 }} and Spectral Statistics, introduced by G. A. Tsekouras and Constantino Tsallis in 2005.{{cite journal |author-last1=Tsekouras |author-first1=G. A. |author-last2=Tsallis |author-first2=C. |title=Generalized entropy arising from a distribution of q indices |journal=Physical Review E |date=2005 |volume=71 |issue=4 |pages=046144 |doi=10.1103/PhysRevE.71.046144 |pmid=15903763 |arxiv=cond-mat/0412329 |bibcode=2005PhRvE..71d6144T|s2cid=16663654 }} Both these entropic forms have Tsallis and Boltzmann–Gibbs statistics as special cases; Spectral Statistics has been proven to at least contain Superstatistics and it has been conjectured to also cover some additional cases.{{Citation needed|date=December 2019}}