well-behaved statistic

More formally the conditions can be expressed in this way. $T$ is a statistic for $\backslash theta$ that is a function of the sample, $\{\; X\; \}\_\{\; 1\; \},...,\{\; X\; \}\_\{\; n\; \}$. For $T$ to be well-behaved we require:

: Condition 1

$\{\; E\; \}\_\{\; \backslash theta\; \}\backslash left(\; T\; \backslash right)$ differentiable in $\backslash theta\; \backslash quad\; \backslash forall\; \backslash quad\; \backslash theta\; \backslash in\; \backslash Theta$, and the derivative satisfies:

$\backslash frac\; \{\; d\; \}\{\; d\backslash theta\; \}\; \backslash int\; \{\; T\backslash left(\; \{\; X\; \}\_\{\; 1\; \},...,\{\; X\; \}\_\{\; n\; \}\; \backslash right)\; \}\; \backslash prod\; \_\{\; i=1\; \}^\{\; n\; \}\{\; f\backslash left(\; \{\; x\; \}\_\{\; i\; \}|\backslash theta\; \backslash right)\; \}\; d\{\; x\; \}\_\{\; 1\; \}...d\{\; x\; \}\_\{\; n\; \}\; =\; \backslash int\; \{\; T\backslash left(\; \{\; X\; \}\_\{\; 1\; \},...,\{\; X\; \}\_\{\; n\; \}\; \backslash right)\; \backslash left[\; \backslash frac\; \{\; \backslash partial\; \}\{\; \backslash partial\; \backslash theta\; \}\; \backslash prod\; \_\{\; i=1\; \}^\{\; n\; \}\{\; f\backslash left(\; \{\; x\; \}\_\{\; i\; \}|\backslash theta\; \backslash right)\; \}\; \backslash right]\; \}\; d\{\; x\; \}\_\{\; 1\; \}...d\{\; x\; \}\_\{\; n\; \}$: Condition 2

In order to derive the distribution law of the parameter T, compatible with $\backslash boldsymbol\; x$, the statistic must obey some technical properties. Namely, a statistic s is said to be well-behaved if it satisfies the following three statements:

1. monotonicity. A uniformly monotone relation exists between s and ? for any fixed seed $\backslash \{z\_1,\backslash ldots,z\_m\backslash \}$ – so as to have a unique solution of (1);
2. well-defined. On each observed s the statistic is well defined for every value of ?, i.e. any sample specification $\backslash \{x\_1,\backslash ldots,x\_m\backslash \}\backslash in\backslash mathfrak\; X^m$ such that $\backslash rho(x\_1,\backslash ldots,x\_m)=s$ has a probability density different from 0 – so as to avoid considering a non-surjective mapping from $\backslash mathfrak\; X^m$ to $\backslash mathfrak\; S$, i.e. associating via $s$ to a sample $\backslash \{x\_1,\backslash ldots,x\_m\backslash \}$ a ? that could not generate the sample itself;
3. local sufficiency. $\backslash \{\backslash breve\backslash theta\_1,\backslash ldots,\; \backslash breve\backslash theta\_N\backslash \}$ constitutes a true T sample for the observed s, so that the same probability distribution can be attributed to each sampled value. Now, $\backslash breve\backslash theta\_j=\; h^\{-1\}(s,\backslash breve\; z\_1^j,\; \backslash ldots,\backslash breve\; z\_m^j)$ is a solution of (1) with the seed $\backslash \{\backslash breve\; z\_1^j,\backslash ldots,\backslash breve\; z\_m^j\backslash \}$. Since the seeds are equally distributed, the sole caveat comes from their independence or, conversely from their dependence on ? itself. This check can be restricted to seeds involved by s, i.e. this drawback can be avoided by requiring that the distribution of $\backslash \{Z\_1,\backslash ldots,Z\_m|S=s\backslash \}$ is independent of ?. An easy way to check this property is by mapping seed specifications into $x\_i$s specifications. The mapping of course depends on ?, but the distribution of $\backslash \{X\_1,\; \backslash ldots,X\_m|S=s\backslash \}$ will not depend on ?, if the above seed independence holds – a condition that looks like a local sufficiency of the statistic S.

The remainder of the present article is mainly concerned with the context of data mining procedures applied to statistical inference and, in particular, to the group of computationally intensive procedure that have been called algorithmic inference.

{{Main|Algorithmic inference}}

In algorithmic inference, the property of a statistic that is of most relevance is the pivoting step which allows to transference of probability-considerations from the sample distribution to the distribution of the parameters representing the population distribution in such a way that the conclusion of this statistical inference step is compatible with the sample actually observed.

By default, capital letters (such as U, X) will denote random variables and small letters (u, x) their corresponding realizations and with gothic letters (such as $\backslash mathfrak\; U,\; \backslash mathfrak\; X$) the domain where the variable takes specifications. Facing a sample $\backslash boldsymbol\; x=\backslash \{x\_1,\backslash ldots,x\_m\backslash \}$, given a sampling mechanism $(g\_\backslash theta,Z)$, with $\backslash theta$ scalar, for the random variable X, we have

:$\backslash boldsymbol\; x=\backslash \{g\_\backslash theta(z\_1),\backslash ldots,g\_\backslash theta(z\_m)\backslash \}.$

The sampling mechanism $(g\_\backslash theta,\backslash boldsymbol\; z)$, of the statistic s, as a function ? of $\backslash \{x\_1,\backslash ldots,x\_m\backslash \}$ with specifications in $\backslash mathfrak\; S$ , has an explaining function defined by the master equation:

: $s=\backslash rho(x\_1,\backslash ldots,x\_m)=\backslash rho(g\_\backslash theta(z\_1),\backslash ldots,g\_\backslash theta(z\_m))=h(\backslash theta,z\_1,\backslash ldots,z\_m),\backslash qquad\backslash qquad\backslash qquad\; (1)$

for suitable seeds $\backslash boldsymbol\; z=\backslash \{z\_1,\backslash ldots,z\_m\backslash \}$ and parameter ?

For instance, for both the Bernoulli distribution with parameter p and the exponential distribution with parameter ? the statistic $\backslash sum\_\{i=1\}^m\; x\_i$ is well-behaved. The satisfaction of the above three properties is straightforward when looking at both explaining functions: $g\_p(u)=1$ if $u\backslash leq\; p$, 0 otherwise in the case of the Bernoulli random variable, and $g\_\backslash lambda(u)=-\backslash log\; u/\backslash lambda$ for the Exponential random variable, giving rise to statistics

:$s\_p=\backslash sum\_\{i=1\}^m\; I\_\{[0,p]\}(u\_i)$

and

:$s\_\backslash lambda=-\backslash frac\{1\}\{\backslash lambda\}\backslash sum\_\{i=1\}^m\; \backslash log\; u\_i.$

Vice versa, in the case of X following a continuous uniform distribution on $[0,A]$ the same statistics do not meet the second requirement. For instance, the observed sample $\backslash \{c,c/2,c/3\backslash \}$ gives

$s\text{'}\_A=11/6c$. But the explaining function of this X is $g\_a(u)=u\; a$.

Hence a master equation $s\_A=\backslash sum\_\{i=1\}^m\; u\_i\; a$ would produce with

a U sample $\backslash \{0.8,\; 0.8,\; 0.8\backslash \}$ and a solution $\backslash breve\; a=0.76\; c$. This conflicts with the observed sample since the first observed value should result greater than the right extreme of the X range. The statistic $s\_A=\backslash max\backslash \{x\_1,\backslash ldots,x\_m\backslash \}$ is well-behaved in this case.

Analogously, for a random variable X following the Pareto distribution with parameters K and A (see Pareto example for more detail of this case),

:$s\_1=\backslash sum\_\{i=1\}^m\; \backslash log\; x\_i$

and

:$s\_2=\backslash min\_\{i=1,\backslash ldots,m\}\; \backslash \{x\_i\backslash \}$

can be used as joint statistics for these parameters.

As a general statement that holds under weak conditions, sufficient statistics are well-behaved with respect to the related parameters. The table below gives sufficient / Well-behaved statistics for the parameters of some of the most commonly used probability distributions.

{{Anchor|SufficientTable}}

class="wikitable" |+ Common distribution laws together with related sufficient and well-behaved statistics. ! Distribution !! Definition of density function !! Sufficient/Well-behaved statistic

Uniform discrete | $f(x;n)=1/n\; I\_\{\backslash \{1,2,\backslash ldots,n\backslash \}\}(x)$ | $s\_n=\backslash max\_i\; x\_i$

Bernoulli | $f(x;p)=p^x\; (1-p)^\{1-x\}\; I\_\{\backslash \{0,1\backslash \}\}(x)$ | $s\_P=\backslash sum\_\{i=1\}^m\; x\_i$

Binomial | $f(x;n,p)=\backslash binom\{n\}\{x\}p^x\; (1-p)^\{n-x\}\; I\_\{0,1,\backslash ldots,\; n\}(x)$ | $s\_P=\backslash sum\_\{i=1\}^m\; x\_i$

Geometric | $f(x;p)=p(1-p)^x\; I\_\{\backslash \{0,1,\backslash ldots\backslash \}\}(x)$ | $s\_P=\backslash sum\_\{i=1\}^m\; x\_i$

Poisson | $f(x;\backslash mu)=\backslash mathrm\; e^\{-\backslash mu\; x\}\; \backslash mu^x\; /\; x!\; I\_\{\backslash \{0,1,\backslash ldots\backslash \}\}(x)$ | $s\_\{M\}=\backslash sum\_\{i=1\}^m\; x\_i$

Uniform continuous | $f(x;a,b)=1/(b-a)\; I\_\{[a,b]\}(x)$ | $s\_A=\backslash min\_i\; x\_i;\; s\_B=\backslash max\_i\; x\_i$

Negative exponential | $f(x;\backslash lambda)=\backslash lambda\; \backslash mathrm\; e^\{-\backslash lambda\; x\}\; I\_\{[0,\backslash infty]\}(x)$ | $s\_\{\backslash Lambda\}=\backslash sum\_\{i=1\}^m\; x\_i$

Pareto | $f(x;a,\; k)=\; \backslash frac\{a\}\{k\}\backslash left(\backslash frac\{x\}\{k\}\backslash right)^\{-a\; -1\}\; I\_\{[k,\backslash infty]\}(x)$ | $s\_\{A\}=\backslash sum\_\{i=1\}^m\; \backslash log\; x\_i;\; s\_K=\backslash min\_i\; x\_i$

Gaussian | $f(x,\backslash mu,\backslash sigma)=\; 1/(\backslash sqrt\{2\; \backslash pi\}\backslash sigma)\; \backslash mathrm\; e^\{-(x-\backslash mu^2)/(2\backslash sigma^2)\}$ | $s\_M=\backslash sum\_\{i=1\}^m\; x\_i;\; s\_\{\backslash Sigma\}=\backslash sqrt\{\backslash sum\_\{i=1\}^m(x\_i-\backslash bar\; x)^2\}$

Gamma | $f(x;r,\backslash lambda)=\; \backslash lambda/\backslash Gamma(r)\; (\backslash lambda\; x)^\{r-1\}\; \backslash mathrm\; e^\{-\backslash lambda\; x\}\; I\_\{[0,\backslash infty]\}(x)$ | $s\_\{\backslash Lambda\}=\backslash sum\_\{i=1\}^m\; x\_i;\; s\_\{K\}=\backslash prod\_\{i=1\}^m\; x\_i$

{{reflist}}

• {{cite journal

| doi=10.1214/aoms/1177728604

| author=Bahadur, R. R.

| authorlink=R. R. Bahadur

|author2=Lehmann, E. L.

| title=Two comments on Sufficiency and Statistical Decision Functions

| journal=Annals of Mathematical Statistics

| volume=26

| year=1955

| pages=139–142

| doi-access=free

}}

{{DEFAULTSORT:Well-Behaved Statistic}}

Category:Statistical inference