From: ikastan@alumni.caltech.edu (Ilias Kastanas) Newsgroups: sci.math Subject: Re: law of large numbers question Date: 6 Nov 1997 20:00:45 GMT In article <63s95u$nrn@freenet-news.carleton.ca>, Angel Garcia wrote: > >David Ullrich (ullrich@math.okstate.edu) writes: >> Angel Garcia wrote: >>> >>> exactly the same as I previously >>> posted with much more PRECISION than yours: >>> "THE VARIANCE tends to infinity as sqrt(N) for N===>infinity !"" >> >> The two statements are _not_ the same - the VARIANCE of a >> random variable X is the expected value of X^2 (or |X|^2 if X is complex), >> not the expected value of |X|. > Variance is another vague word in the long, long history of >statistics; I agree that currently it is taken in the specific sense >of 'average or expected value (x - m)^2', namely, the 'second moment'. >But I did not use it above in such specific sense, but in the generic sense >of 'spread'; perhaps a perfectly precise statement would be >"The variance or spread AS MEASURED by the standard deviation (sigma) > tends to infinity as sqrt(N) for N===>infinity". > > >> What is "imprecise" about saying that the expected value of the >> absolute value of the error tends to infinity? >> > it is 'imprecise' because it is not said in which manner tends to >infinity for N===>infinity. One thing is as N^(1/2) and another as N^3, say !. But saying so is still "imprecise"; what type of convergence is it? From the law of large numbers, S_n /n -> mu (the mean, 0 here), in the sense of "almost-surely" convergence (a.s.). The weak law would have given "convergence in probability". From the CLT, (S_n - mu*n) /sigma*n^(1/2) -> standard normal, "convergence in distribution". ( S_n /n^(1/2) in our case). We do have lim_sup S_n /n^(1/2) = +inf, "a.s.". But S_n /n^(1/2) does not converge in probability. We cannot help that. (Convergence in probability implies "in distribution", but not conversely. Also, "in probability" does not imply "a.s."). All in all, "the sigma of S_n is n^(1/2)" is useful, but maybe less than fully precise. >>> What does it prove ?. >> >> It proves what I said it proves: the error does not tend >> to infinity, it is recurrent. > > If it does not prove anything else...: as the gaussian shape as >enveloping curve with calculated pair of its parameters, namely, >its m (mean or average value) and sigma (its standard deviation) > as function of N (number of tosses)... then it is very poor study. >But probably it does, of course... because all these details are >simple in the case of trivial flip-flop. NOT SO in the genaral case >of ANY discrete distribution repeated N times... a generalization >of the terrible Khinchin's theorem will be needed and now I doubt >that ever has it been done: far too difficult !: one thing is to >say that still sigma will go as sqrt(N) and much more terribly difficult >thing is to ascertain the conditions for such 'large number law' holding >still true. Eh, Khintchine proved the Law of the Iterated Logarithm, lim_sup S_n /(2n * log log n)^(1/2) = 1 a.s., for Bernoulli trials, yes... but his proof of the Law of L.N. was already for any distribution. Anyway, that was in the 20s; all these results have been thus generalized for quite some time now (and taken further, to martingales and beyond). >> What does the question of recurrence have to do with the CLT? >> (The two are certainly related, but they're certainly not the sanme >> thing.) > > of course they are related ! the CLT says precisely in which way >such recurrence takes place with estimation of probability for every >particular NUMBER in your recurrence: 0 is the most probable and >values large (+ or -) are less probable for any finite N. Recurrence involves _infinite_ sequences of +/-1's, and proba- bility as carried by that space ( 'recurrent at x' <=> for every epsilon > 0, the probability of event "|S_n -x| < epsilon, infinitely often" is = 1). It goes beyond the behavior of finite sequences of length n. Finite level n is just that; we can contemplate the S_n of each sequence there, 2^n in all; what matters though is the set of all {S_1, ..., S_n, S_n+1, ...}, not the slice of it at n! The notion "recurrent" is not applicable at, or up to, level n... for any n. For instance: In the case of one-dimensional random walks, condition "S_n /n -> 0, in probability" implies recurrence; true enough. Well, consider walks with "symmetric stable law" (char. function exp(-|t|^a)): If 1 < a < 2, the condition holds and hence we have recurrence. If a = 1 (Cauchy distr.), S_n /n -> 0 fails.. yet the walk is recurrent all the same. If 0 < a < 1, the walks are transient (non-recurrent)... and at the same time they do satisfy lim_inf S_n = -inf, lim_sup S_n = +inf ! They swing back and forth, hitting all values... while the probability of "|S_n| < K infinitely often" is 0. For any K. I should think, then, CLT and the like give part of the picture... but the full story is much more complicated. Ilias ============================================================================== From: ikastan@alumni.caltech.edu (Ilias Kastanas) Newsgroups: sci.math Subject: Re: law of large numbers question Date: 7 Nov 1997 23:10:56 GMT In article <63tjdg$3em@freenet-news.carleton.ca>, Angel Garcia wrote: > >Ilias Kastanas (ikastan@alumni.caltech.edu) writes: >> In article <63s95u$nrn@freenet-news.carleton.ca>, >> Angel Garcia wrote: > >>> it is 'imprecise' because it is not said in which manner tends to >>>infinity for N===>infinity. One thing is as N^(1/2) and another as N^3, say !. >> >> >> But saying so is still "imprecise"; what type of convergence is it? >> > OK. I don't know details: just what Khinchin's > version of CLT in one of his latest books says (Mathematical > Foundations of Statistical Mechanics (Dover, 1949) with spelling of >his name as I do (without 'e' at end as it was in french papers). After >such book he wrote another one (at least) and possibly gave in there >a further refinement of his theorem. Only the set of conditions fill already >more than 1 page. Physicist use it as standard text-book: hs theorems do >not treat the case of discrete distributions (despite that EVERYTHING in >physics is discrete: molecules in a gas, even the field in vacuo is >discrete (Heisenberg's lattice)). If you use F(t) = P(X <= t) you cover all cases, discrete or not. Let S = the set of points with positive probability, P(X=t) > 0; F has a jump at any such point. You might have concentration on S, any P(A) with A disjoint to S being =0 (e.g. Bernoulli)... or S empty (Gaussian)... or neither ("mix"). Going by density f ("dF/dt", or better: F(t) = integral up to t of f) is less general. Beyond "Dirac deltas at jumps", there are continuous F that have no f. The theorems hold for sequences of i.i.d. X_k (independent and each one having the same F); so this includes "discrete". I don't know whether the book states and proves them at this level. Eh, I did not intend to criticize your spelling! I had just looked at a footnote in Feller for years of publication, and that spelling stuck in my mind. But right now I have "Continued Fractions" in my hands; the author is given as "Khinchin"; and I think that's preferable. >> From the law of large numbers, S_n /n -> mu (the mean, 0 here), >in the sense of "almost-surely" convergence (a.s.). The weak law would have >> given "convergence in probability". >> >> From the CLT, (S_n - mu*n) /sigma*n^(1/2) -> standard normal, >> "convergence in distribution". ( S_n /n^(1/2) in our case). >> >> We do have lim_sup S_n /n^(1/2) = +inf, "a.s.". But S_n /n^(1/2) >> does not converge in probability. We cannot help that. (Convergence >> in probability implies "in distribution", but not conversely. Also, >> "in probability" does not imply "a.s."). >> >> >> All in all, "the sigma of S_n is n^(1/2)" is useful, but maybe less >> than fully precise. > Ilias, you go beyond my grasp. How can I assess what you say ?. If >you contend that such > sentence is not 'fully' precise may be you are right, but it is enough >for most cases and EXACTLY expresses the 'law of large numbers'. Sorry I didn't say it clearly. When +/-1 have p = 1/2, sigma^2 for the sum S_n _is_ exactly n, of course. When we look at other F, this is so "in the limit", and we have a convergence issue; e.g., as given above, S_n /n^1/2 => standard normal. >>>>> What does it prove ?. >>>> >>>> It proves what I said it proves: the error does not tend >>>> to infinity, it is recurrent. >>> >>> If it does not prove anything else...: as the gaussian shape as >>>enveloping curve with calculated pair of its parameters, namely, >>>its m (mean or average value) and sigma (its standard deviation) >>> as function of N (number of tosses)... then it is very poor study. >>>But probably it does, of course... because all these details are >>>simple in the case of trivial flip-flop. NOT SO in the genaral case >>>of ANY discrete distribution repeated N times... a generalization >>>of the terrible Khinchin's theorem will be needed and now I doubt >>>that ever has it been done: far too difficult !: one thing is to >>>say that still sigma will go as sqrt(N) and much more terribly difficult >>>thing is to ascertain the conditions for such 'large number law' holding >>>still true. >> >> >> >> Eh, Khintchine proved the Law of the Iterated Logarithm, >> lim_sup S_n /(2n * log log n)^(1/2) = 1 a.s., for Bernoulli trials, >> yes... but his proof of the Law of L.N. was already for any distribution. >> Anyway, that was in the 20s; all these results have been thus generalized >> for quite some time now (and taken further, to martingales and beyond). > Sure he came very late in such 'law' which goes back centuries ago, >but as mentioned he perfected it like anybody else that I know with >his several versions of CLT. But if you know better... I believe it >after all such CLT is, in my and many's view, the most (or almost) >important theroem of mathematics which MODULATES all branches of >Science; since any experiment requires repetitivity thus CLT. The L.L.N. above, S_n /n -> m, even when the X_i do not have finite variance, is due to Khinchin. If they do, and variance is sigma^2, then more is true: CLT (due to Lindeberg) and LIL (the Bernoulli case due to Khinchin). LIL for the general case follows from LIL for Brownian motion (the Wiener-Levy process). Kolmogorov proved LIL for X_i not necessarily identically distributed but bounded. There are lots of variations and extensions, by the above and others. >> I should think, then, CLT and the like give part of the picture... >> but the full story is much more complicated >> Ilias > >I tend to agree... mostly when you go to n-dimensionality. For 1-dimension >the CLT is fairly complete: it gives all orders O(n) for the >centrally approaching gaussian (as envelope if the discrete case is at hand). Note the 1-dim examples I mentioned (symmetric stable laws)... say for a = 1: each X_i is Cauchy (density 1/pi*(1+t^2)). There is no CLT then, and no L.L.N. ... since there is no m in the first place; the expected value EX does not exist. You don't have S_n /n -> anything! But you can show the random walk is recurrent, even though those tools are not available. (It turns out S_n /n itself is Cauchy). Ilias