A Metric on Probabilities, and Products of Loeb Spaces^{∗}

H. Jerome Keisler and Yeneng Sun

Abstract

We first introduce two functions on finitely additive probability spaces that behave well under products: discrepancy, which measures how close one space comes to extending another, and bi-discrepancy, which is a pseudo-metric on the collection of all spaces on a given set, and a metric on the collection of complete spaces. We then apply these to show that the Loeb space of the internal product of two internal finitely additive probability spaces depends only on the Loeb spaces of the two original internal spaces. Thus the notion of a Loeb product of two Loeb spaces is well-defined. The Loeb operation induces an isometry from the nonstandard hull of the space of internal probability spaces on a given set to the space of Loeb spaces on that set, with the metric of bi-discrepancy.

1 Introduction

The most important construction in the application of model theory to probability is the Loeb measure construction from [10]. The Loeb operation converts an internal finitely additive probability space M = (Ω, F, μ) to a complete σ-additive probability space L(M) = (Ω, L(F), L(μ)). It provides a valuable tool for proving mathematical results using model-theoretic methods (see, for example, [1], [8], [9], and [11]). The purpose of this paper is to show that the “Loeb product” of two Loeb spaces is well-defined. That is, given two internal probability spaces, the Loeb space of the internal product depends only on the Loeb spaces of the original internal spaces. Thus, it is appropriate to call the Loeb spaces of the original spaces “factors”, and to call the Loeb space of the internal product the “Loeb product”.

∗AMS subject classification: Primary 03C50, 03H05, 28E05; Secondary 26E35.

For each i = 1, 2, let M_{i} = (Ω_{i}, F_{i}, μ_{i}) be an internal probability space, where F_{i} is an internal algebra of subsets of a nonempty internal set Ω_{i}, and μ_{i} a finitely additive internal measure on F_{i}. There are several ways to construct σ-additive product probability spaces based on the Loeb operations. First, we take the Loeb spaces L(M_{i}) = (Ω_{i}, L(F_{i}), L(μ_{i})) for i = 1, 2 and take their usual σ-additive measure-theoretic product

L(M_{1}) ⊗^{σ} L(M_{2}) = (Ω_{1} × Ω_{2}, L(F_{1}) ⊗^{σ} L(F_{2}), μ_{1} ⊗^{σ} μ_{2}).

Its completion is denoted by (L(M_{1}) ⊗^{σ} L(M_{2}))^{c}. Second, let

M_{1} ⊗ M_{2} = (Ω_{1} × Ω_{2}, F_{1} ⊗ F_{2}, μ_{1} ⊗ μ_{2})

be the internal product space, where F_{1} ⊗F_{2} is simply the internal algebra of all ^{∗}finite disjoint unions of rectangles A_{1} ×A_{2} with A_{i} ∈ F_{i}. The Loeb space L(M_{1} ⊗ M_{2}) of M_{1} ⊗ M_{2} is called the Loeb product space. Third, when the internal probability spaces M_{1} and M_{2} are^{∗}σ-additive, let M_{1}⊗^{σ}M_{2} be the internal ^{∗}σ-additive probability space generated by the internal product M_{1} ⊗ M_{2}. The Loeb space L(M_{1} ⊗^{σ} M_{2}) of M_{1} ⊗^{σ} M_{2} is called the Loeb σ-product space.

As already noted in [2], (L(M_{1})⊗^{σ} L(M_{2}))^{c} ⊆ L(M_{1} ⊗M_{2}). It is shown in [13] that the inclusion is always proper when L(M_{1}) and L(M_{2}) are nonatomic (a specific example can be found in [1] p.74). In fact, Theorem 6.2 in [13] shows that the Loeb product space is very rich in the sense that it can be endowed with independent processes that are not measurable in (L(M_{1}) ⊗^{σ} L(M_{2}))^{c} but have almost independent random variables with any variety of distributions. In addition, it is shown in [3] that there is a continuum of increasing Loeb product null sets with large gaps in the sense that their set differences have L(μ_{1}) ⊗^{σ} L(μ_{2})-outer measure one. Thus, the Loeb product space is much richer than the usual product even on null sets. It is also clear that L(M_{1} ⊗ M_{2}) ⊆ L(M_{1} ⊗^{σ} M_{2}) when both M_{1} and M_{2} are internally ^{∗}σ-additive.

Though the Loeb product (or Loeb σ-product) space is usually much richer than the completion of the usual product of the Loeb spaces, it still has the Fubini property ([9]). This property plays a key role in the discovery of some basic phenomena involving independence (see [13] and [14]). Loeb product spaces are also useful in solving stochastic differential equations ([1]), in chaos decompositions ([4] and Chapter 6 of[11] by Osswald), and in the model theory of stochastic processes ([8], [9]).

Recently, a number of special measure-theoretic properties of Loeb spaces were discovered and formulated in conventional terms; see, for example, [6], [8], [11], [13] and their references. For applications of these special properties, one can simply regard the Loeb space as a primitive object without going through internal operations at all. On the other hand, the definition of Loeb product (Loeb σ-product) relies on the internal product (the internal ^{∗}σadditive product). Also, different internal probability spaces may give the same Loeb space. So the basic question is whether for any two given Loeb spaces, one can still define their Loeb product (or their Loeb σ-product) unambiguously. In this paper, we show that the Loeb product space L(M_{1} ⊗ M_{2}) depends only on the factor Loeb spaces L(M_{1}) and L(M_{2}), and not on the internal spaces M_{1} and M_{2} themselves. We also prove that if M_{1} and M_{2} are^{∗}σ-additive, then the Loeb σ-product L(M_{1}⊗^{σ}M_{2}) depends only on L(M_{1}) and L(M_{2}). Therefore both the Loeb product and the Loeb σ-product are well-defined functions of the factor Loeb spaces.

In Sections 2 and 3, we develop some general methods to study the collection of (finitely additive) standard probability spaces. A standard function d(M, N), which we call the discrepancy, is introduced. It measures how close one (finitely additive) probability space comes to extending another. The discrepancy function is of independent interest, and its properties are studied in detail in Section 2. In particular, given a nonempty set Ω, the bidiscrepancy d_{2}(M, N) = d(M, N) + d(N , M) is a pseudo-metric on the set of all probability spaces on Ω, and a metric on the set of all complete probability spaces on Ω. In Section 3 it is shown that the discrepancy from one product probability space to another product probability space is bounded by the sum of the discrepancies of the respective factor spaces.

In Section 4, we consider Loeb extensions and Loeb equivalence of internal probability spaces, which have useful characterizations in terms of the discrepancy. It shown that the space L_{Ω} of all Loeb spaces on Ω is a complete metric space under bi-discrepancy, and that the Loeb operation induces an isometry from the nonstandard hull of the space of internal probability spaces to L_{Ω}. Section 5 presents the results on the uniqueness of the Loeb product and the Loeb σ-product. Finally, Section 6 deals with hyperfinite probability spaces. It is shown that when one of the factors is the Loeb space of a hyperfinite probability space, the Loeb product and the Loeb σ-product are the same. This leads to the study of spaces that are Loeb equivalent to a hyperfinite probability space.

A few questions are posed in Section 7. For background in nonstandard probability see, e.g., [1] or [11]. As usual, we work in an ω_{1}-saturated nonstandard universe.

This research was supported in part by the Vilas Trust Fund at the University of Wisconsin.

2 Discrepancy of probability spaces

A standard probability space (Ω, F, μ) is only understood to have a finitely additive measure μ on an algebra F of subsets of a nonempty set Ω with μ(Ω) = 1 unless we explicitly assume countable additivity (also termed σadditivity). In this and the next section, we only work with standard probability spaces.

We first give a formal definition of the notion of discrepancy, which measures how close one probability space comes to extending another.

Definition 2.1 Let M = (Ω, F, μ) and N = (Ω, G, ν) be probability spaces. Define the outer measure ν̄ so that

ν̄(B) = inf{ν(C) : B ⊆ C ∈ G}

for any B ⊆ Ω. Define the discrepancy d(M, N) of M from N by

d(M, N) = sup{ν̄(B) − μ(B) : B ∈ F}.

Define the bi-discrepancy between M and N as the sum

d_{2}(M, N) = d(M, N) + d(N , M).

It is easy to see that if N extends M (denoted by M ⊆ N) then d(M, N) = 0, and in particular that d(M, M) = d_{2}(M, M) = 0. Note that the outer measure ν̄ as defined here may be different from the usual outer measure defined in a textbook on measure theory (see [5] or [12]), where a countable covering is used. The following lemma shows that the discrepancy function is non-negative and satisfies the triangle inequality.

Lemma 2.2 Let M = (Ω, F, μ), N = (Ω, G, ν) and P = (Ω, H, ρ) be probability spaces. Then (a) d(M, N) = sup{|ν̄(B) − μ(B)| : B ∈ F}, and 0 ≤ d(M, N) ≤ 1.

(b) The function d(·, ·) satisfies the triangle inequality

d(M, P) ≤ d(M, N) + d(N , P).

Proof. (a) For any B ∈ F, we have ν̄(B) + ν̄(Ω \ B) ≥ 1, and hence

ν̄(Ω \ B) − μ(Ω \ B) ≥ (1 − ν̄(B)) − (1 − μ(B)) = −(ν̄(B) − μ(B)).

This means that if ν̄(B) − μ(B) < 0, then

|ν̄(B) − μ(B)| = −(ν̄(B) − μ(B)) ≤ ν̄(Ω \ B) − μ(Ω \ B) ≤ d(M, N).

When ν̄(B) − μ(B) ≥ 0, it is obvious that |ν̄(B) − μ(B)| ≤ d(M, N). By the arbitrary choice of B, we have sup{|ν̄(B)−μ(B)| : B ∈ F} ≤ d(M, N). The opposite inequality is clear. Hence d(M, N) = sup{|ν̄(B) − μ(B)| : B ∈ F}, which implies that 0 ≤ d(M, N) ≤ 1.

(b) For any B ∈ F, we have ν̄(B) − μ(B) ≤ d(M, N). Let ϵ be any positive real number. There is a set C ∈ G such that B ⊆ C and ν(C)−ϵ/2 ≤ ν̄(B). Similarly, there is a set D ∈ H such that C ⊆ D and ρ(D) − ϵ/2 ≤ ρ̄(C). Add the three inequalities together and rearrange to obtain

ρ(D) − μ(B) ≤ d(M, N) + (ρ̄(C) − ν(C)) + ϵ ≤ d(M, N) + d(N , P) + ϵ.

Since ρ̄(B) ≤ ρ(D), we have ρ̄(B) − μ(B) ≤ d(M, N) + d(N , P) + ϵ, which implies the desired triangle inequality by taking arbitrary choices of B ∈ F and ϵ > 0.

Lemma 2.2(a) shows that in the special case that M, N are complete countably additive probability spaces with the same underlying σ-algebra F = G, the discrepancy d(M, N) is equal to the total variation distance.

Theorem 2.3 The bi-discrepancy d_{2}(·, ·) is a pseudo-metric on the family of all probability spaces on Ω.

Proof. The bi-discrepancy is clearly symmetric, and the triangle inequality follows from Lemma 2.2 (b).

The following example shows that in general, the discrepancy d is not a pseudo-metric because d(M, N) ≠ d(N , M).

Example 2.4 (a) Let N be the probability space such that Ω = [0, 1), G be the set of all finite unions of disjoint sub-intervals [a, b) in [0, 1), and ν be the Lebesgue measure restricted to G. Let M be the Borel probability space on [0, 1) with the Lebesgue measure. Let Q be the set of rational numbers in [0, 1). It is clear that ν̄(Q) = 1 and μ(Q) = 0, and it follows that d(M, N) = 1 but d(N , M) = 0.

(b) Here is a finite example. Let m be a positive integer greater than 3 and Ω = {1, 2, …, m}. Let F be the algebra generated by the partition {{1, 2}} ∪ {{i} : 3 ≤ i ≤ m} and let μ({1, 2}) = 2/m, μ({i}) = 1/m for 3 ≤ i ≤ m. Let G be the algebra of all subsets of Ω and let ν be the uniform counting probability measure. It is easy to check that d(M, N) = 0 and d(N , M) = 1/m.

The outer measure ν̄ associated with a finitely additive measure ν is used to define the discrepancy. We shall also consider the inner measure \underline{ν}. The outer and inner measures are then used to extend ν to its completion ν^{c}.

Definition 2.5 Let N = (Ω, G, ν) be a probability space. Let \underline{ν} be the inner measure such that for any B ⊆ Ω, \underline{ν}(B) = sup\{ν(C) : B ⊇ C ∈ G\}. The completion of N is defined as the probability space N^{c} = (Ω, G^{c}, ν^{c}) where G^{c} = \{B ⊆ Ω : ν̄(B) = \underline{ν}(B)\}, ν^{c} = ν̄|_{G^{c}}. N is said to be complete if N^{c} = N.

Here is a collection of easy facts that will be needed.

Lemma 2.6 (a) N^{c} is a finitely additive probability space extending N.

(b) If N is a countably additive probability space, then so is N^{c}, which is the completion of N in the usual measure-theoretic sense.

(c) N^{c} is complete.

(d) If M ⊆ N, then M^{c} ⊆ N^{c}.

(e) d_{2}(N , N^{c}) = 0 and d(M, N) = d(M^{c}, N^{c}).

(f) d(M, N) = 0 if and only if M ⊆ N^{c}.

(g) d(M^{c}, N^{c}) = 0 if and only if M^{c} ⊆ N^{c}.

(h) d_{2}(M^{c}, N^{c}) = 0 if and only if M^{c} = N^{c}.

Proof. The result (a) is essentially known. We include its proof for the sake of completeness.

We first show that G^{c} is an algebra. For any E ∈ G^{c}, we have

ν̄(Ω \ E) = 1 − \underline{ν}(E) = 1 − ν̄(E) = \underline{ν}(Ω \ E),

and hence (Ω \ E) ∈ G^{c}.

Next, take any B_{i} ∈ G^{c} for i = 1, 2. For any given ϵ > 0, there exist sets C_{i}, D_{i} ∈ G for i = 1, 2 such that D_{i} ⊆ B_{i} ⊆ C_{i} and

ν(C_{i}) − ϵ/4 ≤ ν̄(B_{i}) = \underline{ν}(B_{i}) ≤ ν(D_{i}) + ϵ/4.

It follows that

ν(C_{i} \ D_{i}) = ν(C_{i}) − ν(D_{i}) ≤ ϵ/2.

Let B = B_{1} ∪ B_{2}, C = C_{1} ∪ C_{2}, and D = D_{1} ∪ D_{2}. Then D ⊆ C, and

C \ D = (C_{1} \ D) ∪ (C_{2} \ D) ⊆ (C_{1} \ D_{1}) ∪ (C_{2} \ D_{2}).

Therefore

0 ≤ ν̄(B) − \underline{ν}(B) ≤ ν(C) − ν(D) = ν(C \ D) ≤ ν(C_{1} \ D_{1}) + ν(C_{2} \ D_{2}) ≤ ϵ.

Hence ν̄(B) = \underline{ν}(B), and B ∈ G^{c}.

If B_{1} ∩ B_{2} = ∅, then D_{1} ∩ D_{2} = ∅, and

ν̄(B_{1}) + ν̄(B_{2}) ≤ ν(D_{1}) + ϵ/4 + ν(D_{2}) + ϵ/4 = ν(D) + ϵ/2 ≤ ν̄(B) + ϵ/2,

which implies ν̄(B_{1}) + ν̄(B_{2}) ≤ ν̄(B) by arbitrarily choosing ϵ > 0. Since it is easy to verify that ν̄(B) ≤ ν̄(B_{1}) + ν̄(B_{2}), we have ν̄(B) = ν̄(B_{1}) + ν̄(B_{2}). Hence (a) is proven.

For (b), assume that N is countably additive. Then B ∈ G^{c} if and only if there exist C, D ∈ G such that D ⊆ B ⊆ C and ν(C \ D) = 0. Thus N^{c} is the completion of N in the usual measure-theoretic sense.

For (c), we first show that \overline{ν^{c}} = ν̄. Take any B ⊆ Ω. Since the set {ν^{c}(C) : B ⊆ C ∈ G^{c}} contains the set {ν(C) : B ⊆ C ∈ G}, we have \overline{ν^{c}}(B) ≤ ν̄(B). On the other hand, take any E ∈ G^{c} with B ⊆ E. For any given ϵ > 0, there exists a set D ∈ G such that D ⊇ E and ν(D)−ϵ ≤ ν^{c}(E). Thus, ν̄(B) − ϵ ≤ ν^{c}(E). Then ν̄(B) ≤ ν^{c}(E) follows from the arbitrary choice of ϵ. By taking the infimum on all the E ∈ G^{c} with B ⊆ E, we obtain that ν̄(B) ≤ \overline{ν^{c}}(B). It follows that \underline{ν^{c}} = \underline{ν}. Therefore, (G^{c})^{c} = G^{c} and (ν^{c})^{c} = ν^{c}, i.e., (N^{c})^{c} = N^{c}.

For (d), note that for any B ⊆ Ω, \underline{μ}(B) ≤ \underline{ν}(B) ≤ ν̄(B) ≤ μ̄(B). If B ∈ F^{c}, then we must have \underline{ν}(B) = ν̄(B) = μ̄(B), and hence B ∈ G^{c} and ν^{c}(B) = μ^{c}(B). (d) is shown.

For (e), note that d(N^{c}, N) = sup{ν̄(B) − ν^{c}(B) : B ∈ G^{c}}. By the definition of N^{c}, we know that ν^{c}(B) = ν̄(B) for B ∈ G^{c}. Hence d(N^{c}, N) = 0. Since N^{c} is an extension of N, d(N , N^{c}) = 0. Hence d_{2}(N , N^{c}) = 0, which together with the triangle inequality implies d(M, N) = d(M^{c}, N^{c}).

Now we consider (f). First assume d(M, N) = 0. For any given B ∈ F, Lemma 2.2(a) implies that ν̄(B) = μ(B) . Thus

\underline{ν}(B) = 1 − ν̄(Ω \ B) = 1 − μ(Ω \ B) = μ(B) = ν̄(B),

and hence B ∈ G^{c} and ν^{c}(B) = μ(B). Therefore M ⊆ N^{c}. On the other hand, if M ⊆ N^{c}, then d(M, N^{c}) = 0, and 0 ≤ d(M, N) ≤ d(M, N^{c}) + d(N^{c}, N) = 0. Hence d(M, N) = 0.

If d(M^{c}, N^{c}) = 0, then (e) and (f) imply that d(M, N) = 0 and M ⊆ N^{c}. By (c) and (d), M^{c} ⊆ (N^{c})^{c} = N^{c}. It is obvious that if M^{c} ⊆ N^{c}, then d(M^{c}, N^{c}) = 0. Hence (g) is shown.

(h) follows from (g) easily.

Theorem 2.7 The bi-discrepancy d_{2}(·, ·) is a metric on the family of all complete probability spaces on Ω.

Proof. By Lemmas 2.2 and 2.6.

The following proposition shows that the operation of countably additive extension is a contraction for the discrepancy function.

Proposition 2.8 (a) If N can be extended to a countably additive probability space σ(N) = (Ω, σ(G), ν^{σ}), then d(M, σ(N)) ≤ d(M, N).

(b) Suppose that both M and N can be respectively extended to countably additive probability spaces σ(M) = (Ω, σ(F), μ^{σ}) and σ(N) = (Ω, σ(G), ν^{σ}). Then:

(1) d(σ(M), σ(N)) = d(M, σ(N)), and

(2) d(σ(M), σ(N)) ≤ d(M, N).

Proof. (a) follows easily from the fact that for any B ⊆ Ω,

\overline{ν^{σ}}(B) = inf\{ν^{σ}(C) : B ⊆ C ∈ σ(G)\} ≤ inf\{ν(C) : B ⊆ C ∈ G\} = ν̄(B).

To prove the equality in (b)(1), we first observe that

d(M, σ(N)) = sup\{\overline{ν^{σ}}(B) − μ(B) : B ∈ F\}
≤ sup\{\overline{ν^{σ}}(B) − μ^{σ}(B) : B ∈ σ(F)\}
= d(σ(M), σ(N)).

The proof of the other side of the inequality is more involved. Assume that d(M, σ(N)) < δ. Let E be the set of all B ∈ σ(F) such that there exists a set C ∈ σ(G) with B ⊆ C and ν^{σ}(C) − μ^{σ}(B) ≤ δ. Thus, E contains the algebra F. We show that E is a monotone class.

Let A_{0} ⊇ A_{1} ⊇ ⋯ be a decreasing chain of sets in E and A = ⋂_{n} A_{n}. For each n there is a set C_{n} ∈ σ(G) such that A_{n} ⊆ C_{n} and ν^{σ}(C_{n})−μ^{σ}(A_{n}) ≤ δ. We may take the C_{n} so that C_{0} ⊇ C_{1} ⊇ · · ·. Let C = ⋂_{n} C_{n}. Then A ∈ σ(F), C ∈ σ(G), and

ν^{σ}(C) − μ^{σ}(A) = lim_{n→∞} ν^{σ}(C_{n}) − lim_{n→∞} μ^{σ}(A_{n}) = lim_{n→∞}(ν^{σ}(C_{n}) − μ^{σ}(A_{n})) ≤ δ,

so A ∈ E.

Now let B_{0} ⊆ B_{1} ⊆ ⋯ be an increasing chain of sets in E and let B = ⋃_{n} B_{n}. For each n choose a set D_{n} ∈ σ(G) such that B_{n} ⊆ D_{n} and ν^{σ}(D_{n})−μ^{σ}(B_{n}) ≤ δ. Let E_{n} = ⋂_{m=n}^{∞} D_{m}. Then E_{n} ∈ σ(G), B_{n} ⊆ E_{n} ⊆ D_{n}, and E_{0} ⊆ E_{1} ⊆ · · ·. It follows that ν^{σ}(E_{n}) − μ^{σ}(B_{n}) ≤ δ. Let E = ⋃_{n} E_{n}. We have B ∈ σ(F), E ∈ σ(G), B ⊆ E, and

ν^{σ}(E) − μ^{σ}(B) = lim_{n→∞} ν^{σ}(E_{n}) − lim_{n→∞} μ^{σ}(B_{n}) = lim_{n→∞}(ν^{σ}(E_{n}) − μ^{σ}(B_{n})) ≤ δ,

and hence B ∈ E.

We have shown that E is a monotone class. By the Monotone Class Theorem (e.g. see [5], p.15), E is the σ-algebra σ(F) generated by F. This proves that d(σ(M), σ(N)) ≤ δ, and it follows that d(σ(M), σ(N)) = d(M, σ(N)) as required.

(b)(2) follows easily from (a) and (b)(1).

Example 2.9 Let N and M be the spaces from Example 2.4 (a). N is the restriction of Lebesgue measure to the algebra of finite unions of intervals in [0, 1), M is the restriction of Lebesgue measure to the Borel algebra, and d(M, N) = 1. We note that M = σ(N), so

0 = d(M, σ(N)) = d(σ(M), σ(N)) < d(M, N) = 1.

Thus both the inequalities in (a) and (b)(2) of Proposition 2.8 hold strictly in this case.

3 Discrepancy of product spaces

In this section we show that the discrepancy from one product probability space to another product probability space is dominated by the sum of the discrepancies of the respective factor spaces. The same result also holds for σ-product spaces.

Definition 3.1 Given probability spaces M_{1} = (Ω_{1}, F_{1}, μ_{1}) and M_{2} = (Ω_{2}, F_{2}, μ_{2}), let

M_{1} ⊗ M_{2} = (Ω_{1} × Ω_{2}, F_{1} ⊗ F_{2}, μ_{1} ⊗ μ_{2})

denote the product space. In this product, F_{1} ⊗ F_{2} is the set of all finite unions of rectangles A_{1} × A_{2} where A_{1} ∈ F_{1}, A_{2} ∈ F_{2}, and

(μ_{1} ⊗ μ_{2})(A_{1} × A_{2}) = μ_{1}(A_{1})μ_{2}(A_{2}).

The following result shows that the discrepancy is preserved for product spaces when one factor space is fixed. When both factor spaces are changed, the discrepancy of the product spaces are dominated by the sum of the factor discrepancies.

Theorem 3.2 Let M_{1} = (Ω_{1}, F_{1}, μ_{1}), N_{1} = (Ω_{1}, G_{1}, ν_{1}), M_{2} = (Ω_{2}, F_{2}, μ_{2}), and N_{2} = (Ω_{2}, G_{2}, ν_{2}) be probability spaces. Then

(a) d(M_{1} ⊗ M_{2}, N_{1} ⊗ M_{2}) = d(M_{1}, N_{1}).

(b) d(N_{1} ⊗ M_{2}, N_{1} ⊗ N_{2}) = d(M_{2}, N_{2}).

(c) d(M_{1} ⊗ M_{2}, N_{1} ⊗ N_{2}) ≤ d(M_{1}, N_{1}) + d(M_{2}, N_{2}).

Proof. (a) We first prove d(M_{1} ⊗ M_{2}, N_{1} ⊗ M_{2}) ≤ d(M_{1}, N_{1}). Let δ > d(M_{1}, N_{1}). Let B ∈ F_{1} ⊗ F_{2}. Then B is the union of a finite family of rectangles D_{i} × E_{i}, 1 ≤ i ≤ K for some positive integer K, where each D_{i} ∈ F_{1} and E_{i} ∈ F_{2}. One can arrange things so that the sets E_{i} are pairwise disjoint. For each i we have ν̄_{1}(D_{i}) − μ_{1}(D_{i}) < δ. We may therefore choose sets C_{i} ∈ G_{1} such that for each 1 ≤ i ≤ K, D_{i} ⊆ C_{i} and ν_{1}(C_{i}) − μ_{1}(D_{i}) < δ. Let C = ⋃_{1≤i≤K} C_{i} ⊗ E_{i}. Then C ∈ G_{1} ⊗ F_{2}, B ⊆ C, and (ν_{1} ⊗ μ_{2})(C) − (μ_{1} ⊗ μ_{2})(B) < δ. This proves desired inequality.

Next consider the other side of the inequality. Take any A ⊆ Ω_{1} and any F ∈ G_{1} ⊗ F_{2} with A × Ω_{2} ⊆ F. Then F is the union of a finite family of rectangles G_{i} × H_{i}, 1 ≤ i ≤ L, where each G_{i} ∈ G_{1} and H_{i} ∈ F_{2}. One can arrange things so that the sets H_{i} are pairwise disjoint. Since A × Ω_{2} ⊆ F, we must have ∪_{1≤i≤L}H_{i} = Ω_{2}, and A ⊆ G_{i} for each 1 ≤ i ≤ L. Thus,

(ν_{1} ⊗ μ_{2})(F) = ∑_{1≤i≤L} ν_{1}(G_{i})μ_{2}(H_{i}) ≥ ∑_{1≤i≤L} \overline{ν_{1}}(A)μ_{2}(H_{i})
= \overline{ν_{1}}(A) ∑_{1≤i≤L} μ_{2}(H_{i}) = \overline{ν_{1}}(A).

Hence, \overline{ν_{1} ⊗ μ_{2}}(A × Ω_{2}) ≥ \overline{ν_{1}}(A), which implies that for any A ∈ F_{1},

\overline{ν_{1}}(A) − μ_{1}(A) ≤ \overline{ν_{1} ⊗ μ_{2}}(A × Ω_{2}) − (μ_{1} ⊗ μ_{2})(A × Ω_{2}).

Therefore, d(M_{1}, N_{1}) ≤ d(M_{1} ⊗ M_{2}, N_{1} ⊗ M_{2}), and (a) follows.

The identity in (b) can be proven similarly.

By (a), (b) and the triangle inequality in Lemma 2.2,

d(M_{1} ⊗ M_{2}, N_{1} ⊗ N_{2}) ≤ d(M_{1} ⊗ M_{2}, N_{1} ⊗ M_{2}) + d(N_{1} ⊗ M_{2}, N_{1} ⊗ N_{2})
= d(M_{1}, N_{1}) + d(M_{2}, N_{2}),

and hence (c) is proven.

Corollary 3.3 For any probability spaces M_{1} and M_{2},

(M_{1})^{c} ⊗ (M_{2})^{c} ⊆ (M_{1} ⊗ M_{2})^{c}.

Proof. By Theorem 3.2 (c) and Lemma 2.6 (e),

d((M_{1})^{c} ⊗ (M_{2})^{c}, M_{1} ⊗ M_{2}) ≤ d((M_{1})^{c}, M_{1}) + d((M_{2})^{c}, M_{2}) = 0 + 0 = 0.

The result now follows by Lemma 2.6 (f).

Finally, we consider discrepancy under the operation of σ-product. We first give a formal definition for the σ-product.

Definition 3.4 Let M_{1} and M_{2} be countably additive (also called σ-additive) probability spaces.

M_{1} ⊗^{σ} M_{2} = (Ω_{1} × Ω_{2}, F_{1} ⊗^{σ} F_{2}, μ_{1} ⊗^{σ} μ_{2})

denotes the σ-product space. This is the countably additive probability space generated by the finitely additive product space M_{1} ⊗ M_{2}.

Theorem 3.5 Suppose M_{1}, N_{1}, M_{2}, and N_{2} are countably additive probability spaces. Then

d(M_{1} ⊗^{σ} M_{2}, N_{1} ⊗^{σ} N_{2}) ≤ d(M_{1}, N_{1}) + d(M_{2}, N_{2}).

Proof. Since M_{1} ⊗^{σ} M_{2} = σ(M_{1} ⊗ M_{2}) and N_{1} ⊗^{σ} N_{2} = σ(N_{1} ⊗ N_{2}), Proposition 2.8 (b) and Theorem 3.2 (c) imply that

d(M_{1} ⊗^{σ} M_{2}, N_{1} ⊗^{σ} N_{2}) ≤ d(M_{1} ⊗M_{2}, N_{1} ⊗N_{2}) ≤ d(M_{1}, N_{1})+d(M_{2}, N_{2}),

and the result follows.

4 Loeb extension and Loeb equivalence

From this section onwards, we work in an ω_{1}-saturated nonstandard universe. An internal probability space M = (Ω, F, μ) is understood to have a finitely additive (and hence ^{∗}finitely additive by transfer) measure μ on an internal algebra F of subsets of a nonempty internal set Ω with μ(Ω) = 1, unless we explicitly assume ^{∗}σ-additivity. Note that the values of μ are in ^{∗}[0, 1]. We let ^{o}M = (Ω, F,^{o}μ) be the standard probability space where (^{o}μ)(B) is the standard part of μ(B) for each B ∈ F.

The Loeb probability space generated by an internal probability space M = (Ω, F, μ) is defined as the structure L(M) = (Ω, L(F), L(μ)), where L(F) is the set of all B ⊆ Ω such that

sup{^{o}μ(A) : B ⊇ A ∈ F} = inf{^{o}μ(C) : B ⊆ C ∈ F},

and L(μ)(B) is defined as the above supremum.

Then Definition 2.5 implies that L(M) = (^{o}M)^{c}, which is a finitely additive probability space by Lemma 2.6(a). However, Loeb in [10] showed that L(M) is, in fact, a complete σ-additive probability space. Moreover, Lemma 2.6 (e) shows that d_{2}(^{o}M, L(M)) = 0.

The following definition gives a natural way of comparing two internal probability spaces on the same set.

Definition 4.1 Given two internal probability spaces

M = (Ω, F, μ) , N = (Ω, G, ν),

we say that N Loeb extends M if L(G) ⊇ L(F), and L(ν) ⊇ L(μ) (which simply means that L(ν) is an extension of L(μ)) as a function.

We say that M is Loeb equivalent to N if they generate the same Loeb space, L(M) = L(N).

Remark 4.2 (a) If N is an internal extension of M, i.e., G ⊇ F and ν ⊇ μ, then N Loeb extends M.

(b) M is Loeb equivalent to N if and only if M Loeb extends N and N Loeb extends M.

(c) The Loeb extension relation is transitive.

By transfer, the function ^{∗}d(·, ·) can be defined on the family of internal probability spaces on a nonempty internal set Ω. When there is no ambiguity we will abuse notation by writing d(M, N) instead of ^{∗}d(M, N), dropping the star.

Lemma 4.3 Let M, N be internal probability spaces.

(a) d(^{o}M,^{o}N) = ^{o}(d(M, N)).

(b) d(L(M), L(N)) = ^{o}(d(M, N)).

(c) N Loeb extends M if and only if d(M, N) ≈ 0.

(d) N Loeb extends M if and only if for every B ∈ F there exists C ∈ G such that B ⊆ C and ν(C) ≈ μ(B).

(e) M is Loeb equivalent to N if and only if d_{2}(M, N) ≈ 0.

Proof. (a) It is obvious that for any internal subset B of Ω,

inf{^{o}ν(C) : B ⊆ C ∈ G} = ^{o}(inf{ν(C) : B ⊆ C ∈ G}),

and thus ^{o}(ν̄(B)) = \overline{^{o}ν}(B). Hence

d(^{o}M,^{o}N) = sup\{\overline{^{o}ν}(B) −^{o}μ(B) : B ∈ F\} = sup\{^{o}(\overline{ν}(B) − μ(B)) : B ∈ F\}
= ^{o}(sup\{(\overline{ν}(B) − μ(B)) : B ∈ F\}) = ^{o}(d(M, N))

(b) By the definition of Loeb spaces and by Lemma 2.6(e) and (a) of this lemma,

d(L(M), L(N)) = d((^{o}M)^{c}, (^{o}N)^{c}) = d(^{o}M,^{o}N) = ^{o}(d(M, N)).

(c) N Loeb extends M if and only if L(M) ⊆ L(N), i.e., (^{o}M)^{c} ⊆ (^{o}N)^{c}. By Lemma 2.6(g) and (b) of this lemma, it is equivalent to d((^{o}M)^{c}, (^{o}N)^{c}) = o(d(M, N)) = 0.

(d) If d(M, N) ≈ 0, then Lemma 2.2 (a) implies that for each B ∈ F, ν̄(B) ≈ μ(B). Since ν̄ is an internal outer measure, there exists a set C ∈ G such that B ⊆ C and ν(C) ≈ ν̄(B), and hence ν(C) ≈ μ(B).

Next, assume that for each B ∈ F there exists C ∈ G such that B ⊆ C and ν(C) ≈ μ(B). Take any standard positive real number ϵ. Then ν̄(B) − μ(B) ≤ ν(C) − μ(B) < ϵ, which implies that d(M, N) ≤ ϵ. Since d(M, N) is a non-negative hyperreal number less than any standard positive real number, we have d(M, N) ≈ 0.

(e) follows easily from (c).

In the case that F ⊆ G, the Loeb extension relation behaves in a simple way.

Corollary 4.4 Suppose M, N are such that F ⊆ G.

(a) N Loeb extends M if and only if ν(A) ≈ μ(A) for all A ∈ F.

(b) Let P be the subspace of N with the algebra H = F. N Loeb extends M if and only if P is Loeb equivalent to M.

(c) If F = G, then N is Loeb equivalent to M if and only if N Loeb extends M.

The Loeb extension relation between stars of standard spaces also takes a simple form.

Corollary 4.5 Suppose M = ^{∗}M_{0} and N = ^{∗}N_{0} where M_{0} and N_{0} are standard probability spaces on the same set Ω_{0}.

(a) N Loeb extends M if and only if M_{0} ⊆ (N_{0})^{c}.

(b) M and N are Loeb equivalent if and only if (M_{0})^{c} = (N_{0})^{c}.

Proof. Part (a) follows from Lemma 2.6 (f) and Lemma 4.3 (b) and (c). Part (b) then follows easily.

In this paper, by a hyperfinite set we will mean an internal set whose internal cardinality is a finite or infinite hyperinteger, that is, an element of ∗ℕ. Using the next example, one can easily find different internal probability spaces M, N on a hyperfinite set Ω such that M and N are Loeb equivalent and have the same internal algebra F = G.

Example 4.6 Let Ω be a hyperfinite set. Let M = (Ω, F, μ), N = (Ω, G, ν) be internal probability spaces such that F = G = the set of all internal subsets of Ω, and the sum Σ_{ω∈ Ω} |μ({ω}) − ν({ω})| is positive infinitesimal. Then M and N are Loeb equivalent.

The Loeb extension or Loeb equivalence relation is much more difficult when it is not assumed that F ⊆ G; see the open questions at the end of this paper. The following example shows that one can indeed find different internal probability spaces M, N that are Loeb equivalent, but (1) one is a proper extension of the other, or (2) neither internal algebra is an extension of the other.

Example 4.7 Let Ω be a hyperfinite set. Suppose N = (Ω, G, ν) is an internal probability space such that G is the set of all internal subsets of Ω, and S_{1} and S_{2} are disjoint internal subsets of Ω such that ν(S_{i}) ≈ 0 and card(S_{i}) > 1 for i = 1, 2. For a given i = 1 or 2, let F_{i} be the set of all internal sets U ⊆ Ω such that U either contains S_{i} or is disjoint from S_{i}, and let M_{i} = (Ω, F_{i}, ν|_{F_{i}}). Then M_{1} and M_{2} are Loeb equivalent to each other and to their proper extension N, but F_{1} and F_{2} are not extensions of each other.

Let us now consider the space (I_{Ω},^{∗}d_{2}) of all internal probability spaces on Ω, endowed with the internal pseudo-metric ^{∗}d_{2}. Let L_{Ω} be the space of all Loeb spaces on Ω with the metric of the standard bi-discrepancy d_{2}. Let π be the Loeb operation from I_{Ω} to L_{Ω}, where π(M) = L(M) for each internal probability space M on Ω.

Theorem 4.8 The Loeb operation induces an isometry from the nonstandard hull of the space (I_{Ω},^{∗}d_{2}) of internal probability spaces to the space (L_{Ω}, d_{2}) of Loeb spaces, and (L_{Ω}, d_{2}) is a complete metric space.

Proof. By Lemma 2.2, (I_{Ω},^{∗}d_{2}) is an internal pseudo-metric space. Then ∗d_{2}(M, N) ≈ 0 defines an equivalence relation on I_{Ω}. Let Î_{Ω} be the set of equivalence classes, and let the equivalence class of M be denoted by M̂. For M̂, N̂ ∈ Î_{Ω}, define d̂_{2}(M̂, N̂) = ^{o∗}d_{2}(M, N). Since 0 ≤ ^{∗}d(M, N) ≤ 1, d̂_{2}(M̂, N̂) is finite for all M, N. It is well known (see, for example, [11]) that the nonstandard hull of a metric space is a complete metric space. The same proof shows that the nonstandard hull of an internal pseudo-metric space is also a complete metric space, and hence (Î_{Ω}, d̂_{2}) is a complete metric space.

Define a mapping π̂ from Î_{Ω} to L_{Ω} by letting π̂(M̂) = π(M). By Lemma 4.3(d), π̂ is well-defined and a bijection from Î_{Ω} to L_{Ω}. By Lemma 4.3(b), we have

d̂_{2}(M̂, N̂) = ^{o∗}d_{2}(M, N) = d_{2}(L(M), L(N)) = d_{2}(π̂(M̂), π̂(N̂)),

and hence π̂ is an isometry. Since (Î_{Ω}, d̂_{2}) is a complete metric space, so is (L_{Ω}, d_{2}).

5 Uniqueness of the Loeb product

We show in this section that for any two given Loeb spaces, their Loeb product (or the Loeb σ-product) is uniquely defined. We begin with a formal definition of the internal product.

Definition 5.1 Given internal probability spaces M_{1} and M_{2}, M_{1} ⊗ M_{2} will denote the internal product space, where F_{1} ⊗ F_{2} is the set of all hyperfinite unions of rectangles A_{1} × A_{2}, and (μ_{1} ⊗ μ_{2})(A_{1} × A_{2}) = μ_{1}(A_{1})μ_{2}(A_{2}).

The following theorem shows that the Loeb product is well defined in terms of factor Loeb spaces. That is, the Loeb product space L(M_{1} ⊗ M_{2}) depends only on the Loeb spaces L(M_{1}) and L(M_{2}), and not on the internal probability spaces M_{1} and M_{2} that generate the Loeb spaces L(M_{1}) and L(M_{2}).

Theorem 5.2 (a) If N_{1} Loeb extends M_{1} and N_{2} Loeb extends M_{2}, then N_{1} ⊗ N_{2} Loeb extends M_{1} ⊗ M_{2}.

(b) If M_{1} is Loeb equivalent to N_{1} and M_{2} is Loeb equivalent to N_{2}, then M_{1} ⊗ M_{2} is Loeb equivalent to N_{1} ⊗ N_{2}.

Proof. (a) By Lemma 4.3(c), we have d(M_{1}, N_{1}) ≈ 0 and d(M_{2}, N_{2}) ≈ 0. It follows from Lemma 2.2(a) and Theorem 3.2(c) that d(M_{1} ⊗ M_{2}, N_{1} ⊗ N_{2}) ≈ 0. By Lemma 4.3(c) again, N_{1} ⊗ N_{2} Loeb extends M_{1} ⊗ M_{2}.

(b) follows from Remark 4.2.

We now turn to σ-additive probability spaces.

The following example shows that an internal ^{∗}σ-additive probability space N need not be Loeb equivalent to an internal subspace that ^{∗}σgenerates it.

Example 5.3 Let N be the star of Lebesgue measure on [0, 1], and let M be the star of the subspace of finite unions of half-open intervals in [0, 1]. Then N is not Loeb equivalent to M.

Proof. The set of hyperrational numbers in ^{∗}[0, 1] has measure 0 in N but any superset in F has measure 1 in M.

For a pair of internal ^{∗}σ-additive probability spaces, there is a second notion of product to be considered.

Definition 5.4 Given two internal ^{∗}σ-additive probability spaces M_{1} and M_{2}, M_{1} ⊗^{σ} M_{2} will denote the internal ^{∗}σ-additive probability space generated by M_{1} ⊗ M_{2}

The next theorem shows that the Loeb σ-product is well-defined.

Theorem 5.5 Suppose M_{1}, N_{1}, M_{2}, and N_{2} are ^{∗}σ-additive internal probability spaces.

(a) If N_{1} Loeb extends M_{1} and N_{2} Loeb extends M_{2}, then N_{1} ⊗^{σ} N_{2} Loeb extends M_{1} ⊗^{σ} M_{2}.

(b) If N_{1} is Loeb equivalent to M_{1} and N_{2} is Loeb equivalent to M_{2}, then N_{1} ⊗^{σ} N_{2} is Loeb equivalent to M_{1} ⊗^{σ} M_{2}.

Proof. The procedure for proving this theorem is exactly the same as that of Theorem 5.2 except using Theorem 3.5 instead of Theorem 3.2(c).

6 Hyperfinite probability spaces

The Loeb σ-product is especially simple when one of the factor spaces is hyperfinite. A hyperfinite probability space is an internal probability space M such that F is a hyperfinite set. (Note that we do not require Ω to be hyperfinite.)

Remark 6.1 If at least one of the internal ^{∗}σ-additive probability spaces M_{1} and M_{2} is hyperfinite, then the internal product space is the same as the internal σ-product space.

Corollary 6.2 Suppose that M, N are internal^{∗}σ-additive probability spaces and either M or N is Loeb equivalent to a hyperfinite probability space. Then M ⊗^{σ} N is Loeb equivalent to M ⊗ N.

Proof. By Theorems 5.2 and 5.5 and Remark 6.1.

In view of Corollary 6.2, it is natural to ask which internal spaces are Loeb equivalent to hyperfinite probability spaces. The rest of this section deals with that question.

Proposition 6.3 Suppose M and N are internal probability spaces, and M is hyperfinite.

(a) If N Loeb extends M, then N has a hyperfinite subspace P that Loeb extends M.

(b) If N is Loeb equivalent to M, then N has a hyperfinite subspace that is Loeb equivalent to M.

Proof. We prove (a) first. By Lemma 4.3, for each B ∈ F there is a set F (B) = C ∈ G such that B ⊆ C and ν(C) ≈ μ(B). The function F may be taken to be internal. Let H be the hyperfinite subalgebra of N generated by the range of F, ρ = ν|_{H} and P = (Ω, H, ρ). Then by Lemma 4.3, P Loeb extends M.

For (b), note that L(M) ⊆ L(P) ⊆ L(N) ⊆ L(M). Hence L(M) = L(P).

An atom of M is a set B ∈ F such that B ≠ ∅ but there is no A ∈ F such that ∅ ⊂ A ⊂ B. Note that this is the notion of an atom in the Boolean algebra on F, which is different from the notion of an atom in the measure algebra on F modulo the null sets.

Proposition 6.4 Suppose N is an internal probability space. Then the following are equivalent.

(a) N is Loeb equivalent to some hyperfinite probability space.

(b) There is a hyperfinite set S of atoms of N with ν(⋃_{S∈S} S) ≈ 1.

Proof. Assume (a). By Proposition 6.3, N has a Loeb equivalent hyperfinite subspace M. Let S be the set of all atoms of M that are also atoms of N. We claim that ν(⋃_{S∈S} S) ≈ 1, so (b) holds. Suppose not. Let D = Ω \ ⋃_{S∈S} S. Since M is hyperfinite, D is the union of a hyperfinite set U of atoms of M, so D ∈ F. No element of U is an atom of N, so there is a set E ∈ G such that E ⊆ D, and for each set H ∈ U, ∅ ⊂ (E ∩ H) ⊂ H. Then whenever A, C ∈ F and A ⊆ E ⊆ C, we must have A = ∅ and D ⊆ C, so ^{o}ν(C \ A) ≥ ^{o}ν(D) > 0. This means that E ∉ L(F), which contradicts the assumption that M is Loeb equivalent to N. Hence it proves the claim.

Now assume (b). As in the last paragraph, let D = Ω \ ⋃_{S∈S} S. Let M be the hyperfinite subspace generated by the partition S ∪ {D}. Then F ⊆ G. Let B ∈ G. Then B = (B ∩ D) ∪ ⋃_{S∈S}(B ∩ S). For each atom E of N, either E ⊆ B or E ∩ B = ∅. Hence B = (B ∩ D) ∪ ⋃_{E∈S,E⊆B} E. Let A = ⋃_{E∈S,E⊆B} E and C = A ∪ D. Then B = A ∪ (B ∩ D) ⊆ C with ν(C \ A) = ν(D) ≈ 0. Hence B ∈ L(F). By Lemma 4.3, M is Loeb equivalent to N and (a) holds.

Corollary 6.5 If N is an internal probability space with no atoms of internally positive measure, then no hyperfinite probability space M is Loeb equivalent to N.

Example 6.6 Let N be the star of Lebesgue measure on [0, 1]. Then the above corollary implies that no hyperfinite probability space M is Loeb equivalent to N.

7 Some questions

We include a few open questions in this final section. In each question, M and N are assumed to be internal probability spaces.

Question 7.1 Suppose N Loeb extends M. Must N have an internal subspace that is Loeb equivalent to M? What if M is assumed to be hyperfinite?

Assuming a negative answer to Question 7.1, one can ask the following.

Question 7.2 Suppose N Loeb extends M and N is hyperfinite. Must M be Loeb equivalent to a hyperfinite probability space?

Question 7.3 Suppose M is Loeb equivalent to N, and let H be the internal set algebra generated by F ∪G. Must there be an internal probability measure λ on H such that M is Loeb equivalent to (Ω, H, λ)? What if M and N are assumed to be hyperfinite?

We conclude with one more open question.

Question 7.4 Let M and N be internal ^{∗}σ-additive probability spaces. Must the product M ⊗ N be Loeb equivalent to the σ-product M ⊗^{σ} N?

In the case that at least one of the spaces M, N is Loeb equivalent to a hyperfinite probability space, the answer is “yes” by Corollary 6.2.

In the case that the spaces M and N are stars of standard spaces, M = ∗M_{0} and N = ^{∗}N_{0}, by definition we have

M ⊗ N = ^{∗}(M_{0} ⊗ N_{0}), M ⊗^{σ} N = ^{∗}(M_{0} ⊗^{σ} N_{0}).

Therefore, in view of Corollary 4.5, Question 7.4 takes a standard form.

Let M_{0} and N_{0} be standard countably additive probability spaces. Must (M_{0} ⊗ N_{0})^{c} = (M_{0} ⊗^{σ} N_{0})^{c}?

We have not been able to find an answer to this question in the literature, but we conjecture that the answer is “no” in the case that M_{0} and N_{0} are both equal to the unit interval [0, 1] with Lebesgue measure.

References

[1] S. Albeverio, J. E. Fenstad, R. Hoegh-Krohn, and T. L. Lindstrom, Nonstandard Methods in Stochastic Analysis and Mathematical Physics, Academic Press, Orlando, Florida, 1986.

[2] R. M. Anderson, A nonstandard representation of Brownian motion and Itô-integration, Israel J. Math. 25 (1976), 15-46.

[3] J. Berger, H. Osswald, Y. N. Sun, J. L. Wu, On nonstandard product measure spaces, Illinois J. Math. 46 (2002), 319-330.

[4] N. J. Cutland and S. A. Ng, A nonstandard approach to the Malliavin calculus, in Proc. of the Blaubeuren Conf., Univ. Tübingen, Walter de Gruyter, 1995.

[5] J. L. Doob, Measure Theory, Springer-Verlag, 1994.

[6] S. Fajardo and H. J. Keisler, Neometric spaces, Advances in Math. 118 (1996), 134-175.

[7] S. Fajardo and H. J. Keisler, Existence theorems in probability theory, Advances in Math. 120 (1996), 191-257.

[8] S. Fajardo and H. J. Keisler, Model Theory of Stochastic Processes, Lecture Notes in Logic, Association for Symbolic Logic (in press).

[9] H. J. Keisler, Hyperfinite model theory, in Logic Colloquium 76 (R. O. Gandy and J. M. E. Hyland eds.) North-Holland, Amsterdam, 1977.

[10] P. A. Loeb, Conversion from nonstandard to standard measure spaces and applications in probability theory, Trans. Amer. Math. Soc. 211 (1975), 113-122.

[11] P. A. Loeb and Manfred Wolff, eds. Nonstandard Analysis for the Working Mathematician, Kluwer Academic Publishers, Dordrecht, 2000.

[12] H. L. Royden, Real Analysis, Macmillan, New York, 1968.

[13] Y. N. Sun, A theory of hyperfinite processes: the complete removal of individual uncertainty via exact LLN, J. Math. Econ. 29 (1998), 419-503.

[14] Y. N. Sun, The almost equivalence of pairwise and mutual independence and the duality with exchangeability, Probability Theory and Related Fields 112 (1998), 425-456.

H. Jerome Keisler

Department of Mathematics, University of Wisconsin-Madison, 480 Lincoln Drive, Madison, Wisconsin 53706-1388, USA.

e-mail: keisler@math.wisc.edu

Yeneng Sun

Institute for Mathematical Sciences, National University of Singapore, 3 Prince George’s Park, Singapore 118402, Republic of Singapore.

and

Department of Mathematics, National University of Singapore, 2 Science Drive 2, Singapore 117543, Republic of Singapore.

e-mail: matsuny@nus.edu.sg