Recover the full experiment from a partial view

4 Robust reconstruction of joint experiments

4.1 The information lost by deleting a view

The obstruction concerns exact target selection, not the existence of stable measures of joint information. A different question is available: after retaining only part of an observation, can one common reconstruction procedure reproduce the full experiment for every unknown preparation?

Let $\Theta$ be a nonempty finite preparation set. An experiment has laws $P_{J,\theta}$ on a finite joint alphabet $\mathcal Y_J$. A retained view is the deterministic projection $\pi_L:\mathcal Y_J\to\mathcal Y_L$, with image alphabet $\mathcal Y_L$ and marginal laws $P_{L,\theta}$.

Definition 4.1 (Directed reconstruction deficiency)

The loss of reconstructing $J$ from $L$ is

$$\delta_P(J\mid L)=\min_{G:\mathcal Y_L\rightsquigarrow\mathcal Y_J} \max_{\theta\in\Theta}\TV(P_{J,\theta},P_{L,\theta}G).$$(4.1)

The stochastic row kernel $G$ is the same for all unknown $\theta$. It may use a declared public context, but not the unknown source label.

This is directed statistical deficiency in a target-first, half-$L^1$ convention, not a new information measure. Statistical comparison has its classical Blackwell/Le Cam lineage [6, 16]; deficiency has also been applied directly to feature quality [24]. Argument order, row versus column kernels, and the factor of two in variational divergence must be translated when comparing formulas.

The decoder reproduces laws. In particular it may resample even the retained coordinates. It need not recover the same realized event, preserve a particular archive carrier, respect a deadline, or be available as a native physical operation. Those stronger requirements belong to the execution contract in Section 5, Section 6. They are not consequences of Equation 4.1.

Proposition 4.2 (Finite comparison bounds)

The minimum in Equation 4.1 exists, lies in $[0,1]$, and is a rational linear-program value for rational finite data. For any $\theta,\theta'$,

$$\delta_P(J\mid L)\ge \tfrac12\bigl[ \TV(P_{J,\theta},P_{J,\theta'})- \TV(P_{L,\theta},P_{L,\theta'})\bigr]_+.$$(4.2)

For experiments $\mathsf E,\mathsf F,\mathsf H$ on the same parameter set,

$$\delta(\mathsf E\mid\mathsf H) \le \delta(\mathsf E\mid\mathsf F)+\delta(\mathsf F\mid\mathsf H).$$(4.3)

More informative retained observations cannot increase reconstruction loss, nor can postprocessing the target. If $\max_\theta\TV(P_{J,\theta},\widehat P_{J,\theta})\le\epsilon$, with the same projection and preparation labels, then

$$\bigl|\delta_P(J\mid L)-\delta_{\widehat P}(J\mid L)\bigr|\le2\epsilon.$$(4.4)

All these statements preserve the nominated experiment and decoder permissions.

Proof

Finite stochastic matrices form a compact polytope and the objective is continuous. The absolute-value epigraph gives a finite linear program with rational coefficients; a rational optimum can be chosen. Insert $P_{L,\theta}G$ and $P_{L,\theta'}G$ between the two target laws and use TV contraction under $G$ to obtain Equation 4.2. Composing two reconstruction kernels proves Equation 4.3; the same argument gives the monotonicities. For a fixed $G$, replace both target and input laws by their hatted versions. The first replacement costs at most $\epsilon$, and the second at most $\epsilon$ after contraction. Take infima in both directions to obtain Equation 4.4.

□

For a decision loss valued in $[0,1]$, any decision procedure based on $J$ can be emulated from $L$, using an optimal $G$ followed by that procedure, with additional risk at most $\delta_P(J\mid L)$ for every preparation and hence every prior. This forward implication follows from the TV bound on bounded expectations. No full statistical-comparison converse is needed below.

One may record the entire table $\delta_P(J\mid L)$. A derived deletion diagnostic is

where the empty view is a one-point alphabet. Positive $u_P(J)$ means that each one-coordinate deletion loses experiment information. It is not an additive information decomposition, a physical edge attribution, or an exhaustive notion of integration. In particular, it should not be identified with a partial-information-decomposition atom; unique-information constructions answer their own specified comparison questions [5].

4.2 Genuine informational padding

Proposition 4.3 (Zero loss and common-conditional padding)

For a deterministic view $Y_L=\pi_L(Y_J)$, $\delta_P(J\mid L)=0$ if and only if there is one conditional kernel $A(dy_J\mid y_L)$, supported on $\pi_L(y_J)=y_L$, such that

$$P_{J,\theta}=P_{L,\theta}A\qquad\text{for every }\theta.$$(4.6)

In particular, if

$$P_\theta(y,z)=P_\theta(y)A(z\mid y)$$(4.7)

with the same conditional kernel in every preparation, then

$$\delta((Y,Z)\mid Y)=0, \qquad \delta((Y,Z)\mid L)=\delta(Y\mid L)$$(4.8)

for any common retained experiment $L$.

Proof

A conditional reconstruction immediately proves the forward zero-loss claim. Conversely, an optimal zero-loss kernel reproduces the full experiment, though it need not initially preserve the view. Put a strictly positive prior on the finite $\Theta$. The reconstructed Markov chain $\Theta\to Y_L\to Y_J^*$ has the original $(\Theta,Y_J)$ law at its endpoints. Data processing gives $I(\Theta;Y_J)\le I(\Theta;Y_L)$. Since $Y_L$ is originally a deterministic function of $Y_J$, the reverse inequality holds. Thus $I(\Theta;Y_J\mid Y_L)=0$ in the original experiment. Its conditional law supplies $A$, common to every preparation on the nonnull support. Values null for every preparation can be extended arbitrarily within their nonempty projection fibres. For padding invariance, reconstruct $Y$ and append $A$ for one bound, and project away $Z$ for the other.

□

Independent noise and a copy of an already retained record satisfy this informational criterion. A coordinate whose marginal merely ignores the source need not satisfy it. Nor does snapshot redundancy imply that the coordinate is causally idle later: an otherwise redundant backup may become indispensable after the original register is overwritten.

4.3 The exact masked-encoder loss

The two rows in Equation 3.3, Equation 3.4 remain valid statistical experiments on the extended interval $0\le\eps\le1$. Only the original graph argument was restricted to $0\le\eps\le\tfrac12$.

Theorem 4.4 (Exact reconstruction loss of the masked encoder)

For the extended experiment,

$$\begin{align}\delta_P(12\mid1)&=\max\left\{\tfrac12-\eps,\frac{1-\eps}{4}\right\},\tag{4.9}\\ \delta_P(12\mid2)&=\tfrac12\max\{\eps,1-\eps\}. \tag{4.10}\end{align}$$

The first formula changes branch at $\eps=\tfrac13$. Both losses are continuous and have explicit optimal reconstruction kernels.

Proof

For $\eps\le\tfrac12$, the joint contrast is $1-\eps$ and the first-coordinate contrast is $\eps$, so Equation 4.2 gives $\tfrac12-\eps$. For a second bound on the whole interval, put $q_+=(1+\eps)/2$, $q_-=(1-\eps)/2$. The reconstruction rows obey

$$Q_0=q_+G_0+q_-G_1,\qquad Q_1=q_-G_0+q_+G_1.$$

Therefore $Q_1(00)\ge(q_-/q_+)Q_0(00)$. If both reconstruction errors are at most $t$, the target masses give $Q_0(00)\ge\tfrac12-t$ and $Q_1(00)\le t$. Hence $t\ge(1-\eps)/4$. The two kernels in Appendix A attain the larger lower bound on the respective intervals. For output 2, the retained law is the same fair bit under both sources. Every decoder gives one common target law; the midpoint of the two targets attains half their TV diameter. Direct subtraction gives diameter $\max\{\eps,1-\eps\}$.

□

Figure 1 contrasts this continuous information loss with the inherited support selection. The new diagnostic does not reproduce the old graph while somehow removing its discontinuity: it deliberately asks a different, quantitative question.

4.4 Why a single TV increment is insufficient

Proposition 4.5 (Marginal independence and scalar increments)

No rule can discard every marginally source-independent coordinate while retaining all joint-only information. For a fixed contrast, or a common family of maximized contrasts,

$$\sigma(J)=d(J)-\max_{L\subsetneq J}d(L)\ge0,$$

but $\sigma(J)=0$ does not imply that some proper view reconstructs $J$ exactly.

Proof

Let $(Y,Z)=(\xi\oplus b,\xi)$ with $\xi$ fair. Both marginals are independent of $b$, while $Y\oplus Z=b$. Thus marginal independence cannot identify irrelevant padding. Nonnegativity of $\sigma$ follows from contraction under projection, including after taking suprema over one common contrast family. At $\eps=1/2$, the masked experiment has full and best-singleton contrasts both $1/2$, yet Equation 4.9 gives $\delta_P(12\mid1)=1/8$ and Equation 4.10 gives $\delta_P(12\mid2)=1/4$.

□

There is a concrete decision witness. With source prior $\Prb(b=0)=3/4$, optimal binary classification has Bayes error $1/8$ from the pair and $1/4$ from its first coordinate. The calculation is in Appendix A. Equal-prior pairwise discrimination is only one decision problem, so preserving its best score does not preserve the full experiment.

Redundant representations can also overlap. In $(Y_1,Y_2,Y_3)=(\xi,\xi\oplus b,\xi)$, the pairs $12$ and $23$ each have deletion diagnostic $1/2$, while $u_P(13)=u_P(123)=0$. The full triple has not lost the bit: it contains redundant ways to retain it. A disjoint partition of informative representations is not implied by statistical sufficiency.