Every discrete, finite image is uniquely determined by its dipole histogram

A finite image I is a function assigning colors to a finite, rectangular array of discrete pixels. Thus, the information directly encoded by I is purely locational. Such locational information is of little visual use in itself: perception of visual structure requires extraction of relational image information. A very elementary form of relational information about I is provided by its dipole histogram DI. A dipole is a triple, ((dx, dy), alpha, beta), with dx and dy horizontal and vertical, integer-valued displacements, and alpha and beta colors. For any such dipole, DI((dx, dy), alpha, beta) gives the number of pixel pairs ((x1, y1), (x2, y2)) of I such that I[x1, y1] = alpha, I[x2, y2] = beta, and, (x2, y2) - (x1, y1) = (dx, dy). Note that DI explicitly encodes no locational information. Although DI is uniquely determined by (and easily constructed from) I, it is not obvious that I is uniquely determined by DI. Here we prove that any finite image I is uniquely determined by its dipole histogram, DI. Two proofs are given; both are constructive, i.e. provide algorithms for reconstructing I from DI. In addition, a proof is given that any finite, two-dimensional image I can be constructed using only the shorter dipoles of I: those dipoles ((dx, dy), alpha, beta) that have magnitude of dx < or = ceil((# columns in I)/2) and magnitude of dy < or = ceil((# rows in I)/2), where ceil(x) denotes the greatest integer < or = x.


Introduction
Formally, a finite image I is a function that assigns colors (coded here by real numbers) to a finite, rectangular array of locations in space (coded here by ordered pairs of integers).Thus the elementary information explicitly represented in any image is purely locational.In itself, however, such locational information is of little visual import.Sensitivity to any sort of visual structure or pattern requires extraction of relational image information -information about the constellations of colors occurring jointly throughout the stimulus field.
A data structure easily computable from I that explicitly encodes the most elementary relational information in I is the dipole histogram D I of I.A dipole is a triple, ((d x , d y ), h, i), with d x and d y horizontal and vertical, integer-valued displacements, and h and i real numbers (color values).For any such dipole, D I ((d x , d y ), h, i) gives the number of pixel pairs ((x 1 , y 1 ), (x 2 , y 2 )) of I such that I[x 1 , y 1 ]= h, I[x 2 , y 2 ]= i, and, (x 2 , y 2 )− (x 1 , y 1 )= (d x , d y ).Note that D I explicitly encodes no locational information.
Although D I is uniquely determined (and easily constructed from) I, it is not immediately clear whether I is uniquely determined by D I .Indeed, previous research in the field of texture perception has tended to suggest the contrary.Julesz (1962) is primarily responsible for awakening interest in dipoles.Julesz, Gilbert, Shepp and Frisch (1973) conjectured that any textures I and J with the same 'second-order statistics' would fail to be preattentively discriminable.(Throughout this paper, we use boldface letters to denote random images -i.e.images whose pixel values should be construed not as fixed colors, but rather as jointly distributed random colors.)Many voices have entered into this discussion (Julesz, 1962;Julesz et al., 1973;Julesz, Gilbert & Victor, 1978;Victor & Brodie, 1978;Gilbert, 1980;Julesz, 1981;Diaconis & Freedman, 1981;Gagalowicz, 1981;Julesz & Bergen, 1983;Yellott, 1993;Victor, 1994;Victor, Conte, Purpura & Katz, 1995).There are at least two possible interpretations that can be given to the term 'second-order statistics,' one probabilistic, the other deterministic.Under the deterministic interpretation, images are said to have 'identical second-order statistics' if and only if their dipole histograms are identical.The probabilistic interpretation treats images as random objects and their pixel values as jointly distributed random variables.Two such random images I and J are said to have probabilistically identical second-order statistics if the following condition holds: for any two pixels (x 1 , y 1 ) and (x 2 , y 2 ), the joint distribution of the pair (I[x 1 , y 1 ], I[x 2 , y 2 ]) of random variables is identical to the joint distribution of the pair (J[x 1 , y 1 ], J[x 2 , y 2 ]).Many pairs of textures I, J with probabilistically identical second-order statistics have been demonstrated which segregate preattentively.For example, the 'even', 'odd' and 'coinflip' textures (Julesz et al., 1978) all have probabilistically identical second-order statistics, yet are easily discriminated.
Interest in the dipole histograms of specific texture images (i.e.deterministic, second-order statistics) began with the work of Gagalowicz (1981), who argued that the even and odd textures do not provide a convincing test of the Julesz hypothesis.He observed first that texture segregation is induced by the single stimulus that one is viewing at the moment; thus, the process that produces this segregation must be based on properties inherent in that individual stimulus (i.e.not on properties of the probabilistic ensemble of stimuli).Thus, a pair of discriminable textures, I and J, would constitute a counterexample to the Julesz conjecture only if the dipole statistics in any given instances of I and J were approximately equal.As Gagalowicz noted, the odd and even textures fail this test for the following reason: the dipole histogram of any given finite patch of either the odd or even texture tends to deviate significantly from the expectation of its dipole histogram.Therefore, with high probability, any given patch of the even texture will have a dipole histogram dramatically different from that of a corresponding patch of odd texture.
Accordingly, Gagalowicz set out to construct more compelling counterexamples to the Julesz conjecture: pairs of visually distinct, spatially homogeneous textures with nearly identical dipole histograms.The examples he produced were readily discriminable, spatially homogeneous, and had very similar dipole histograms (at least for the limited set of dipoles checked by Gagalowicz).Gagalowicz' success at producing pairs of preattentively discriminable textures with approximately equal dipole histograms might be taken to suggest that distinct images P and Q could be produced with identical dipole histograms.However, this question was not explicitly addressed in his paper.Yellott (1993) extended the argument of Gagalowicz, pointing out in addition that if distinct, finite images I and J have deterministically identical third-order statistics, then I and J must be identical (up to a spatial translation).He focused on binary images (coding white by 0 and black by 1), and provided a method for constructing distinct pairs, I and J, of binary images with identical black-to-black dipole histograms (i.e.such that D I ((d x , d y ), 1, 1)=D J ((d x , d y ), 1, 1) for all displacements (d x , d y )).(Any such images, I and J, are easily seen to have identical autocorrelation functions.) It is easy to check, however, that the examples offered by Yellott have dipole histograms that differ in their white-to-black, black-to-white and white-to-white histograms.Thus, although Yellott's emphasis on the third-order statistics of an image might be taken to suggest that an image is not uniquely determined by its dipole histogram, Yellott's examples did not establish this.
And indeed they could not because, as we show here, any discrete, finite image is uniquely determined by its dipole histogram.D I can thus be viewed as an alternative representation of I.The proof is simple and generalizes in an obvious way to images of arbitrarily high dimensionality (e.g. to motion pictures).
Our interest in the question addressed here derives from the following reflections.As Julesz was quick to realize, the first-order statistics (i.e. the color histogram) of an image provide no purchase in grasping visual structure.After all, first-order statistics are invariant with respect to arbitrary permutations of pixel values.Vision, however, is concerned primarily with interpreting relationships between colors across space and time.Thus, the dipole emerges as a minimal element of visual structure.This suggests that the dipole histogram D I may be a useful data object for purposes of analyzing the visual meaning of an image I.For example, Doner (1999) has argued that human sensitivity to some aspects of spatial pattern is reflected by various measures of dipole histogram entropy.Nonetheless, the dipole histogram D I would be of only limited value in analyzing image structure if I were not uniquely determined by D I .On the other hand, if it is true that I is uniquely determined by D I , then one can dispense altogether with the locational information encoded explicitly by I, and focus exclusively on the relational information embodied by D I .

Dipoles and the dipole histogram of an image
Here we use the term image to refer to a discrete, finite image.Throughout the discussion it will be convenient to fix integers N, M and let X={0, 1,…, N} and Y= {0, 1 ,…, M}.Then, a one-dimensional image is simply a function I:X R. For any xX, I[x] de-notes the value assigned by I to x.A two-dimensional image is a function I mapping the Cartesian product X × Y into R.For any (x, y) X ×Y, I[x, y] denotes the value assigned by I to (x, y).Regardless of whether I is one-or two-dimensional, points in the domain of I will be called pixels.
A one-dimensional dipole is a triple, (d, h, i), with d an integer-valued displacement, and h and i real numbers.We shall say that a (one-dimensional) dipole (d, h, i) bridges a pair (x 1 , x 2 ) of pixels in and I[x 2 ]= i.The dipole histogram D I assigns to each dipole (d, h, i) the number of distinct pairs in I bridged by (d, h, i).Thus, if D I (4, 0, 2) =16, then there are 16 pixels x of I such that I[x] =0, and I[x +4] =2.(Note that this definition of D I requires that the image I be finite.) A two-dimensional dipole is a triple, (d, h, i), with d = (d x , d y ), for d x a horizontal, and d y a vertical, integer-valued displacement, and h and i real numbers.For any two-dimensional image As in the one-dimensional case, the dipole histogram D I assigns to each dipole (d, h, i) the number of pixel pairs in I bridged by (d, h, i).
For any dipole (d, h, i), regardless of the dipole's dimensionality, d is called the dipole's displacement; h is called its h-6alue, and i is called its i-6alue.
We begin with the one-dimensional case, giving two proofs.Then we show that the two-dimensional version of the proposition follows as a corollary from the one-dimensional case.

Proposition
Any finite one-dimensional image is uniquely determined by its dipole histogram.
Proof 1.Let I be a one-dimensional image.It will be convenient to write h,i to indicate a sum ranging over all pairs of pixel values of I. (Note that this summation is over a finite number of elements since I is finite in size.)We use this notation to define: and Now, for k= 0, 1 ,…, N, we have: (3) Thus, immediately: and with I[0] in hand, we recursively obtain I[k] for k= 1, 2 ,…, N as follows: In proof 2 of proposition 2.1, we show that the information in D I is redundant for purposes of constructing I. Specifically (writing ceil(6) for the smallest integer greater than or equal to real number 6, and floor(6) for the greatest integer less than or equal to 6), we show that I can be computed from the restriction of D I to either (i) the subset of dipoles with displacements] floor(N + 1/2), or (ii) the subset of dipoles with displacements5 ceil((N + 1)/2).
Proof 2. For any given displacement d, there are exactly N−d+ 1 dipoles of I with displacement d.These dipoles bridge the pixels (0, d), (1, d+ 1) ,…, (N− d, N).Note that the h-values of these dipoles are assigned to pixels 0, 1 ,…, N− d, and the i-values are assigned to pixels d, 1 ,…, N. Now define: and Observe that: and Moreover, for d=1 ,…, N, I[d] can be recovered in either of two ways: (10) also, however, Hence, for k =floor(N/2), and Note that in the sequence of equations above, each of I's pixel values is computed with reference solely to D I ; moreover, each equation involves only dipoles with displacements] N −floor(N/2)= floor((N +1)/2).This proves that I can be computed from the restriction of D I to the subset of dipoles with displacements greater than or equal to floor((N +1)/2).
Proofs 1 and 2 of proposition 2.1 are instructive in different ways.Proof 2 makes clear that the image I can be constructed from either the 'long' dipoles alone or the 'short' dipoles alone.However, proof 2 is curious in the following respect: although we succeed in reconstructing I from the dipole histogram D I , the statistics used to accomplish this goal (the functions A and B) make no use of the relational information contained in any given dipole.

A[d] depends only on the h-values in the dipoles with displacement d, and B[d] depends only on the i-values.
By contrast, proof 1 shows that I can be constructed using autocorrelation-like statistics that make full use of the information in the dipoles they depend on.Indeed, C 11 is precisely the autocorrelation function of I, whose Fourier transform is the power spectrum of I. Since combining C 10 with C 11 uniquely determines I, and thus its phase spectrum as well as its power spectrum, it is natural to wonder about the relationship between the phase spectrum and C 10 alone.Thinking along Fourier lines, one might conjecture that C 10 is unconstrained by C 11 in the same way as the phase spectrum of a function is not constrained by its power spectrum.However, this is not so, as one can easily see by considering a binary image I (with each pixel value either 0 or 1) for which in which case we must have C 10 [N] =0.This shows that C 10 is partially constrained by C 11 .

Corollary
Any finite, two-dimensional image I is uniquely determined by its dipole histogram.
Proof.Let maxd x be the maximal horizontal displacement in any dipole of I. Then the number of columns in I is C= maxd x + 1.From I we can produce a one-dimensional image J I by concatenating I's rows.Thus, any pixel (x, y) of I spawns pixel x+Cy of J I , from which it follows that any dipole ((d x , d y ), h, i) of I spawns a dipole (Cd y + d x , h, i) of J I .Of course, the correspondence, here, is not one-to-one.For example, for any fixed h and i, the dipoles ((1, 0), h, i) and ((−maxd x , 1), h, i) each spawn a dipole (1, h, i) of J I .For our purposes, however, the important point is that from the dipole histogram of J I , we can generate the entire set of dipoles of J I , and hence derive the dipole histogram of J I .It follows that by referring only to the dipole histogram of I we can determine (1) the number C of columns in I, and (2) the dipole histogram of J I .By proposition 2.1, J I is uniquely determined by its dipole histogram.It is obvious, however, that J I uniquely determines I. Specifically, the first C values of J I compose the top row of I; the second C values of J I compose the second row of I, etc.
The reader will note that the strategy used in the proof of corollary 2.2 can easily be applied to show that an image of any finite dimension is uniquely determined by its dipole histogram.It is also possible to prove corollary 2.2 using algorithms, analogous to those used in the proofs of proposition 2.1, that reconstruct I's pixel values row by row and column by column.Such a proof is given in Appendix A.
The proof in Appendix A is instructive in another way.In the second of the two proofs of proposition 2.1, we showed that a finite, one-dimensional image comprising N+ 1 pixels could be constructed using only those of its dipoles with displacements shorter than or equal to ceil((N + 1)/2).One might wonder how this result generalizes to the two-dimensional case.The proof given in Appendix A shows that any finite, discrete, two-dimensional image I comprising M+ 1 rows and N+1 columns can be constructed using only those of its 'short' dipoles: i.e. those dipoles (d y , d x ) with d y 5 ceil((M + 1)/2) and d x 5 ceil((N + 1)/2).

What light is shed by the current result on models of texture processing?
Perhaps not much.Recent models of texture processing (e.g.Malik & Perona, 1990;Bergen & Landy, 1991;Landy & Bergen, 1991;Chubb, Econopouly & Landy, 1994;Graham, 1994) rely on spatially local, non-linear image transformations to reveal perceptual differences between textures.Under such models, texture regions are discriminable if some of these hypothesized transformations yield responses whose between-region variance is significantly greater than their within-region variance.Even if these transformations are modeled simply as linear filters followed by pointwise non-linearities, it is easy to construct transformations sensitive to image statistics of order higher than 2. Specifically, this will generally be the case if the receptive field used by the linear filter gives non-zero weight to more than two points in the image, and the subsequent pointwise non-linearity is anything other than a squaring transformation.Thus, current models of texture processing give us no reason to expect second-order statistics to be decisive in determining the perceptual impact exerted by a patch of texture.
These remarks, however, should not be taken to imply that isodipole textures (textures I, J such that ) are of no use in the study of visual perception.On the contrary because isodipole textures elude simple models based on texture energy, they enable one to isolate higher-order non-linear processes.Indeed, over the past decade, such textures have proven to be extremely useful psychophysical tools in analyzing specific spatial non-linearities mediating texture discrimination (Victor & Conte, 1989, 1991, 1996).

Why has this not been noticed before?
Given the simplicity of proposition 2.1, it is surprising that it has not been previously observed.Although we cannot speak for other researchers in the field, we can testify that until recently, we had tacitly assumed that proposition 2.1 is false.Perhaps others shared this misconception.
Our personal confusion can be traced to the earliest attempts to construct textures with probabilistically identical second-order statistics.The first construction methods (e.g. the '4-dot' method, Julesz et al., 1973) used binary textures comprising sparse groupings of dots on a homogeneous background.Let us imagine that dots are black on a white background (as was true in many of the figures from the early papers by Julesz and collaborators).By accentuating the figure/ground relationship of dots vs. background, such textures gave us the impression that the spatial relationships between the dots (as reflected solely by the black-to-black dipoles) uniquely determined the perceptual impact produced by the texture.
The impression that the black-to-black dipoles exclusively determined the percept elicited by a texture was bolstered by the following mathematical observation, due to Julesz et al. (1973): the 'second-order statistics' of a binary (random) texture are completely determined by the black-to-black 'second-order statistics' of the texture.Julesz et al. had in mind the probabilistic second-order statistics; at that time, no explicit distinction had yet been drawn between second-order statistics construed in the probabilistic vs. the deterministic sense.
To see the point made by Julesz et al. (1973), let I be a randomly generated binary image in which black is coded by 1 and white by 0. Write p 1 (x) for the probability that a given pixel x gets painted black (i.e.takes value 1), and p 11 (x, y) for the probability that pixels x and y both get painted black.Note first that: and Then: and Thus, all of the probabilistic second-order statistics of I are determined by the probabilistic black-to-black second-order statistics.
Note in addition that if I is viewed as a stationary stochastic process, then, for any displacement d: where I is the autocorrelation function of I. Thus the probabilistic second-order statistics of I are equivalent to the autocorrelation function of I.
If we now shift perspective and treat I not as a random image, but rather as a specific picture, we may (incorrectly) retain the impression that I's ensemble of black-to-black dipoles suffice to nail down I's deterministic second-order statistics (i.e.D I ).In this case, since we have already absorbed the assumption that we need concern ourselves only with the black-to-black dipoles, we allow the white background to be large enough that we can consider it infinite.Then, the black-to-black dipoles once again are seen to be equivalent to the autocorrelation function of I, defined now by: Indeed, as Yellott (1993) demonstrated, it is possible to construct physically distinct, deterministic binary images with identical ensembles of black-to-black dipoles.The existence of such image pairs coupled with the tacit assumption that D I is determined by the ensemble of I's black-to-black dipoles bolstered our impression that proposition 2.1 was false.However, as can be checked, although the black-to-black dipoles of Yellott's images are identical, the dipole histograms of these images differ (as proposition 2.1 implies must be the case).
In summary, we speculate that at least part of the reason that proposition 2.1 has not been previously observed is because, like us, other researchers have tacitly assumed it to be false.Our personal misconception resulted from a confusion in the use of the term 'second-order statistics'.It is true that the probabilistic second-order statistics of a (randomly generated) image I are completely determined by the probabilisitic, blackto-black, second-order statistics.This early observation (Julesz et al., 1973), strongly influenced our own thinking about isodipole textures.Unfortunately, no clear distinction was drawn between probabilistic vs. deterministic second-order statistics.Thus, we formed the impression that the dipole histogram of a binary image was completely determined by the ensemble of black-toblack dipoles in the image.This (false) impression, conjoined with examples such as those given by Yellott (1993) of distinct binary images with identical ensembles of black-to-black dipoles, seemed to suggest that proposition 2.1 was false.

Final remarks
Dipoles have exerted, and continue to exert (e.g.Doner, 1999) a powerful influence on thinking about visual cognition.In a finite image I, relational information is encoded implicitly.Such information must be inferred from locational information, which is explicitly represented.However, the reverse is true in the dipole histogram: in D I it is precisely the elementary, relational information that is explicitly encoded.The primary contribution of this paper has been to prove that locational information can indeed be uniquely retrieved from D I .Thus, D I can be viewed as an alternative representation of I, a representation that may be of some visual importance, given that vision is primarily concerned with interpreting relational information in an image.Indeed, Doner (1999) has argued that human sensitivity to some aspects of spatial pattern in deterministic images is reflected by various measures of dipole histogram entropy.
On the other hand, there are two important factors that argue against the relevance of this result for purposes of understanding the biological basis of human vision.First, for all but the tiniest images I, the dipole histogram of I is of cardinality much greater than I. Thus, D I would seem to provide a much less efficient representation than I itself.Second, the results we have provided suggest that to fully represent I, it may be necessary to preserve information about dipoles with long offsets.Specifically, we have shown in the appendix, for two-dimensional I with M columns and N rows, that I can be constructed from the restriction of D I to those dipoles with offsets of horizontal length less than or equal to ceil((M + 1)/2) and vertical length less than or equal to ceil((N + 1)/2).Although this is not as long as it might have been, it is still much too long to play a role in the biological basis of vision.Note also