Dipole statistics of discrete finite images: two visually motivated representation theorems.

,


INTRODUCTION
A discrete finite image I͓r, c͔ is a function that assigns real numbers (representing colors) to a finite, rectangular array of ordered pairs (r, c) of integers (representing spatial locations).Thus I directly codes purely locational information.For many purposes, however, locational information is of little direct visual utility.For example, locational information is irrelevant for identifying a face or recognizing an object or assessing a property of some surface texture.More important for such purposes are the spatial relationships between intensities in the visual field.][3][4] Here we explore some basic properties of relational image representations.This is a class of image-coding schemes that may prove useful for pattern recognition and other visual judgments that depend primarily on sensitivity to internal relationships between components of a pattern.Moreover, the representations in this class lend themselves gracefully to size-invariant assessments as well.(Face-selective neurons in monkey STS are remarkably size invariant. 5,6Indeed, in their study of 33 such face-selective neurons, Rolls and Baylis 6 found that the median size change tolerated with a response of greater than half the maximal response was 12 times.) A basic example of a relational representation is the dipole histogram D I of an image I.A dipole is a triple, ((d R , d C ), ␣, ␤), where d R and d C are horizontal and vertical, integer-valued displacements and ␣ and ␤ are real numbers (color values).In any such dipole, ␣ is called the initial value, and ␤ is called the terminal value.For any such dipole, D I ((d R , d C ), ␣, ␤) gives the number of pixel pairs ((r 1 , c 1 ), ͑ r 2 , c 2 ͒) of I such that I͓r 1 , c 1 ͔ ϭ ␣, I͓r 2 , c 2 ͔ ϭ ␤, and (r 2 , c 2 ) Ϫ (r 1 , c 1 ) ϭ (d R , d C ). (Thus D I has the obligate symmetry, D I ((d R , d C ), ␣, ␤) ϭ D I ((Ϫd R , Ϫd C ), ␤, ␣).) Note that the information directly encoded by D I is exclusively relational.
][20][21][22] It is easy to see how D I can be constructed from I.Although it may not be quite so obvious, I can also be uniquely constructed from D I . 23Thus any image I can be represented either in terms of purely locational information or in terms of the purely relational information embodied in D I .
The purely relational nature of the information directly coded in the dipole histogram makes it an intriguing alternative image representation.However, there are ob-vious reasons to suppose that the visual system does not in fact construct dipole histograms of its input.First, the dipole histogram is typically a much larger object than the corresponding image-that is, the cardinality of D I tends to be much greater than that of the original input image I (especially in the case in which most of I's pixel values are distinct).Second, although we can imagine artificial images comprising only a few different discrete colors, the images encountered in daily life typically comprise continuously varying colors, almost none of which are likely to be identical between different pixels within a given image.Moreover, even when an image viewed by the eye nominally contains only a small number of discrete color values, photon noise and the random nature of photon absorption in photoreceptors imply that the effective neural image will contain a far greater (and unknown) range of values.To compute the dipole histogram of such an image requires a level of precision in intensity resolution that we have no reason to think is achieved by the visual system.
In this paper, after basic definitions in Section 2, we shall begin in Section 3 by generalizing the notion of the dipole histogram of an image I to that of a ''soft'' dipole representation of I. Soft dipole representations relax the requirement (implicit in the definition of the dipole histogram) that intensities within an image be precisely registered, allowing instead a ''softened'' or ''graded'' registration of intensity.In this respect, at least, soft dipole representations may claim greater biological plausibility than dipole histograms.Moreover, as we shall show, precise registration of image intensity can be sacrificed with no loss of information: A soft dipole representation of a given input image uniquely determines that image.
After introducing soft dipole representations, and showing that they uniquely determine the images to which they correspond, we return in Section 4 to (nonsoft) dipole histograms.Our purpose in Section 4 is to show that relational representations need not be exorbitantly large compared with the images to which they correspond.Specifically, we show that for any image I (of any dimension) comprising N Ͼ 1 pixels, there always exists a restriction (term defined in Subsection 4.A) Q I of D I to a subset H I of at most N Ϫ 1 dipoles with the following property: For any image J comprising N pixels, if D J () ϭ Q I () for all dipoles H I , then J ϭ I.In other words, the restriction Q I uniquely determines I (provided that one knows the number of pixels in I).
In summary, our aim is twofold: First, we will show that the notion of the dipole histogram can be generalized to a family of representations that do not require precise registration of intensity; second, we will show that relational representations (specifically, dipole representations) need not be huge in comparison with the images they represent.We hope that these two results will stimulate further research in this area.

A. One-Dimensional Images
Let R ϭ ͕0, 1 ,..., N Ϫ 1͖, for some integer N.Then, a one-dimensional image is a function I:R → R. Elements of R are called pixels, and for any pixel r, I͓r͔ denotes the value assigned by I to r.
A one-dimensional dipole is a triple, (d, ␣, ␤), with d a nonnegative integer-valued displacement and ␣ and ␤ real numbers.We say that a dipole (d, ␣, ␤) bridges a pair (r 1 , r 2 ) of pixels in I if r 2 Ϫ r 1 ϭ d, I͓r 1 ͔ ϭ ␣, and I͓r 2 ͔ ϭ ␤.The dipole histogram D I assigns to each dipole ϭ (d, ␣, ␤) the number of distinct pairs in I bridged by .Thus, if D I (6, 1, Ϫ4.3) ϭ 16, then there are 16 pixels r of I such that I͓r͔ ϭ 1 and I͓r ϩ 6͔ ϭ Ϫ4.3.We write Support(D I ) for the set of all dipoles such that D I () Ͼ 0.

B. Two-Dimensional Images
Let R ϭ ͕0, 1 ,..., N Ϫ 1͖ and C ϭ ͕0, 1 ,..., M Ϫ 1͖ for integers N and M. A two-dimensional image is a function J:R ϫ C → R, where R ϫ C denotes the Cartesian product of R and C. The elements of R ϫ C are called pixels, and J͓r, c͔ denotes the value assigned by J to a given pixel (r, c).
A two-dimensional dipole is a triple, (d, ␣, ␤), with displacement d ϭ (d R , d C ) comprising a nonnegative, integer-valued vertical displacement d R and a (possibly negative) integer-valued, horizontal displacement d C .(We need to allow either negative row or column displacements in order to capture all varieties of two-dimensional displacement, each in one direction; as a matter of convention, we let column displacements vary in sign.) As in the one-dimensional case, ␣ and ␤ are real numbers.We say that a dipole (d, ␣, ␤) bridges a pair ͓(r 1 , c 1 ), (r 2 , c 2 )͔ of pixels in J if d ϭ (r 2 , c 2 ) Ϫ (r 1 , c 1 ), J͓r 1 , c 1 ͔ ϭ ␣ and J͓r 2 , c 2 ͔ ϭ ␤.As in the one-dimensional case, the dipole histogram D J assigns to each dipole ϭ (d, ␣, ␤) the number of distinct pairs of pixels in J bridged by , and we continue to write Support(D J ) for the set of all dipoles such that D J () Ͼ 0.

SOFT DIPOLE REPRESENTATIONS
One reason to question the biological relevance of dipole histogram representations is the following: Even when an image viewed by the eye nominally contains only a small number of discrete color values, photon noise and the random nature of photon absorption in photoreceptors imply that the effective neural image will contain a far greater (and unknown) range of values.In this section we explore the possibility of relaxing the assumption implicit in the definition of the dipole histogram that image intensities are precisely registered by the visual system.
To take an example, let I be an image, d a displacement, and ␣ and ␤ intensities.Then D I (d, ␣, ␤) gives the total number of pixels r for which I͓r͔ ϭ ␣ and I͓r ϩ d͔ ϭ ␤.By contrast, let neighborhood(␣) and neighborhood(␤) be the open intervals (␣ Ϫ 1/2, ␣ ϩ 1/2) and (␤ Ϫ 1/2, ␤ ϩ 1/2), and consider the function S I (d, ␣, ␤) giving the total number of pixels r such that I͓r͔ neighborhood(␣) while I͓r ϩ d͔ neighborhood(␤).S I is an example of a soft dipole representation.As is evident in its definition, S I embodies information similar to that embodied by D I ; however, S I forgoes the precise registration of image intensities implicit in the definition of D I .Nonetheless, as we show here, I is uniquely determined by S I .

A. Soft Dipole Representations of One-Dimensional Images
It will be convenient to obtain some basic results about one-dimensional images and then extend them to higherdimensional cases.Accordingly, let G:R → R be a probability density function with mean 0.Then, for any onedimensional image I with N pixels, any ␣, ␤ R, and d ϭ 0, 1 ,..., N Ϫ 1, define [Note that the sum in Eq. ( 1) ranges over all pixels r for which I͓r͔ and I͓r ϩ d͔ are both defined.]Suppose, for example, that In this case, as discussed above, S I behaves as an intensity-tolerant version of D I .The crucial difference is as follows: D I (d, ␣, ␤) counts precisely those occurrences of pixels r for which I͓r͔ ϭ ␣ and I͓r ϩ d͔ ϭ ␤; by contrast, S I does not require strict equality between I͓r͔ and ␣ or between I͓r ϩ d͔ and ␤: for any given ␣ and ␤, S I (d, ␣, ␤) gives the total number of pixels r in Dom(I) such that I͓r͔ is within distance 1/2 of ␣ and I͓r ϩ d͔ is within distance 1/2 of ␤.
Another natural choice for G is a Gaussian density with mean 0: With G given by Eq. ( 3), it is not as easy to give a precise verbal characterization of S I (d, ␣, ␤) [as with G defined by Eq. ( 2)].It is clear, however, that as long as is relatively small in comparison with the range of pixel values occurring in I, then S I (d, ␣, ␤) continues to reflect the degree to which I tends to assign values near ␣ to pixels r while assigning values near ␤ to pixels r ϩ d.
The choices of G given in Eqs. ( 2) and (3) yield definitions of S I similar in spirit to D I .However, there are many choices of G for which this is not true.For example, one might choose a bimodal (zero-mean) density G with G(0) ϭ 0. Such a choice of G would yield an S I largely insensitive to the information reflected by D I .Specifically, S I (d, ␣, ␤) would be utterly uninfluenced by occurrences of pixels r for which I͓r͔ ϭ ␣ and I͓r ϩ d͔ ϭ ␤-precisely the events that D I (d, ␣, ␤) counts.
However, regardless of the value that G assigns to 0, if G is a density function with mean 0, then I is uniquely determined by S I .Note, for example, that implying that one way of retrieving I from S I is to take The reader will note, however, that although S I uniquely determines I, S I is uncountable in cardinality, being continuous in each of ␣ and ␤.In general, finite approximations of S I will correspond only to approximations of I.

B. Soft Dipole Representations of Two-Dimensional Images
Let I be a two-dimensional image with N rows and M columns.Then [by analogy to Eq. ( 1)] define Note that in Eq. ( 6), d r is constrained to be in the range ͕0, 1 ,..., N Ϫ 1͖, while d c is allowed to take all values in ͕1 Ϫ M, 2 Ϫ M ,..., Ϫ1, 0, 1 ,..., M Ϫ 1͖.
Let J be the one-dimensional image obtained by concatenating the successive rows of I. Specifically, In this case, reflection on Eqs. ( 6) and (1) shows that whereas otherwise, Thus S I uniquely determines S J .Equation ( 5) thus implies that J is uniquely determined by S I .In addition, we can infer the number M of columns in I by surveying Support(S I ): Specifically, M is equal to 1 plus the maximum column displacement in any dipole in Support(S I ).This final item of information extracted from S I makes possible the construction of I. We need only assign the first M pixel values of J to the first row of I, the second M pixel values of J to the second row of I, etc.This shows that any two-dimensional image I is uniquely determined by the corresponding soft dipole representation S I .Moreover, it should be clear that this argument generalizes straightforwardly to images of any dimension.

C. Possible Roles of Soft Dipole Representations in Visual Processing
Equation (5) shows that soft dipole representations preserve complete information about the input image I even though they sacrifice precise registration of image intensity.However, for any given displacement d, S I (d, ␣, ␤) varies continuously over all ␣, ␤ R.Moreover, the method used in Eq. ( 5) to reconstruct I from S I makes use of the entire continuum of values S I (d, ␣, ␤), d ϭ 0, 1 ,..., N Ϫ 1, and ␣, ␤ R.This might be taken to suggest that infinitely many neurons would be required to fully encode the information in I (even though I itself comprises only N ϩ 1 pixels).
Several comments are in order.First (and most important), it is unlikely that visual mechanisms using soft dipole representations would require those representations to preserve complete information about the input.Recognition of a face, for example, might well depend on the extraction of only a portion of the relational information inherent in the input image [e.g., only the values S I (d, ␣, ␤) corresponding to a restricted set of crucially informative dipoles, (d, ␣, ␤)].
Second, for natural choices of G [e.g., G given by Eq. (3)], reflection on Eq. (1) suggests that for any given displacement d, the function S I (d, ␣, ␤) will vary slowly in each of the arguments ␣ and ␤.For example, if G is a Gaussian function, we can expect S I (d, ␣, ␤) to be lowpass filtered in each of ␣ and ␤ and hence band limited.In this case, however, it seems likely that nearly all the information in I may be preserved by a relatively lowdimensional representation of S I .
This issue remains to be explored in detail.We shall, however, address a closely related question in the next section.

SMALL DIPOLE REPRESENTATIONS
Recall that any function is a set of ordered pairs.This is true in particular of D I although each individual element of D I is rather complicated: An element of D I is an ordered pair (, D I ()), where ϭ ((d R , d C ), ␣, ␤) is a dipole and D I () is the number of occurrences of in I.We call any subset Q ʕ D I a dipole representation of I if the following condition holds: For any image J, if J has the same number of pixels as I, and Q ʕ D J , then J ϭ I.In this case, we shall sometimes (equivalently) say that Q uniquely determines I.
The cardinality of any finite set is simply the number of elements in that set.Thus, for a dipole representation Q of some finite image I, the cardinality of Q (denoted ͉Q͉) is the number of ordered pairs (, D I ()) in Q. Note, however, that a given dipole can occur in at most one ordered pair of Q; thus ͉Q͉ gives the number of dipoles on which Q is defined.
For any image I, the dipole histogram D I is itself a dipole representation of I. 23 However, D I is typically greater in cardinality than the original image I.For example, consider an image I with N rows by M columns of pixels that assigns a different color to each pixel.In this special case, every dipole occurring in I differs from every other dipole in either its initial or its terminal value.If we limit redundancy in D I by allowing only displacements d ϭ (d R , d C ) such that d R у 0 and d C у 0 whenever d R ϭ 0, then each pair of (possibly identical) pixel locations in I corresponds to a single type of dipole of I. Thus the set of dipoles over which D I is nonzero is of cardinality N pixels (N pixels ϩ 1)/2 for N pixels ϭ NM; and for each dipole Our purpose in this section is to show that for any discrete, finite image I comprising N Ͼ 1 pixels there always exists a dipole representation of I of cardinality at most N Ϫ 1 (provided that one also knows the number of pixels in I).The proof we offer is constructive; that is, we show how to produce such a representation for any given image I.The representation we construct is not necessarily the smallest that can be obtained (though in some cases it is).Moreover, we make no claim that the particular representation we derive is biologically plausible or useful.Our proof merely establishes an upper bound on the size of a minimal representation.

A. Proper Subsets of D I That Uniquely Determine I
Chubb and Yellott 23 showed that any one-dimensional image I is uniquely determined by D I (and extended this result to images of arbitrary dimensionality).We include a simple proof of this fact (in the one-dimensional case) in Appendix A. However, it often turns out that D I is highly redundant: We can usually find proper subsets of D I that uniquely determine I.
Let H be a subset of Support(D I ), and define Q:H → R by setting Q() ϭ D I () for any dipole H.The subset Q of D I is called the restriction of D I to H.For a given image I, there often exist proper subsets of D I that suffice to determine all the pixel values of I.In other words, it often turns out that D I is redundant.
To take an example, let I be a one-dimensional image all of whose pixels are assigned distinct values.It is easy to see that I will be uniquely determined by the restriction Q of D I to the set H of dipoles (d, ␣, ␤) such that d ϭ 1.There will exist in H a unique dipole (1, ␣ 1 , ␤ 1 ) such that ␣ 1 appears as the terminal value in none of the other dipoles of H.It is easy to see that I͓1͔ ϭ ␣ 1 , and that I͓2͔ ϭ ␤ 1 .Moreover, there will be a unique dipole (1, Clearly, I͓3͔ ϭ ␤ 2 , and so forth, until all of I's pixel values are determined.Of course, we must also be given the information that I has N pixels; otherwise, we cannot rule out the possibility that some of I's dipoles with displacement 1 are missing from H. To take another simple example, suppose all of I's pixels take the same value, ␥.In this case, Support(D I ) comprises only the N dipoles, (0, ␥, ␥), (1, ␥, ␥) ,..., (N Ϫ 1, ␥, ␥).However, D I is nonetheless highly redundant.Indeed, I is uniquely determined by the restriction Q of D I to the singleton ͕(0, ␥, ␥)͖.The knowledge that Q(0, ␥, ␥) ϭ N in conjunction with the information that I has N pixels tells us immediately that all of I's pixels take value ␥.In this case, then, the minimal dipole representation of I is smaller in cardinality than I itself.
In both of the examples we have considered, we exploited special properties of the given image I to extract a restriction of D I sufficient to determine I.The main point of this section is to show that special properties are not required.A relatively small restriction of D I can always be derived that suffices to determine I.The proof that we offer is constructive and addresses directly the multidimensional case.
An (M ϩ 1)-dimensional dipole is a triple, (d, ␣, ␤), with initial and terminal values ␣, ␤ R and displacement d ϭ (d M , d MϪ1 ,..., d 0 ) comprising integer coordinate values.We say that a dipole (d, ␣, ␤) bridges a pair (x, y) of pixels in I if d ϭ y Ϫ x, I͓x͔ ϭ ␣ and I͓ y͔ ϭ ␤.As in the one-dimensional case, the dipole histogram D I assigns to each dipole ϭ (d, ␣, ␤) the number of distinct pairs of pixels in I bridged by ; we continue to write Support(D I ) for the set of all dipoles such that D I () Ͼ 0.

C. There Exists a Relatively Small Dipole Representation of Any Image
The proof of the main result of this Subsection (Proposition 4.C.2) makes use of two functions, and its inverse , which we now define.Note that the pixel lattice X comprises N ϭ N M N MϪ1 ...N 0 (10)   pixels.We assign integers 0, 1 ,..., N Ϫ 1 to the pixels in X by setting for any pixel x ϭ (x M , x MϪ1 ,..., x 0 ) X.
For any pixels x, y X, one quickly sees that ͓ y͔ у ͓x͔ iff y lexicographically dominates x.It is also clear that is 1Ϫ1 from X onto ͕0, 1 ,..., N Ϫ 1͖, imply- ing that the inverse function Ϫ1 is well defined.It will be convenient to write for Ϫ1 ; thus, for any n ͕0, 1 ,..., N Ϫ 1͖, (n) is the nth pixel of X under lexi- cographic ordering.
A few observations will facilitate the proof of Proposition 4.C.2.

Lemmas
1.For any x, y X, To derive (a), note that the vector x ϩ y will be an element of X only if x i ϩ y i р N i Ϫ 1, for i ϭ 0, 1 ,..., M.
In this case, the fact that Eq. ( 11) is linear in x implies that ͓x ϩ y͔ ϭ ͓x͔ ϩ ͓ y͔.The argument for (b) is similar.
We are now equipped to prove the main result of this section.

Proposition
For some positive integer M, let I be an image on the pixel lattice X with dimensions N M , N MϪ1 ,..., M 0 .Then there exists a dipole representation of I of cardinality at most N Ϫ 1, where N ϭ N M N MϪ1 ...N 0 .
Proof.For simplicity, suppose that N is even; that is, suppose that N ϭ 2K for some integer K. Then let (In Eq. ( 18) we take the liberty of writing ͓k͔ sometimes to denote the pixel x for which ͓x͔ ϭ k and at other times to denote the displacement d whose offsets are equal to the coordinate values of x.)Note first that ) happen to be identical, for one or more j (i.e., if I͓͓0͔͔ ϭ I͓͓ j͔͔ and I͓͓N Ϫ j Ϫ 1͔͔ ϭ I͓͓N Ϫ 1͔͔).We want to show that the restriction Q I of D I to H I is a dipole representation of I. Accordingly, let J be an image with N pixels, and suppose that Q I ʚ D J .
The argument begins with a demonstration that for almost all pairs x, y of pixels in X separated by displacement ͓N Ϫ 1 Ϫ j͔, both of J͓x͔ and J͓ y͔ are determined by the induction hypothesis.Specifically, we shall show that there are only two pairs of pixels x 1 , y 1 and x 2 , y 2 separated by displacement ͓N Ϫ 1 Ϫ j͔ for which this is not true: The first such pair has x 1 ϭ ͓0͔ and y 1 ϭ ͓N Ϫ 1 Ϫ j͔ (y 1 is undetermined); the second such pair has x 2 ϭ ͓ j͔ and y 2 ϭ ͓N Ϫ 1͔ (x 2 is undetermined).Note that for each of these two pixel pairs Eq. ( 14) above implies that the displacement from x to y is ͓N Ϫ 1 Ϫ j͔.
Consider then some x X for which 0 Ͻ ͓x͔ Ͻ j, and let y ϭ x ϩ ͓N Ϫ 1 Ϫ j͔.It is possible that y may not be a pixel in X.However, by Eq. ( 12), if y X, then ͓ y͔ ϭ ͓x͔ ϩ N Ϫ 1 Ϫ j Ͼ N Ϫ 1 Ϫ j.In this case, the induction hypothesis implies that J͓x͔ ϭ I͓x͔ and J͓ y͔ ϭ I͓ y͔.Thus, as asserted at the outset, there exist only two pairs of pixels x 1 , y 1 X and x 2 , y 2 X separated by displacement ͓N Ϫ 1 Ϫ j͔ for which the induction hypothesis does not directly suffice to determine that J͓x͔ ϭ I͓x͔ and J͓ y͔ ϭ I͓ y͔.The first such pair has x 1 ϭ ͓0͔ and y 1 ϭ ͓N Ϫ 1 Ϫ j͔ (here we do not yet know whether J͓ y 1 ͔ ϭ I͓ y 1 ͔); the second has x 2 ϭ ͓ j͔ and y 2 ϭ ͓N Ϫ 1͔ (here we do not yet know whether J͓x 2 ͔ ϭ I͓x 2 ͔).
However, as we now show, because Q I ʚ D J , we are able to infer that indeed J͓͓ j͔͔ ϭ I͓͓ j͔͔ and J͓͓N Ϫ 1 Ϫ j͔͔ ϭ I͓͓N Ϫ 1 Ϫ j͔͔.Specifically, the crucial information is provided by the values For any dipole ϭ (d, ␣, ␤), it will be convenient to let IndCnt() be the number of pairs x, y of pixels in X such that 0 Ͻ ͓x͔ Ͻ j, y ϭ x ϩ d, J͓x͔ ϭ ␣ and J͓ y͔ ϭ ␤.
Because Q I ʚ D J , we know that is greater by 2 than IndCnt().Similarly, if H I contains two distinct dipoles 1 and 2 with displacement ͓N Ϫ 1 Ϫ j͔, then for each of i ϭ 1 and 2, Q I ( i ) must be greater by 1 than IndCnt( i ).
If H I contains only a single dipole ϭ ͓͓N Ϫ 1 Ϫ j͔,␣,␤͔, then (as argued above) Q I 's construction ensures that Q I () is greater by two than IndCnt().Since the only pixel pairs with displacement ͓N Ϫ 1 Ϫ j͔ that remain unaccounted for are (͓0͔, ͓N Ϫ 1 Ϫ j͔) and (͓ j͔, ͓N Ϫ 1͔), we infer that This completes the proof in the case in which N is assumed to be even.No important complexities arise if N is odd.

FINAL REMARKS
Visual processes such as recognition of an object or face are concerned primarily with the spatial relations between intensities composing the input image.The dipole histogram of an image encodes purely relational information about that image.It is also true that the dipole histogram of any discrete, finite image uniquely determines that image. 23These considerations make the dipole histogram a promising way of representing an image, at least for purposes of recognition.
However, it does not seem likely that human vision encodes its input images in terms of their dipole histograms.First, the definition of a dipole histogram implicitly requires an unrealistic level of precision in registration of image intensity, point by point throughout the stimulus field.Second, the dipole histogram of a given image is typically of much larger cardinality than the image itself.
In this paper we have attempted to address these two difficulties in order to explore the possibility that some form of purely relational image representation might subserve recognition processes in human vision.In Section 3 we introduced soft dipole representations.These representations are similar to dipole histograms; however, they forgo the precise registration of image intensity required by the dipole histogram.We also showed that soft dipole representations uniquely determine the images to which they correspond.
It should be noted, however, that the soft dipole representations we have considered here tend to be large.An important open question (raised in Subsection 3.C) is the following: Do there exist small soft dipole representa-tions that suffice to uniquely determine the images to which they correspond?
That there do, in fact, exist small dipole representations was demonstrated in Section 4. In that section we provided a method for constructing a purely relational representation Q I of any discrete, finite image I with N Ͼ 1 pixels, such that ͉Q I ͉ р N Ϫ 1.Given the information that (a) I has N pixels and (b) Q I is a subset of D I (the dipole histogram of I), one can uniquely determine the pixel values of I.This result demonstrates that one can always obtain a purely relational representation of a given image that is no greater in order of complexity than the original image.
It should be noted, however, that the construction we have provided does not guarantee that the resulting relational representation will be minimal in cardinality.There may well exist smaller subsets of D I that uniquely determine I.For example, as discussed in Subsection 4.A, in the special case in which all of I's pixels take the same value, I is uniquely determined by a restriction Q ʚ D I for which ͉Q͉ ϭ 1 (as usual, provided that one knows the number of pixels in I).The method we have described yields representations of cardinality at least ceil(N/2), for ceil(N/2) the greatest integer less than or equal to N/2.
Thus the following problem presents itself: Given an arbitrary input image I, how can one find a dipole representation of I of minimal cardinality?That is, is there a general method for constructing a restriction Q of D I such that (1) Q uniquely determines I and (2) ͉Q͉ is minimal over the set of all restrictions satisfying (1)?
We speculate that a solution to this problem may yield interesting insights into the nature of visual coding.Doner 24 investigated the relationship between perceptual properties of images I and the corresponding dipole histograms D I .In particular, he examined the ways in which various measures of dipole histogram entropy relate to different sorts of pattern judgments.Although this idea is enticing, we suspect that it may be important to focus on the relationships between visual judgments and nonredundant dipole representations.
To help clarify our intuitions we offer the following definition: For any discrete, finite image I, define the dipole complexity of I as the minimum of ͉Q͉ across all restrictions Q of D I that uniquely determine I.
What might the visual significance be (if any) of the dipole complexity of an image?We have already observed that the dipole complexity of a one-dimensional image of uniform color is 1.More generally, it seems reasonable to suspect that images of low dipole complexity may turn out to be structurally ''simple'' in some sense echoed by human pattern perception, whereas those of high dipole complexity may turn out to be visually ''complex.'' The current paper shows that the dipole complexity of any image I is less than or equal to ͉I͉ Ϫ 1.At present, however, the problem of how to determine the dipole complexity of a given image remains unsolved.

APPENDIX A
In this appendix, for the sake of building intuition, we supply a proof, previously presented, 23 that any one-dimensional image is uniquely determined by its dipole histogram.This is, of course, a corollary of Proposition 4.C.2.However, the following proof is simpler than that of Proposition 4.C.2.
Proof.Let I be a one-dimensional image.It will be convenient to write ͚ ␣,␤ to indicate a sum ranging over all pairs of pixel values of I. We use this notation to define (A5) Address correspondence to Charles Chubb at the location on the title page or by e-mail, cfchubb@uci.edu;phone, 949-824-1481; or fax, 949-824-2517.
Note, then, that any dipole ((d r , d c ), ␣, ␤) of I corresponds uniquely to the dipole (d r M ϩ d c , ␣, ␤) of J.Moreover, for any dipole (d, ␣, ␤) of J, there exist at most two dipoles of I that correspond to (d, ␣, ␤).If they both exist, these two dipoles are ((d r , d c ), ␣, ␤) and ((d r ϩ 1, d c Ϫ M), ␣, ␤), where d r is the greatest integer such that d r M р d and d c ϭ d Ϫ d r M. If either d c ϭ 0, or d r ϭ N Ϫ 1, then ((d r ϩ 1, d c Ϫ M), ␣, ␤) will be undefined, in which case only the dipole ((d r , d c ), ␣, ␤) of I will correspond to dipole (d, ␣, ␤) of J.