Surface Roughness Measurement Using Shadows Produced by Circulating Light Sources

We propose a new measurement method for a degree of roughness of a given object surface. This method is not to measure the degree of roughness of the object surface directly, but to estimate the roughness from surface images. Named as circulating light sources (CLS), its multiple light sources aligned in a circle illuminate sequentially, and produce images including shadow of the object surface. As the shadows on the images reﬂect a shape of the object surface, the shapes of the surface, concavo-convex shape, can be estimated by these shadows. In this paper, features of surface roughness are extracted by a Wavelet Multiresolution Analysis (MRA) from the shadow images produced by the CLS, and are classiﬁed by a Self-Organizing Map (SOM). A roughness of an unknown surface can be estimated by the SOM after learning.


Introduction
Physiological and psychological studies about human (animal) brain dynamics for cognition of three-dimensional (3D) objects have been reported. It was proposed by Gibson 1 that human visual system utilizes various kinds of depth cues to perceive the 3D shape of an object from two-dimensional (2D) retinal images. Visual depth is perceived not only with binocular cues of disparity, but also with monocular cues of linear perspective, texture gradient, and shading. Similarly, Marr 2 postulated in his computational theory of vision that the description of the geometry of a visible surface by integrating various depth cues is a critical step in visual information processing for forming a representation of a 3D shape. Many psychological studies have shown that in addition to binocular disparity, monocular depth cues, such as the shadows and lighting 3,4,5 , the linear perspective of contours 6,7,8,9 and texture gradients 10,11,12,13,14 , are also important for the perception of the surface orientation. It was reported that such monocular depth cues interact with binocular depth cues during the perceptual process 15 . In general, binocular depth cues are innate and biological while monocular depth cues are learned and environmental.
Shikata et al. identified surface-orientation-selective (SOS) neurons in the caudal part of the lateral intraparietal sulcus (area CIP) of the monkey and found that they are sensitive to binocular disparity 16 . It was found that these neurons are sensitive to the gradients of disparity across the surface and/or those along the contour, and that they integrate these disparity signals to represent 3D surface orientation 17 . It was shown by further study that some of these SOS neurons are also sensitive to monocular cues for depth such as linear perspective and texture gradient 18 .
Many researches for modeling of the neruodynamics of the human stereopsis and realizing by computer systems have been reported 19 . There are two main types of these researches; one, is to set two cameras parallel to each other 20,21 , and the other one is to set two cameras to intersect at a point on the object 22,23,24,25 . In both methods a structural matching is one of the all pervading problems in computer vision. It arises in 2D and 3D object recognition from 2D and 3D image descriptions.
As mentioned above, some physiological and psychological evidences have been reported (Fig. 1). We have focused attention on the psychological evidences for the shading, which explains cognition of 3D shapes. One of the most famous optical illusions in human visual perception is the apparent inversion of reliefs that can occur when shaded images are viewed upside down as shown in Fig. 2. The human visual system can rapidly and accurately derive the three-dimensional shapes of surfaces by using variations in image intensity alone. This ability to perceive shapes from shading is one of the most important aspects of human vision, and has been studied extensively 3,4,5 . The details of neurodynamics of this phenomenon remain to be clarified, but it would appear that a human recognize the 3D shapes by higher brain functions based on his/her knowledge and experiences. From a viewpoint of engineering, the 3D cognition by only shadows information is very attractive. It means that a 3D cognition system can be realized using only one camera and a few illuminations.
Consequently, we develop a new imaging device with circulating light sources (CLS) inspired by the 3D cognition mechanism of human. A proposed method based on the CLS imaging device is able to measure a degree of surface roughness using shadow information. The surface roughness measurement mentioned in this study is a method for measuring the degree of roughness of concavo-convex surface with texture. In this measuring method it is need to eliminate information except shadows, e.g. texture, because only the shadows of concavo-convex shapes have importance. The proposed method can measure the degree of roughness by a shadow extraction method which is described in Section 4 regardless of surface texture. The CLS imaging device is equipped with one camera and eight illuminations, and produces eight shadow images. If the CLS imaging device is equipped with fewer illuminations (in extreme case, only one illumination), it is highly possible that regions of the shadow consistently arise in the shadow images. As a result, a  determination of whether the region is the shadow or surface texture is difficult. Thus the large number of illuminations is reasonable for improving the accuracy of the 3D cognition system.
In this study, the features of surface roughness are extracted by a Wavelet Multiresolution Analysis (MRA) 35,36,37,38 from the shadow images produced by the proposed the CLS imaging device, and are classified by Self-Organizing Map (SOM) 39,40,41,42 . The roughness of unknown surfaces can be estimated by the SOM after learning. This paper is organized into seven sections. Section 2 mentions general surface roughness measurements. Section 3 explains a structure of the CLS imaging device. Section 4 provides extraction and elimination of foreign patterns. Section 5 describes a structure of the MRA and the SOM. Section 6 discusses experimentations to verify the proposed roughness measurement method and its results. And we summarize the conclusions in Sec. 7.

Surface Roughness Measurements
The surface roughness is an important parameter for characterizing the surface properties of materials. In particular, it is closely related to the tribological char-  acteristics of the material surface such as those relating to friction, wear, fretting and lubrication, which affect the mechanical properties of the materials and their reliability. 26 Therefore, in order to ensure the desired quality in the material surfaces, surface roughness must be evaluated quantitatively. Two methods, stylus profiling and optical scattering, are well accepted as good conventional methods for measuring surface roughness. 27 In the stylus method, the measurements are performed using a diamond stylus that can profile the detailed geometry of the material surface. Although this technique has been widely used in industry because of its reliability and accuracy, the measurement speed is rather slow and the stylus often deteriorates the quality of the material surface by scratching.
On the other hand, the optical scattering method can provide non-contact measurements based on light beam irradiation. In certain cases, the optically measured roughness has a good correlation with the one mechanically profiled using a stylus. 28 However, the upper limit of the measurable range of the roughness parameter by the optical method is less than 1 mm, while the roughness parameter of engineering material surfaces generated by machining processes often ranges from 0.5 mm to 20 mm or more. 29,30 Therefore, it is desirable to have an alternative non-contact technique for measuring a relatively large range of roughness.
It would appear that the contact or non-contact technique is not an appropriate method for measuring the mucosa surface roughness. Because the mucosa surface is very soft and the mucosa has a shiny surface due to salivary flow. Therefore we believe that our image processing technique suits well for the measurement of the mucosa surface roughness.

Circulating Light Sources
The structure of the circulating light sources (CLS) is shown in Fig. 3. Multiple light sources (white color LEDs) which aligned in a circle illuminate on concavo-convex surface sequentially, and produce shadow images along concavo-convex shapes. Figure 4(a) shows the external view of prototype CLS imaging device. The CLS imaging device is covered with a steel case to block out lights other than the CLS. Figure 5 illustrates the shadow images of a nut produced by the CLS as shown in Fig.4(b). The shadow images produced by the CLS contain concavo-convex information. If the shadow area is large, the degree of surface roughness is large. On the other hand, if the shadow area is small, the degree of surface roughness is small. The features of surface roughness are extracted from the shadow images including concavo-convex information.

Eliminating Irregular Luminance of Surface by Light Source
LEDs used in the CLS are a point light source. In a point light source illumination, a luminance of a surface element is inversely proportional to the two power of a distance between the point light source and the surface 31 . In other words, the surface near the light source is brighter than the surface far away from the light source. Thus, the irregular luminance is visible in the shadow images produced by CLS as shown in Fig. 5. This irregular luminance may negatively affect the measurement of surface roughness. A powerful yet simple method for eliminating the irregular luminance is as follows: (i) Taking original shadow images including object surface by the CLS.
(ii) Subtracting shadow images without object, prepared beforehand, from original shadow images.  (iii) Correcting the luminance turned dark. Figure 6 shows the result of irregular luminance elimination. As shown in Fig.6, we confirmed that the irregular luminance is eliminated from the original shadow image.

Foreign Patterns Extraction and Elimination
Other patterns (foreign pattern) except for shadow patterns (original texture) also would sometimes appear in original shadow images produced by CLS. The foreign patterns will affect the surface roughness measurement based on the shadow patterns produced by CLS. Thus, the foreign patterns should be eliminated from the original shadow images.  (ii) Focusing on a pixel (P n i,j ), and summing pixel value of eight images (S i,j ). Where i and j are pixel location.

Extracting Foreign Patterns from Shadow Images
(iii) Deciding a pixel value of the foreign patterns image (F i,j ) is as follows: Figure 8(b) shows an example of foreign pattern extraction. In this case, we used a sandpaper with foreign pattern such as Fig. 8(a).

Eliminating Foreign Patterns
After extracting foreign pattern region, this region must be filled with ideal patterns. The image completion, also known as an image inpainting, is a challenging problem in computer graphics and computer vision. The image completion aims at filling in missing pixels in a large unknown region of an image in a visually plausible way 32,33,34 .
In this paper, we used simple method instead of the image completion. The luminance of the foreign pattern area is corrected to conform the average luminance   of the foreign pattern region to the average luminance of the original texture region. Figure 8(c) shows an example of the foreign patterns extraction.

Feature Extraction and Data Classification
The features of surface roughness are extracted by Wavelet Multi-Resolution Analysis (MRA) as wavelet coefficients. And then, the features are classified by Self-Organizing Map (SOM) to estimate the roughness of unknown surfaces.

Wavelet Multiresolution Analysis for Feature Extraction
The idea of wavelets is explained in details by Daubechies 35,36,37,38 who stated that wavelets are functions that are used as the basis to represent other functions. This one is called motherwavelet. A set of functions can be generated by translations and dilations of the mother function. Suppose that ψ(x) is a mother function, the translations and dilations will be ψ( x−b a ), while a ∈ + and b ∈ . The values of a and b can be calculated using a = 2 −j and b = k · 2 −j while k and j are integers.
Wavelets Multiresolution Analysis (MRA) is based on applying 2D wavelets transform to the image and a set of four different coefficients are produced in each level of decomposition. The produced coefficients are Three levels of 2D wavelets decompositions are illustrated in Fig.9. In this study, we use a level-3 decomposition based on Daubechies-8 wavelet.
The MRA resolves original signal to a high-frequency component and a lowfrequency component. Then the resolved low-frequency component is also resolved into the high-frequency component and the low-frequency component by the MRA.
The features of surface roughness are extracted by the MRA as wavelet coefficients.

Self-Organizing Map for Data Classification
The Self-Organizing Map (SOM) 39,40,41,42 is biologically motivated by the ability of the brain to map outside world onto the cortex, where nearby stimuli are coded on nearby cortical areas. The SOM algorithm is a simplified model of such a mapping process.
The structure of the SOM is shown in Fig. 10(a). The SOM consists of the input and the competitive layers that include M and N units, respectively. The j-th unit in the competitive layer is connected to all units in the input layer by the weight vector w j = [w j1 , · · · , w ji , · · · , w jM ]. The SOM is trained using learning data set {x l | l = 1, · · · , L}. We abbreviate the subscript l except special cases. The procedure of learning of the SOM is as follows: (i) The weight vectors of all units in the competitive layer are initialized using random values. (ii) The input vector x = [x 1 , · · · , x i , · · · , x M ] is randomly selected from the set of learning data and it is applied to the input layer. (iii) The distance between the input vector x and weight vector w j of the j-th unit in the competitive layer is calculated. Euclidean distance is usually employed as distance measure, thus distance between x and w j is represented by: The unit which has the minimum Euclidean distance is defined as the winner unit c by: The units which are located near the winner unit in the competitive layer are assigned as neighboring units as shown in Fig. 10(b). In Fig. 10(b) N c (t) is defined as neighborhood area. The distance between units neighboring each other is defined as 1. (v) The weight vectors of the winner unit and neighboring units are updated by: where t represents a learning step, and w j (t + 1) and w j (t) are the weight vectors after and before updating, respectively. α(t) is a learning coefficient.  (vi) The procedures (i) to (iv) are repeated until that all data are selected. In the iteration the learning rate α(t) and neighborhood area N c (t) decrease with learning steps. For example, they are defined by: where α i and α f are the initial and final values of the learning coefficient, respectively. T is total number of learning steps. N c,i and N c,f are the initial and final values of the neighborhood area, respectively.
Applying the input vectors to the input layer of the SOM after the learning, an input vector pairs with a small distance between them make the corresponding winner units to be close to each other. In other words, input vectors are mapped to two-dimensional competitive layer based on the distance between them. Using this feature, the SOM is successfully applied to pattern classification, data analysis, vector quantization, and so on.
The wavelet coefficients of various surface roughness extracted by the MRA are classified by the SOM. The roughness of unknown surfaces can be estimated by the SOM after learning.

Simulations and Results
In order to verify performances of the proposed measurement of surface roughness, it is applied to five kinds of the sandpaper; P16, P24, P36, P40 and P50 are standardized by JIS 43 . Some shadow images of these sandpapers are shown in Fig. 11.
A procedure of the proposed method is shown in Fig. 12 , and is as follows: (i) Eight shadow images without foreign patterns are shot by the CLS imaging device (Fig. 13). (ii) Each shadow image is decomposed by the MRA, and the wavelet coefficients are calculated. Figure 14 shows     (a) 5 testing vector without foreign patterns (one input vectors per a kind of sandpaper) (b) 5 testing vector with foreign patterns as shown in Fig. 8(a) (one input vectors per a kind of sandpaper) (c) 5 testing vector eliminated above foreign patterns as shown in Fig. 8(c) (one input vectors per a kind of sandpaper) Table 6 shows the parameters of the SOM. Figure 15 Fig. 15, the sandpaper data of similar type are located close together on the feature map. And, the testing data (P16 T ∼ P50 T ) are also located in the category of same sandpaper on the feature map. As a result of Fig. 15, the roughness of unknown surfaces could be estimated by the SOM after learning.
As shown in Fig. 16, the sandpaper data after eliminating of the foreign pattern (P16 E ∼ P50 E ) are located in the category of same sandpaper. However, the sandpaper data before eliminating of the foreign pattern (P16 BE ∼ P50 BE ) are not located in the category of same sandpaper except P16.
As a result of Fig. 16, the elimination of the foreign patterns is effective for the system.

Conclusions
In this paper, we proposed the new measurement method for degree of roughness of object surface. The proposed method has been taken a hint from the 3D cognition mechanism of human, particularly the shading of the psychological evidences. The CLS produces shadow images along surface shape by multi light sources. The features of surface roughness are extracted by the MRA from the shadow images, and are classified by the SOM. As a result of simulations, the roughness of unknown surfaces can be estimated by the SOM after learning.
Practical applications of the CLS imaging device include diagnosis of squamous cell carcinoma. Some characteristics of the squamous cell carcinoma are as follows: a coarsely-granular surface, nonuniform white macules, nonuniform bumps, an eroded or ulcerous surface and induration surface. An especially-important characteristic among these characteristics is that a mucosa surface has a coarsely-granular (concavo-convex) shape. Therefore, affected parts are diagnosed by a measurement of the mucosa surface roughness for determining whether the affected part is squamous cell carcinoma or not. From now on, experiments based on actual carcinoma data are needed to evaluate this system performance.