Structural Health Monitoring for Civil Structures: From the Lab to the Field
View this Special IssueResearch Article  Open Access
ImageBased Framework for Concrete Surface Crack Monitoring and Quantification
Abstract
In the engineering community, nondestructive imaging has been widely used for damage identification by capturing anomalies on the surface or inside of structural elements. In this paper, we focus on one of the most common damage types observed in civil engineering, namely, concrete surface cracks. To identify this type of damage, we propose an imagebased framework, whereby optical cameras provide the source images. The framework involves several advanced image processing methods, including: (i) the determination of damage occurrence using timeseries images, (ii) the localization of damage at each image frame, and (iii) the geometric quantification of damage. Challenges that may arise when images are obtained in the laboratory or field environment are addressed. Two application examples are provided to demonstrate the use and effectiveness of the proposed approach.
1. Introduction
In the communities of structural health monitoring (SHM) and nondestructive evaluation (NDE), development of an automated structural damage identification solution has been a key objective. Most SHM methodologies rely on sensing of onedimensional vibration signals, which are used to extract global modal features as signatures of structural integrity by using system identificationbased or statistical pattern recognitionbased methods [1, 2]; therefore, the awareness of structural damage is usually based on classifying these indirect signatures. Different from these SHM methodologies, NDE methods, especially when 2 or higherdimensional imaging methods are employed, are able to provide a direct characterization of local structural damage. Depending on the nature of images, spatial extent and spectral variation of damage usually manifest themselves in the captured images. Subsequently, one can employ photogrammetric or image analysis methods to extract these damage characteristics quantitatively.
A variety of imaging technologies have been developed in the NDE community in an effort to detect local structural damage to civil/mechanical systems or components. Widely used imaging devices include infrared thermography, microwave imaging, acoustic imaging, Xray imaging, and other radiographybased methods [3]. Common types of structural damage encountered in practice include external or internal cracks, voids, delamination, and ablation, to name a few. To detect these different types of damage, the imaging device must often be customized to capture the characteristics associated with the physical damage; hence the resulting equipment may be expensive. In civil engineering, however, optical imaging by means of commercially inexpensive cameras has become an important, economically feasible utility for recording structural damage. Its use has been extensive in the laboratory setting; however, use in the field has also recently become commonplace due to the low cost of highresolution optical cameras and associated support equipment. To expedite and automate damage recognition using the captured digital image, computational image processing tools have been attempted to detect or track structural damage in static images or dynamic image frames [4–8].
In this paper, we consider one of the most commonly observed types of damage in civil engineering, namely, concrete surface cracks. Concrete is the most widely used manmade material for constructed systems, and surface cracks are a reality due to its low tensile strength and environmental sensitivity. As a result, it is well known that a primary damage modality of concrete structures is that of surface cracks. In engineering practice, the identification of concrete surface cracks is an indispensable task in the process of conducting damage or failure prevention structural upgrading. Very narrow to hairline cracks, however, as may occur due to the natural process of shrinkage for example, are usually not significant enough to change the global structural integrity of the element hence they are not identifiable using SHM technologies. However, the appearance of these cracks and their propagation in concrete structures may indicate potential inception of degradation of structural integrity or associated softening of the system. In practice, visual inspection of the concrete surface is a common approach. For example, a considerable amount of human effort is devoted to periodic visual inspection of concrete bridges in the United States [9].
In Figure 1, two groups of manually prepared image mosaics are shown. In Figure 1(a), the example surfaces display normal concrete finish whereas in Figure 1(b) concrete surfaces with evident cracks are shown. Generally, the surface objects in optical images display distinct features in terms of their spectral intensities and their topographical patterns. More specifically, the normal concrete surface generally can be viewed as a type of texture with lighter intensities and visually identifiable spotted air pockets. Cracks, on the other hand, are usually darker with approximately uniform spectral intensities. In addition cracks in images can be geometrically described as elongated “narrow” 2D objects with closed boundaries. From an image analysis point of view, the concrete surface in images provides rich and diverse patterns for exploring different analysis methods.
(a)
(b)
1.1. Problem Definition and Previous Work
Detection of cracks with images of concrete surfaces has been explored by many researchers using a variety of methods. However, a comprehensive imagebased damage identification framework is still lacking in the literature. Such a framework needs to provide a complete solution to the final goal, which is damage quantification. Herein, we define damage identification as a threelevel probing process, which includes (1)temporal monitoring of the inception or propagation of damage using timeseries images (the occurrence determination problem); (2)detection of the boundary of damage at each image frame (the localization problem); (3)quantitative, geometric characterization of individual damage objects (the quantification problem).
If only static images instead of multiple images in a temporal sequence are available, identification reduces to the problems of damage boundary localization and damage quantification (Levels 2 and 3). In previous work, efforts have stressed more on boundary detection [4, 5, 10–12]. Among these efforts, the simplest solution is to determine the intensity gradients across the image, and boundaries between different image objects are extracted based on local variations of gradient magnitudes and orientations. This detection process in image processing is termed gradientbased edge detection. Common gradientbased edge detectors include the Roberts, Sobel, Prewitt, and Canny edge detector [13]. In these detectors, the approximation of intensity gradients is based on the concept of convolution with a set of directional derivative masks at a specified scale. To deal with the multiresolution nature of edges, researchers have proposed an automatic scaleselection approach [14]; however, this approach has not been investigated in the crack detection literature. Another approach to overcom the multiresolution problem is to use waveletbased image transform. Wavelets are special basis functions, localized in both spatial and frequency domains. Shortduration wavelets can isolate fine variations of image intensity while longduration wavelets can isolate coarse variations. Image structures, such as edges, at different scales are obtained by computing wavelet coefficients at different resolutions [15]. Subsequently, the obtained wavelet coefficients are thresholded to determine edges at different scales.
1.2. Scope of This Work
Regardless of the previous methods adopted, a major difficulty lies in the postprocessing steps. When geometric quantities of detected cracks are to be extracted, the processing usually becomes a tedious step with many manual interventions. This is due to the lack of a geometric model for the cracks. On the other hand, there is no previous work that has addressed an imagebased monitoring problem in the context of concrete surface crack identification. In what follows, first, we discuss the implementation challenges when raw laboratory or field images are used. Subsequently, a unified framework is proposed, which involves (i) a manifolddistance computing algorithm for imagebased damage monitoring and (ii) a levelset active contour modelbased method for damage boundary detection and geometric quantification. To demonstrate the use of the proposed framework, two application examples are presented. One example uses a field image, and the other uses timeseries images taken in a controlled laboratory environment. The later provides direct comparison with analog measurements for validation of the analysis results.
2. Practical Challenges
Although image samples shown in Figure 1 are valuable in the context of building library of damage patterns and testing new crack detection methods, significant practical difficulties exist when raw images captured in the laboratory or field are considered. Considering Figure 2, an image taken in the field of a shear crack at a bridge abutment, although it is visually easy to identify the crack in the image, many noisy artifacts are found, which prevent from realizing an automated, algorithmbased identification. These noisy artifacts include, for example, decoloration, occlusion, and solar illuminationrelated photometric variabilities (e.g., shadows). In addition, the relative large size of image compared to the cracked subregion impedes a computationally efficient implementation.
Even in a controlled environment, for example, laboratory testing, although test configuration is adjustable, the optimal one is often not for imaging. Hence, use of images for a quantitative damage identification faces significant challenges as well. Figures 3(a) and 3(b) show two images, which display the same concrete wall specimen that has been subject to biaxial loading at two different cyclic displacement peaks [16]. From Figure 3(a) to Figure 3(b), it can be seen that additional cracks were developed due to the increased displacement demand. One may attempt to use a boundary detection method to locate the cracks in either of the two images. However, it is not difficult to reason that the obtained boundaries would include many falsealarms (e.g., those associated with the diagonal displacement transducers), thereby preventing from an automated damage quantification.
(a)
(b)
3. Proposed Framework
A straightforward approach to addressing the aforementioned adverse factors may be achieved by performing nonsupervised image segmentation. However, in most image segmentation methods, the number of image modalities must be known a priori [17]. Therefore, for a field image as shown in Figure 2 where the number of different texture regions is arbitrary, segmentation approach is not practical.
This difficulty may be solved if continuous imaging through routine inspection is conducted, or more ideally, permanent camera sensors are installed to monitor the surface change of a critical structural component. Therefore, multitemporal images for the same structural surface can be obtained. This situation admits a digital change detectionbased solution [18]. The applicability of a change detection method is based on a reasonable assumption that two temporally adjacent images may only differ in illuminationinduced photometric distortion (e.g., solar or laboratory illumination variations), structural motion or camera motioninduced geometric distortion, and most importantly, structural damage. Meanwhile, most of the image artifacts in the scene, including decoloration and other texture objects, remain in the image sequence; hence, simple differencing operations can theoretically remove their influence if the image distortions are handled appropriately. To deal with geometric or photometric distortions, relatively simple models can be used; if these models or model parameters are estimated, a measure that is sensitive to the occurrence of structural damage by distortion can be subsequently extracted (i.e., distortioninvariant measure). As shown in the later formulation, this is the basic idea underlying the proposed manifold distancebased structural damage monitoring.
From the reasoning above, one can see that an appropriate modeling for geometric or photometric distortions becomes the critical path in realizing a distortioninvariant damage detection. For mitigating illuminationinduced photometric distortion, a common technique is normalization of spectral intensities in small image regions through subdividing the image domain to many small subdomains. For geometric distortion, the prevailing adverse factor that complicates this concept is the motion of structural objects. For a structure under loading, any point within the structure potentially undergoes some spatial motion across time. To extract a robust feature representing crack damage from a time series of images, modeling the motion of structural objects in the images is a critical issue.
With the background information introduced in the above, we propose an imagebased structural damage monitoring and quantification framework. As illustrated in Figure 4, the proposed framework has three operational components. First, image acquisition is conducted. The success of this element relies upon reliable assembly of hardware and careful design of acquisition software, which is not in the scope of this paper. It should be noted that computing is necessary in this step, which usually includes camera calibration and image correction. When a 3D scene is projected into an image plane, geometric distortion is associated with the projection determined by the intrinsic and extrinsic parameters of the digital camera [17]. In practice, a process known as camera calibration has to be conducted. Once calibration parameters are estimated, acquired images can be easily corrected to remove this distortion. In the following, we describe the subsequent two key elements to the framework, which are temporal imagebased damage monitoring and geometric imagebased quantification of damage.
4. ManifoldDistance Concrete CrackMonitoring Using TimeSeries Images
We approach our imagebased damage monitoring problem as a temporal change detection problem, wherein structural damage is treated as a type of local anomaly with its amount measurable between temporal image frames. Due to the aforementioned adverse effects of structural motion in images, a seemly straightforward approach—image differencing—is not appropriate since it requires strict spatial alignment between the temporal image pair. A simple remedy is to first extract the underlying motion parameters, then with the estimated motion parameters, the temporal image frames can be coregistered, such that image differencing can be performed, and motioninvariant features useful for representing concrete cracks can be defined. This approach was proposed in our previous work in [19]. In this section, we will briefly present this combined motion estimation and feature extraction procedure.
4.1. Assumptions about Structural Motion in Images
Theoretically, the displacement field of the structural member is governed by the material's stressstrain relation and the geometric boundary condition of the structural member, which may be in a form or a combination of bending, axial, shear deformation, and rigidbody motion. However, the motion of the structural members cannot be fully captured without loss of accuracy due to the limit of image resolution. Suppose that a resolution of may suffice for the purpose of monitoring crack damage for concrete structures. With this resolution and a common image size of , the field of view (FoV) on the surface of the target member is approximately a rectangular zone. For normalweight concrete under compression, the ultimate compressive strain is about 0.003 [20]. At this limit state, the maximum relative deformation due to compression within this FoV is about . Since the original image domain is usually divided into multiple subdomains (e.g., subdomains) in order to minimize photometric distortions, the maximum within a subdomain is much less than . A direct consequence is that the relative deformation captured in an image subdomain can be reasonably assumed to be negligible. It is noted that the above reasoning is based on the maximum compressive strain; however, cracks are due to local extensive tensile strain, which is much smaller. This strengthens that local deformation is generally not captured by optical cameras.
We also limit the study in this paper to consider the structural motion plane only approximately parallel with the image plane, resulting in a linearly scaled planar motion in the image domain. Therefore, we assume that the structural motion considered is rigidbody motion only, which includes the two orthogonal translations and one inplane rotation.
4.2. Formulation of Manifold Distance Measure
To present the formulation of both motion estimation and damage feature extraction using a time series of images, we use two notations to describe an image acquired at time . First, is used to denote a continuous image function or an intensity value at the location and the time . Second, if a digital image is treated as a vector, it is denoted as with dimension of indicating the number of pixels. For a sequence of image vectors across time, they can be denoted by .
Given a current intensity value at time by and its temporally previous value by , the current intensity value can be obtained by evolving through a spatial transformation of the coordinates if there is no photometric transformation. Without loss of generality, we use to replace and to replace for notational simplicity. Therefore, the above transformation is written as where the function defines a generic transformation model parameterized by (an vector quantity with ). This expression in the visual tracking literature is traditionally called the image constancy assumption [21]. The parameterized transformation model is differentiable with respect to both and . Additionally, an initialization is usually adopted for , which is . If the assumed transformation holds globally in the spatial domain , then any image can be fully determined by another image and the spatial transformation model between them. Formally, images like these can be treated as a lowerdimensional manifold, denoted by , embedded in the original highdimensional observation space . In this case, the intrinsic dimension of such a manifold space is the number of motion parameters , which has six parameters for an affine transformation model or three parameters for a rigidbody motion model. Note that the intrinsic dimension is generally far less than the dimension of the image observation space .
The geometric structure of an image manifold is usually augmented and complicated by some lightinginduced photometric perturbations and the development of cracks across the image sequence. Technically, the photometric perturbation can be modeled by some parametric intensity transformation [22]. As a result, more intrinsic dimensions are added into the previous dimensions. In this paper, we accept the fact that lighting is time varying across the image sequence however, we assume that it is statistically stationary within a small image domain at one acquisition time. Therefore, this type of perturbation can be removed through a simple intensity standardization (i.e., subtracting the sample mean intensity from the raw image) to remove constant variation. Cracks may also be viewed as a sort of geometric deformation occurring at some intrinsic dimensions. However, the irregular topological shape of cracks in the image domain indicates that they have highly nonlinear curvature locally in both observation and manifold spaces. Therefore, it is hard to find an explicit parametric model defined globally in the image domain to represent the cracks.
Leaving the potential crack development not modeled at a pixel , we treat it as a residue expressed by and denote the total residue over the spatial domain by , where contains the ground truth values of motion parameters that fully accounts for the spatial transformation occurring in the image frames from to . Ideally, if there are no cracks developing at time (more exactly, within a duration from to ), but only spatial transformation, the total residue will yield a value of zero. If crack damage does occur, the total residue should be larger than zero, and the same amount of cracks corresponds to the same value of the total residue yet with possibly different motion parameters s. Therefore, is theoretically an ideal feature of crack occurrence that is invariant to the underlying motion of the structural component.
If one replaces with an unknown motion parameter vector in the previous definition of the total residue, a parameterized total residue is yielded, , which is essentially a varying Euclidean distance (ED) between a parameterized image vector with a base at and the reference image vector . On the other hand, if the image vectors are viewed in the manifold space, then this distance metric quantifies the distance between the manifold and the reference point . It is reasoned that given some potential crack occurrence left unmodeled and a certain amount of motion determined by , the quantity is therefore the minimum distance between and , which is called manifold distance (MD) in the literature [23, 24]. In this paper, we define an MD measure at time given and : In particular, if is equal to , that is, there is no spatial motion, degrades to the constant Euclidean distance between and , which is . Therefore, the seeking of a motioninvariant crack damage feature entails a proper estimate of , which leads to the following optimization problem:
4.3. Estimation of Motion Parameters
The objective function in (3) unlikely owns a convex surface due to the highly nonlinear nature of an image function in terms of both and . To address this, one may approximate the functional in the manifold space at by carrying out a Taylor series expansion by considering the parameterized model . Suppose that the translation vector is denoted by , and the rotation angle is , which constitutes the motion parameter vector . The motion function is therefore With some steps of vector calculus computation, one arrives at an approximate form of (3): where the constant coefficients are integrations over the image domain . Detailed derivation of these constant coefficients may be found in [19]. The approximate squared residue in (6) is a fourthorder polynomial; hence it provides a tractable basis for studying the solution of this optimization problem defined in (3) and (4). Although analytical forms of the solution are available, they are extremely lengthy. A more common approach in practice is to use the standard Newton method for the numerical solutions, which is expressed an iterative procedure: where and are the Hessian matrix and the gradient vector of (6) with respect to , respectively, and their analytical forms are found in [19]. Upon convergence of the iteration, the MD measurement is computed by where is the estimated motion parameter vector from (7).
If one is given a time series of images ,, there are two ways to compute a sequence of timevarying MD measurements. First, one can compute an MD measurement for any temporally adjacent image frames, that is, between and . In this paper, MD measurements resulting from this method can be used to monitor the inception of concrete cracks. Another method is that upon cracks having appeared at time , the subsequent MD measurements at time are all computed relative to the time , that is, is obtained based on and where is fixed. This approach can be used to monitor the continuous growth of cracks.
4.4. Damage Localization
The MD measurement computed at an arbitrary time in essence offers a lump sum indication of occurrence for the concrete cracks in an image domain. However, if the image domain is relatively large, it does not provide an indication regarding the location of cracks. In this paper, we localize concrete cracks in images by dividing the 2D image domain into multiple subdomains before computing the MD measurements. Another advantage of this treatment is that it can minimize the disturbance of photometric distortions. The criterion for the subdivision is that the size of the subdomain should be much larger than the potential motion change between two temporal image patches upon which an MD measurement is computed. In practice, one needs a priori knowledge about the magnitude of the potential motion change in order to make such subdivision.
Suppose that the image domain is divided into subdomains, therefore, a matrix () of MD measurements can be obtained at an arbitrary acquisition time. With the MD measurements at different times, one can perform a nonsupervised outlier analysis to determine which MD measurement statistically guarantees an instance of crack occurrence. The outlier analysis, in its simplest form, is implemented by assuming that the underlying features follow a normal distribution (for more rigorous treatment about modeling the distribution of outliers, one may refer to [25]). The subdomains that are determined statistically to have crack occurrence hence provide a description of the initial location of damage.
5. LevelSet Representation of Concrete Cracks and Quantification
The imageprocessing procedure presented in the previous section offers a means of monitoring the inception or propagation of structural damage; meanwhile, a simple method for approximately locating the potential cracks is given. In the following, a method for geometric boundary detection and detailed damage quantification of concrete surface cracks is presented, in which we proposed a levelset active contour modelbased approach for this purpose.
5.1. Active Contour Models for Boundary Detection
The basic idea in active contour models is to evolve a curve subject to constraints in a given image, and eventually the curve converges to the closed boundary of objects in the image. Since the invention of the classical active contour model [26], many geometric versions have been proposed, such as the Geodesic active contour model [27]. In the expressions of the resulting partial differential equations in these contour models, a stopping function is used to stop the curve evolution, which is usually defined based on image gradients.
In cases of imagebased detection of concrete cracks, another cue is that concrete surface images can be approximated by piecewise, multiphase cartoon images. For a cracked concrete surface image, the simplest form contains two phases, the normal background or the cracked areas. This implication inspires us to use the MumfordShah segmentation model [28], which is further extended by Chan and Vese in a levelset framework [29]. A levelset method is a general technique for evolving curves (surfaces) that undergo complex topographical changes such as merging and pinching [30, 31]. It has been proven efficient in recent years in problems ranging from tracking, modeling, and simulating motion of dynamic surfaces in the fields of graphics, image analysis, fluid dynamics, fracture mechanics, and optimal topology control.
The ChanVese levelset active contour model can detect object boundaries without relying on image gradients, eliminating the need to define a gradientbased stopping function. In addition, the most appealing aspect of the ChanVese model is that it has a levelset representation, which offers the following benefits: (i) automatic handling of topographical merging and pinching of evolving contours, (ii) detection of interior contours, (iii) initial contours can be anywhere in images, and (iv) its levelset representation offers the capability to extract geometric quantities of concrete damage within a formal mathematical framework.
5.2. LevelSet Representation
Suppose that an original concrete surface image is denoted by defined in a 2D domain ; an evolving curve attempting to locate objects of interest in the image is denoted by ; one further denotes the open subdomains inside and outside of by and , respectively. In the spirit of levelset methods, the curve is embedded as the zero level set of a Lipschitz continuous function , denoted by . Correspondingly, is written as and as . Let be the Heaviside function. The energy ChanVese functional is where and are the average intensities across the inside () and the outside () regions, is the associated length scale, and is a pseudo time variable. Generally, the parameter controls the smoothness of the curve; therefore is a regularizer in the above functional. The timedependent (a pseudocomputational time will be introduced in evolving ) constants and vary as the curve evolves in .
By minimizing (9) with respect to , , and , one arrives at the following EulerLagrange equations: where is the Dirac delta function defined by . With the initial contour denoted by , one can see that (10)–(12) construct a recursive procedure when solving for the implicit levelset function .
5.3. Extraction of Geometric Properties
In this paper, the numerical schemes for solving (10) are not included; one may refer to [32] for more details. To summarize, by iteratively integrating (10) numerically, the levelset function will eventually converge at a time, denoted by ; hence, we obtain the final levelset function . Many geometric quantities of detected cracks, such as perimeter (total length of closed boundary), area, curvature, and normal direction of crack boundaries, can be computed directly with . However, we believe that the area, perimeter, and width are of primary engineering interest.
5.3.1. Area and Perimeter of Cracks
In the ChanVese model, the computation of areas and perimeters of detected objects has been embedded in (10)–(12): the area , and the perimeter . However, for the purpose of computing areas and perimeters, a more “smearedout” approximation will offer more numerical accuracy, which is where the parameter is selected comparable to the used space step. In the work of Osher and Sethian [31], is recommended.
5.3.2. Extraction of Crack Width
Another important geometric quantity of engineering interest is crack width. However, unlike the computation of areas or perimeters for concrete cracks, the extraction of crack width is a nontrivial problem. First, the concept of width is empirical. Recall that we adopt a nonrigorous way to describe cracks, namely, cracks in images are spatially narrow, elongated objects with strong contrast with the background, where the qualitative terms “narrow” or “elongated” are related to “width” without explicit definitions. In engineering practice, first, one needs to manually identify two end points for a single crack or multiple end points for a crack with a complex pattern; then by starting any of these end points, widths can be measured along the cracks continuously. To compute the crack widths analytically, a rigorous definition for crack width is necessary.
Considering a simple case (Figure 5), where a crack, with its closed boundary denoted by , has two end points and , therefore, the boundary of this crack can be split into two open boundaries, and , which both start at and end at . The width at a pixel along one side of the crack boundaries, say, , is the shortest Euclidean distance ( distance) from to a point in the other side of boundaries . For cracks with complex patterns, for example, a continuous crack with “”, “+” shapes, or other multiple angle splits, more than two end points have to be specified. One can break these cracks into several segments, wherein each segment has two end points, hence two sides of boundaries. The Euclidean distancebased definition for widths associated with these crack segments can be defined similarly.
5.3.3. Automated Extraction Method
The preceding definition of crack width may give rise to an exact solution to compute widths with known end points for a given crack. However, as the topographical patterns of cracks become more complex, the manual intervention for specifying the possible end points will be more tedious. Herein, we propose an approximate method for width extraction. This approximate method relies on the extraction of centerlines of detected cracks as well as the construction of a socalled signed distance function (SDF). An SDF, denoted by , defines the distance values from to the evolving zero levelset , which are positive inside of and negative outside of , and satisfies everywhere. The steady state of is obtained through solving where is a signum function, and is another pseudotime variable added in the dependents of . With the initial condition , the zero level set will be preserved during evolving . For centerline extraction, although a level setbased extraction of centerlines is possible [33], we employ a simple morphological skeleton operator that can preserve the connectivity of the original topographical patterns [32]. Theoretically, for objects with general topographical shapes, morphological skeltonization does not usually produce the ideal centerlines; nonetheless, for narrow, elongated 2D objects, such as cracks, morphological skeltonization is sufficiently accurate.
The proposed width extraction method is summarized in the following: (1)from , a binary image is obtained, wherein the detected cracks are marked by unity;(2) perform skeletonization on , and a new binary image is produced wherein the centerlines are unity;(3)solve for an SDF using (14) with ;(4)the shortest distances from the centerlines to the zero level set are obtained by a simple Boolean operation, which is denoted by , , the widths along cracks are approximated by .
In this approximate method, there is no restriction to the complexity of topographical patterns of cracks as well as no need to manually specify the end points of cracks. Regarding the accuracy of this method, the approximate widths will be slightly greater than the exact values for a simple crack similar to Figure 5.
6. Application I: Field Image
In routine field inspection of concrete structures, a large amount of digital images are usually collected. Wellorganized image archives as well as textbased documents provide an informative means of reporting the structural integrity of civil infrastructure objects. However, these field images are mostly used as a visual media only—readers or analysts read the images visually and may subjectively correlate visual identification with other engineering knowledge. To obtain more quantitative information, an effective image analysis method is needed. In what follows, we present a comparative study to demonstrate that the proposed framework provides a quantitative imagemining solution to this problem, which is unparalleled compared to other noncontourbased image analysis methods.
Figure 6 displays a field image of the surface of a concrete region that has clearly observed cracks. One first notices that the surface textures vary spatially showing different modalities (e.g., lighter versus darker texture). Second, there are a large number of small air pockets on the surface. Finally, cracks are not fully connected; rather discontinuities are observed between crack segments. The presence of these artifacts results in significant adverse effects and prevents from realizing an automatic identification of concrete cracks.
We first attempt to use a traditional approach. In Figure 7, two crack detection results are shown using the classical Canny edge detector [34]. The two results are presented using different analysis parameters. It is seen that the two results are dramatically different, and it is very difficult to differentiate the boundaries of the concrete cracks from the noisy edges in the background. Although some statistical methods can mitigate this parameter selection problem (e.g., [35]), the critical difficulty that prohibits subsequent geometric quantification is that the detected edges are merely nonclosed segments rather than closed contours. This difficulty exists similarly if other nonmodelbased edge detectors (e.g., a Sobel detector or wavelet transformbased detectors) are used.
(a)
(b)
The proposed levelset active contour approach can avoid this difficulty. Before applying this approach, we note that the purpose of crack identification in this situation is to obtain subsequent quantitative characteristics of the cracks rather than to solely determine the presence of the cracks (i.e., the analyst knows that there are cracks in the image). Therefore, we assume that the analyst interactively selects a region of interest (ROI) in the image domain then seeks a quantitative characterization. In Figure 8(a), such an ROI is shown, which contains multiple intersecting cracks. Starting with the initialization in Figure 8(b), we apply the levelset based detection, which iteratively converges to the final detected image contours as shown in Figures 8(c), 8(d), and 8(e). In the final result of Figure 8(e), one observes that small contours that come from air pockets are detected as well. These small noncrack boundaries can be easily removed by a labeling and area thresholding process, and the final crack contours are shown in Figure 8(f).
(a)
(b)
(c)
(d)
(e)
(f)
Different from the detection results shown in Figure 7, each detected crack segment in Figure 8(f) is featured with a closed contour (and an underlying mathematical levelset function). This property determines that the quantitative extraction of crack width and other geometric properties can be readily produced using the proposed methods in the previous section. We demonstrate this application thoroughly in the next section.
7. Application II: Laboratory Images
To demonstrate a complete process of concrete surface crack monitoring and damage quantification using the proposed framework, a laboratory beam is loaded and monitored using an array of cameras. As shown in Figure 9(a), the experiment involves a simply supported concrete beam loaded at midspan controlled with linearly increasing displacements. A 2point arrangement of loading is provided resulting in a region of constant moment. The beam span is (), and its section size is (). It was designed to fail in a shear mode using () concrete with no shearresisting stirrups. The front view of the beam was monitored during loading by four digital cameras. The resolution of the image frames in the recorded video streams is , and the frequency of camera capture was set at .
(a)
(b)
(c)
To illustrate the use of imagebased crack damage monitoring and quantification, we select a short sequence of image frames generated from Camera C1, which were continuously captured within a duration, resulting in image frames. In Figure 10, we illustrate six frames that are representative of the temporal development of crack occurrence. Note that the images shown here are cropped from the raw image frames, resulting in a smaller image domain, which is .
(a)
(b)
(c)
(d)
(e)
(f)
7.1. Crack Damage Monitoring
7.1.1. Computation of TimeSeries Damage Measures
In Figure 10(a), we show that the image domain is divided into subdomains; each of the subdomains has pixels except the subdomains in the far right column which are of size . For each subdomain, we compute the timevarying MD measurements (as well as the motion estimates). Two methods of computing MD measurements are conducted in parallel. First, we compute the MD measurements using temporally adjacent image patches. In Figure 11(a), MD measurements resulting from this method are plotted against time. For completeness, we illustrate the extracted progressive motion quantities across time in Figure 11(b), which are the average motion quantities based on the center image patches. The second method of computing MD measurements, which uses a fixed image frame at a certain time as the base, will not be started until crack damage is detected.
(a)
(b)
From Figure 11(a), it can been seen that all MD measurements computed before from the image patches are in a narrow band (about ). At , peak values of the MD measurements from some subdomains emerge, which are a strong indication of crack damage occurrence. The evolution of extracted motion quantities in the center of the beam agrees with the time of this crack event (Figure 10(c)). However, we see that these motion quantities, if used as features of crack occurrence, are not as discriminative as the MD measures across time.
7.2. Outlier Analysis and Initial Damage Localization
The above conclusion regarding when the crack event occurs is based on subjective observation. In a practical structural damage monitoring context, an outlier analysis may be employed to automate this process. An outlier analysis, in its simplest form, is implemented by assuming that the underlying features follow a normal distribution (for more rigorous treatment about modeling the distribution of outliers and other approaches, one may refer to [25]). Herein, we take advantage of the fact that there is no crack in the second image frame (at ), therefore the first group of MD measures at this moment are used to extract the sample mean and the sample standard deviation of MD measures, denoted by and . At the next image frame (at ), the MD measures () are used to test the hypothesis that if holds for a fixed threshold . Any s that fail the test are treated as outliers indicating crack damage; otherwise, they are normal MD measures, and are used to update the sample mean and the sample standard deviation . In addition, as the hypothesis test goes, at each time, a binary damage map for the divided image domain is generated progressively, where the subdomain is set unity if it is determined as a damaged region. In Figure 12, this initial damage localization is illustrated.
(a)
(b)
7.3. Crack Damage Boundary Detection and Quantification
In Figure 13, images showing the cracked regions at later times are shown. These are automatically cropped based on the initial localization information presented in Figure 12(b). In the following numerical studies, we generally choose the parameters as follows: space step , time step for (10) and for (14), and and for the approximate Heaviside functions. The length scale in principle should vary relative to the scale of the target objects, and empirical investigation needs to be conducted to determine the suitable range of this parameter. In this paper, we use the average of these bounds, approximately , throughout this work.
(a)
(b)
(c)
7.3.1. Damage Detection for Regions of Interest
Recall that the ChanVese active model in this paper is derived for twophase images, namely, assuming that the images contain only two objects, the normal background and the potential structural damage. This limitation in practice should be considered when selecting the regions of interest in structural components. Thus, we only consider a small area in the left shear zone of the beam as shown in Figure 9(c). Also, since a large number of image frames were recorded, we subsampled and selected three representative image frames in this work. Figures 13(a)–13(c) show the three images captured at different times for the same target region as shown in Figure 9(c). In Figures 14(a)–14(c), the boundaries of the cracks are successfully detected by applying the ChanVese active contour model. It is notable that in Figure 14(a), an interior noncracked area appears, which visually can be seen in Figure 13(a).
(a) 1
(b) 2
(c) 3
(d) 1
(e) 2
(f) 3
7.3.2. Extraction of Geometric Quantities: Areas and Perimeters
By visually inspecting the detected cracks in Figures 14(a)–14(c), one can observe that their topographical patterns change significantly with time () due to increasing midspan displacements. We first compute the areas and perimeters of these cracks using the methods discussed previously. In Table 1, these results are reported as well as their corresponding true values considering the image resolution. The true values are determined by manually summing the pixels. It is interesting to note that the areas of the cracks increase; however, the perimeters of the cracks with increasing midspan displacements do not necessarily increase. This is due to the higher topographical variations of concrete cracks at lower displacement levels.

7.3.3. Extraction of Crack Widths
In accordance with the approximate procedure for extraction of crack width, we first perform skeltonization over the binary detection results at time , and . Figures 14(d)–14(f) illustrate the centerlines of these multitemporal cracks. By solving (14) for the SDFs and conducting Boolean operations to extract distance values at the centerlines, one obtains the approximate widths along cracks, denoted by , , and . Table 2 summarizes the statistics of these crack widths. Since it is difficult to plot extracted crack widths along the crack in a 2D domain, for width values in , we plot the pairs , that is, the coordinate versus the maximum width in direction given . These plots are shown in Figures 15(a)–15(c).

(a) 1
(b) 2
(c) 3
(d) 1
(e) 2
(f) 3
To evaluate the accuracy of the above approximate crack widths, we use an explicit searching method to extract the exact crack widths: first, we explicitly start traveling along one side of the two boundaries (e.g., the in Figure 5) from one end point, which can be realized by searching the 8 connected neighborhoods repeatedly; then at each pixel , the shortest distance to the other side of the two boundaries () is computed by minimizing the distance values from to all the pixels in . This explicit searching method yields the exact values of crack width along a simple crack. Certainly this method suffers from the limitations that manual intervention is needed to provide the end points, and further generalization to complex crack patterns is difficult. Nonetheless, it provides the baseline for comparison with the approximate method. The bottom two rows of Table 2 summarize the width statistics of the obtained exact crack widths, and in Figures 15(d)–15(f) the crack widths are again plotted along the axis.
By comparing the statistics results from the two extraction methods in Table 2, one can see that the basic statistics of extracted crack widths at the three different times are considerably close. One can further compare the plots of extracted width along axis in Figure 15; visually, at each time of , or , the obtained widths from the two methods agree very well. Quantitatively, we compute the rootmeansquareerror () over the results from the exact method and from the approximate method: at , at , and at , . At time , slightly higher error is produced because in the exact method, the small interior noncracked region within the detected crack is ignored, therefore, higher widths are yielded around this region while in using the level setbased approximate method, the interior noncracked region is automatically excluded in computing the width.
7.3.4. Correlation with Displacement and Load Amplitudes
In this laboratory experiment, the linearly increasing displacements at the midspan and associated load amplitudes from the actuator were recorded at . Based on the synchronized computer times of the imageacquisition workstation and the actuatorcontroller workstation, each captured image frame can be mapped to a simultaneous midspan displacement and load amplitude. In Figure 16(a), the linearlyincreased midspan displacement versus time is plotted, with the times when the crack images used previously were captured articulated. In Figure 16(b), the loaddisplacement curve is provided, wherein there simultaneous values of average crack width at times are annotated.
(a)
(b)
These correlation plots illustrate that cracks in a critical region of a structural component can be captured continuously with a proper camera array setup, and if available, this data can be synchronized with other measurements. It is noted that this continuous sampling of realistic 3D objects, as made possible with the 2D images, is not possible with any other current measurement methods that are discontinuous in space. With an offline analysis proposed in this paper, the positions of cracks and the associated geometric quantities can be tracked continuously in time. Hence, one can correlate the crack characteristics to other quantities of interest, such as load amplitude, displacement, or other measured response data.
8. Concerns Prior to Practical Application
8.1. Desired Resolution
In the previous two sections, the proposed framework and the component algorithms are applied in two different practical situations. In either of these example applications, if engineering properties are to be extracted, the resolution of the images must be carefully considered. In the field, with a large amount of (potentially manually captured) images, it may not be feasible to reliably configure the camera capture hardware to accomplish consistent and reasonable resolution. One of the easiest solutions is to place a small pattern, for example, drawn onto paper, within the field of view. Using this reference pattern, a simple calculation based on the known size of the pattern and its size in the image domain may give an approximate estimate of the image resolution (e.g., in terms of length unit per pixel).
The laboratory application example demonstrates a complete application of the proposed framework. In this experiment, the image resolution is determined to be 0.89 mm/pixel. It is noted that narrower, hairline cracks, which may have widths as small as 0.1 mm cannot be captured with this setup. This is primarily a limitation of the hardware setup. Alternative camera type and placement relative to the field of view of interest can resolve this issue. If the width of the cracks is still less than the image resolution, a detection method with subpixel accuracy may be necessary. The levelset active contour method as presented in this paper has not been calibrated to solve this problem, as it internally relies on the calculation of the mean intensity values within contoured image areas. Multiple overlapping fields of view have been used with success in other applications to achieve subpixel accuracy; however, this would be a subject of future study using the currently presented methods.
8.2. Image Noise and Size
A practical problem in any imagebased analysis situation is that noisy image artifacts may exist in the domain. In the case of concrete surface images, these artifacts include large decolored, roughed, or spalled concrete surface areas. In this situation, prior to turning to other image segmentation or object recognition methods, we propose that a subdomain approach may be used first. Such an approach may be in the form of dividing the image domain into multiple subdomains or in the form of identifying userselected ROIs. It is noted that the influence of noisy artifacts will appear in the first application example if a different ROI is selected (Figure 6). Another practical issue, with a related solution, is that of image size. If images with very large sizes are encountered, computational cost may increase significantly; meanwhile robustness to noisy artifacts likely decreases. In this situation, the subdomain approach is an effective solution as well.
8.3. Computational Cost
The algorithms in the proposed framework, including the manifoldbased monitoring algorithm and the level setbased crack detection and quantification algorithms, comprise a large number of basic vector or matrix operations with the complexity order at or . As a result, the associated computational cost is considerably larger than the traditional simple filteringbased methods. Computational cost is a concern if the framework is needed for online monitoring and quantification of rapidly captured image sequences. For offline processing of single images, however, the computation time is acceptable based on our implementation of the proposed algorithms in a Matlab environment [36]. If the framework is used as an online monitoring solution, fine tuning of the image capture rate and the image processing time is subject to further investigation. A plausible solution may be to subsample the captured images (i.e., only process a fraction of the images). It is noted that most of the time consumed from an operational point of view is spent tuning the internal parameters. This issue is expected to be mitigated when more empirical knowledge is obtained.
9. Conclusions
In this paper, a novel imagebased framework is proposed to monitor and quantify one of the most common types of structural damage in civil engineering, namely, concrete surface cracks. In this case, optical camera images are used. Using illustrative examples, we first discuss the technological challenges in implementing any imagebased damage identification approach using field or laboratory images. These include various instances of photometric and geometric distortions. For the latter, structural motion in the images forms one of the major challenges.
By means of integrating two advanced image modeling algorithms, the proposed framework provides a full solution to the common damage identification objectives encountered in practice. First, through the novel use of manifolddistance computation based on multitemporal images, motioninvariant features of crack occurrence are extracted. Second, a level setbased active contour algorithm is applied to realize a modelbased boundary detection for concrete surface cracks. Based on the obtained levelset crack boundaries, an approximate method for determining continuous crack widths is developed. Two example applications are provided to demonstrate use of the proposed framework. Results show the success in monitoring temporally and quantifying geometrically the concrete cracks.
Acknowledgments
Support for this work was provided by a National Science Foundation (NSF) Grant (CMS0510802) and a research assistantship from the University of California, San Diego. The authors would like to thank Dr. KaiUwe Doerr and Proffessor Falko Kuester for assistance with the image capture during the laboratory experiment and Mr. Robert Kazanjy and laboratory staff at University of California, Irvine for assistance during testing. Opinions, findings, and conclusions expressed in this paper are those of the authors and do not necessarily reflect those of the sponsoring organizations.
References
 S. W. Doebling, C. R. Farrar, and M. B. Prime, “A summary review of vibrationbased damage identification methods,” Shock and Vibration Digest, vol. 30, no. 2, pp. 91–105, 1998. View at: Google Scholar
 C. R. Farrar and K. Worden, “An introduction to structural health monitoring,” Philosophical Transactions of the Royal Society A, vol. 365, no. 1851, pp. 303–315, 2007. View at: Publisher Site  Google Scholar
 P. Shull, Nondestructive Evaluation: Theory, Techniques, and Applications, CRC Press, Boca Raton, Fla, USA, 2002.
 I. AbdelQader, O. Abudayyeh, and M. E. Kelly, “Analysis of edgedetection techniques for crack identification in bridges,” Journal of Computing in Civil Engineering, vol. 17, no. 4, pp. 255–263, 2003. View at: Publisher Site  Google Scholar
 T. C. Hutchinson and Z. Chen, “Improved image analysis for evaluating concrete damage,” Journal of Computing in Civil Engineering, vol. 20, no. 3, pp. 210–216, 2006. View at: Publisher Site  Google Scholar
 T. C. Hutchinson, F. Kuester, K.U. Doerr, and D. Lim, “Optimal hardware and software design of an imagebased system for capturing dynamic movements,” IEEE Transactions on Instrumentation and Measurement, vol. 55, no. 1, pp. 164–175, 2006. View at: Publisher Site  Google Scholar
 T. Zhang and G. Nagy, “Surface tortuosity and its application to analyzing cracks in concrete,” in Proceedings of the 17th International Conference on Pattern Recognition (ICPR '04), vol. 2, pp. 851–854, Cambridge, Mass, USA, August 2004. View at: Google Scholar
 Z. Zhu and I. Brilakis, “Detecting air pockets for architectural concrete quality assessment using visual sensing,” Electronic Journal of Information Technology in Construction, vol. 13, pp. 86–102, 2008. View at: Google Scholar
 M. Moore, D. Rolander, B. Graybeal, B. Phares, and G. Washer, “Highway bridge inspection: stateofthepractice survey,” Tech. Rep. FHWARD01033, Federal Highway Administration, McLean, Va, USA, 2001, http://www.tfhrc.gov/hnr20/nde/pdfs/01033.pdf. View at: Google Scholar
 Y. Fujita, Y. Mitani, and Y. Hamamoto, “A method for crack detection on a concrete structure,” in Proceedings of the 18th International Conference on Pattern Recognition (ICPR '06), vol. 3, pp. 901–904, Hong Kong, August 2006. View at: Publisher Site  Google Scholar
 A. Ito, Y. Aoki, and S. Hashimoto, “Accurate extraction and measurement of fine cracks from concrete block surface image,” in Proceedings of the 28th Annual Conference of the IEEE Industrial Electronics Society, vol. 3, pp. 2202–2207, Seville, Spain, November 2002. View at: Google Scholar
 G. De Schutter, “Advanced monitoring of cracked structures using video microscope and automated image analysis,” NDT & E International, vol. 35, no. 4, pp. 209–212, 2002. View at: Publisher Site  Google Scholar
 R. Gonzalez and R. Woods, Digital Image Processing, PrenticeHall, Upper Saddle River, NJ, USA, 2002.
 T. Lindeberg, “Edge detection and ridge detection with automatic scale selection,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '96), pp. 465–470, San Francisco, Calif, USA, June 1996. View at: Google Scholar
 S. Mallat and S. Zhong, “Characterization of signals from multiscale edges,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 7, pp. 710–732, 1992. View at: Google Scholar
 T. Soppe and T. Hutchinson, “Experimental damagegas flow correlations for cyclically loaded reinforced concrete walls,” Tech. Rep. SSRP 09/07, Department of Structural Engineering, University of California, San Diego, Calif, USA, 2009. View at: Google Scholar
 D. A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, PrenticeHall, Upper Saddle River, NJ, USA, 2002.
 R. J. Radke, S. Andra, O. AlKofahi, and B. Roysam, “Image change detection algorithms: a systematic survey,” IEEE Transactions on Image Processing, vol. 14, no. 3, pp. 294–307, 2005. View at: Publisher Site  Google Scholar
 Z. Chen, B. Chang, and T. C. Hutchinson, “Imagebased monitoring of structural damage: concrete surface cracks,” in Smart Sensor Phenomena, Technology, Networks, and Systems, vol. 6933 of Proceedings of SPIE, San Diego, Calif, USA, March 2008. View at: Publisher Site  Google Scholar
 ACI Committee 318, Building Code Requirements for Structural Concrete, American Concrete Institute, Farmington Hills, Mich, USA, 2008.
 B. Horn, Robot Vision, MIT Press, Cambridge, Mass, USA, 1996.
 P. W. Hallinan, “A lowdimensional representation of human faces for arbitrary lighting conditions,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '94), pp. 995–999, Seattle, Wash, USA, June 1994. View at: Google Scholar
 P. Y. Simard, Y. A. Le Cun, J. S. Denker, and B. Victorri, “Transformation invariance in pattern recognition: tangent distance and propagation,” International Journal of Imaging Systems and Technology, vol. 11, no. 3, pp. 181–197, 2000. View at: Publisher Site  Google Scholar
 N. Vasconcelos and A. Lippman, “A multiresolution manifold distance for invariant image similarity,” IEEE Transactions on Multimedia, vol. 7, no. 1, pp. 127–142, 2005. View at: Publisher Site  Google Scholar
 V. J. Hodge and J. Austin, “A survey of outlier detection methodologies,” Artificial Intelligence Review, vol. 22, no. 2, pp. 85–126, 2004. View at: Publisher Site  Google Scholar
 M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: active contour models,” International Journal of Computer Vision, vol. 1, no. 4, pp. 321–331, 1988. View at: Publisher Site  Google Scholar
 V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic active contours,” International Journal of Computer Vision, vol. 22, no. 1, pp. 61–79, 1997. View at: Google Scholar
 D. Mumford and J. Shah, “Optimal approximation by piecewise smooth functions and associated variational problems,” Communications in Pure and Applied Mathematics, vol. 42, no. 1, pp. 577–685, 1989. View at: Google Scholar
 T. F. Chan and L. A. Vese, “Active contours without edges,” IEEE Transactions on Image Processing, vol. 10, no. 2, pp. 266–277, 2001. View at: Publisher Site  Google Scholar
 S. Osher and R. Fedkiw, Level Set Methods and Dynamic Implicit Surfaces, Springer, New York, NY, USA, 2002.
 S. Osher and J. A. Sethian, “Fronts propagating with curvaturedependent speed: algorithms based on HamiltonJacobi formulations,” Journal of Computational Physics, vol. 79, no. 1, pp. 12–49, 1988. View at: Google Scholar
 P. Soille, Morphological Image Analysis: Principles and Applications, Springer, New York, NY, USA, 1999.
 M. S. Hassouna and A. A. Farag, “Robust centerline extraction framework using level sets,” in Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), vol. 1, pp. 458–467, San Diego, Calif, USA, June 2005. View at: Publisher Site  Google Scholar
 J. Canny, “A computational approach to edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679–698, 1986. View at: Google Scholar
 Y. Yitzhaky and E. Peli, “A method for objective edge detection evaluation and detector parameter selection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 1027–1033, 2003. View at: Publisher Site  Google Scholar
 MathWorks, Inc, 2010, http://www.mathworks.com/.
Copyright
Copyright © 2010 ZhiQiang Chen and Tara C. Hutchinson. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.