Abstract
To further extend the scope of stereo-digital image correlation (stereo-DIC) to more challenging environments, a novel Scheimpflug camera-based stereo-DIC is developed for full-field 3D deformation measurement, wherein the Scheimpflug condition, consisting of tilting the sensor plane with respect to the lens plane for the sake of larger depth of field (DOF) of the camera, is employed. The geometric imaging model of the Scheimpflug camera is described, on the basis of which a robust and effective stepwise calibration strategy is performed to calculate the intrinsic and extrinsic parameters of the stereo Scheimpflug rig. With the aid of a specially tailored stereo triangulation method and well-developed subset-based DIC algorithms, the three-dimensional shape and displacement of the specimen can be retrieved. Finally, practical experiments, including rigid motion tests and three-point bending tests, demonstrate the effectiveness and accuracy of the proposed approach.
1. Introduction
Since Kahn-Jetter and Chu [1] and Luo et al. [2] laid the foundation and established the main framework based on image correlation and stereovision in the early 1990s, stereo-DIC has been significantly improved in the aspects of accuracy, computational efficiency, and robustness [3, 4]. Actually, stereo-DIC has evolved into a quite mature noncontact image-based optical technique for full-field 3D shape, displacement, and deformation measurements. Due to its outstanding superiorities, stereo-DIC has been extensively adopted in various fields, ranging from regular metal to composite materials [5] or even biological tissues [6], from nano- [7] to macroscopic scale [8], from static or quasistatic loading to high-speed dynamic loading [9], and from laboratory conditions to industrial extreme environments [10].
Nevertheless, the application of stereo-DIC might be constrained in view of the limited field of view (FOV) in the required working distance. In order to deal with the challenge, multicamera stereo-DIC, in which three or more cameras are employed to capture the surface images of the specimen from multiviews, has been developed [11–14]. Generally, the multi-DIC system can be regarded as a combination of multiple regular stereo-DIC systems, firstly each stereo-DIC measures the local 3D shape at different regions of the specimen surface, and then the obtained results via individual stereo-DIC are stitched or mapped to the universal reference system with precalibration parameters. In spite of the fact that the multi-DIC system produces a sufficiently larger effective FOV, the uncertainty of the system might increase and the robustness deteriorates with the growth of the camera number. Moreover, multicamera calibration with high precision and flexibility is still a challenging problem.
In this work, an alternative approach of the Scheimpflug camera-based stereo-DIC method is provided. The Scheimpflug camera, adopting the Scheimpflug condition by tilting the lens with respect to the image plane, enables extending the measurement range of the stereo-DIC system without zooming the aperture, which has been widely employed in the field of particle image velocimetry (PIV) [15], line structured light [16], and portable 3D laser scanner [17]. However, to the best of our knowledge, Scheimpflug cameras have been rarely involved in stereo-DIC literatures so far.
The outline of the paper is as follows. Section 2 elaborates the geometric imaging model including lens distortion of the Scheimpflug camera, on the basis of which a robust and effective stepwise calibration strategy is presented to calculate the intrinsic and extrinsic parameters of the stereo Scheimpflug rig. In Section 3, the tailored Scheimpflug stereo triangulation method and well-developed subset-based DIC algorithms utilized in this work are introduced in detail. For validation, practical experiments, including rigid motion tests and three-point bending tests, are performed in Section 4 to evaluate the performance of the proposed method. Finally, brief conclusions are drawn in Section 5.
2. Scheimpflug Camera Model and Stereo Calibration
2.1. Scheimpflug Camera Model
As depicted in Figure 1(a), the Scheimpflug principle, traditionally credited to Theodor Scheimpflug in 1902, states that the object plane (the plane that is in focus), the thin lens’s plane, and the image plane must all meet in a single line, the Scheimpflug line [18, 19]. The principle is applicable to both thin prism and thick prism models, with minor corresponding modifications. To ensure its further high-quality measurement in stereo-DIC systems, accurate calibration of the Scheimpflug stereo rig is the prerequisite and most essential step. Yet the conventional calibration methods are not valid in this case because the assumptions used by classical calibration methodologies are not satisfied anymore for cameras undergoing the Scheimpflug condition [20]. Therefore, more and more researchers devote themselves to the related research, and a variety of methods have been proposed to accommodate various applications [21–25]. Inspired by the conclusion drawn in literature [26], the Scheimpflug imaging model based on RMM (rotation matrix model) is adopted in this section.
(a)
(b)
RMM models the lens-image sensor configuration by an explicit rotation matrix about the optic axis and includes it as a part of the intrinsic calibration parameter set. As illustrated in Figure 1(b), is the camera coordinate system with the origin at the optical center, while the intersection of the optical axis with the ideal image plane is referred to as the principal point. Denoted as , the image coordinate system of the tilted Scheimpflug image plane shares the same origin with the in the image plane . And the symbols and indicate the rotational angles about axes and , respectively. The 3D world point projects on at point , and the light ray intersects at point .
The conventional perspective projection can be briefly expressed as where and indicate the rotation matrix and translation vector of the camera frame relative to the world frame, respectively. Usually, extrinsic parameters refer to the three angle components defining the successive rotation about axes and the three translation components. Besides, is the projective depth, and is the camera intrinsic matrix with respect to the ideal image plane, which is defined as where and denote the horizontal and vertical focal length in pixel units, respectively.
Taking the tilt effect of the Scheimpflug condition into account, the rotation which maps the plane onto the plane can be split into successive rotation about axes and with angles and .
Therefore, the projection in the tilted Scheimpflug image plane of the world point can be modeled as follows, and the detailed derivation process can be found in Reference [25]. where is the arbitrary scale factor and is the Scheimpflug array:
Substituting Equations (1), (2), (3), and (4) and letting the normalized coordinates in be denoted as , thus, the transformation that relates to the image point can be achieved.
Likewise, is the arbitrary scale factor. In view of the inevitable lens distortion, the radial and tangential lens distortions are taken into account on the ideal image plane [38]. where and represent the radial and tangential distortion coefficients, respectively. Thus, relying on Equations (1), (2), (3), (4), (5), (6), and (7), the Scheimpflug imaging model including lens distortion is established.
2.2. Stepwise Calibration of the Stereo Scheimpflug Camera
In consideration of the sensitivity and dependency of the Scheimpflug camera’s parameters, the stepwise calibration strategy analogous to the literature [23] is employed to allow an adequate scope for parameter values. Besides, on account of the unique advantages of the planar calibration object, a planar checkerboard is adopted in this section. Sufficient images of the calibration object ought to be acquired by two cameras synchronously from multiple views.
A stereo rig composed of two Scheimpflug cameras is considered. As depicted in Figure 2, and are the optical centers of the left and right cameras, respectively. A 3D world point projects on at point , the light ray intersects at point in the left images, and the camera baseline, respectively, intersects image planes and at and ; likewise, in the right images. Given all the calibrated intrinsic and extrinsic parameters of the stereo rig (), we have
Supposing that a set of image correspondences is given, then the corresponding world point set can be calculated combining Equations (8) and (9).
In general, the stepwise calibration scheme of stereo Scheimpflug cameras can be divided into five consecutive steps.
Step 1. Initialization of the intrinsic and extrinsic parameters of the two individual cameras. In terms of intrinsic parameters, the Scheimpflug angles (, ) can be roughly estimated according to literature [25] or just obtained from the datasheet. Meanwhile, identical with the frequently used method, the principal point is initialized at the center of the image plane. Furthermore, the distortion coefficients are assumed to be zero. For the extrinsic parameters , they can be calculated from the homographies of all captured images using Zhang’s method [20], without taking the Scheimpflug condition and lens distortion into account.
Step 2. Estimation of the Scheimpflug angles of two individual cameras. The projection from to the Scheimpflug image plane, depending on Equations (1), (2), (3), (4), (5), (6), and (7), can be denoted by . To estimate the Scheimpflug angles, the following objective function is minimized: with the corresponding featured image point number , the captured image number , and the optimization parameter vector . is the observation of marker for position . It should be noted that the parameters of the principal point and distortion are not involved in this optimization. Moreover, the minimizations in this step and hereinafter are performed by means of a suitable version of the Levenberg-Marquardt algorithms.
Step 3. Estimation of the principal points of two individual cameras. A similar objective function as in Step 2 is performed, while the optimization parameters are supposed to be changed into . Likewise, the Scheimpflug angles and distortion coefficient parameters are excluded from optimization this time.
Step 4. Determination of the distortion coefficients. Given the results obtained from the precious steps, the distortion parameters are taken into consideration in this step. And the optimization parameters are supposed to be .
Consequently, the intrinsic parameter vector η, including , of the two cameras can be acquired through the separated stepwise calibration procedure.
Step 5. Calculation of the relative orientation and position between the two cameras. Suppose that the optimized intrinsic and extrinsic parameter sets of two individual cameras are defined as and , respectively. Without loss of generality, the left camera is assumed to be the reference camera. Hence, initialization of the right camera pose with respect to the left can be expressed as Thus, the previous results can be merged in the reference coordinate system, on the basis of which an additional optimization is implemented. where refers to the number of cameras. And the optimization parameter vector converts into . Note that the optimized intrinsic parameters set is employed for the optimization. Hence, all the intrinsic and extrinsic parameters of the stereo Scheimpflug cameras have been obtained.
3. Stereo-DIC with Scheimpflug Cameras
To accurately determine the full-field surface displacement and strain of the specimen, the well-developed subset-based DIC algorithms are utilized to register the images of the specimen coated with a random speckle pattern in the reference and deformed states and to match the same region between the stereo images. Typically, a square reference subset centered at the predefined calculation point is chosen from the left reference image and employed to search its corresponding location in the right image. To this end, a robust zero-mean normalized sum of squared difference (ZNSSD) criterion [27], integrated with a second-order shape function [28], is adopted to quantitatively assess the similarity between the reference and target subsets. To enhance subpixel registration accuracy, the state-of-the-art inverse compositional Gauss-Newton (IC-GN) algorithm [29] and a biquintic B-spline interpolation scheme are utilized. Giving top priority to the accuracy rather than efficiency, the matching strategy adopted here is to correlate all the deformed images of the stereo rig to the left reference image [4].
Therefore, the desired disparity data of the region of interest (ROI) can be obtained. Together with the precalibrated parameters of the stereo Scheimpflug cameras, the 3D world coordinates of the measurement points on the specimen surface can be recovered via the tailored bundle adjustment-based stereo triangulation. It should be noted that the lens distortion must be taken into consideration during the triangulation process. Furthermore, the three displacement components, resulting from external loading, can be calculated by subtracting the 3D coordinates of the same interest points at the initial state from those of the deformed. In common with literature [30], a variation of the triangular Cosserat point element method proposed in [31] is utilized to calculate the full-field strain distribution. It is worth noting that the average rigid-body motion of the specimen should be subtracted to give a better insight into the relative deformation of the specimen surface.
4. Experiment Verification
4.1. Experimental Apparatus
To verify the performance of the developed Scheimpflug camera-based stereo-DIC system, a series of experiments, including stereo calibration of Scheimpflug cameras, rigid motion tests, and three-point bending tests of the specimen, were performed under controlled laboratory conditions.
As shown in Figure 3(a), experiments in this study have been conducted with a wide range of system configurations, which mainly include (1) two 8-bit IMPERX CCD Cameras (IGV-B2520M-SC000) employed to simultaneously record the images of the specimen surface from two directions, which have the spatial resolution of pixels, and equipped with Kowa lenses (focal length 50 mm) and custom-made Scheimpflug adapters, (2) a static loading frame (CTLD-2) of 2 kN capacity, (3) a high-precision vertical linear stage (Newport GTS30V) with the precision of 0.1 μm, and (4) a customized aluminum alloy three-point bending specimen, which is sketched in Figure 3(b), and the surface indicated by a yellow rectangle in the middle as depicted in Figure 3(c) is chosen as the ROI.
Note that all equipment aforementioned is mounted on a Newport optical bench, and specimen preparation consists of spraying black/white colour paint over the specimen surface to yield a suitable random speckle pattern. In order to obtain entirely focused images with small distortions, the cameras and lenses are adjusted with the aid of the Scheimpflug arrangement. Besides, uniform illumination of the specimen surface is guaranteed via holding two LED lamps on each side of the camera. Moreover, the specially tailored experimental data processing software is developed on the basis of the well-developed open-source toolbox Ncorr [32] and MultiDIC [30].
An example is presented in Figure 4 to compare the imaging performance of the Scheimpflug camera with that of the conventional camera under the same experimental configuration. The conventional camera here refers to the same cameras indicated in Figure 3(a) but without tilting the lens. Figures 4(a) and 4(b), respectively, show the specimen surface image obtained from the left camera based on the Scheimpflug imaging model and that based on the conventional pinhole imaging model; meanwhile, the enlarged local images are depicted on their right.
(a)
(b)
As illustrated in Figure 4(b), it is apparent that a larger range of sharp images of the ROI on the specimen surface can be acquired using the Scheimpflug camera. Besides, an effective global parameter (Mean Intensity Gradient (MIG)) [33] is adopted to further quantitatively assess the overall quality of the obtained speckle patterns. Twenty image pairs obtained via the Scheimpflug camera and the conventional camera are utilized for comparisons, and the MIG of captured speckle patterns from the Scheimpflug camera (16.8765) is obviously higher than that from the conventional camera (8.7836), which indicates that the Scheimpflug camera accordingly produces the smaller mean bias error and standard deviation error in the further DIC measurements. Thus, it can be deduced that the imaging performance of the Scheimpflug camera, under the experimental configuration in this paper, is superior to that of the conventional camera.
4.2. Stereo Scheimpflug Camera Calibration
Prior to specimen testing, the Scheimpflug camera-based stereo-DIC system should be well calibrated. As sketched in Figure 3(a), two Scheimpflug cameras are symmetrically arranged, with the normal to the specimen surface such that it approximately bisects the stereo angle. In the calibration experiment, a printed checkerboard pattern attached to a cardboard rather than high-precision calibration targets is adopted. The calibration pattern consists of a plane of grids with 5 mm checkerboard spacing. Totally, 30 images of the checkerboard under different orientations and positions are captured. Employing the proposed stepwise stereo camera calibration technique, the intrinsic and extrinsic parameters, as well as the reprojection errors of the stereo rig, can be retrieved in Table 1.
To further verify the calibration results of the Scheimpflug rig, the 3D coordinates and structure of 20 arbitrarily placed checkerboards are reconstructed via the rig with the calibrated parameters and simultaneously obtained image pairs. And we fit the reconstructed coordinates with an ideal flat plane and calculated the distance between the reconstructed points and the ideal plane. Moreover, a typical reconstruction result is given in Figure 5.
(a)
(b)
Figure 5(a) depicts the reconstructed 3D checkerboard points and the fitted plane while the error distribution of the reconstructed checkerboard points is presented in Figure 5(b). As indicated in Figure 5(a), the reconstructed 3D points agree well with the fitted plane in spite of minor deviations. Furthermore, it can be found out in Figure 5(b) that the deviations of reconstructed points are approximately symmetrical about the center of the fitted plane, while the deviations at the corners are more significant. Nonetheless, the maximum deviation of the reconstructed points is approximately 0.030 mm, which is still quite small compared with the depth of the checkerboard. Besides, the RMSE (Root Mean Squared Error) of the distance between all the reconstructed 3D points and the corresponding fitted planes under 20 different poses is 0.0133 mm. Consequently, the experimental results verify the effectiveness and accuracy of the calibration method.
4.3. Rigid-Body Motion Test
To quantify the metrological performance of the developed Scheimpflug camera-based stereo-DIC system, a null strain test was performed on the specimen subjected to a rigid-body motion [34]. As these fields are theoretically equal to zero, we can thus obtain the overall error of the displacement measurements. The three-point bending specimen is positioned on the high-precision vertical linear stage, and the stage is moved along the vertical direction from 0 to 10 mm and back to 0 mm by steps of 1 mm with regard to the reference position. Stereo images are simultaneously captured for each position of the specimen. According to the criterion developed in [35], a subset size of pixels coupled with a step size of 5 pixels is chosen during the stereo-DIC processing; therefore, the full-field displacements and strains of the specimen surface can be calculated.
Given that the reference frame of the vertical linear stage is not identical to the reference frame of the proposed stereo-DIC system, the scale of the displacement , rather than the 3D displacement components, is compared between the proposed stereo-DIC measurements and the readout of the vertical linear stage.
As shown in Figure 6(a), the average displacement measurements of the specimen are compared with reference values obtained from the vertical linear stage, and the corresponding standard deviation of displacements is illustrated in Figure 6(b).
(a)
(b)
It can be observed that the displacement measurements for the Scheimpflug-based stereo-DIC system is in well conformity with the reference imposed vertical displacements. It is worth noting that as the imposed vertical displacements increase from 1 to 10 mm, the average errors of the measured specimen displacements are enlarged from 5.8 μm (unit of micron) to 50.9 μm. In the meanwhile, the standard deviations of the specimen climb from 1.9 μm to 10.3 μm. Despite all these, the average errors and standard deviations of displacement measurements are rather low, even in the case of large vertical translations. As the translation is increased, the percent of average errors and standard deviations in displacement measurements has remained approximately constant. Consequently, these results indicate that the displacement measurement for the Scheimpflug-based stereo-DIC system is fairly accurate.
4.4. Three-Point Bending Experiment
To further verify the validity of the proposed method, a three-point bending experiment is performed on the specially customized specimen, which is widely adopted to identify the flexural and tensile strength of quasibrittle materials. The experimental setup is exhibited in Figure 3, the specimen is positioned on the center of the loading frame, and a pair of images captured at a small load (about 20 N) is taken as the reference configuration. It needs to be emphasized that the small load applied here is to overcome excessive rigid-body motion due to any slack in loading fixtures [36]. As the applied load increases from 20 up to 2000 N with the increment of 200 N, a series of image pairs are recorded as the deformed configuration. The same settings as in the rigid-body motion test are applied to the subset-based matching operation. Therefore, full-field distributions of the specimen surface displacement and deformation can be obtained and utilized to explain the underlying deformation mechanisms in the three-point bending test. Moreover, Figures 7(a)–7(c), respectively, depict the typical evolution of full-field distributions of displacement components in three directions along with the increase in the applied load (left 600 N, middle 1200 N, and right 1800 N).
It is worthwhile to notice that all the experimental results in this paper are obtained in the left camera coordinate system. As shown in Figure 3(a), the -axis points to the specimen surface, while the -axis is vertically downward and the -axis forms a right-hand coordinate system. Even though the small angle between the -axis and the normal of the specimen surface exists, for convenience of analysis, it is ignored and assumed that the -axis is perpendicular to the specimen surface, while the -axis and -axis are, respectively, parallel to the longitudinal and transverse directions of the ROI. Namely, the static loading direction is approximately parallel to the -axis.
Globally, displacement maps of the components in three directions in Figures 7(a)–7(c) exhibit typical patterns that are characteristics of three-point bending tests. As illustrated in Figure 7(a), the displacement fields of the -components are distributed symmetrically in the center and the displacements in the middle part of the specimen surface are relatively low and gradually increase towards both ends of the specimen surface. With the increase in the applied loads, the upper surface of the specimen tends to move towards the middle, while the lower surface moves towards the opposite direction. As for -components in Figure 7(b), they are axially symmetrically distributed, and the middle parts of the specimen surface move downward along the -axis, while the sides move upward. In the case of the displacements of -components, they are much smaller than those in the other two directions. In general, the displacements at the edge of the specimen surface are larger than those at the central part. Actually, for the arrangement in these experiments, the displacements of -components can be approximated as the out-of-plane displacements on the specimen surface. Moreover, Figure 7(d) shows the full-field distribution of the specimen surface displacement magnitude. It can be observed that two fixed minimum points appear between the three loading heads and do not change as the applied loads increase.
Besides, the full-field distribution of Lagrangian strain magnitude maps corresponding to the typical applied loads is shown in Figure 8(a); meanwhile, Figure 8(b) illustrates the relative surface area change of the specimen. It can be found that the strain mainly occurs at the upper and lower parts of the specimen surface, while the strain on the neutral plane is rather close to zero. Further, as revealed in Figure 8(b), the upper surface area of the specimen is compressed while the lower surface area is stretched, and this trend is more pronounced as the applied load increases. Considering the geometric size of the aluminum alloy specimen and the limited capacity of the static loading frame, the deformation and strain levels of the specimen are relatively small. Despite all that, the experimental results of three-point bending further validate the effectiveness of the proposed method. Further analysis of the specimen’s mechanical properties is beyond the scope of this work but can be found in [37].
5. Conclusions
In this article, a novel Scheimpflug camera-based stereo-DIC method is developed for the full-field 3D shape and deformation measurement. In contrast with conventional cameras, the Scheimpflug camera enables extending the measurement range of the stereo-DIC system without zooming the aperture, which exhibits significant advantages over conventional cameras in stereo-DIC measurements. Also, a robust and effective stepwise calibration approach is presented to calculate the intrinsic and extrinsic parameters of the stereo Scheimpflug rig. Consequently, with the aid of the tailored stereo triangulation method and well-developed subset-based DIC algorithms, the three-dimensional shape and deformation of the specimen can be retrieved. Finally, experimental results of rigid-body motion tests and three-point bending tests successfully demonstrate the accuracy and effectiveness of the proposed approach. Due to the attractive advantages of employing Scheimpflug cameras, the proposed technique offers new avenues for performing 3D deformation measurements and indicates great potentials in further fields of stereo-DIC measurements.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This research was funded by the National Natural Science Foundation of China (NSFC) under Grant Nos. 11872070, 11727804, and 11902349.