Abstract
Grass is one of the most important resources in the ecosystem for the sustainable development of human beings. However, the studies focusing on grass identification, which were traditionally implemented by experts with low efficiency and precision, cannot meet the requirements of modern grassland management. In this study, we proposed cubic interpolation LBP (CILBP) and dbN wavelets for grass identification based on leaf images. A low-frequency component of leaf images decomposed by dbN wavelets was used as the input of CILBP for more subtle texture extraction. The novelty of the proposed method was that CILBP can better describe the texture features from the low-frequency subimage, as compared with the original bilinear LBP. The effectiveness in identification accuracy of the proposed method for grass leaf was demonstrated by the experimental results.
1. Introduction
Grass is one of the most important resources in our ecosystem. Grass identification is still a difficult and challenging task for the investigators because of its variations in shape and size during different growing stages and environments. Grass has its own life cycle from seeds to seeds, including sprouting, forming leaves, blooming, and fruitification. Among the cycle, leaves are more obvious and long-existed organs. They are almost always used to supply features including shape, texture, and venation for classification in botany [1]. Shape and appearance of leaves give an intuitive impression for the observers in the earlier stage, which are often seen in the references of botany classifications [2]. Until now, most of botany classification is dependent on the knowledge of the experts because the shape and appearance are ambiguous among the closely related species. Texture provides an alternative way in the recognition. Zhang et al. [3] proposed a fusion of superpixel, K-means, and PHOG in a plant’s diseased leaf segmentation and recognition. Recently, deep learning has been applied in plant leaf classification for a promising recognition result [1].
As an important part of botany, grass identification based on leaves has attracted the attention of the investigators. For example, Wang et al. [4] classified leguminous forage based on shape features of leaf images. The global shape features include the axis ratio, rectangularity, and 7 invariant moments of leaf images. The roughness of the leaf edges are extracted as the local features. Both global and local features are used as the input to probabilistic neural network (PNN) and back propagation network (BP) for classification, yielding correct classification rates of 85% and 82.4%, respectively, in a database of 560 training samples and 1,400 testing samples comprising 14 species.
However, even though some advanced processing tools such as deep learning have been applied in the area, the tedious parameter setting procedure makes the identification complex and time-consuming. At the same time, some classical methods such as local binary pattern (LBP) transform can be feasible for these practical tasks. LBP [5] is an effective texture feature extraction algorithm with invariance to grayscale changes of monotonous image. Marko et al. [6] investigated leaf identification using LBP, Hu Monuments, and SVM for a higher identification rate, suggesting the effectiveness of LBP in leaf texture.
Therefore, the texture features of grass leaf are still the focus of our study. We proposed a robust algorithm by aggregating cubic interpolation LBP (CILBP) and wavelet transform for grass identification. Wavelet transform is the mathematical basis for low-dimensional representation of images, which can decompose the images into multiresolution and multiscale components. Three main steps are involved in the algorithm. Firstly, the ROI images of grass are segmented by mathematical morphology. Then, the images are decomposed into multiresolution components by wavelet transform. Finally, the images are transformed with CILBP to achieve more precise presentation. The effectiveness of the proposed method is demonstrated by the experiments. The novelty lies in the application of cubic interpolation in LBP, describing more detailed texture in grass leaf.
The remainder of this study is organized as follows. Section 2 predominantly focuses on the image-acquisition and image-preprocessing procedure of the database for experiments. Section 3 mainly introduces the algorithm for grass identification of grass leaves in our study: dbN wavelet and CILBP. The experimental results and discussions are listed in Section 4. Section 5 highlights the conclusions.
2. Materials
The grass leaf samples were taken directly from the suburbs of Huhhot. In order to ensure the credibility of the experiment, the leaves should be of different size and intact. All the preprocessing and later experiments were executed on Mathworks Matlab R2016a and Windows10 (Intel (R) Core(TM) i7-8700 CPU 3.20 GHz, 16 GB RAM).
2.1. Image Acquisition
The specific image acquisition process was as follows. First of all, leaves of grass were picked up and pressed inside a book to ensure smooth surfaces. Then, the images were acquired by using the built-in camera of a Samsung mobile phone with a resolution of 2560 × 1920 pixels inside the lab, where the camera should be kept in the same acquisition distance of about 30–50 cm from all samples. Finally, the image samples were uniformed into 512 × 512 or 512 × 256 in JPG format, according to its ratio of blade shape and length-width. Accordingly, the leaf image database composed of 150 leaf images of 15 common grass species constructed for identification experiments. Some sample images in the database are shown in Figure 1.
(a)
(b)
(c)
(d)
2.2. Image Preprocessing
The preprocessing stage included image grayscale and denoising. Owing to that the colors of grass vary with time and growing stages, texture of the leaves were preferred to the colors as more stable features. In addition, the storage capacity and computing expense on color image were very large. Therefore, the color images were converted into grayscale ones for further simple and time-saving operations.
Because the leaves in the shooting process were vulnerable to be affected by equipment cleanliness and other environmental factors, there existed image noises. Therefore, they must be removed from the images, where a 3 × 3 mean filter was adopted for denoising. Figure 2 illustrates the preprocessing procedure of a grass leaf image, in which the left, middle and right ones correspond to 512 × 512 original JPG images, grayscale image and denoised image, respectively.
(a)
(b)
(c)
3. Methods
The proposed algorithm used dbN wavelet and CILBP to extract feature vectors of preprocessed image samples. The flowchart of the proposed algorithm is shown in Figure 3.
3.1. Wavelet Transform
Wavelet transform decomposes an image into a low-frequency band and three high-frequency bands, including a LL approximation component, LH horizontal component, HL vertical component, and HH diagonal component. Because different wavelet basis leads to various results, the criteria for selecting a wavelet function usually include support length, symmetry, and regularity. A wavelet with symmetry does not produce phase distortion, and a good regularization wavelet can easily obtain smooth reconstruction curves and images with minor error. Considering all the above factors, we chose the wavelet function dbN proposed by Ingrid Daubechies, in which N is the vanishing moment of this wavelet function [7]. Although the larger the vanishing moment, the smoother the wavelet function and the longer the support length, and the consumption will also increase accordingly [8]. And except for N = 1, dbN has no symmetry. Considering the criteria for determining wavelets and the effect of the algorithm, “db1” was obtained in this study.
Figure 4 illustrates four component images decomposed by the wavelet. The first one is a low-frequency subimage, which contains most of the details of the leaf image, and the other three are high-frequency subimages corresponding to the edges and contours of leaf. Focusing on the leaf texture information, the high-frequency components were discarded in the study. The identification results with different frequency components of wavelet decomposition were compared in the experimental section.
(a)
(b)
(c)
(d)
3.2. CILBP
Local binary patterns (LBP) were first proposed by Ojala et al. [9] for texture features extraction of an image. The LBP originated from the initial 8-neighborhood analysis was developed into block and partition LBP within a circular neighborhood. When the local neighborhood texture feature of the grayscale image was extracted in the circular neighborhood, bilinear interpolation was a traditional way to deal with the neighborhood which can not accurately fall to the center of the pixel. A linear interpolation was to linearly determine the value of a point on a line of two pixels based on their respective values. A bilinear interpolation was the two-dimensional extension of linear interpolation, which can be obtained by a series of first-order linear interpolations. The output value was the weighted average of the grayscales from the nearest 2 × 2 neighborhood of the sampled point in the input image.
However, the smoothing effect of bilinear interpolation will degrade the details of the image in some geometric operations. Meanwhile, the discontinuity leads to an unexpected outcome. These disadvantages could be overcome by a higher-order interpolation, in which the output value is the weighted average of the grayscales from the nearest 4 × 4 neighborhood of the sampled point in the input image. Considering the time complexity, a cubic interpolation was applied to calculate the dots that can not fall precisely on the centers of the grayscale image in our study. To distinguish the LBP with different interpolations, we abbreviated the cubic interpolation and common bilinear interpolation LBP as CILBP and BILBP in this paper.
In image processing, the weighted interpolation of 16 adjacent pixels around the pixel (i + u, j + ) was taken into consideration to calculate the gray value of the target pixel (x, y). The calculation formulas are as follows:wherewhere
Figures 5 and 6 correspond to the encoding images of a leaf sample using BILBP and CILBP, respectively. The four images in Figures 5 and 6 correspond to block numbers 1, 2, 3, and 4, respectively. Comparing these two groups of figures, it can be clearly observed that the CILBP-encoding image was smoother, manifesting that cubic interpolation better maintain the details of the image. The more fitting points involved in the calculation, the more accurate output of the interpolated pixel can be obtained. Inevitably, the number of pixels involved in the calculation affected the computational complexity as well. The compromise between accuracy and time complexity made the cubic interpolation feasible in this study.
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
CILBP can improve the eigenvector at the same time. Table 1 compares the feature vectors of a low-frequency subimage, extracted by BILBP and CILBP. The BILBP feature vectors contain 24 zeros with some missing information, whereas the statistical data CILBP are distributed more evenly, which contributed a higher identification accuracy.
3.3. Feature Matching
Euclidean distance in the two-dimensional space is the distance between two points, which is commonly used for distance metric. The concrete formula is
The nearest neighbor classifiers were used for identification, in which the minimum Euclidean distance between the test samples and training sample determined the category.
4. Experimental Results and Discussion
In order to verify the effectiveness of the proposed integration of dbN wavelet transform and CILBP for grass leaf identification, the following three groups of experiments were executed. Firstly, the experimental performances of the blocks BILBP and CILBP were compared. Secondly, block and partition were taken into account in LBP (referred to as the block LBP and partition LBP in the following context) for comparison. Thirdly, the identification performances of LBP based on low-frequency image and reconstructed image after wavelet decomposition were compared. The identification rate and average identification time were the main focuses of the identification performance, in which the former was the ratio of the correctly recognized image number to the total image numbers, and the latter was the average time required to identify each test sample.
4.1. Identification Performances of the Blocks BILBP and CILBP
Table 2 lists the identification rates of BILBP and CILBP based on low-frequency subimages decomposed by dbN wavelet. The correct identification rates of BILBP were 88% and 84% corresponding to the block numbers 1 and 2, respectively. Comparatively, the identification rates of CILBP reached 91% and 89%, 3% and 5% higher than those of BILBP. The improvements came from better description of texture using cubic interpolation in LBP. The identification rates of BILBP and CILBP both decreased when the block number increased to 3. The larger block size aroused by greater block number with rougher textural graininess led to more texture information loss. Especially, the identification rates of CILBP were 79% when the block number was 4, 8% lower than that of BILBP, both failing to meet the needs of identification accuracy because of the tough texture description. It can be concluded from above that CILBP can greatly improve the identification accuracy as compared with original BILBP when the block number is from 1 to 3.
Average identification time was also compared in Table 2. Generally speaking, CILBP required relatively more time because of more complicated calculation of cubic interpolation as compared with bilinear interpolation, which coincides with the average identification time consumption as listed in Table 2. The increase in block number results in less average time needed for rougher graininess analysis of larger block size. The identification time of CILBP and BILBP dropped to 0.37 s and 0.30 s when the block number was 3. Taking into consideration of both identification accuracy and average time, CILBP with block number 2 was preferred for an overall optimal performance of 89% identification rate and 0.72 s average time.
4.2. Identification Performance of Block and Partition CILBP
Table 3 shows the performance of block and partition CILBP and BILBP based on low-frequency components decomposed by dbN wavelet. According to the biological characteristics of grass leaf texture, in which more coarse and obvious textures are close to the petiole, and more delicate and abundant texture with an overall symmetry occur on the leaf edges, vertical partition 1 × 4 was chosen in this experiments. As shown in Table 3, block and partition CILBP also can greatly improve the identification accuracy as compared with block and partition BILBP when the block numbers were from 1 to 3.
The top identification rate of CILBP in Table 3 reaches 96%, 5% higher than the data in Table 2 under the same block condition, and average time was shorter than that in Table 2. In addition, Figures 7 and 8 correspond to the comparison of identification rates and average time listed in Tables 2 and 3. The improvements demonstrated that proper partition can effectively enhance the texture information of the local differences in the image.
To make further comparison between the performance of CILBP and BILBP, their ROC curves under the condition of 1 4 partition and 1 block are plotted in Figure 9. Each of the images in the testing set was matched with all images in the training set. Since each class had 5 samples in the training set, there were 5 correct and 70 incorrect matches for each sample in the testing set. If the matching distance between two images was smaller than the threshold incorrect matches, the match was a genuine acceptance. Figure 9 shows that genuine acceptance rate (GAR) of CILBP is higher than that of BILBP at the same false acceptance rate (FAR).
4.3. LBP Based on Reconstructed Image
Table 4 lists the recognition results of CILBP and BILBP based on reconstructed images whose components were composed of three high-frequency subimages and the low-frequency subimage. Figure 10 illustrates the comparison of identification rates of CILBP listed in Tables 3 and 4. Compared with Table 3, it was found that the identification rates were not significantly improved when the high frequency information was utilized. Therefore, the high frequency component can be discarded with no obvious improvements.
5. Conclusions
In this paper, we proposed cubic interpolation LBP (CILBP) to extract more subtle texture contained in the low-frequency component decomposed by dbN wavelets for grass leaf identification. A first-level decomposition of the image can obtain the low-frequency image, 1/4 of the original size, which shortened computation time on feature extraction. Meanwhile, the reconstructed high-frequency components were abandoned for no obvious influence on the identification effect while extending identification time. CILBP can effectively highlight the grass leaf texture features from the low-frequency subimage with cubic interpolation when the gray values of the original images were highly similar, as compared with bilinear interpolation of common LBP. The experimental results showed the effectiveness in identification accuracy of the proposed algorithm for grass leaf.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Disclosure
Fan Han and Xue Qiao are co-first authors.
Conflicts of Interest
The authors declare that this article content has no conflicts of interest.
Authors’ Contributions
Fan Han and Xue Qiao contributed equally to this work.
Acknowledgments
This work was supported by the National Natural Science Foundation of China under Grant Nos. 61562067 and 61962048.