Detection and Assessment of Hull Plate Corrosion Damage Based on Image Recognition Techniques

Corrosion is a major cause of failure in aging ship structures,¹  and has increasingly attracted the attention of relevant researchers. Corrosion damage can lead to a decrease in the ultimate strength and carrying capacity of ship structures, and even result in buckling or fracture. In the past decade, image recognition and processing techniques based on corrosion damage datasets as well as convolutional neural network (CNN) models have played an increasingly important role in the field of corrosion damage detection and assessment due to the rapid development of artificial intelligence and deep learning.

For the detection and assessment of corrosion damage, current research can be mainly divided into three aspects. First, traditional digital image processing methods are used for macroscopic corrosion image analysis, which started in the 1960s and has since become very mature. When analyzing pitting corrosion, the corrosion images are usually obtained through a scanner and then subjected to threshold processing. The pitting rate is calculated by counting the number of pits to describe and evaluate the pitting corrosion situation.²  In addition to traditional digital image processing methods, combining fractal theory with digital image processing to present the clear morphology of macroscopic corrosion images is an important research direction. By using a binary image method and establishing a fractal model, the macroscopic corrosion morphology features can be extracted and quantitatively analyzed to assess corrosion characterstics.³ 

The second aspect is the analysis of the microscopic morphology of corrosion images. For example, by analyzing the metallographic structure of the images, the relationship between the depth of the corrosion pits and the corrosion area can be established, and a three-dimensional distribution map of the corrosion pits can be obtained.⁴  In addition, microscopic corrosion images can be observed under a microscope and the wavelet transform method can be applied to analyze the morphological features of corrosion pits. This is also a commonly used method in the field of corrosion detection.⁵  In recent years, as obtaining high-resolution images has become easier, multiscale statistical analysis methods based on high-resolution and large-field-of-view optical image recognition for the spatial distribution of corrosion pits have seen wider applications.⁶ 

The third aspect is the current research hotspot of pattern recognition and neural network applications in corrosion image recognition and processing. Initial neural network research mainly utilized artificial neural networks and applied them to established corrosion damage models, which in turn assessed changes in structural ultimate strength.⁷  In recent years, with the development of deep learning, CNNs have been increasingly applied by researchers in the field of corrosion damage detection. Related studies have shown that the performance of CNNs is superior to that of multilayer perceptron networks based on visual corrosion detection developed using texture and color analysis.⁸  Sufficient image datasets are required to support the use of CNNs. A common research method is to establish a relevant database of the research object, conduct testing on the CNN structure, and modify, train, validate, and test the CNN architecture using this database.⁹  In addition, the structure of the CNN is usually adapted to the specific research objectives to make it more suitable for corrosion detection of the target structure. Algorithms such as overlapping windows are also combined to identify and locate the corrosion regions.¹⁰ 

In the last decade, many researchers have devoted themselves to making full use of computer technology in ship corrosion inspection and have achieved certain results.^11-12  For example, by introducing a reliable and cost-effective corrosion damage model development technique to predict the residual life of ship structures with nonlinear corrosion damage over time.¹³  The random field theory is used to describe the defects of the ship structure, and the artificial neural network is used to evaluate the variability of the ultimate strength of the VLCC cruise ship under the action of random geometric defects and the ultimate strength of the ship beam with different defect models is predicted.⁷  In China, there is a faster regional CNN architecture and VGG-19 model for deep transfer learning to achieve rapid corrosion detection in the field of ships.¹⁴ 

CNNs are now commonly recognized not only in engineering but also in medicine. Relying on its great signal processing capability, the automatic extraction of local features from biomedical signals through convolutional layers shows significant advantages in recognizing anisotropic square waves in heart failure, which greatly facilitates the recognition and diagnosis of heart failure.^15-16  In malignant tumors, CNNs can help doctors in diagnosis by recognizing features in images and are applied to predict the disease risk and prognosis level of patients.¹⁷ 

Currently, research mainly focuses on the detection and assessment of corrosion damage in metal materials,^18-19  but the results are not suitable for engineering applications. This paper analyzes the factors that affect the accuracy of the evaluation of CNN models and obtains a corrosion damage evaluation model for underwater hull plating. A corrosion damage evaluation system is established which can be used for corrosion damage detection and assessment of ship structures.

In the experiment, the entire Q345D steel plate was laser-cut into 50 mm × 50 mm × 4 mm specimens. Plastic compartment boxes were used to avoid corrosion of the immersion tank by the corrosive solution. Relevant corrosion solutions and weighing operations were required in the experiment, so instruments such as measuring cups and electronic balances were prepared. According to the current relevant research results, 3.5% sodium chloride solution is the concentration that has the highest corrosion rate on steel.²⁰  In addition, H₂O₂ was added to seawater to strengthen the cathode depolarization to simulate and accelerate the corrosion in the seawater immersion zone of hull steel. The test results were similar to those in a seawater system, and the composition of the rust layer was the same.²¹  The experiment selected a mixed solution of 0.8 mol/L hydrogen peroxide and 3.5% sodium chloride solution as the accelerated corrosion solution for simulating a seawater immersion zone. As this study is only concerned with the recognition of corrosion from images and the temperature of the corroded samples does not directly affect the image recognition results, the effect of external environment parameters such as temperature has not been directly considered.

Next, the prepared accelerated corrosion solution was poured into the corrosion box, and the surface-treated specimens were completely immersed in the accelerated corrosion test box. The height of the solution was approximately two-thirds of the height of the test box. The specimens were kept from touching the walls of the box and were stably immersed in the corrosion solution, as shown in Figure 1(b). The cycle of the accelerated corrosion test was set to 10 d, 20 d, 30 d, 40 d, 50 d, and 60 d.

At the end of each corrosion period, the corroded specimens were carefully removed and placed on a table to dry until the surface moisture evaporated. The specimens were then photographed and their weights were measured. All specimens were then subjected to surface rust removal, weighed again, and their weight was recorded. After replacing the corrosion solution, all specimens were returned to their original positions for the next corrosion cycle. These steps were repeated until the completion of six accelerated corrosion cycles.

From the analysis of rust morphology, in the initial stage, the rust layer on the surface of the specimen was thin and had poor adhesion, which was loose and allowed more oxygen to enter, accelerating the corrosion process. In the middle stage, the corrosion rust layer gradually thickened, and the inner rust layer increased, but less than the outer rust layer. The outer rust layer was loose, while the inner rust layer was dense. In the late stage of corrosion, the outer rust layer increased slowly and was almost as dense as the inner rust layer, hindering the entry of oxygen and slowing down the corrosion rate.

From the analysis of corrosion products, the outer rust layer mainly consists of loose and porous β-FeOOH and γ-FeOOH. The continuous and dense Fe₃O₄ and α-FeOOH are the main components of the inner rust layer. In the initial stage of corrosion, the outer layer of the product occupies most of the space, and the corrosion product increases rapidly. As the corrosion continues, the inner layer of the product occupies more and more space, and the corrosion rate gradually slows down.

Increasing the diversity of the corrosion image database is crucial to avoid the uniformity of images and improve the training of corrosion damage assessment models. This can be achieved by considering several aspects such as distance, lighting conditions, shooting angles, and postprocessing techniques.

The corrosion image database contains information on the experimental conditions and data augmentation methods used in the precorrosion experiments, along with the storage path of the images. The database is mainly divided into several categories based on different experimental periods, each containing all samples. Each sample is numbered and includes the corresponding images, shooting conditions, and data augmentation methods.

In terms of shooting distance, both far and close-range shooting methods were used. For lighting conditions, bright and dim conditions were designed to simulate the uncertainty of lighting conditions during actual shooting, which enables the corrosion assessment models to recognize images affected by various lighting conditions. To approach real-world scenarios, the shooting angles were diversified by tilting the camera.

Corrosion images obtained from experiments were subjected to recognition and processing using digital image processing techniques and unsupervised learning clustering methods to extract corrosion feature parameters. OpenCV^† was used to implement the corrosion parameter extraction from the images in digital image processing techniques. The techniques included gray transformation, edge detection, and contour extraction, to obtain the corrosion grayscale image and corrosion binary image. Edge detection and contour extraction methods were used to further identify the corrosion regions in the images. By calculating the number of pixels in the corrosion region, the area of the corrosion region and the ratio of the total area of the corrosion can be obtained.

Digital image processing techniques can provide accurate and efficient analysis of corrosion damage, which is crucial for ensuring the safety and reliability of structures and components in various industries. The use of unsupervised learning methods can also help automate the analysis process and reduce human errors.

For complex corrosion images with ambiguous corrosion boundaries, unsupervised learning clustering methods can be used for image recognition and segmentation. The self-organizing neural network algorithm is used to segment the corrosion area, and the color difference between the corroded and noncorroded areas is used for clustering. The area of the corroded region and the ratio of the total corroded area to the total area are calculated by counting the number of pixels in each region.

In the second step, the initial clustering result is further refined by performing one or two additional rounds of clustering to filter out the ambiguous corrosion areas. Finally, the corroded regions are comprehensively clustered based on the selected features, resulting in a fully labeled corrosion map, as shown in Figures 6(c) and (d).

This paper’s corrosion assessment method is mainly based on the proportion of the corroded area to the entire material surface, referring to the coating inspection and grading rules in GB8923-88 “Surface Rust and Rust Removal Grades for Steel Before Coating” and Section 8.2 of the “Guidelines for Corrosion Inspection of Ship Structures” by China Classification Society. In cases where the thickness of the rust layer in a rust image is difficult to distinguish, or where there is a large area of rust but not much rust loss or gain, it is not possible to determine the degree of rust based on the area ratio alone. Therefore, the mass method can be used for further evaluation. This paper uses the weight gain ratio as one of the parameters for corrosion damage assessment. The weight gain ratio is calculated by measuring the weight change of the test piece before and after corrosion and using Equation (2).

Combining the weight gain ratio and the average corrosion rate characterized by the depth method, a set of evaluation standards for the degree of corrosion damage has been developed, as shown in Table 1. Based on the comprehensive evaluation of these two parameters, the degree of corrosion damage is classified into five levels according to the values of weight gain ratio and average corrosion rate. Level 0 corresponds to extremely slight corrosion or no corrosion, while level 1 corresponds to mild corrosion. As the level increases, the corrosion becomes more severe, and when reaching levels 3 or 4, the material becomes unusable due to the severity of the corrosion.

The specific corrosion damage assessment method is presented in Table 2 and is divided into five levels denoted as 0, 1, 2, 3, and 4. The table provides a set of standards for evaluating the degree of corrosion damage based on the weight gain ratio and the average corrosion rate characterized by the depth method. The lower portion of the table lists example corrosion images corresponding to each corrosion damage level.

In level 1 corrosion, all area ranges of floating rust are included, which means that very slight corrosion can only be seen on the surface of the material as slight rust, and it can even be easily wiped off, as shown in Figure 7(b). Level 1 corrosion also includes situations where a small amount of pitting corrosion occurs on the material surface, which means that the material has been corroded by a small area of trace rust, as shown in Figure 7(g). This kind of corrosion has a very minor impact on the material properties. In level 2 corrosion, trace rust occupies nearly half of the area, as shown in Figure 7(c), or the material surface exhibits relatively slight layered rust, as shown in Figure 7(h). Layered rust is a thick corrosion product on the material surface that seriously damages the original structure of the material surface, leading to increasingly severe corrosion. Therefore, layered rust is a very dangerous presence for the material.

In level 3 corrosion, trace rust covers more than half of the area and begins to spread on the material surface, as shown in Figure 7(d), or layered rust begins to increase and gradually approaches half of the material surface area, as shown in Figure 7(i). This belongs to a relatively severe corrosion situation. In level 4 corrosion, the material surface is covered with layered rust, and it is almost impossible to see any completely uncorroded surface, as shown in Figures 7(e) and (j). This is a very severe corrosion situation and requires immediate protective measures. By using the above classification method, the degree of corrosion damage can be subdivided in detail, and the existing corrosion images can be assessed based on this table.

In the training of CNN, the images are divided into a training set and test set. The test dataset is the images that have not been entered into the training set and the ratio of the two is 7:3. Because the process of making the test set is similar to that of the training set, it will not be explained too much.

Regarding the construction of CNN model, there are many good CNN models given by many researchers through a lot of research and analysis, including LeNet,²³  AlexNet,²⁴  VGGNet,²⁵  GoogleNet,²⁶  ResNet,²⁷  and these classical networks have been used by the researchers a lot. The LeNet and AlexNet structures are simpler and have less room for improvement, while the GoogleNet and ResNet are more complicated to improve. As VGG networks use deeper hierarchies and smaller convolutional kernels, they can extract richer image features and perform better for complex visual tasks. On the other hand, the structure of VGG networks is relatively simple and easy to understand and modify. Researchers can adjust the network structure according to specific task requirements to improve performance or adapt to different application scenarios.

Corrosion image features are designed for corrosion pits and other corrosion pits with depth features The VGG network can better identify such corrosion pit features and just like the VGG structure it is easy to adjust and can be adjusted by a small number of parameters to achieve the effect of recognition of corrosion images. Therefore, this paper establishes a CNN model that considers the accuracy and efficiency, takes the CNN model VGG as the basis, and adjusts its structure so that it is suitable for the corrosion damage level classification task of corrosion images.

For the input layer, on the one hand, because its corrosion products are more complex, its corrosion features are more reflected in texture and color, etc., and corrosion features are not obvious; on the other hand, for different corrosion damage levels of the image features are more similar, if the use of a low resolution may make the subsequent convolution operation can not be able to extract enough initial corrosion features, leading to the poor model training effect, so the corrosion image The input size is adjusted to 320 × 320.

The number of neurons in the fully connected layers of the original VGG network structure was adjusted to avoid overfitting, so the number of neurons in the fully connected layers was adjusted to 1,000 in layers F1 and F2. As the ultimate goal of this model is to evaluate the input corrosion image to one of the five corrosion damage levels. the F3 layer is set to five neurons. The output layer is implemented using the activation function Softmax. The specific structure of the final CNN model is shown in Table 3..

This paper focuses on the role of three parameters, namely learning rate, batch size, and training rounds, in the training process of the k CNN model. The hyperparameters used for training this model and their settings are shown in Table 4. The learning rate setting for the training of this model uses the following standard: first set a large learning rate for observation, so that the model loss value decreases significantly, and then gradually decrease the learning rate to prevent the global optimum from being skipped. The learning rate is set to 0.001, 0.0005, 0.0001, and 0.000054 cases to train the CNN model, respectively.

Large batches of training can make the loss function converge faster, but the loss value may fall into a local minimum; selecting small batches for training can introduce randomness, which is conducive to the improvement of the model training effect, but may lead to oscillation of the loss function and failure to converge to the minimum value. Comprehensive computer configurations, 4, 16, and 32 batches, were selected for training in this paper to analyze the effects of different batch sizes on the model training effect. The accuracy of the test set in the model training results is used to adjust the size of the training rounds in time. For the neural network structure of corrosive image classification, four training rounds of 15, 25, 50, and 75 are selected for model training.

Table 5 shows the training results of the various hyperparameters adjusted corresponding to the CNNs. According to the training results in Table 5, when the learning rate was large, i.e., 0.001 and 0.0005, both models showed that the accuracy rate remained below 50% with the increasing number of training rounds, which is a gradient explosion, and the weight parameters in the neural network could not be further optimized at this time.

When the learning rate is small, i.e., the learning rate of 0.00005 is selected, both models show an extremely slow increase in the accuracy rate with the increase of training rounds, which indicates that the optimization parameters of the model corresponding to this learning rate take too long and still cannot reach the global minimum after several iterations.

As can be seen from Figure 9, when the learning rate is 0.0001, the accuracy of both models is higher than that of their respective models under the other three learning rates, and the accuracy of the improved VGG11 is higher than that of the original VGG11. From the above analysis, it can be seen that the training effect is best when the learning rate is selected to be 0.0001, and the accuracy of the test set is the highest.

The analysis of the above table shows that the other four CNN models, all perform relatively well, and the highest accuracy of the test set is over 80% and close to 90%. The accuracy of the test set of the improved VGG is higher than the other four classical neural network models, the training time of the improved VGG is shorter, and the training efficiency is higher than the other four neural network models. The accuracy of the model was verified by comparing it with other classical CNN structures.

In the context of corrosion damage assessment, the database containing the corrosion images and associated parameters can be used to train and test corrosion damage assessment models. The availability of a diverse and comprehensive database can help improve the accuracy and reliability of corrosion damage assessment, which is crucial for ensuring the safety and longevity of structures and components in various industries. Correctly detecting and assessing the corrosion damage on the outer plate of the underwater hull can reduce the probability of structural damage caused by corrosion damage on the hull of the ship and thus the occurrence of safety accidents. The trained model is applied to the actual corrosion damage assessment, the corrosion damage image is input into the assessment model, the corrosion is detected and assessed, and the corresponding protective measures are given according to the assessment results.

The corrosion damage level assessment system is mainly composed of three parts. The first part is the accelerated corrosion test of the hull structure and the corrosion image database established on the basis, which mainly provides a large number of corrosion damage images and related corrosion parameters. The second part is the corrosion image recognition and processing system, which is mainly divided into two aspects—one is the procedure based on the traditional digital image processing method and the other is the image clustering and segmentation procedure based on unsupervised learning. The two are used in combination to do corrosion image recognition and processing to extract corrosion damage parameters. According to the corrosion damage images and parameters obtained by the corrosion image recognition processing system, the corrosion damage dataset is constructed. In the third part, the corrosion damage dataset is imported into the structurally adjusted CNN model for training, and the corrosion damage level assessment model is established and applied. Finally, the corrosion damage level assessment of corroded ship hull pictures is achieved.

In this paper, the accelerated corrosion test images were detected and processed to obtain the corrosion parameters, and the corrosion damage assessment criteria and image corrosion damage level dataset were established. The dataset was introduced into the CNN model for training, and the effects of the number of training rounds, iteration number, and learning rate on the accuracy of the model were analyzed The CNN model for corrosion damage level assessment was obtained and an interface for corrosion damage level assessment was established. The specific findings are as follows.

• Corrosion damage characterization parameters were obtained. Using traditional digital image processing methods, a digital image recognition and processing console application was written to realize image preprocessing, thresholding, edge detection, and contour extraction; an improved self-organizing neural network algorithm was used for image corrosion area identification and segmentation. The parameters of the corrosion center, corrosion area, and total corrosion area ratio are obtained by two methods of image recognition and processing. Combined with the average corrosion rate and corrosion weight loss ratio parameters calculated from the test data and the corrosion parameters obtained by the image identification and processing methods, the corrosion damage level assessment standard was established.

• The corrosion damage level assessment model is established. In the construction of the CNN model, the VGG network was adjusted from the input layer, convolutional layer, and output layer to make it more suitable for corrosion damage level assessment. In the construction of the corrosion damage dataset, the corrosion damage level assessment criterion established in the previous stage was used to calibrate the corrosion images and add corrosion damage level labels to them. iIn the training of the CNN model, three hyperparameters of learning rate, batch size, and training rounds were selected and the original VGG11 and the improved VGG11 were compared for training. The accuracy of the model was verified by comparing it with other classical CNN structures.

• A corrosion damage level assessment system for hull plates was formed. The system contains a corrosion damage database and uses image recognition and CNN to save and call the trained corrosion damage level assessment model to realise the corrosion damage level assessment of the corroded hull plate.

The authors would like to thank the financial support from the Aeronautical Science Foundation of China (Grant No. 2023M031077001) and the Open Project Program of Shandong Marine Aerospace Equipment Technological Innovation Center, Ludong University (Grant No. MAETIC202209).