Detecting Welding Defects in Steel Plates using Computer Vision Algorithms

#machinelearning #deeplearning #computervision #tutorial

Introduction
Prerequisites
Image Segmentation
Image Moments
Understanding the Data
Method and Algorithm Used
Results
Conclusion
Co-Authors
References

1. Introduction

Welding defects can be defined as weld surface irregularities, discontinuities, imperfections, or inconsistencies that occur in welded parts. Defects in weld joints could result in the rejection of parts and assemblies, costly repairs, significant reduction of performance under working conditions and, in extreme cases, catastrophic failures with loss of property and life.

Furthermore, there are always certain flaws in the welding due to the inherent weakness in welding technology and the characteristics of metals.

It is important to evaluate the weld quality, as welded joints are often the locations of crack initiation due to inherent metallurgical geometrical defects, as well as heterogeneity in mechanical properties and the presence of residual stresses.

In practice, it is practically impossible to obtain a perfect weld and, in most circumstances, it is not necessary to provide the adequate service functions required. However early detection and segregation is always preferred to mishaps.

Using our algorithms, we can easily detect faults in welding by the images and also measure the severity of each fault precisely. This will further help in faster image recognition and avoid adverse situations from arising.

It was found, using Convolutional Neural Networks algorithm and U-Net architecture making the process much more efficient. Resulting in an accuracy of 98.3% by the end of the work.

2. Prerequisites

Basic understanding of Machine Learning
Basic idea of Convolutional Neural Networks
Understanding of Convolution, Max Pooling and Up Sampling operations
Idea of U-Net architecture
Basic understanding of skip connections as in residual blocks (Optional)
Working knowledge of ConvNets with Python, TensorFlow and Keras library (Optional)

3. Image Segmentation

Segmentation partitions an image into distinct regions containing pixels with similar attributes. To be meaningful and useful for image analysis and interpretation, the regions should strongly relate to depicted objects or features of interest.

The success of image analysis depends on reliability of segmentation, but an accurate partitioning of an image is generally a very challenging problem.

A chest x-ray with the heart (red), lungs (green), and clavicles (blue) are segmented

4. Image Moments

An image moment is a certain particular weighted average of the image pixels’ intensities. Image moments are useful to describe objects after segmentation.

Simple properties of the image which are found via image moments include:

Area (or total intensity)
Centroid
Information about its orientation.

5. Understanding the Data

The dataset contains two directories. The raw images are stored in images directory and the segmented images are stored in labels directory.

Let’s visualize the data:

These raw images above are RGB images which have to be used for training the model as well as testing the model. These pictures vary in dimensions. Intuitively, the darker portions are welding defects. The model needs to perform Image Segmentation on these images.

These images from labels directory are binary images or ground truth labels. This is what our model must predict for the given raw image. In binary images the pixels have either a high value or a low value. The white region or the high values denote the defective regions and the black region or the low values denote no defect.

6. Method and Algorithm Used

The architecture we are going to use for this problem is U-Net. We are going to detect faults and measure the severity of these welding images in three broad steps:

Image Segmentation
Showing Severity using Colour
Measuring Severity using Image Moments

Training the Model

The following is the U-Net architecture we used for our model:

Points to Note:

Each blue box corresponds to a multi-channel feature map
The number of channels is denoted on the top of the box.
The (x,y) dimensions are provided at the lower left edge of the box.
The arrows denote different operations.
The name of the layer is provided below the layer.
C1, C2, …. C7 are the output layers after Convolutional operation
P1, P2, P3 are the output layers of Max Pooling operation
U1, U2, U3 are the output layers of up-sampling operation
A1, A2, A3 are the skip connections.
The left-hand side is the contraction path, where regular convolutions and max pooling operations are applied
The size of the image gradually reduces while the depth gradually increases.
The right-hand side is the expansion path, where (Up Sampling) transposed convolutions along with regular convolutions operations is applied
In the expansion path, the size of the image gradually increases and the depth gradually decreases
To get better precise locations, at every step of the expansion we use skip connections by concatenating the output of the transposed convolution layers with the feature maps from the Encoder at the same level: A1 = U1 + C3 A2 = U2 + C2 A3 = U3 + C1
After every concatenation we again apply regular convolutions so that the model can learn to assemble a more precise output.

Model is compiled with Adam optimizer and we use binary cross entropy loss function since there are only two classes (defects and no defects).

We use a batch size of 10 with 100 epochs(The number of times the model is run on all inputs).

Please note that there could be a lot of scope to tune these hyper-parameters and further improve the model performance.

Testing the model

Since the model takes input of dimension 512x512x3, we have resized the input to that dimension. Next, we have normalized the image by dividing it by 255 for faster computation.

The image has been fed to the model for predicting the binary output. In order to amplify the intensities of the pixels the binary output has been multiplied by 1000.

The image is then converted to 16-bit integer for easy image manipulations. After which an algorithm detects the defects and visually marks the severity of the defects through color grading along with assigning weights to the pixel with defects depending on their severity. The Image Moment is then calculated on this image considering the weighted pixels.

The image is finally converted back to 8-bit integer and the output image is displayed with color grading and its severity value.