High-Speed Ship Detection in SAR Images Based on a Grid Convolutional Neural Network 20190904

Abstract

问题：
The speed of SAR ship detection is extraordinarily significant, especially in real-time maritime rescue and emergency military decision-making.
本文解决方法：
In order to solve this problem, this paper proposes a novel approach for high-speed ship detection in SAR images based on a grid convolutional neural network (G-CNN). G-CNN is a brand new network structure proposed by us and it is mainly composed of a backbone convolutional neural network (B-CNN) and a detection convolutional neural network (D-CNN).
实验过程：
First, SAR images to be detected are divided into grid cells and each grid cell is responsible for detection of specific ships. Then, the whole image is input into B-CNN to extract features. Finally, ship detection is completed in D-CNN under three scales.
实验结果:
We experimented on an open SAR Ship Detection Dataset (SSDD) used by many other scholars and then validated the migration ability of G-CNN on two SAR images from RadarSat-1 and Gaofen-3. The experimental results show that the detection speed of our proposed method is faster than the existing other methods, such as faster-regions convolutional neural network (Faster R-CNN), single shot multi-box detector (SSD), and YOLO, under the same hardware environment with NVIDIA GTX1080 graphics processing unit (GPU) and the detection accuracy is kept within an acceptable range.

1. Introduction

SAR images的特点:

1. different from optical images, it is difficult to interpret SAR images intuitively;
2. due to the special imaging mechanism of SAR and the existence of speckle noise, subjective error in artificial interpretation is inevitable.
   The main contributions of our work are as follows:
3. The detection speed of our proposed method is faster than the existing other methods.
4. We established a brand new network structure G-CNN.

2. Methodology

2.1.Dataset

SAR Ship Detection Dataset (SSDD).
数据集介绍：
There are 1160 SAR images in the SSDD from RadarSat-2, TerraSAR-X, and Sentinel-1 in Yantai, China and Visakhapatnam, India. The size of each image sample was about 500×500. parameters (x,y,w,h) of real ships where (x, y) was the coordinate of the upper left corner of the rectangular box, w is the width, and h is the height.

Figure 1. Image annotation process. (a) 1160 original synthetic aperture radar (SAR) images in the SAR Ship Detection Dataset (SSDD); (b) LabelImg software;©1160 annotated SAR images.

Moreover, SAR images are formed by coherent processing of echoes from the continuous radar pulses, so SAR images are affected by speckle noise. In order to decrease negative effects by some bad samples with large speckle noise, we preprocessed these SAR images and constructed an enhanced SSDD named ESSDD by an adaptive refine lee filter.

Figure 2. The detection template and edge direction window of the refined lee filter.

2.2 G-CNN

G-CNN is mainly composed of B-CNN and D-CNN as shown in Figure 4. First, the SAR image to be detected was resized to 416×416. Then the features of the ship were extracted by B-CNN in 1024 feature maps with a scale of 52×52. Finally, the output of B-CNN was connected to D-CNN to complete ship detection. B-CNN is the backbone of G-CNN, which is used to extract ship features. There are some network structures that can be used as the backbone of a target detector, such as visual geometry group-16(VGG-16)[22],GoogLeNet[23],andsoon. However,these network structures have more parameters and consume more computer memory, which will increase detection time. Therefore, we use a lightweight network MobileNet [24] to extract features. MobileNet is an efficient model for mobile and embedded devices. Based on streamlined architecture, MobileNet uses depthwise separable convolution (dw) to construct lightweight deep neural networks, which can greatly reduce the amounts of parameters and calculations.

Figure 4. The grid convolutional neural network (G-CNN) ship detection system. (a) Overall framework; (b) backbone convolutional neural network (B-CNN); © detection convolutional neural network (D-CNN).

Figure5. Depthwise separable convolution. (a)Depthwise convolutional filters;(b)pointwise convolutional filters;©depthwise separable convolution;(d)activation function: Rectified linear unit(ReLU).

**Batch normalization (BN)**算法

2.3. Model

Moreover, the dataset must have been correctly labeled. Additionally, the dataset was divided into: (1) a training set which was used to establish a detection model; (2) a test set which was used to evaluate the detection model.

Figure 7. G-CNN ship detection model. (a) Original image; (b) S×S grids; © find out these grid cells which are responsible for predicting ships; (d) bounding boxes + score; (e) class probability map; (f) final detetions.
Figure 8. G-CNN ship detection flow chart

2.4. Anchor Box

Obviously, if ships are very small and densely distributed, they may appear in the same grid cell, and as a result, many ships will be missed,which will inevitably reduce the accuracy of detection. To solve this problem,we can setup two anchor boxes shown in Figure 9c. According to the previous method, the detection tasks of ship 1 and ship 2 are assigned to the grid cell G. Now, we further assigned the detection task of ship 1 to anchor box 1 and that of ship 2 to anchor box 2, according to the center of the ship. Therefore, the grid cell G can detect two ships at the same time.(这里有点没看懂???)

Figure9. Anchorbox. (a)Groundtruth;(b)imagemesh;©the grid cell responsible for predicting ships

The size of the anchor box can be obtained by the K-means algorithm.Distance metric计算公式

The cluster centroids were significantly different than hand-picked anchor boxes. There were fewer short, wide boxes and taller, thin boxes. The results of K-means is shown in Figure 10 and the size of each anchor box for three scales are shown in Table 2.

2.5.Evaluation Indicator

Recall & Precision

In SAR ship detection, TP means that ships are correctly detected, FP means miss-detection, and FN means false alarm.
Mean average precision (mAP) is defined by:

3. Experiments and Results

Our experimental platform was a personal computer with Intel® i7-8700 CPU @3.20GHz processor, 16G memory, and NVIDIA GTX1080 graphics card with 8G memory.

3.1. Establishment of Training Model

We randomly divided the dataset into a training set, validation set, and test set according to the ratio of 7:2:1, where the validation set was used to adjust the model’s hyperparameters to avoid over-fitting. Then, the G-CNN network was established based on the Keras framework. The adaptive moment estimation (ADAM) [32] algorithm similar to stochastic gradient descent (SGD) [33] was used to update the weights and biases in the network.Batch size = 8 which means that every 8 training samples were sent into the network to complete parameter updating. We set the learning rate of the first 100 iterations to 0.001. the learning rate will automatically be reduced when the loss of the verification set does not decrease any more beyond three times.

3.2. Feature Maps

in Section 2.2, the outputs of 2-layer were 208 × 208 × 64 feature maps, where 208×208 was the size of the feature map and 64 was the number of feature maps. Additionally, the feature maps were104×104×128 for 6-layer and 52×52×256 for 10-layer.

Figure 11. The process of feature extraction. (a) Origin image; (b) two feature maps of 2-layer; © two feature maps of 6-layer; (d) two feature maps of 10-layer; (e) detection results.

3.3. Results of G-CNN

We set score = 0.3 (score∈[0,1]) as the detection threshold. We also set IoU = 0.5 as another detection threshold.
There were 183 real ships in the test set. The ship test results between the SSDD and ESSDD are shown in Table 3. The precision and recall (P-R) curve of SSDD and ESSDD is shown in Figure 13. From Table 3, our enhanced dataset ESSDD by refined lee filter has better performance than the origin dataset SSDD. The AP of ship detection on ESSDD reaches 90.16%, which is an acceptable range in practical applications. The test time per image on the SSDD and ESSDD is about 21 ms.

3.4. Results of Different Methods

Figure 17. The ship detection results of different methods. (a) Ground truth; (b) faster-regions convolutional neural network(FasterR-CNN);©you only look once(YOLO);(d)single shotmulti-box detector(SSD);(e)G-CNN.Themiss-detection is marked in red and the false alarm is marked in yellow.

3.5. Actual Ship Detection for RadarSat-1 and Gaofen-3

We divided the two big SAR images into 64 sub-images, and then we resized the 64 sub-images into 416 × 416. From the test results, the performance of Image 2 was inferior to that of Image 1, because the ships in Image 2 were too dense and the background was more complex. In short, most ships were accurately detected .


Figure 20. The ship detection results of RadarSat-1 and Gaofen-3. (a) Image 1; (b) Image 2. Correct ship detection is marked in blue, the miss-detection is marked in red, and the false alarm is marked in yellow.

4. Discussion

5. Conclusions

We confirmed the validity of the proposed method on a public SSDD and then constructed an enhanced ESSDD through refined lee filtering to improve accuracy. The ship detection accuracy of our G-CNN ship detection system is maintained within an acceptable range for practical application. More importantly, our method is superior to other existing methods in ship detection speed, under a similar hardware environment with NVIDIA GTX1080 GPU. Our proposed method realizes the high-speed ship detection in SAR images with only 21 ms detection time per image, authentically. The proposed method can satisfy real-time SAR ship detection and is of great value in maritime distress rescue and emergency military strategy formulation.
Future works: Our G-CNN SAR ship detection system has a slightly lower performance for small and dense ships. Therefore, further improvement is needed to solve this problem.