Object Detection Algorithm for Wheeled Mobile Robot Based on an Improved YOLOv4

Abstract

In practical applications, the intelligence of wheeled mobile robots is the trend of future development. Object detection for wheeled mobile robots requires not only the recognition of complex surroundings, but also the deployment of algorithms on resource-limited devices. However, the current state of basic vision technology is insufficient to meet demand. Based on this practical problem, in order to balance detection accuracy and detection efficiency, we propose an object detection algorithm based on a combination of improved YOLOv4 and improved GhostNet in this paper. Firstly, the backbone feature extraction network of original YOLOv4 is replaced with the trimmed GhostNet network. Secondly, enhanced feature extraction network in the YOLOv4, ordinary convolution is supplanted with a combination of depth-separable and ordinary convolution. Finally, the hyperparameter optimization was carried out. The experimental results show that the improved YOLOv4 network proposed in this paper has better object detection performance. Specifically, the precision, recall, F1, mAP (0.5) values, and mAP (0.75) values are 88.89%, 87.12%, 88.00%, 86.84%, and 50.91%, respectively. Although the mAP (0.5) value is only 2.23% less than the original YOLOv4, it is higher than YOLOv4_tiny, Eifficientdet-d0, YOLOv5n, and YOLOv5 compared to 29.34%, 28.99%, 20.36%, and 18.64%, respectively. In addition, it outperformed YOLOv4 in terms of mAP (0.75) value and precision, and its model size is only 42.5 MB, a reduction of 82.58% when compared to YOLOv4’s model size.

1. Introduction

Object detection can be regarded as target positioning and image classification. There are usually only one or a fixed number of targets in target positioning, whereas the type and quantity of objects in the object detection image are not fixed.

Before 2013, the majority of object detection relied on the manual extraction of features. By creating complicated models and multi-model integration based on low-level feature expression, most people gradually improved detection accuracy. The researchers [1,2,3,4,5,6,7] noticed that CNN was able to learn characteristic representations with robustness and certain expressive power when it was shown in the 2012 ILSVRC Image Classification project. Thus, Girshick et al. proposed the regions with CNN features (R-CNN) model [8] in 2014. From that point on, object detection research accelerated at an unparalleled rate. R-CNN makes a significant contribution by introducing deep learning into object detection, boosting the mAP on the Pascal VOC 2007 dataset from 35.1% to 66.0%. Because the input and output sizes of the entire connection layer after the convolutional layer are fixed when R-CNN transmits the candidate region to the CNN, the CNN likewise requires a fixed input size, limiting the size of the input image adjusted arbitrarily. Furthermore, since candidate regions frequently overlap, the approach of feeding each candidate region into CNN results in a significant number of repeated calculations. In 2014, Kaiming He et al. proposed the SPP-Net [9] (space pyramid pooling network) as a solution to these two issues. Girshick et al. proposed an enhanced version of the fast R-CNN [10] based on the R-CNN in 2015. Similar to the SPP layer, the fast R-CNN pools a RoI (region of interest) layer. For synchronization training, SPP adds the classification and bounding box regression phases to a deep network. While the fast R-CNN increases speed and accuracy, it also requires the use of external algorithms to extract target candidate boxes. Immediately after the fast R-CNN was proposed, Ren Shaoqing et al. proposed the faster R-CNN model [11] which combines the process of extracting an object candidate box into a deep network. The faster R-CNN model is the first deep learning object identification method that is really end-to-end, as well as the first deep learning object detection algorithm that is quasi-real-time. The design of a regional candidate network is R-CNN’s significant innovation (RPN). In 2016, Dai Jifeng et al. introduced the R-FCN (region-based fully convolutional network) model [12], where location-sensitive RoI pooling was added to improve object detection accuracy. They are all detection methods based on candidate areas in object detection, from R-CNN to R-FCN. In most cases, there are two steps to the implementation. The two-stage target identification approach involves extracting deep characteristics from an image and calculating candidate regions, followed by locating each candidate region (including classification and regression). Despite its excellent detecting accuracy, there is still a delay between speed and real-time.

The researchers proposed a one-stage object detection technique to enable object detection to satisfy real-time requirements. The approach of “coarse detection + refinement” is abandoned and replaced by the method of “anchor point + correction” (this method only performs one feedforward network calculation, which is very fast and can achieve real-time effects). Joseph and Girshick et al. introduced the first one-stage object detection algorithm called YOLO [13] (you only look once) in 2015. The introduction of YOLO has revealed fresh information on the speed, which can now approach 45 frames per second. However, the limitations are clear, namely, demonstrating the limited number of network energy detection targets and the poor small object detection effect. Wei Liu et al. proposed the SSD [14] method in the same year. SSD incorporates YOLO’s quick detection concept, merges RPN’s advantages in faster R-CNN, and improves multi-size target processing. Most previous methods create predictions using semantically rich high-level convolutional features; however, the high-level features will result in the loss of some detailed information, and the target location will be unclear. Tsung-Lin et al. proposed the FPN [15] (feature pyramid networks) at the end of 2016. FPN improves the network processing capabilities for small targets by combining high-level, low-resolution, strong semantic information with low-level, high-resolution, weak semantic information. The FPN can be used in conjunction with either a one-stage or two-stage object detection approach. At the same time, YOLO released YOLOv2 [16], which added a batch normalization layer for each convolutional layer to increase convergence while removing the fully connected layer from the network.

Although one-stage object detection is significantly faster than two-stage object detection, the detection accuracy is not as good as two-stage object detection. As a result, in 2017, Tsung-Yi Lin proposed the RetinaNet [17] detection model. The focus loss function, proposed by RetinaNet, decreases the learning weight of simple background samples during network training. Zhang Shifeng et al. proposed the RefineDet [18] in late 2017. The ARM (anchor refinement mould), the ODM (object detection module), and the TCB (transfer connection block) are all proposed by RefineDet. YOLOv3 [19] was updated in early 2018, replacing the softmax function with numerous independent classifiers for multi-dimensional prediction, comparable to the feature pyramid network.

Even though both two-stage and one-stage object detection algorithms have made significant progress on high-performance devices, they need significant processing resources to run. As a result, these algorithms are challenging to implement on some resource-constrained devices. Some lightweight object detection algorithms, such as YOLOV3_tiny [20], YOLOV4_tiny [21], Mobilenetv1 [22], Mobilenetv2 [23], Mobilenetv3 [24], and others [25,26,27,28,29,30] have been offered for this problem. However, the identification accuracy suffers as a result of the network model’s compression, making it impossible to detect some complicated sceneries and small target items.

With the continuous development of object detection technology and robotics, several object detection methods are deployed to robots and applied to various fields, for example. agriculture [31,32,33,34,35,36], industry [37,38,39,40], health [41], unmanned vehicles [42], sports [43,44], and so on.

The RoboMaster AI Challenge brings together deep neural network enthusiasts from all around the world to research robotics. It is also envisaged that the findings would be applied to field rescue, unmanned driving, autonomous logistics, and other industries in order to benefit human life. A wheeled mobile robot should be able to recognize, track, and shoot targets in the RoboMaster AI Challenge. The key to success is figuring out how to make the object detection algorithm on the mobile robot ensure detection accuracy. This paper takes this task as the specific application context for the study.

The main contributions of our work can be summarized as follows:

(1)

This study proposes an improved YOLOv4 object detection algorithm that replaces the YOLOv4 backbone extraction network with a trimmed GhostNet lightweight network. This algorithm improves performance over the YOLOv4-related algorithm while maximizing accuracy retention.

(2)

This study uses a combination of deep separable convolution and ordinary convolution to improve ordinary convolution, which significantly reduces the number of parameters and makes the improved network more lightweight and efficient, suitable for deployment on resource-constrained mobile robotic devices, with great potential for applications.

(3)

To demonstrate the feasibility of the algorithm, we experimentally validate the object detection using a wheeled movable robot in the context of DJI’s RoboMaster AI challenge application.

(4)

The experimental dataset in this paper is from DJI. This dataset has relatively complex scenes, identifies more small target objects, and the official dataset is more valuable for research. We also design a method to enhance the dataset by image processing and demonstrate experimentally that it can effectively improve the detection accuracy.

2. Materials and Methods

2.1. Experimental Device

The robot platform used in this paper is the generic omnidirectional wheeled mobile robot development platform created by the ICRA RoboMaster AI Challenge, as shown in Figure 1. There are five modules in total: chassis, expansion, cloud platform, launch, and referee system. To implement the robot’s movement, the robot chassis module has a set of McNham wheels (the McNham wheel is a type of omnidirectional wheel that allows for omnidirectional movement in both XY directions as well as for combined movement); the Robotics Expansion Module is an expansion platform mounted on a chassis module with an external controller (two devices are used in this paper, the Manifold 2-G and the Dell OptiPlex 3060 (the Dell OptiPlex 3060 is manufactured by Dell and originates from China), and their specific parameters are shown in

Table 1. Configuration parameters for the robot controller. (The robot controller is the CPU of the robot and provides various interfaces to connect different external devices and handle the associated computing operations).

Name	Processor	System	Memory	GFLOPS
Manifold 2-G	NVIDIA Jetson TX2	Ubuntu 18.04	8 GB 128 bit	1260
Dell OptiPlex 3060	Intel(R) Pentium(R) Gold G5400T CPU@ 3.10 GHz	Windows 10	16 GB 64 bit	99.2

); the robot head module can complete pitch and yaw rotation motion, flexibility, and confrontation; the robot launch mechanism module can launch a RoboMaster 17 mm projectile; users can use it flexibly to create a unique automatic robot solution. We chose the Mercury Series 1.31 MP MER-131-75GM/GC DaHeng Image Industrial Camera as an experimental camera.

2.2. Object Detection Algorithm Design

2.2.1. YOLOv4 Structure

The YOLOv4 [45] network is based on the YOLOv3 network and showed significant improvements in identifying small and blurred targets, with AP and FPS values increasing by 10% and 12%, respectively. YOLOv4 enriches the data on the input side by randomly resizing, cropping, and aligning the four photographs. This greatly enhanced the robustness of the network, greatly enriched the detection dataset and increased the number of small targets. YOLOv4 builds the CSPDarknet53 in the main network section, and its structure splits Darknet53 into two parts: the backbone part continues the original stack of residual blocks; the other part is connected directly to the end similar to a residual edge with a small amount of processing. Its structure is shown in Figure 2, and it modified the activation function of DarknetConv2D from LeakyReLU to Mish. Since the Mish activation function is unbounded (i.e., positive values can reach any height) to avoid saturation because of capping, there is no problem of gradient disappearance during training. In addition, the smoothing nature of the Mish activation function gives it better performance in terms of solving and model generalization compared to other activation functions. The equation is as follows, with x being the input value.

$$\mathrm{Mish}=\mathrm{x}\times \tanh \left(\ln (1+{\mathrm{e}}_{}^{\mathrm{x}})\right).$$

(1)

The SPP structure and PANet structure are used in the enhanced feature extraction network part, which can greatly increase the perceptual field and separate out the most significant contextual features. The prediction network YoloHead uses a more efficient feature layer to obtain predictions. The full network structure of YOLOv4 is shown in Figure 3.

2.2.2. GhostNet Structure

GhostNet [46] is a lightweight network model that employs Ghost modules instead of regular convolutions.The Ghost module is divided into three steps: convolution, Ghost generation, and feature map combination. The first step is to use the traditional convolution to obtain the feature maps. Then, the intrinsic feature map of each channel is processed by depth-separable convolution to generate the Ghost feature map. Finally, the intrinsic feature map obtained in the first step is combined with the Ghost feature map obtained in the second step, and the final result output is obtained. The Ghost module is shown in Figure 4.

Ghost bottleneck similar to the basic residual block in ResNet, it integrates multiple convolutional layers, and the Ghost bottleneck consists mainly of two stacks of the Ghost modules.The Ghost bottleneck is shown in Figure 5.

The Ghost bottleneck significantly reduces the computational volume and size of the model and achieves a good mix between processing, memory size, and accuracy. The GhostNet network structure is shown in Figure 6.

2.2.3. Improved Network

GhostNet was first used to find valid feature layers of the same height and width as YOLOv4’s CSPDarknet53, and then it was pruned according to these feature layers, as shown in Figure 7. The reduced GhostNet then replaces CSPDarknet53. To extract the enhanced features, the GhostNet feature layer is added to the enhanced feature extraction network.

Although GhostNet replaces the CSPDarknet backbone feature extraction network and reduces the number of parameters, most of the computation is performed on the enhanced feature extraction network PANet. Depthwise separable convolution is a combination of depthwise (DW) and pointwise (PW) components used to extract feature maps, with a lower number of parameters and lower computational cost than conventional convolution operations. Thus, we replaced traditional convolution with a combination of depth separation and traditional convolution.

Based on the specificity of the application and algorithm, we named the improved algorithm YOLOV4_RMLight. Figure 8 depicts the structure of the final improved network.

2.3. Dataset

This experiment used the dataset named “The DJI Robomaster Objects in Context (DJI ROCO)”, which is the official RoboMaster dataset released by DJI in 2019. We used 2065 tagged photos from the “Robomaster Central China Competition” folder as the base dataset; an example figure is shown in Figure 9. The dataset uses the Pascal VOC dataset format and consists of three parts, JPEGImages (which holds all the images from the training and testing), Annotations (which holds the XML file corresponding to each image after it has been tagged), and ImageSets (which has a main folder containing txt files with the names of the images, divided into training and testing). The objects to be recognized are divided into five categories in total, namely, car (robot), watcher (sentry robot), base (base), ignore (ignore), and armor (armor plate). The recognition effect is shown in Figure 10.

2.4. Data Augmentation

With the following image processing method, (1) Gaussian noise; (2) change brightness; (3) image translation; (4) image rotation; (5) image flipping; (6) image scaling; (7) cutout, random combinations process dataset pictures to form new images. Training with the enhanced dataset allows the network to increase its robustness. We augmented to triple the original dataset by this method. Experiments demonstrate that following data improvement, identification accuracy improves dramatically.

2.5. Experimental Design

2.5.1. Effectiveness Verification Experiments Design

To demonstrate the effectiveness of the improved method, we use the idea of the controlled variable method to design effectiveness experiments.

(1)

The YOLOv4, SSD, and YOLOv5x are trained separately using the dataset, and the evaluation results are obtained to prove the effectiveness of YOLOv4.

(2)

The original data were augmented using the data augmentation method in Section 2.4. The yolov4_RMLight network was used to train with the original data and the augmented data separately. The results before and after the augmented data demonstrated the effectiveness of the data augmentation method. We named the data-enhanced model yolov4_RMLight_ADD.

(3)

The yolov4_RMLight_ADD backbone feature extraction network remains unchanged, and its augmented feature extraction network is restored to the original YOLOv4 augmented feature extraction network. It is used to demonstrate the effectiveness of the improvements to the augmented feature extraction network.

(4)

The yolov4_RMLight_ADD enhanced feature extraction network is unchanged, and its backbone network is replaced with mobilenetv1, mobilenetv2, mobilenetv3, resnet [47], and vgg [48] to obtain the evaluation results and prove the effectiveness of using ghostnet.

2.5.2. Experimental Process Design

The device for training the algorithm model uses the CPU processor of Intel (R) Xeon (R) CPU E5-2678 v3 @ 2.50 GHz, equipped with GEFORCE RTX 3090 graphics card, 32 GB memory, and the deep learning framework of pytorch. Two devices were chosen as mobile robot controllers (as shown in Table 1) to test the performance of the algorithm on different devices. The object detection experiment of the wheeled mobile robot consists of two main parts: the training process and the testing process. The whole experimental process is shown in Figure 11.

Training process. Firstly, we downloaded the dataset from the official RoboMaster website. Secondly, the dataset was augmented by the data augmentation method in Section 2.4. Then, a cross-validation procedure was used to divide the dataset into a training set and a test set in a 9:1 ratio, and the training set was divided into a training set and a validation set in a 9:1 ratio. Finally, the training set was input into the network model for training with parameters set to batch size = 32 and epoch = 300. The trained loss function is shown in Figure 12.

Testing process. Firstly, the trained optimal model is deployed to the mobile robot device. Secondly, cameras are connected to mobile robotic devices. Then, the test set is input into a network model on a mobile robot device for testing. Finally, accuracy tests and performance tests are performed, and the experimental results are saved.

2.5.3. Evaluating Indicator

This research evaluates the algorithm’s quality in two ways: the accuracy of the identification and the performance of the algorithm.

The mAP (mean average precision) is commonly used to assess the algorithm’s accuracy. To begin, you must first determine the precision and recall, which are determined as follows:

$$\mathrm{precision}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FP}},$$

(2)

$$\mathrm{recall}=\frac{\mathrm{TP}}{\mathrm{TP}+\mathrm{FN}},$$

(3)

In the formula, TP (true positive) means that it is divided into positive samples and correct. FP (false positive) means that it is positive but wrong (in fact, this sample is a negative sample). FN (false negative) means that it is divided into negative samples but wrong (in fact, this sample is this). AP is calculated for n samples of a certain classification; assuming that it has m positive cases, each positive case corresponds to a recall R value (1/m, 2/m, ..., 1). Maximum accuracy P was calculated for each recall, and then averaged for m p-values. The expression for calculating the AP is as follows:

$$\mathrm{AP}=\frac{1}{\mathrm{m}}{\sum }_{\mathrm{i}}^{\mathrm{m}}{\mathrm{P}}_{\mathrm{i}}=\frac{1}{\mathrm{m}}\times {\mathrm{P}}_1+\frac{1}{\mathrm{m}}\times {\mathrm{P}}_2+\cdots +\frac{1}{\mathrm{m}}\times {\mathrm{P}}_{\mathrm{m}}.$$

(4)

While the AP is particular to a single class, a dataset often comprises a large number of classifications, and the mAP is calculated by averaging the AP of all classes in the datasets. The following is the formula for determining the mAP:

$$\mathrm{mAP}=\frac{1}{\mathrm{C}}{\sum }_{\mathrm{C}}^{\mathrm{j}}{\mathrm{P}}_{\mathrm{j}}.$$

(5)

A boundary regression task is included in the object detection task, and its accuracy is commonly tested by IoU. The IoU is calculated using the following formula:

$$\mathrm{IoU}=\frac{{\mathrm{S}}_1}{{\mathrm{S}}_2}.$$

(6)

S₁ is the area of overlap between the predicted boxes and actual boxes; S₂ is the total area occupied by the predicted boxes and actual boxes. Thus, mAP (0.5) refers to the value of mAP at IoU = 0.5. The mAP (0.75) is the value of the mAP at IoU = 0.75.

The F1-score is a measure of a classification problem and is often used as the final metric for multi-classification problems; it is the summed average of precision and recall. F1-score can be calculated using the following formula:

$$\mathrm{F}1=2\times \frac{\mathrm{recall}\times \mathrm{precision}}{\mathrm{recall}+\mathrm{precision}}.$$

(7)

We set the performance metric to the time it takes to collect the mAP values in order to better discover the algorithm’s time performance because the time it takes to calculate the mAP comprises both the time it takes to infer the image and the time it takes to calculate the aforementioned series of numbers. It can correctly display the algorithm’s performance on a variety of devices. We also use model size, number of parameters, and flops as a measure of the performance of the algorithm on hardware.

3. Results

3.1. Effectiveness Experiment Results

In order to further analyze the effectiveness of the application methods proposed in this paper, we used the experimental setup in Section 2.5.1 and measured these methods through evaluation metrics.

The experimental results of the effectiveness of YOLOv4 are shown in

Table 2. YOLOv4 validity experiment results.

Algorithms	mAP (0.5)	Model Size/MB	Flops
YOLOv4	88.63%	244	30.17 GB
SSD	57.36%	92.6	31.35 GB
YOLOv5x	68.52%	333	109 GB

. Compared to mainstream object detection networks such as SSD and YOLOv5x, YOLOv4 performs very well on this dataset. The mAP (0.5) values for YOLOv4 are 31.27% and 20.11% higher than SSD and YOLOv5x, respectively. Moreover, the flops value of YOLOv4 is lower than both of them, indicating that the model is relatively less computationally intensive.

Data augmentation experiment results are shown in

Table 3. Data augmentation experiment results.

Algorithms	mAP ( 0.5)	mAP (0.75)
yolov4_RMLight_ADD	86.84%	50.91%
yolov4_RMLight	81.35%	44.80%

. After data-enhancement experiments using yolov4_RMLight, it can be seen that the values of mAP (0.5) and mAP (0.75) have increased by 5.49% and 6.11%, respectively, after data enhancement. The effectiveness of the data-enhancement method is proved.

The experimental results to improve the effectiveness of the enhanced feature extraction network are shown in

Table 4. The experimental results of improving the effectiveness of the enhanced feature extraction network.

Algorithms	mAP (0.5)	Model Size/MB	Parameter Quantity/Million
yolov4_RMLight_ADD	86.84%	42.50	10.80
yolov4_RMLight_ADD_0	70.31%	145	38.38

Yolov4_RMLight_ADD_0 denotes the network model after reduction to the enhanced feature extraction network of YOLOv4.

. It is evident from the results that the improvements in the enhanced feature extraction network substantially reduce the number and size of parameters in the model. This makes the improved network much lighter.

GhostNet was used as the backbone feature extraction network in yolov4_RMLight_ADD. To verify the effectiveness of GhostNet, the backbone feature network was replaced using an existing mainstream classification network. The experimental results are shown in

Table 5. The experimental results for improving the effectiveness of the backbone feature extraction network.

Algorithms	mAP (0.5)	Model Size/MB	Flops
yolov4_RMLight_ADD	86.84%	42.50	3.53 GB
yolov4_mobilenetv1	79.46%	51.10	5.27 GB
yolov4_mobilenetv2	79.68%	46.60	4.08 GB
yolov4_mobilenetv3	79.68%	46.60	3.80 GB
yolov4_resnet50	65.36%	127	33.68 GB
yolov4_vgg16	66.93%	90	23.94 GB

Yolov4_mobilenetv1 indicates that the backbone feature extraction network is replaced with mobilenetv1. Yolov4_mobilenetv2 indicates that the backbone feature extraction network is replaced with mobilenetv2. Yolov4_mobilenetv3 indicates that the backbone feature extraction network is replaced with mobilenetv3. Yolov4_resnet50 indicates that the backbone feature extraction network is replaced with Resnet50. Yolov4_vgg16 denotes replacing the backbone feature extraction network with VGG16.

. The experimental results clearly show that GhostNet has a clear advantage as a backbone feature extraction network in terms of both accuracy and performance. The non-lightweight Resnet50 and VGG16 far exceeded the lightweight networks in terms of computational effort and model size, which made deployment on mobile robotic devices more difficult, and several popular lightweight networks do not perform as well as GhostNet in terms of accuracy. GhostNet was therefore chosen as the backbone extraction network.

3.2. Contrast Experimental Results

In order to better analyze our proposed improved algorithm, we conducted comparative experiments using some lightweight object detection algorithms. The quantitative analysis was also carried out in terms of both the accuracy of recognition and the performance of the algorithms. The results of the comparison experiments are shown in

Table 6. Accuracy results for different detection models.

Algorithms	Precision	Recall	mAP (0.5)	mAP (0.75)	F1
yolov4_RMLight_ADD	88.89%	87.12%	86.84%	50.91%	88.00%
YOLOv4	88.63%	89.48%	89.07%	49.20%	89.40%
YOLOv4_tiny	57.92%	56.28%	57.50%	44.92%	57.00%
Eifficientdet-d0	78.83%	53.27%	57.85%	51.99%	56.20%
YOLOv5n	75.72%	63.37%	66.48%	52.07%	67.20%
YOLOv5s	75.88%	65.47%	68.20%	52.35%	69.00%

and

Table 7. Performance results for different detection models.

Algorithms	Model Size/MB	Parameter Quantity/Million	Flops
yolov4_RMLight_ADD	42.50	10.80	3.53 G
YOLOv4	244	64.36	30.17 G
YOLOv4_tiny	22.25	6.06	3.47 G
Eifficientdet-d0	15	3.87	2.55 G
YOLOv5n	6.95	1.88	2.33 G
YOLOv5s	27.1	7.28	8.53 G

.

Based on the results of the contrast experiments, the following conclusions can be reached.

(1)

In Table 6, the proposed model’s precision, recall, mAP (0.5), mAP (0.75) values, and F1-scores were 88.89%, 87.12%, 50.91%, and 88.00%, respectively.

(2)

From Table 6, we can see that our proposed yolov4_RMLight_ADD model has high precision and mAP (0.5) values, with precision values higher than the lightweight network YOLOv4_tiny, Eifficientdet-d0 [49], YOLOv5n, and YOLOv5s by 30.97%, 10.06%, 13.17%, and 13.01%. The mAP (0.5) values are 29.34%, 28.99%, 20.36%, and 18.64% higher than those of the lightweight networks YOLOv4_tiny, Eifficientdet-d0, YOLOv5n, and YOLOv5s, respectively, and the precision value is 0.26% higher than the large network YOLOv4. The mAP (0.5) value is only 2.23% lower than YOLOv4. This shows that our proposed yolov4_RMLight_ADD model has excellent object detection accuracy.

(3)

From Table 6, we can see that our proposed yolov4_RMLight_ADD model has higher recall and F1 values, with recall values 30.84%, 33.85%, and 23.75% higher than the lightweight networks YOLOv4_tiny, Eifficientdet-d0, YOLOv5n, and YOLOv5s (21.65%). The F1 values are 31%, 31.8%, 20.8%, and 19% higher than those of the lightweight networks YOLOv4_tiny, Eifficientdet-d0, YOLOv5n, and YOLOv5s, respectively, and recall and F1 values are only 2.36% and 1.4% lower than those of YOLOv4. This shows that our proposed yolov4_RMLight_ADD model has effect on object recognition.

(4)

As can be seen from Table 7, our proposed model size of yolov4_RMLight_ADD is 42.5 MB, the number of parameters is 10.8 million, and the flops is 3.53 GB. The model size is 201.5 MB less than YOLOv4, the number of parameters is 83.21% less, and the flops are 88.30% less. Although the model size is 20.25 MB, 27.5 MB, 35.55 MB, and 15.4 MB more than the lightweight networks YOLOv4_tiny, Eifficientdet-d0, YOLOv5n, and YOLOv5s, respectively, the flops are similar to the lightweight networks, which means that the computational consumption on the hardware is is about the same.

(5)

Overall, the yolov4_RMLight_ADD model proposed in this paper has the highest precision, recall, mAP value, and F1-score among the lightweight networks; moreover, although the model size and number of parameters are more than the lightweight networks, the flops are similar. Considering the computational performance of the mobile robot device and the excellent performance of the yolov4_RMLight_ADD model in terms of recognition accuracy far above other lightweight networks, the yolov4_RMLight_ADD model is more suitable for deployment on mobile robots.

In addition to detection accuracy, the time performance of the object detection algorithm on mobile robotic devices is also a key metric. We obtained the time to calculate the mAP value on different devices by deploying the object detection method on different devices, as shown in Figure 13. We also used the camera on the robot to test the real-time detection capability of the object detection algorithm, and Figure 14 shows the results of the FPS tests on different devices.

Based on the experimental results of the time performance, the following conclusions can be drawn.

(1)

As can be seen from Figure 13, the three models consume similar times on the PC. On the Manifold 2-G device, our proposed model consumes 100.12 s, which is 53.09% lower than YOLOv4 and only 24.24 s higher than the lightweight network YOLOv4_tiny. On the Dell OptiPlex 3060 device, our proposed model consumes 534.55 s, which is 88.62% lower compared to YOLOv4 and only 434.42 s higher than the lightweight network YOLOv4. 88.62% and only 434.42 s higher than the lightweight network YOLOv4_tiny. It can be seen that on high-performance devices, the time spent by the different models is similar. On resource-limited devices, the fewer computational resources the device has, the greater the gap in computation time between different levels of algorithms. It is clear that our proposed model drastically reduces the computation time relative to the large network YOLOv4. Although not as small as that of the lightweight network YOLOv4_tiny, they are both of the same order of magnitude. Combined with the accuracy, our proposed algorithm is more advantageous.

(2)

As can be seen from Figure 14, our proposed model can achieve close to 30 FPS for real-time detection on pc devices, while YOLOv4 can only maintain around 20 FPS, and the lightweight network YOLOv4_tiny can achieve around 45 FPS. On the Manifold 2-G device, our proposed model achieves close to 10 FPS, while YOLOv4 only stays at around 2 FPS and the lightweight network YOLOv4_tiny achieves around 25 FPS. On the Dell OptiPlex 3060 device, our proposed model stays at around 2 FPS, while YOLOv4 is essentially 0 FPS, and lightweight network YOLOv4_tiny only reaches around 5 FPS. It is clear that the large model YOLOv4 is largely unavailable on resource-constrained devices, while our proposed model is not as good as the lightweight network, but is still generally available. Combined with the accuracy, our proposed model is very advantageous.

4. Conclusions

This paper presents an object detection algorithm based on a combination of the improved YOLOv4 and the improved GhostNet. The algorithm maximizes the detection accuracy while maintaining high accuracy. The improved principle of the algorithm is to replace the backbone feature extraction network of YOLOv4 with a portion of GhostNet, and then replace the normal convolution in the enhanced feature extraction network with a depth-separated convolution, and we also devise data-enhancement methods to extend the dataset. We designed effectiveness experiments using the idea of the controlled variable method, and the experimental results showed that each of our proposed improvements was effective.

We also compared the algorithm with other existing lightweight object detection networks, and our accuracy, recall, mAP (0.5), and F1 values were the highest compared to other lightweight networks, although the model size was slightly higher than theirs, but the flops were similar. It is also in the same order of magnitude as the lightweight network in terms of real-time performance tests. Combined with a far higher accuracy than other lightweight networks, our proposed yolov4_RMLight_ADD is a better choice.

However, the proposed algorithm has some limitations. Its performance in real-time detection tasks needs to be improved and it needs to be studied in conjunction with some robot control tasks, and future work will focus on combining these two points.