Next Article in Journal
Radiomic Analysis for Human Papillomavirus Assessment in Oropharyngeal Carcinoma: Lessons and Pitfalls for the Next Future
Next Article in Special Issue
Early Fire Detection System by Using Automatic Synthetic Dataset Generation Model Based on Digital Twins
Previous Article in Journal
Development of Adaptive Control System for Aerial Vehicles
Previous Article in Special Issue
A Blockchain-Based Model for the Prevention of Superannuation Fraud: A Study of Australian Super Funds
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Efficient Fire Detection with E-EFNet: A Lightweight Deep Learning-Based Approach for Edge Devices

by
Haleem Farman
1,
Moustafa M. Nasralla
1,*,
Sohaib Bin Altaf Khattak
1 and
Bilal Jan
2
1
Smart Systems Engineering Lab, Department of Communications and Networks, Prince Sultan University, Riyadh 66833, Saudi Arabia
2
Department of Computer Science, FATA University, Kohat 26100, Pakistan
*
Author to whom correspondence should be addressed.
Appl. Sci. 2023, 13(23), 12941; https://doi.org/10.3390/app132312941
Submission received: 19 October 2023 / Revised: 25 November 2023 / Accepted: 29 November 2023 / Published: 4 December 2023

Abstract

:
Fire detection employing vision sensors has drawn significant attention within the computer vision community, primarily due to its practicality and utility. Previous research predominantly relied on basic color features, a methodology that has since been surpassed by adopting deep learning models for enhanced accuracy. Nevertheless, the persistence of false alarms and increased computational demands remains challenging. Furthermore, contemporary feed-forward neural networks face difficulties stemming from their initialization and weight allocation processes, often resulting in vanishing-gradient issues that hinder convergence. This investigation recognizes the considerable challenges and introduces the cost-effective Encoded EfficientNet (E-EFNet) model. This model demonstrates exceptional proficiency in fire recognition while concurrently mitigating the incidence of false alarms. E-EFNet leverages the lightweight EfficientNetB0 as a foundational feature extractor, augmented by a series of stacked autoencoders for refined feature extraction before the final classification phase. In contrast to conventional linear connections, E-EFNet adopts dense connections, significantly enhancing its effectiveness in identifying fire-related scenes. We employ a randomized weight initialization strategy to mitigate the vexing problem of vanishing gradients and expedite convergence. Comprehensive evaluation against contemporary state-of-the-art benchmarks reaffirms E-EFNet’s superior recognition capabilities. The proposed model outperformed state-of-the-art approaches in accuracy over the Foggia and Yar datasets by achieving a higher accuracy of 0.31 and 0.40, respectively, and its adaptability for efficient inferencing on edge devices. Our study thoroughly assesses various deep models before ultimately selecting E-EFNet as the optimal solution for these pressing challenges in fire detection.

1. Introduction

Disaster management has been the focus of research across various fields, including computer science, health sciences, environmental sciences, and business. Such disasters can be classified as either technological or natural, according to the Federal Emergency Management Agency [1]. Technological disasters include incidents such as hazardous materials, terrorism, and nuclear disasters, while natural disasters encompass events like earthquakes, floods, and forest fires. Regardless of the type of disaster, early detection, preventive measures, and timely notification of relevant departments are crucial [2]. Fire disasters, often caused by system failures or human error, can result in significant human and ecological losses and economic damage [2,3,4]. For example, it was reported that in June 2013 in Arizona (USA), 19 firefighters were killed and 100 houses were burned by a wildfire [5]. To cope with this, researchers have presented various methods for fire detection based on environmental and visual sensors [3,6]. Environmental sensor-based systems that detect close-range fires are feasible for indoor environments and need human intervention [7,8].
On the other hand, vision-based methods offer numerous advantages compared to different approaches. They can cover a large geographical area, allowing for more comprehensive space monitoring. Furthermore, they can detect fire in the initial stages and provide a rapid response, which is crucial in emergencies. Another advantage of vision-based fire detection methods is their ability to perform effectively under different environmental conditions, making them a robust solution for fire detection [3,9].
Surveillance-based fire detection can be broadly categorized into machine- and deep-learning-based methods. The machine learning-based methods rely on color-based features or motion, such as YCbCr [10], RGB [11,12], YUV [13], and HSV [14], for fire detection. These methods use a set of images to match objects, similar to that of fire. These methods are computationally efficient and can be used in real-time systems. Certain researchers have employed statistical color models in combination with background subtraction techniques to identify pixels indicating the presence of fire [15,16,17]. However, these methods have a high false alarm rate and are sensitive to illumination changes. To address the difficulties of fire detection, certain researchers have integrated color attributes and motion information to identify a fire’s shape [18,19,20,21]. These studies reduce false alarm rates; however, their accuracy is challenging, and they cannot identify long- or short-distance fires. Moreover, manually designed features in these methods are not optimal due to variations in fire shapes, lighting conditions, and fire colors [22]. Thus, the researchers have explored fire detection techniques that utilize deep learning methods.
Deep learning provides numerous applications in various fields, such as segmentation, detection, and classification [2,3,23,24]. Over the past few years, using convolutional neural network (CNN)-based techniques for fire detection has gained popularity, proving effective in uncertain and certain environmental surveillance systems. The application of CNN-based techniques has resulted in notable enhancements to the resilience of fire detection systems in various environments and reduced the frequency of false alarms. Researchers have developed different CNN-based architectures for smoke and fire detection. For example, a study [25] used ResNet50 and VGG-based models to evaluate their performance over a customized dataset, while Frizzi et al. [26] developed a 9-layered architecture for the underlying task. A study by Muhammad et al. [27] involved adjusting CNN models for improved early detection of fires and integrating a prioritization system for each node within a monitoring environment. Several researchers have studied the application of deep models in smoke and fire detection, and various methodologies have been developed to achieve accurate and efficient results [28,29]. One of the approaches involves using CNN architectures, such as VGG and customized CNNs, to extract fine details and features for fire detection [3].
Attention-based networks have also been used to enhance the accuracy of fire detection, with evaluations performed on custom datasets and compared with state-of-the-art CNN architecture [30]. Other deep-learning-based strategies for fire detection include the attention- and squeeze-customized CNN [31], U-shape network [31], Adam network [32], CNN-SA [33], and Adams predictor–corrector color weights network [34]. Moreover, Khan et al. [35] proposed the ConvNeXtTiny model for fire detection in a real-world environment. They performed their experiments on two benchmarks, the Yar and Foggia datasets, and obtained better performance. A vision transformer-based fire scene classification model is developed in [36]. Dishad et al. [37] proposed a lightweight CNN model inspired by the VGG model and achieved promising performance [37]. A modified CNN model with an attention mechanism for effective fire detection is proposed in [3,9,38,39]. Furthermore, Zhu et al. [40] proposed an efficient network for small object detection, and some other researchers have created fire localization techniques using advanced detection methodologies that rely on deep learning. For instance, Yar et al. [4] employed the YOLOv5 model with several modifications, such as using focus and stem modules in the backbone, replacing a larger filter in spatial pyramid pooling with a smaller one, and adding a p6 module in the head part. This modification obtained higher performance with lower model complexity. Some other approaches include Fire YOLO [41], RCNN [42], Faster RCNN [43], YOLOv2 [44], YOLO V5 [45], YOLOv4 [46], YOLOX [47], and sparse residual network YOLO [48]. Despite achieving SOTA accuracy, CNN-based models face several challenges.
Implementing fire detection systems using complex machine learning models raises concerns regarding their computational complexity, which requires more extended training and testing times. Due to this, their deployment on edge devices is questionable. Furthermore, the precision of these models is currently inadequate for practical implementation, and they have relatively high false alarm rates, necessitating further enhancements. Additionally, these methods rely on traditional weight initialization methods, which can lead to vanishing gradient problems and increased computational costs.
To tackle these challenges, we developed a lightweight CNN-driven model for fire detection that delivers superior accuracy and minimizes false alarm rates. Our approach employs a backbone architecture based on EfficientNet, combined with stacked, encoded layers to improve accuracy and reduce false-positive rates. We chose EfficientNet as the backbone architecture because it incorporates compound scaling techniques that allow consistent adjustments to the network’s depth, width, and resolution. This approach ensures optimal performance and efficiency compared to other CNN architectures that adjust these factors manually.
Our research makes the following significant contributions:
  • Introduction of E-EFNet: The primary contribution of this research lies in the design of E-EFNet, a novel fire detection model. E-EFNet is designed to excel in fire recognition while reducing the occurrence of false alarms. This model leverages the lightweight EfficientNetB0 as a foundational feature extractor, augmented by a series of stacked autoencoders for refined feature extraction before the final classification phase.
  • Addressing False Alarms: E-EFNet’s contribution lies in reducing false alarms in fire detection. Adopting dense connections, as opposed to conventional linear connections, significantly enhances its effectiveness in identifying fire-related scenes, a critical challenge in fire detection systems.
  • Faster Convergence: To expedite the convergence process and mitigate issues related to slow convergence and increased training time, E-EFNet introduces a randomized weight initialization strategy that contributes to the model’s efficiency and effectiveness.
  • Efficient Inferencing on Edge Devices: The study demonstrates that E-EFNet is adaptable for efficient inferencing on edge devices. It significantly contributes to the Internet of Things (IoT) and resource-limited environments, where efficient fire detection is crucial.
  • Comprehensive Evaluation: The research contributes by thoroughly assessing various deep models before ultimately selecting E-EFNet as the optimal solution for the pressing challenges in fire detection. It demonstrates a rigorous evaluation process and the model’s superiority over contemporary benchmarks.
Our proposed model was subjected to comprehensive experimentation, which revealed its superior performance in terms of increased recognition accuracy and reduced false alarm rates. We conducted benchmark testing and compared our results to those achieved by recent state-of-the-art models. The findings indicated that our model outperformed the existing alternatives in these critical performance metrics.
In the remainder of this paper, we delve into the details of our proposed method. In Section 2, we comprehensively describe the approach we have developed. Subsequently, Section 3 is dedicated to presenting the results obtained through the application of our method, shedding light on the outcomes and findings of our study. Finally, in Section 4, we draw our conclusions, summarizing the key insights from our research and discussing their implications in the broader context of the study area.

2. The Proposed Method

In the literature, researchers have developed several techniques for fire detection that are essential for preserving both lives and property. However, some methods, such as CNN, are computationally expensive, while others have higher false alarm rates and low accuracy, such as those based on motion or color. Therefore, we developed E-EFNet, a lightweight CNN model that is computationally efficient and accurate and can be deployed on edge devices. Our E-EFNet architecture, shown in Figure 1, uses EfficientNetB31 as a backbone for feature extraction from input frames. We design densely connected encoding layers to process the resulting feature vector. In the following sections, we provide a more detailed explanation of the architecture of EfficientNetB3 and our proposed E-EFNet.

2.1. Feature Extraction and Encoding

Researchers have developed several CNN-based models for various purposes, including applications in photovoltaics [49], crowd estimation [50], deep learning in big data [51], classification and detection [2,52,53], medical data/healthcare [54,55], renewable energy [56], energy consumption [57], IoT-based smart cities [58,59,60], and fire recognition [61,62,63,64]. Examples of these architectures include AlexNet, SqueezeNet, GoogleNet, MobileNet, etc. However, each model has limitations and strengths, and researchers are constantly investigating new architectures for performance improvements. To address the issue of fire detection, we have studied the EfficientNet architecture, which is designed to scale all the network dimensions via a compound scaling method. EfficientNet employs a multi-objective network, which prioritizes the optimization of both FLOPs and accuracy. This architecture utilizes search space and uses scaling coefficients, namely, phi, gamma, and delta, to represent the model’s FLOPs and accuracy as optimization tools, where alpha and beta monitor the trade-off between accuracy and FLOPs. The EfficientNet architecture comprises numerous convolutional layers, each with varying kernel sizes. The model takes RGB inputs with a size of 150 × 150 and reduces the size of the feature maps by scaling down the hidden layers. Additionally, the network width is increased to improve accuracy. This design methodology ensures that the model extracts and utilizes relevant features from the input data. To further reduce false alarms and enhance accuracy, we process the output features of EfficientNet by passing them through our proposed densely connected autoencoder layers. This step facilitates feature encoding, allowing for the selection of the most optimal features from the output of EfficientNet.
The autoencoder learns the underlying representation of input data in a feature map in an unsupervised manner. An autoencoder typically consists of input, hidden, and output layers, depicted in Figure 2, with the encoder and decoder serving as the two essential components. The encoder compresses the input into a lower-dimensional feature map while the decoder reconstructs the original input from the compressed feature map. Assuming we have a dataset of input samples ( x n ) N n = 1 , where each sample x n belongs to r m x l , the encoder takes the input sample x n and maps it to a lower-dimensional feature map o n , which can be calculated using Equation (1):
h n = F ( w 1 x n + b 1 )
where w 1 , F , and b 1 are the encoder’s weight, activation function, and bias, respectively.
The decoder then takes the compressed feature map h n and reconstructs the original input sample x n from it, as shown in Equation (2):
o n = G ( w 2 x n + b 2 ) ,
Here, w 2 , G , and b 2 are the weights, activation, and bias of the decoder, respectively. By minimizing the difference between the original input sample x n and the reconstructed sample o n , the autoencoder can learn a lower-dimensional representation of the input data that captures its essential features.
The autoencoder’s encoding phase transforms the input data into a compressed feature representation, which is then fed into the decoding stage of the autoencoder to reconstruct the original input. By reducing the dimensionality of the input data, the encoding phase can capture all the critical features in a compressed format. Herein, we use the encoder part of the autoencoder to encode the features for further processing.

2.2. Weight Initialization

CNNs have three primary layers: convolutional, pooling, and fully connected. The convolutional layer extracts spatial features by convolving multiple filters of different sizes with the input data. Proper initialization of the weights and biases is critical to obtaining meaningful features. However, during training, problems like vanishing or exploding gradients may occur due to the different hyperparameter settings, such as the learning rate. Researchers have explored various hyperparameter configurations to fine-tune the model weights and optimize performance.
There are three types of weight initialization methods. The first method uses a constant set of weights for network initialization, which can prevent the learning algorithm from updating the network weights. The second method uses distribution-based initialization, such as uniform or Gaussian distribution, to assign random values to the distribution matrices. However, setting appropriate parameters for the network, such as the standard deviation and mean of the distribution, is challenging. This can impact the model’s training and result in vanishing gradient problems. The third approach utilizes random initialization based on prior knowledge.
The traditional CNN models use the backpropagation error approach to fine-tune parameters, resulting in slow convergence and a prolonged search for local minima, which needs longer training times. To address this issue, neural networks that leverage random weight initialization have been proposed in the literature. Currently, deep learning approaches have demonstrated promising results across a range of domains. However, these models also present challenges such as higher computation costs, specific task-oriented parameter tuning, and low convergence rates.
Heuristic approaches randomly initialize the layer weights and activation functions to cope with these issues. Heuristic strategies involve problem-solving without the use of an optimal solution method. This type of randomization allocates the normal distribution variance based on the input shape, which reduces the problem of vanishing or exploding gradients. As a result, the model achieves faster convergence and mitigates the oscillation of minima.

2.3. Architecture

The E-EFNet model proposed in this work is an extension of the EfficientNetB3 architecture, designed to obtain meaningful patterns from the data. The architecture consists of a densely connected network that incorporates three encoding layers. The output of EfficientNetB0, a 1280-dimensional feature vector, is passed through these encoding layers to produce a lower-dimensional feature vector while preserving essential information. We use two encoding layers to transform the initially encoded 1280-dimension feature vector to 640 and 320. This process allows for a more accurate classification of the input data. The densely connected network is based on a mechanism in which each layer receives input from all preceding layers. In the current study, the output of every encoding layer is merged with the input of the previous layer to preserve the feature vector’s dimensionality, resulting in better classification features. A SoftMax classifier is employed to perform the classification task. We trained the model for 100 epochs with an SGD optimizer. The loss function is binary cross-entropy, with a learning rate of 1 × 10−4 and a momentum of 0.9. After conducting various tests, we selected the ideal combination of learning rate, optimizer, and epochs.

3. Results

In this segment, we will examine the evaluation metrics, implementation configuration, datasets, and comparisons of E-EFNet in terms of accuracy, false alarms, and computational complexity. We implemented the E-EFNet using Python V3.6.4 on a GeForce RTX 3060 GPU with 6 GB memory. We utilized the Keras framework with TensorFlow as the backend. To assess the effectiveness of the proposed method, we used two benchmarks: Foggia (FGG) [65] and Yar (YR) [3]. More detailed information on the topics mentioned above is provided below.

3.1. Metrics of Evaluation

In this study, the E-EFNet model is compared to other lightweight deep learning models using three standard evaluation metrics: F1-score, accuracy, false negative rate (FNR), precision, false positive rate (TPR), and recall, as evaluated by other studies [3,66]. These metrics are widely used in the literature to assess the performance of fire recognition models. The TPR metric, also known as sensitivity, measures the system’s ability to detect the presence of fire in an input frame. The second metric, TNR, also known as specificity, measures the system’s ability to identify non-fire frames correctly and is calculated by dividing the number of true negative predictions by the sum of true negative and false positive predictions. The third metric, accuracy, measures the overall classification performance of the system for both fire and non-fire frames and is calculated by dividing the sum of true positive and true negative predictions by the total number of predictions.

3.2. Datasets

The datasets utilized in this research include FGG and YR to evaluate the performance of our fire detection model. FGG contains 31 videos captured from indoor and outdoor environments, with 17 videos having normal scenes and 14 having fire scenes. The dataset includes 62,690 frames, with the details in [2]. On the other hand, YR is a small-scale dataset consisting of 2000 images of normal scenes, with 1000 images belonging to the fire scenes and 1000 images belonging to the normal scenes. The dataset presents a challenge due to objects like colored fire, i.e., light and sun. The datasets are divided into three sets for testing, validation, and training, consisting of 20%, 10%, and 70% of the data, respectively, as performed by previous studies [66].

3.3. Ablation Study

This section compares the E-EFNet performance of different lightweight models on the FGG and YR datasets. The compared models include ResNet50, MobileNet, Inception, NASNet, EfficientNet, and E-EFNet, where each model’s results are reported in Table 1 over both datasets. The results shown in Table 1 show that EfficientNet surpassed other methods by achieving higher accuracy and lower false alarms. However, the performance of E-EFNet was significantly better than all the other models, achieving an FPR of 0, FNR of 0.22, and accuracy of 99.91% on FGG, while achieving 98.74% precision, 98.70% recall, 98.40% accuracy, and 98.74% F1-score on the YR dataset. The E-EFNet achieved superior performance compared to other state-of-the-art deep learning models on FGG and YR datasets. E-EFNet achieved high accuracy, precision, recall, and F1-score with a low false-positive and false-negative rate, indicating its effectiveness for effective fire detection. The ablation study also demonstrated the impact of the proposed modifications to the EfficientNet architecture, resulting in improved performance. Therefore, E-EFNet can be considered a promising model for fire detection. The effectiveness of the proposed E-EFNet model is further enhanced by accurately classifying challenging samples, such as fire-like objects and distant fires, which is demonstrated through the visualized results shown in Figure 3 and Figure 4 for the FGG and YR datasets, respectively. These figures provide clear evidence that the model is highly capable of successfully categorizing difficult instances, which is a significant achievement in fire detection.

3.4. Comparison of E-EFNet Performance with Baseline

The effectiveness of the proposed model is assessed using accuracy, TPR, and FNR metrics, as per the evaluation performed by [28], for both FGG and YR. In FGG, the proposed method outperformed other state-of-the-art models, as depicted in Table 2. Table 2 presents a performance comparison of the proposed E-EFNet model with other state-of-the-art models on the FGG dataset. The evaluation metrics used in this comparison were false negative rate (FNR), accuracy, and false positive rate (FPR). The results show that the proposed E-EFNet model achieved a very low FNR of 0.22%, the second-best performance after the DFAN model with an FNR of 0.58%. E-EFNet also achieved the highest accuracy, 99.91%, among all the compared models, demonstrating its high effectiveness in accurately detecting fire in images.
Moreover, the FPR for E-EFNet was 0%, the lowest among all the compared models, indicating that the model has a shallow rate of falsely identifying non-fire objects as fire. Among the other models, several achieved high performance, including EMNFire, LWCNN, DFAN, and SE-CANet, with FNRs below 1% and accuracy above 95%. However, some models, such as CANetB0 and CNNFire, had relatively low accuracy and high FPRs, indicating that they may not be as effective in detecting fire in images. Overall, the results in Table 2 suggest that the proposed E-EFNet model outperforms most of the state-of-the-art models on the FGG dataset in terms of FNR, accuracy, and FPR and is highly effective in detecting fire.
Table 3 presents a performance comparison of the proposed E-EFNet model with other state-of-the-art models on the YR dataset. The evaluation metrics used in this comparison are F1 score, precision, recall, and accuracy. The results show that E-EFNet achieved better performance compared to other models, including ResNetFire [25], LW [3], EFDNet [22], and DFAN [2], in terms of all evaluation metrics. Specifically, our model obtains a precision of 98.82%, recall of 98.70%, F1 score of 98.74%, and accuracy of 98.75%. These results show the higher effectiveness of the proposed model in accurately detecting fire in images, which is a critical requirement for fire safety and prevention. Compared to other models, EFDNet obtained the second-best performance, with precision, recall, F1 score, and accuracy of 94.11%, 96.00%, 95.00%, and 95.00%, respectively. LW and DFAN models achieved comparable results but slightly lower accuracy and F1 scores than EFDNet. The ResNetFire obtained the most inadequate performance, with precision, recall, F1 score, and accuracy of 88.00%, 86.00%, 86.00%, and 86.67%, respectively. Overall, the results in Table 3 suggest that the proposed E-EFNet model is highly effective in detecting fire and outperforms the existing models on the YR dataset.

3.5. Time Complexity

Time complexity is another evaluation criterion that shows how long it takes to perform a particular computation or task. In the context of deep learning models for image classification, the time complexity is often expressed in terms of frames per second (FPS), which means the model can process this number of images in one second. Table 4 compares E-EFNet with other state-of-the-art models regarding computational complexity over robust systems and resource-constrained devices. The table shows the FPS achieved by each model on both robust systems and edge devices. As we can see, E-EFNet achieved a high FPS on both the Core-i5 CPU and Raspberry Pi B3+ (RPIB3+) edge devices, with 51 and 8, respectively. This suggests that E-EFNet is a good choice for image classification tasks in IoT environments, where edge devices are often used due to their low cost and energy efficiency. It is worth noting that some other models, such as those of Fogia et al. [65] and Lascio et al. [18], achieved higher FPS on robust systems because these methods are based on traditional machine learning techniques. The computational complexity of these methods is lower. However, the performance of evaluation metrics is much worse than that of other deep models, as shown in the comparison tables. However, E-EFNet outperforms these models in terms of FPS on more powerful systems. Therefore, the choice of model will depend on the specific requirements of the application and the resources available.

4. Conclusions

Detecting fires in surveillance videos is crucial to taking prompt action and preventing damage and loss of life. Various machine learning and deep learning models have been proposed to detect fires in surveillance videos, but some face limitations regarding accuracy and computational efficiency. We proposed a highly efficient and effective fire detection model called E-EFNet to cope with these issues. The E-EFNet uses EfficientNetB0 for feature extraction, which is then passed to a densely connected, stacked encoding network for feature refinement, followed by classification. E-EFNet has achieved superior performance through massive evaluation compared to detailed ablation studies and existing methods. This architecture is designed to enhance accuracy and reduce false alarms, yielding results that outperform previous state-of-the-art models in both accuracy and false alarm rate. Furthermore, the time complexity analysis demonstrated that the proposed E-EFNet model achieved high FPS rates on both robust systems and resource-constrained edge devices. This capability makes it a promising solution for real-time fire detection in various IoT applications.

Author Contributions

Conceptualization, H.F. and B.J.; methodology, H.F.; validation, H.F., B.J. and S.B.A.K.; formal analysis, H.F. and B.J.; investigation, H.F.; data curation, B.J. and S.B.A.K.; writing—original draft preparation, H.F.; writing—review and editing, M.M.N. and B.J.; supervision, M.M.N.; project administration, H.F. and M.M.N.; funding acquisition, M.M.N. All authors have read and agreed to the published version of the manuscript.

Funding

The authors would like to acknowledge the support of Prince Sultan University for paying the article processing charges (APCs) of this publication.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Data is contained within the article.

Acknowledgments

The authors would like to acknowledge Prince Sultan University and Smart Systems Engineering lab for their valuable support.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. FEMA55. Coastal Construction Manual; Federal Emergency Management Agency: Washington, DC, USA, 2000; Volume 2, pp. 1–400.
  2. Yar, H.; Hussain, T.; Agarwal, M.; Khan, Z.A.; Gupta, S.K.; Baik, S.W. Optimized Dual Fire Attention Network and Medium-Scale Fire Classification Benchmark. IEEE Trans. Image Process. 2022, 31, 6331–6343. [Google Scholar] [CrossRef] [PubMed]
  3. Yar, H.; Hussain, T.; Khan, Z.A.; Koundal, D.; Lee, M.Y.; Baik, S.W. Vision sensor-based real-time fire detection in resource-constrained IoT environments. Comput. Intell. Neurosci. 2021, 2021, 5195508. [Google Scholar] [CrossRef] [PubMed]
  4. Yar, H.; Khan, Z.A.; Ullah, F.U.M.; Ullah, W.; Baik, S.W. A modified YOLOv5 architecture for efficient fire detection in smart cities. Expert Syst. Appl. 2023, 231, 120465. [Google Scholar] [CrossRef]
  5. Toulouse, T.; Rossi, L.; Akhloufi, M.; Celik, T.; Maldague, X. Benchmarking of wildland fire colour segmentation algorithms. IET Image Process. 2015, 9, 1064–1072. [Google Scholar] [CrossRef]
  6. Yar, H.; Imran, A.S.; Khan, Z.A.; Sajjad, M.; Kastrati, Z. Towards smart home automation using IoT-enabled edge-computing paradigm. Sensors 2021, 21, 4932. [Google Scholar] [CrossRef] [PubMed]
  7. Jan, H.; Yar, H.; Iqbal, J.; Farman, H.; Khan, Z.; Koubaa, A. Raspberry pi assisted safety system for elderly people: An application of smart home. In Proceedings of the 2020 First International Conference of Smart Systems and Emerging Technologies (SMARTTECH), Riyadh, Saudi Arabia, 3–5 November 2020; IEEE: Piscataway, NJ, USA, 2020. [Google Scholar]
  8. Harkat, H.; Nascimento, J.M.; Bernardino, A.; Ahmed, H.F.T. Fire images classification based on a handcraft approach. Expert Syst. Appl. 2023, 212, 118594. [Google Scholar] [CrossRef]
  9. Majid, S.; Alenezi, F.; Masood, S.; Ahmad, M.; Gündüz, E.S.; Polat, K. Attention based CNN model for fire detection and localization in real-world images. Expert Syst. Appl. 2022, 189, 116114. [Google Scholar] [CrossRef]
  10. Celik, T.; Demirel, H. Fire detection in video sequences using a generic color model. Fire Saf. J. 2009, 44, 147–158. [Google Scholar] [CrossRef]
  11. Rafiee, A.; Dianat, R.; Jamshidi, M.; Tavakoli, R.; Abbaspour, S. Fire and smoke detection using wavelet analysis and disorder characteristics. In Proceedings of the 2011 3rd International Conference on Computer Research and Development, Shanghai, China, 11–13 March 2011; IEEE: Piscataway, NJ, USA, 2011. [Google Scholar]
  12. Khan, Z.A.; Ullah, W.; Ullah, A.; Rho, S.; Lee, M.Y.; Baik, S.W. An Adaptive Filtering Technique for Segmentation of Tuberculosis in Microscopic Images. In Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval, Dubai, United Arab Emirates, 18–19 February 2023. [Google Scholar]
  13. Marbach, G.; Loepfe, M.; Brupbacher, T. An image processing technique for fire detection in video images. Fire Saf. J. 2006, 41, 285–289. [Google Scholar] [CrossRef]
  14. Chen, T.-H.; Wu, P.-H.; Chiou, Y.-C. An early fire-detection method based on image processing. In Proceedings of the 2004 International Conference on Image Processing, ICIP’04, Singapore, 24–27 October 2004; IEEE: Piscataway, NJ, USA, 2004. [Google Scholar]
  15. Kim, Y.-H.; Kim, A.; Jeong, H.-Y. RGB color model based the fire detection algorithm in video sequences on wireless sensor network. Int. J. Distrib. Sens. Netw. 2014, 10, 923609. [Google Scholar] [CrossRef]
  16. Celik, T.; Demirel, H.; Ozkaramanli, H.; Uyguroglu, M. Fire detection using statistical color model in video sequences. J. Vis. Commun. Image Represent. 2007, 18, 176–185. [Google Scholar] [CrossRef]
  17. Celik, T.; Ozkaramanli, H.; Demirel, H. Fire pixel classification using fuzzy logic and statistical color model. In Proceedings of the 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP’07, Honolulu, HI, USA, 15–20 April 2007; IEEE: Piscataway, NJ, USA, 2007. [Google Scholar]
  18. Di Lascio, R.; Greco, A.; Saggese, A.; Vento, M. Improving fire detection reliability by a combination of videoanalytics. In Proceedings of the International Conference Image Analysis and Recognition, Porto, Portugal, 29 September–1 October 2004; Springer: Berlin/Heidelberg, Germany, 2004. [Google Scholar]
  19. Borges, P.V.K.; Izquierdo, E. A probabilistic approach for vision-based fire detection in videos. IEEE Trans. Circuits Syst. Video Technol. 2010, 20, 721–731. [Google Scholar] [CrossRef]
  20. Mueller, M.; Karasev, P.; Kolesov, I.; Tannenbaum, A. Optical flow estimation for flame detection in videos. IEEE Trans. Image Process. 2013, 22, 2786–2797. [Google Scholar] [CrossRef] [PubMed]
  21. Dimitropoulos, K.; Barmpoutis, P.; Grammalidis, N. Spatio-temporal flame modeling and dynamic texture analysis for automatic video-based fire detection. IEEE Trans. Circuits Syst. Video Technol. 2014, 25, 339–351. [Google Scholar] [CrossRef]
  22. Li, S.; Yan, Q.; Liu, P. An efficient fire detection method based on multiscale feature extraction, implicit deep supervision and channel attention mechanism. IEEE Trans. Image Process. 2020, 29, 8467–8475. [Google Scholar] [CrossRef] [PubMed]
  23. Parez, S.; Dilshad, N.; Alghamdi, N.S.; Alanazi, T.M.; Lee, J.W. Visual intelligence in precision agriculture: Exploring plant disease detection via efficient vision transformers. Sensors 2023, 23, 6949. [Google Scholar] [CrossRef]
  24. Parez, S.; Dilshad, N.; Alanazi, T.M.; Lee, J.-W. Towards Sustainable Agricultural Systems: A Lightweight Deep Learning Model for Plant Disease Detection. Comput. Syst. Sci. Eng. 2023, 47, 515–536. [Google Scholar] [CrossRef]
  25. Sharma, J.; Granmo, O.-C.; Goodwin, M.; Fidje, J.T. Deep convolutional neural networks for fire detection in images. In Proceedings of the International Conference on Engineering Applications of Neural Networks, Athens, Greece, 25–27 August 2017; Springer: Berlin/Heidelberg, Germany, 2017. [Google Scholar]
  26. Frizzi, S.; Kaabi, R.; Bouchouicha, M.; Ginoux, J.-M.; Moreau, E.; Fnaiech, F. Convolutional neural network for video fire and smoke detection. In Proceedings of the IECON 2016—42nd Annual Conference of the IEEE Industrial Electronics Society, Florence, Italy, 24–27 October 2016; IEEE: Piscataway, NJ, USA, 2016. [Google Scholar]
  27. Muhammad, K.; Ahmad, J.; Baik, S.W. Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 2018, 288, 30–42. [Google Scholar] [CrossRef]
  28. Muhammad, K.; Ahmad, J.; Mehmood, I.; Rho, S.; Baik, S.W. Convolutional neural networks based fire detection in surveillance videos. IEEE Access 2018, 6, 18174–18183. [Google Scholar] [CrossRef]
  29. Muhammad, K.; Ahmad, J.; Lv, Z.; Bellavista, P.; Yang, P.; Baik, S.W. Efficient deep CNN-based fire detection and localization in video surveillance applications. IEEE Trans. Syst. Man Cybern. Syst. 2018, 49, 1419–1434. [Google Scholar] [CrossRef]
  30. Huang, L.; Liu, G.; Wang, Y.; Yuan, H.; Chen, T. Fire detection in video surveillances using convolutional neural networks and wavelet transform. Eng. Appl. Artif. Intell. 2022, 110, 104737. [Google Scholar] [CrossRef]
  31. Zhang, J.; Zhu, H.; Wang, P.; Ling, X. ATT squeeze U-Net: A lightweight network for forest fire detection and recognition. IEEE Access 2021, 9, 10858–10870. [Google Scholar] [CrossRef]
  32. Deng, Z.; Hu, S.; Yin, S.; Wang, Y.; Basu, A.; Cheng, I. Multi-step implicit Adams predictor-corrector network for fire detection. IET Image Process. 2022, 16, 2338–2350. [Google Scholar] [CrossRef]
  33. Sarkar, S.; Menon, A.S.; Gopalakrishnan, T.; Kakelli, A.K. Convolutional Neural Network (CNN-SA) based Selective Amplification Model to Enhance Image Quality for Efficient Fire Detection. IJ Image Graph. Signal Process. 2021, 5, 51–59. [Google Scholar] [CrossRef]
  34. Zhang, R.; Zhang, W.; Liu, Y.; Li, P.; Zhao, J. An efficient deep neural network with color-weighted loss for fire detection. Multimed. Tools Appl. 2022, 81, 39695–39713. [Google Scholar] [CrossRef]
  35. Khan, T.; Aslan, H.İ. Performance Evaluation of Enhanced ConvNeXtTiny-Based Fire Detection System in Real-World Scenarios. 2023. Available online: https://openreview.net/forum?id=A-E41oZCfrf (accessed on 1 October 2023).
  36. Yar, H.; Hussain, T.; Khan, Z.A.; Lee, M.Y.; Baik, S.W. Fire Detection via Effective Vision Transformers. J. Korean Inst. Next Gener. Comput. 2021, 17, 21–30. [Google Scholar]
  37. Dilshad, N.; Khan, T.; Song, J. Efficient deep learning framework for fire detection in complex surveillance environment. Comput. Syst. Sci. Eng. 2023, 46, 749–764. [Google Scholar] [CrossRef]
  38. Nadeem, M.; Dilshad, N.; Alghamdi, N.S.; Dang, L.M.; Song, H.-K.; Nam, J.; Moon, H. Visual Intelligence in Smart Cities: A Lightweight Deep Learning Model for Fire Detection in an IoT Environment. Smart Cities 2023, 6, 2245–2259. [Google Scholar] [CrossRef]
  39. Khan, S.U.; Lee, S.; Yar, H.; Lee, M.Y.; Khan, H.; Baik, S.W. An Efficient Fire Detection Using a Smart Surveillance System. Available online: https://www.earticle.net/Article/A433523 (accessed on 1 October 2023).
  40. Zhu, Z.; Wang, S.; Gu, S.; Li, Y.; Li, J.; Shuai, L.; Qi, G. Driver distraction detection based on lightweight networks and tiny object detection. Math. Biosci. Eng. 2023, 20, 18248–18266. [Google Scholar] [CrossRef]
  41. Zhao, L.; Zhi, L.; Zhao, C.; Zheng, W. Fire-YOLO: A Small Target Object Detection Method for Fire Inspection. Sustainability 2022, 14, 4930. [Google Scholar] [CrossRef]
  42. Chopde, A.; Magon, A.; Bhatkar, S. Forest Fire Detection and Prediction from image processing using RCNN. In Proceedings of the 7th World Congress on Civil, Structural, and Environmental Engineering, Virtual, 10–12 April 2022. [Google Scholar]
  43. Pan, J.; Ou, X.; Xu, L. A collaborative region detection and grading framework for forest fire smoke using weakly supervised fine segmentation and lightweight Faster-RCNN. Forests 2021, 12, 768. [Google Scholar] [CrossRef]
  44. Saponara, S.; Elhanashi, A.; Gagliardi, A. Real-time video fire/smoke detection based on CNN in antifire surveillance systems. J. Real-Time Image Process. 2021, 18, 889–900. [Google Scholar] [CrossRef]
  45. Wen-ping, J.; Zhen-cun, J. Research on early fire detection of Yolo V5 based on multiple transfer learning. Fire Sci. Technol. 2021, 40, 109. [Google Scholar]
  46. Mukhiddinov, M.; Abdusalomov, A.B.; Cho, J. Automatic Fire Detection and Notification System Based on Improved YOLOv4 for the Blind and Visually Impaired. Sensors 2022, 22, 3307. [Google Scholar] [CrossRef]
  47. Zhang, J.; Ke, S. Improved YOLOX Fire Scenario Detection Method. Wirel. Commun. Mob. Comput. 2022, 2022, 9666265. [Google Scholar] [CrossRef]
  48. Li, Y.; Shen, Z.; Li, J.; Xu, Z. A Deep Learning Method based on SRN-YOLO for Forest Fire Detection. In Proceedings of the 2022 5th International Symposium on Autonomous Systems (ISAS), Hangzhou, China, 8–10 April 2022; IEEE: Piscataway, NJ, USA, 2022. [Google Scholar]
  49. Khan, Z.A.; Hussain, T.; Baik, S.W. Dual stream network with attention mechanism for photovoltaic power forecasting. Appl. Energy 2023, 338, 120916. [Google Scholar] [CrossRef]
  50. Haroon, U.; Ullah, A.; Hussain, T.; Ullah, W.; Sajjad, M.; Muhammad, K.; Lee, M.Y.; Baik, S.W. A multi-stream sequence learning framework for human interaction recognition. IEEE Trans. Hum. Mach. Syst. 2022, 52, 435–444. [Google Scholar] [CrossRef]
  51. Khan, M.; Jan, B.; Farman, H. Deep Learning: Convergence to Big Data Analytics; Springer: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
  52. Jan, B.; Farman, H.; Khan, M.; Imran, M.; Islam, I.U.; Ahmad, A.; Ali, S.; Jeon, G. Deep learning in big data analytics: A comparative study. Comput. Electr. Eng. 2019, 75, 275–287. [Google Scholar] [CrossRef]
  53. Ullah, W.; Hussain, T.; Ullah, F.U.M.; Lee, M.Y.; Baik, S.W. TransCNN: Hybrid CNN and transformer mechanism for surveillance anomaly detection. Eng. Appl. Artif. Intell. 2023, 123, 106173. [Google Scholar] [CrossRef]
  54. Yar, H.; Abbas, N.; Sadad, T.; Iqbal, S. Lung Nodule Detection and Classification Using 2D and 3D Convolution Neural Networks (CNNs). In Artificial Intelligence and Internet of Things; CRC Press: Boca Raton, FL, USA, 2021; pp. 365–386. [Google Scholar]
  55. Nasralla, M.M.; Khattak, S.B.A.; Ur Rehman, I.; Iqbal, M. Exploring the Role of 6G Technology in Enhancing Quality of Experience for m-Health Multimedia Applications: A Comprehensive Survey. Sensors 2023, 23, 5882. [Google Scholar] [CrossRef] [PubMed]
  56. Khan, Z.A.; Ullah, A.; Ullah, W.; Rho, S.; Lee, M.; Baik, S.W. Electrical energy prediction in residential buildings for short-term horizons using hybrid deep learning strategy. Appl. Sci. 2020, 10, 8634. [Google Scholar] [CrossRef]
  57. Khan, Z.A.; Hussain, T.; Baik, S.W. Boosting energy harvesting via deep learning-based renewable power generation prediction. J. King Saud Univ. Sci. 2022, 34, 101815. [Google Scholar] [CrossRef]
  58. Khattak, S.B.A.; Nasralla, M.M.; Farman, H.; Choudhury, N. Performance Evaluation of an IEEE 802.15. 4-Based Thread Network for Efficient Internet of Things Communications in Smart Cities. Appl. Sci. 2023, 13, 7745. [Google Scholar] [CrossRef]
  59. Khattak, A.; Bin, S.; Nasralla, M.M.; Esmail, M.A.; Mostafa, H.; Jia, M. WLAN RSS-based fingerprinting for indoor localization: A machine learning inspired bag-of-features approach. Sensors 2022, 22, 5236. [Google Scholar] [CrossRef] [PubMed]
  60. Hazarika, A.; Poddar, S.; Nasralla, M.M.; Rahaman, H. Area and energy efficient shift and accumulator unit for object detection in IoT applications. Alex. Eng. J. 2022, 61, 795–809. [Google Scholar] [CrossRef]
  61. Khan, Z.A.; Hussain, T.; Ullah, A.; Ullah, W.; Del Ser, J.; Muhammad, K.; Sajjad, M.; Baik, S.W. Modelling Electricity Consumption During the COVID19 Pandemic: Datasets, Models, Results and a Research Agenda. Energy Build. 2023, 294, 113204. [Google Scholar] [CrossRef]
  62. Ahmad, K.; Khan, M.S.; Ahmed, F.; Driss, M.; Boulila, W.; Alazeb, A.; Alsulami, M.; Alshehri, M.S.; Ghadi, Y.Y.; Ahmad, J. FireXnet: An explainable AI-based tailored deep learning model for wildfire detection on resource-constrained devices. Fire Ecol. 2023, 19, 54. [Google Scholar] [CrossRef]
  63. Almasoud, A.S. Intelligent Deep Learning Enabled Wild Forest Fire Detection System. In Computer Systems Science & Engineering; Tech Science Press: Henderson, NV, USA, 2023; Volume 44. [Google Scholar]
  64. Alqourabah, H.; Muneer, A.; Fati, S.M. A smart fire detection system using IoT technology with automatic water sprinkler. Int. J. Electr. Comput. Eng. 2021, 11, 2994–3002. [Google Scholar] [CrossRef]
  65. Foggia, P.; Saggese, A.; Vento, M. Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion. IEEE Trans. Circuits Syst. Video Technol. 2015, 25, 1545–1556. [Google Scholar] [CrossRef]
  66. Khan, Z.A.; Hussain, T.; Ullah, F.U.M.; Gupta, S.K.; Lee, M.Y.; Baik, S.W. Randomly initialized CNN with densely connected stacked autoencoder for efficient fire detection. Eng. Appl. Artif. Intell. 2022, 116, 105403. [Google Scholar] [CrossRef]
  67. Lascio, R.D.; Greco, A.; Saggese, A.; Vento, M. Improving Fire Detection Reliability by a Combination of Videoanalytics; Springer: Cham, Switzerland, 2014. [Google Scholar]
  68. Muhammad, K.; Khan, S.; Elhoseny, M.; Ahmed, S.H.; Baik, S.W. Efficient fire detection for uncertain surveillance environment. IEEE Trans. Ind. Inform. 2019, 15, 3113–3122. [Google Scholar] [CrossRef]
  69. Hashemzadeh, M.; Zademehdi, A. Fire detection for video surveillance applications using ICA K-medoids-based color model and efficient spatio-temporal visual features. Expert Syst. Appl. 2019, 130, 60–78. [Google Scholar] [CrossRef]
  70. Li, Y.; Zhang, W.; Liu, Y.; Jin, Y. A visualized fire detection method based on convolutional neural network beyond anchor. Appl. Intell. 2022, 52, 13280–13295. [Google Scholar] [CrossRef]
  71. Habiboğlu, Y.H.; Günay, O.; Çetin, A.E. Covariance matrix-based fire and flame detection method in video. Mach. Vis. Appl. 2012, 23, 1103–1113. [Google Scholar] [CrossRef]
Figure 1. The main framework of E-EFNet incorporates three main steps: data feeding, model training, and output.
Figure 1. The main framework of E-EFNet incorporates three main steps: data feeding, model training, and output.
Applsci 13 12941 g001
Figure 2. Diagram of the internal structure of the autoencoder, which is the basic building block of the proposed model.
Figure 2. Diagram of the internal structure of the autoencoder, which is the basic building block of the proposed model.
Applsci 13 12941 g002
Figure 3. Visual results of E-EFNet over FGG dataset: (a) images with fire (b) images without fire.
Figure 3. Visual results of E-EFNet over FGG dataset: (a) images with fire (b) images without fire.
Applsci 13 12941 g003
Figure 4. Visual results of the E-EFNet model over the YR dataset: (a) represents the fire images accurately classified by the proposed model, whereas (b) shows the normal images.
Figure 4. Visual results of the E-EFNet model over the YR dataset: (a) represents the fire images accurately classified by the proposed model, whereas (b) shows the normal images.
Applsci 13 12941 g004
Table 1. Comparison of E-EFNet with state-of-the-art model over FGG and YR dataset.
Table 1. Comparison of E-EFNet with state-of-the-art model over FGG and YR dataset.
FGG DatasetYR Dataset
MethodFNRAccuracyFPRPrecisionRecallAccuracyF1-Score
ResNet500.6897.850.8184.8580.2183.9281.74
MobileNet2.1891.871.8592.3185.7188.4688.89
Inception0.5497.450.8390.9194.3492.5992.84
NASNet1.0396.820.9190.4589.9990.9190.91
EfficientNet0.4598.210.1794.3496.1595.2495.23
E-EFNet0.2299.91098.7498.7098.4098.74
Table 2. Comparison of E-EFNet with state-of-the-art models over the FGG dataset.
Table 2. Comparison of E-EFNet with state-of-the-art models over the FGG dataset.
MethodFNRAccuracyFPR
FD-CSM [65]093.611.7
FD-CV [67]092.913.3
ANetFire [27]2.1394.49.07
GNetFire [28]1.5094.40.054
CNNFire [29]2.1294.58.87
EMNFire [68]0.1495.90
ICA-K [69]4.5395.34.83
VIT [36]1.0497.862.63
LWCNN [3]0.9297.20
CNN (Anchor) [70]1.9294.78.65
DFAN [2]0.5899.60
SE-CANet [66]0.0397.20.04
CANetB0 [9]1.9143.57.48
E-EFNet0.2299.910
Table 3. Comparison of E-EFNet with existing models over the YR dataset.
Table 3. Comparison of E-EFNet with existing models over the YR dataset.
BackboneF1-ScoreRecallPrecisionAccuracy
EFDNet [22]95.0096.0094.1195.00
LW [3]95.0494.0095.0094.50
DFAN [2]97.0097.0098.0097.50
ResNetFire [25]86.0086.0088.0086.67
FCAN [39]98.2098.0098.5098.00
E-EFNet98.7498.7098.8298.40
Table 4. Comparative Analysis of FPS Achieved by State-of-the-Art Models on Different Devices.
Table 4. Comparative Analysis of FPS Achieved by State-of-the-Art Models on Different Devices.
Ref.FPSParameters (Millions)Edge Device (FPS)System Specifications
Muhammad et al. [68]344.35Nvidia TITAN X GPU
RPIB3+
Muhammad et al. [29]20604Nvidia TITAN X
RPIB3+
Fogia et al. [65]60--3Dual-core
RPIB3
Lascio et al. [18]70----
Habiboğlu et al. [71]20---Dual-core CPU
DFAN [2]7083.630.83GeForce-RTX-3090
RPIB3+
E-EFNet5112.38GeForce-RTX-3060
RPIB3+
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Farman, H.; Nasralla, M.M.; Khattak, S.B.A.; Jan, B. Efficient Fire Detection with E-EFNet: A Lightweight Deep Learning-Based Approach for Edge Devices. Appl. Sci. 2023, 13, 12941. https://doi.org/10.3390/app132312941

AMA Style

Farman H, Nasralla MM, Khattak SBA, Jan B. Efficient Fire Detection with E-EFNet: A Lightweight Deep Learning-Based Approach for Edge Devices. Applied Sciences. 2023; 13(23):12941. https://doi.org/10.3390/app132312941

Chicago/Turabian Style

Farman, Haleem, Moustafa M. Nasralla, Sohaib Bin Altaf Khattak, and Bilal Jan. 2023. "Efficient Fire Detection with E-EFNet: A Lightweight Deep Learning-Based Approach for Edge Devices" Applied Sciences 13, no. 23: 12941. https://doi.org/10.3390/app132312941

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop