Information monitoring system based on intelligent image classification of video streams

Abstract

The development of an information system for monitoring fire and environmental safety based on the analysis of video data received from unmanned aerial vehicles is an urgent task, since timely detection of a fire source can reduce both material and human losses. The purpose of the study is to develop an information system for monitoring the fire situation based on the analysis of images obtained from RGB cameras using autonomous intelligent agents (AIA) and machine learning models. A method for classifying images of a video sequence has been developed, which consists of two stages. At the first stage, the images are segmented into rectangular segments of a given size. In this case, three classes of images are analyzed: smoke, flame, indifferent class. To classify the selected segments, "weak" and "strong" classifiers are used. "Weak" classifiers are built on the use of descriptors. The descriptors were formed on the basis of the Walsh-Hadamard transformation. Descriptors are calculated for three "weak" classifiers. The first "weak" classifier uses the Walsh-Hadamard spectral coefficients for the entire segment window. The second classifier implements descriptors for a window whose size is 2 times less than the selected analyzed segment. Accordingly, the third classifier uses the descriptors computed in a window whose size is four times smaller than the window size of the original selected segment. The classifier consists of three independently trained neural networks - "weak" classifiers. To combine the outputs of neural networks, an ensemble averaging block is used. An information system has been developed that allows: to form a database of images of segments for the classes "smoke", "flame", "indifferent"; determine the two-dimensional Walsh-Hadamard spectrum of image segments obtained from unmanned aerial vehicles; train fully connected neural networks; conduct exploratory analysis. Experimental studies on the classification of video data containing flames and smoke showed an average accuracy of 86% for smoke detection, and 89.5% for flames. Errors of the second kind in smoke detection averaged 13%, and in flame detection - 4.5%. To set up the classifiers, data from CCTV cameras in open spaces were used.