Seeing Danger: How AI-Based Computer Vision Detects Firearms to Boost Security

• c++
• opencv

This is not just a project; it's the foundation of a startup I'm building to revolutionize security solutions. The goal is to create an advanced AI-powered surveillance system capable of identifying and responding to potential threats in real-time. Using the YOLO/Darknet framework, I built and trained a specialized neural network designed to detect active shooter scenarios with exceptional precision. By leveraging a convolutional neural network (CNN), this project is designed to enhance safety and reliability in real-world security applications. ¹

---

Technologies Used

• YOLO/Darknet: High-performance convolutional neural network framework.
• DarkMark: Annotation and training tool for fine-tuning models.
• FlTK: Used FLTM framework to write C++-based GUI I engineered for minimal memory consumption.
• C++: Core programming language for the system's backend.
• HTML: For explaining my goal for this startup/project.
• Twilio Voice API: Facilitates instant voice alerts for critical situations.

---

---

Application Interface Gallery

Explore the C++ GUI built with FLTK framework for real-time firearm detection and security monitoring

---

How it works:

1. Trained millions of Postive and Negative Datasets for Machine learning task
2. After Training, deploy the model weights into the C++ program
3. My C++ program consists of using the OpenCV framework. Using this framework helped me create a clear goal for object detection.

And yup, that works. But not without jumping through a few hoops:

• We can use this basic technology to save many lives. During active threat situations on school grounds, the attacker often comes from an entry point around the school, which can catch staff, teachers, and students off guard. Having a detection system can cut response time in half and save many lives. In situations like this, every second really matters.

                                  
                                      
                                          
    #include  opencv2/opencv.hpp  #include opencv2/highgui/highgui.hpp 
  #include opencv2/imgproc/imgproc.hpp 
  #include iostream 
  #include opencv2/objdetect/objdetect.hpp    
  
  using namespace std; 
  using namespace cv;
  
   
  #include "FL/Fl.H"
  #include "FL/Fl_Window.H"
  #include "darknet.hpp"
  #include "darknet_cfg_and_state.hpp"
  #include "FL/Fl_Box.H"
  #include "FL/Fl_Button.H"
  #include "FL/Fl_Toggle_Button.H"
  #include "opencv2/opencv.hpp"
  #include "iostream"
  #include "thread"
  #include "chrono"
  #include "iomanip"
  
      class WebcamWindow : public Fl_Window {
          private:
              Fl_Box* videoBox; // Pointer to a box widget to display the video feed
              cv::VideoCapture cap; // OpenCV object to capture video from the desired camera  
              cv::Mat frame, resizedFrame; // Matrices to hold the current frame and the resized version of it for display
              Fl_RGB_Image* img; // Pointer to an image object to hold the current frame for display the video frame in rhe GUI 
              bool stopFlag; // Video Capture Flag
              cv::VideoWriter videoWriter; 
              bool isRecording; 
              std::string currentVideoFile;
          
              std::string getTimestamp()
              {
                  auto now = std::chrono::system_clock::now();
                  auto in_time_t = std::chrono::system_clock::to_time_t(now);
                  std::stringstream ss; 
                  ss << std::put_time(std::localtime(&in_time_t), "%Y%m%d_%H%M%S");
                  return ss.str();
              }
          void startRecording() {
                  currentVideoFile = "recording_" + getTimestamp() + ".avi";
                  int fourcc = cv::VideoWriter::fourcc('M','J','P','G'); // Motion-JPEG codec
                  double fps = estimated_fps;
                  cv::Size frameSize(cap.get(cv::CAP_PROP_FRAME_WIDTH), 
                                    cap.get(cv::CAP_PROP_FRAME_HEIGHT));
                  // button camera works
                  if(videoWriter.open(currentVideoFile, fourcc, fps, frameSize)) {
                      isRecording = true;
                      recordButton->label("Stop");
                      recordButton->color(fl_rgb_color(255, 0, 0)); // Red when recording
                      std::cout << "Recording started: " << currentVideoFile << std::endl;
                  } else {
                      std::cerr << "Failed to initialize video writer!" << std::endl;
                  }
              }
          
              void stopRecording()
               {
                  isRecording = false; 
                  videoWriter.release(); 
                  recordButton->label("Record");
                  recordButton->color(fl_rgb_color(34, 139, 34)); // Original green color
                  std::cout << "Recording saved: " << currentVideoFile << std::endl;
          
               }
               static void record_button_callback(Fl_Widget* widget, void* data)
               {
                  WebcamWindow* win = static_cast(data);
                  if(!win->isRecording)
                  {
                      win->startRecording();
                  } else {
                      win->stopRecording();
                  }
               }
          
          
              // Neural network
              Darknet::NetworkPtr net; // Pointer to the neural network object detection 
          
              // FPS and statistics
              double estimated_fps;
              size_t frame_counter;
              size_t total_objects_found;
              std::chrono::high_resolution_clock::time_point timestamp_start;
          
              // Top bar widgets
              Fl_Box* topBar; // Pointer to the top bar widget
              Fl_Button* exitButton; // Pointer to the exit button widget
          
              // Create the action button
              Fl_Button* actionButton;
              Fl_Button* alertButton; 
              Fl_Button* settingsButton;
              Fl_Button* recordButton;
              Fl_Button* CameralistButton;
              Fl_Button* viewButton;
              // Strings for GuardiansafeAI Neural Network Model paths(cfg, names, weights)
              const std::string MODEL_CFG = "/home/zeshan/src/darknet/src-examples/Shooter.cfg";
              const std::string MODEL_NAMES = "/home/zeshan/src/darknet/src-examples/Shooter.names";
              const std::string MODEL_WEIGHTS = "/home/zeshan/src/darknet/src-examples/Shooter_best.weights";
          
  
      double estimate_camera_fps(cv::VideoCapture & cap) {
          std::cout << "Estimating FPS..." << std::endl;
  
          // Read and discard a few frames to allow the camera to stabilize
          cv::Mat mat;
          for (int i = 0; i < 5; i++) {
              cap >> mat;
          }
  
          // Estimate FPS by reading several consecutive frames
          size_t frame_counter = 0;
          const auto ts1 = std::chrono::high_resolution_clock::now();
          for (int i = 0; cap.isOpened() and i < 5; i++) {
              cap >> mat;
              if (!mat.empty()) {
                  frame_counter++;
              }
          }
          const auto ts2 = std::chrono::high_resolution_clock::now();
  
          const double actual_fps = static_cast(frame_counter) / std::chrono::duration_cast(ts2 - ts1).count() * 1000000000.0;
          return actual_fps;
      }
  
      void drawAppleStyleText(cv::Mat& frame, const std::string& text, cv::Point position, double fontScale, int thickness) {
          // Calculate the size of the text
          int baseline = 0;
          cv::Size textSize = cv::getTextSize(text, cv::FONT_HERSHEY_SIMPLEX, fontScale, thickness, &baseline);
  
          // Define the background rectangle for the text
          cv::Rect bgRect(
              position.x, // X-coordinate of the top-left corner
              position.y - textSize.height, // Y-coordinate of the top-left corner
              textSize.width, // Width of the rectangle
              textSize.height + baseline // Height of the rectangle
          );
  
          // Create a semi-transparent black background for the text
          cv::Mat overlay = frame.clone();
          cv::rectangle(overlay, bgRect, cv::Scalar(0, 0, 0), cv::FILLED); // Black rectangle
          cv::addWeighted(overlay, 0.3, frame, 0.7, 0, frame); // Blend with the original frame (30% opacity)
  
          // Draw the text with a subtle shadow
          cv::putText(frame, text, position + cv::Point(2, 2), cv::FONT_HERSHEY_SIMPLEX, fontScale, cv::Scalar(0, 0, 0), thickness); // Shadow
          cv::putText(frame, text, position, cv::FONT_HERSHEY_SIMPLEX, fontScale, cv::Scalar(255, 120, 255), thickness); // Main text
      }
  
      void updateFrame() {
          if (!cap.read(frame)) {
              std::cerr << "Error: Failed to capture frame!" << std::endl;
              return;
          }
  
          // Process the frame through the neural network
          const auto results = Darknet::predict_and_annotate(net, frame);
          total_objects_found += results.size();
  
          // Calculate FPS
          const auto now = std::chrono::high_resolution_clock::now();
          const double elapsed_seconds = std::chrono::duration_cast(now - timestamp_start).count() / 1000000000.0;
          const double current_fps = frame_counter / elapsed_seconds;
  
          // Prepare multi-line text
          std::stringstream stats;
          stats << "FPS: " << std::fixed << std::setprecision(1) << current_fps << "\n"
                << "Objects: " << total_objects_found << "\n"
                << "Frame: " << frame_counter;
  
          // Split the text into lines
          std::vector lines;
          std::string line;
          while (std::getline(stats, line, '\n')) {
              lines.push_back(line);
          }
  
          // Render each line with Apple-style text
          int y_offset = 30; // Starting Y position for the first line
          for (const auto& text_line : lines) {
              drawAppleStyleText(frame, text_line, cv::Point(10, y_offset), 0.7, 2);
              y_offset += 30; // Increment Y position for the next line
          }
  
          // Convert the frame to RGB for display
          cv::cvtColor(frame, frame, cv::COLOR_BGR2RGB);
          cv::resize(frame, resizedFrame, resizedFrame.size());
  
          if (img) {
              delete img;
          }
          img = new Fl_RGB_Image(resizedFrame.data, resizedFrame.cols, resizedFrame.rows, 3);
  
          videoBox->image(img);
          videoBox->redraw();
  
          frame_counter++;
      }
  
      static void captureLoop(void* userdata) {
          WebcamWindow* win = static_cast(userdata);
          while (!win->stopFlag) {
              win->updateFrame();
              Fl::check();
              std::this_thread::sleep_for(std::chrono::milliseconds(1000 / 30)); // Aim for 30 FPS
          }
      }
  
      void startCapture() {
          std::thread captureThread(captureLoop, this);
          captureThread.detach();
      }
  
      ~WebcamWindow() {
          stopFlag = true;
          if (img) {
              delete img;
          }
          cap.release();
          Darknet::free_neural_network(net);
      }
  };
  
  int main(int argc, char* argv[]) {
      WebcamWindow window(640, 480, "FLTK Webcam", argc, argv);
      window.show();
      window.startCapture();
      return Fl::run();
  }
  
  
  
  
  
  
                                      
  
                                      
                                          
                                             
                                          
                                      
                                      
                                          
    
                                          
                                      
                                      
                                          
                                              
                                          
                                      
                                  
                              

  For now it works well enough - lets build :D

  Footnotes

  ---

  ²

  For more info, check e.g. OpenCv Documentation

  ³

  https://unix.stackexchange.com/questions/366797/grep-slow-to-exit-after-finding-match - so that 's why pipes sometimes seem to get "stuck "- never thought about how things work. TIL.