Machine Learning to Improve Mine Safety
Grad researcher Carlos Olmos de Aguilera tests the program by standing in front of the camera with PPE (helmet).
The University of Arizona’s mine automation lab is taking mining into the future. Dr. Nathalie Risso, Assistant Professor of Mining and Geological Engineering, is leading a developing research project utilizing artificial intelligence to help mines with PPE (Personal Protective Equipment). In a mine, workers must wear a variety of protective equipment, like hard hats, reflective vests, gloves, and other elements depending on the mine site. Safety is essential to a mine, but the lab is taking it a step further, with a computer program designed to use cameras to see if workers are wearing the proper equipment.
“Mining companies see a need for verifying safety compliance,” says Risso. The aim of this project is to implement the computer program into preexisting surveillance cameras in a mine that will trigger an alarm if it detects anyone in the mine not wearing the proper PPE. While the program is in its early stages, it has made a lot of progress.
How does it work?
AI works similar to the way a human eye works but with machine learning. What Risso and Carlos Olmos de Aguilera, Graduate Researcher, did first with the program is train an algorithm by feeding it various pictures of people wearing PPE. Using these examples, the AI then learns how to recognize if someone is wearing PPE.
Going in front of the camera, without any PPE, the algorithm has been programmed to say “head.” Around the person’s face is a box and under it shows the accuracy level. When putting on a hard hat, it says “helmet,” with the given accuracy. At the recent Mines for Limitless Minds event [link], students and visitors could try on a variety of hard hats and other funny hats, like chef’s hats, pirate hats and other options to see if the computer could see through the hat and recognize the person was still not wearing PPE. By feeding the algorithm so many photos, it teaches it to generalize and pay less attention to a miner’s individual features, but rather the goal which is the PPE.
Grad Researcher Carlos Olmos de Aguilera tests Helmet recognition by program.
However, with the photos available, there are several challenges. One is the available photos themselves. Originally, they were “working with a data set lacking diversity, with little variability on race and gender representation. When I created the algorithm on that data set, it didn’t have a good performance, and more important it wasn’t representative of the many faces in today’s mining,” says Olmos de Aguilera. The algorithm needs a representative data set in order to increase its accuracy in the growingly diverse mining industry. The lab is interested in asking students and other people from the mining industry to come in to help create an inclusive dataset which will allow them to create solutions for a more representative industry. The algorithm is currently being expanded to apply to different kinds of hard hats and other PPE, still there is some development to be made, as it is in the early stages.
Mining has a unique set of challenges. There is dust, low visibility with variable light conditions, and more. Computer vision hopes to conquer these challenges. The computer vision system can also be applicable to above ground construction sites. Risso and her team will be taking the current prototype into the San Xavier Mining Laboratory to test. This program will be a foundational element for mine safety. Other developments will be able to build off this program. Mine companies would be able to add and personalize the program, programming to the individual site features and the people who are allowed to enter the site or identifying mobile equipment entering a section of the mine.
Getting involved
Risso encourages current students to reach out and get involved with the lab. There are several projects under development at the automation lab like an app for mine hazards using computer vision being developed by Dr. Angelina Anani, Associate Professor of Mining and Geological Engineering, and Dr. Nathalie Risso, and robotics projects, where autonomous machines are being created for mine sites.
“Students can contact Dr. Risso, Dr. Anani, or me via email to learn more and to coordinate a visit to the Mine Automation and Autonomous System Laboratory. We are currently working on installing a security camera to have a demo ready for visitors to test the project and for data capture. We will be featuring a demonstration at the SME MineXchange 2023 conference,” says Olmos de Aguilera.
Machine learning, automation, and data analytics are growing rapidly in the mining industry. At the University of Arizona, if students can’t get involved in a lab, there are classes being offered on data analytics with mining applications, such as MNE 420/520: Data Analysis and Application Development for Mining Engineers and stay tuned for a new course offering by Professor Nathalie Risso!
Contacts: Nathalie Risso, Angelina Anani, Carlos Olmos de Aguilera