Leverage machine learning to improve site security

A project funded by Innovate UK aims to improve site safety by using machine learning technologies to identify hazards in real time and send alerts.

In the [construction] industry, we think one death is too many,” says Olugbenga Akinade, associate professor at the University of the West England (UWE). He adds that accidents on site also lead to a loss of working time and therefore of income for subcontractors.

These are the reasons for the industry’s drive to improve workplace health and safety over the past few decades. In addition to adhering to stricter regulations and producing detailed corporate guidelines, companies in the industry have invested in various types of innovations designed to support their efforts.

It helps you determine if there is an imminent risk or if all is well

UWE’s Big Data Enterprise & Artificial Intelligence Laboratory (Big-Deal) investigated the use of technology by the construction industry for health and safety purposes and contacted prime contractor Winvic to obtain an overview of the market on the subject.

They found that existing vision-based approaches to construction site monitoring focus only on areas such as site safety and the use of time-lapse movies to show progress.

There was little use of this technology to improve security.

Additionally, health and safety managers and officers have traditionally relied on self-declaration or warnings from co-workers to indicate safety risks.

Anticipate accidents

“The challenge with this approach is that you can’t anticipate an injury or a safety issue,” says Akinade, who says what’s needed is a proactive approach, not a reactive one.

To develop a product with such an approach, Big-Deal collaborated with Winvic and Bristol-based smart video solutions provider One Big Circle.

They developed the Autonomous and Intelligent Video Response Lookout (AIVR), initially called Computer Vision Smart, for which they won a £600,000 smart grant from UK innovation agency Innovate UK.

AIVR Lookout leverages real-time imaging and machine learning technologies and is already undergoing on-site trials (see box). It extracts information from camera systems on construction sites and, using machine learning, identifies hazards – from mobile heavy machinery to overhead work to people working without the proper personal protective equipment. The system maps the zones corresponding to each type of danger and codes them by color according to the level of risk.

We are now looking at mobile cameras that look at a smaller area of ​​the site and potentially require less calibration

When a hazard has been identified, push notifications – with an image and information about the hazard – are sent to the site team to enable a rapid response to behavior or activity that could cause an accident.

AIVR Lookout will have two interfaces where push notifications will be delivered, one for computers and one for mobile devices.

Sam Low, product manager at One Big Circle, says the innovation “acts like a guardian angel for the site team” as it recovers different types of equipment and building materials, as well as people.

The system’s ability to identify objects and people on site solely through the images provided by the camera, makes this approach less intrusive compared to installing sensors on all construction equipment, says Akinade.

He explains that what makes AIVR Lookout innovative is that it goes beyond tracking and computing, using an “ontology-based approach”. Ontology is the establishment of a network of relations – in this case by representing the relationships between different objects and people on the spot.

“It helps you determine if there’s an imminent risk or if everything is okay,” says Akinade.

Lessons learned

A big challenge for the development of AIVR Lookout is calibration, due to the constant changes that occur on construction sites.

Akinade explains, “How do you allow the model to understand the heights of objects and the distances between objects? Usually we use a reference point, but because the layout changes frequently […] We have to imagine another way to calibrate the model. He adds that changes to venues mean the location of cameras must change.

Depending on the stage of construction, there are very few straight lines and there may be a hole or a hill. This means that it is difficult to make a simple assumption as one would when calibrating a camera or converting 2D to 3D for a flat surface.

For these reasons, a camera system on an easily movable structure on site is considered a better solution than the static camera proposed in the initial concept.

Mobile cameras

“We are now looking at mobile cameras that look at a smaller area of ​​the site and potentially require less calibration. So we’re actually changing the way we do some of that alarm detection, so that it doesn’t require any calibration,” says Low.

Mobile camera equipment was delivered to the test site (see box) in early January, the camera will be mounted on a 6m high CCTV mast with a wheelbase. There is a common belief among project teams that the ability to use deployable cameras for AIVR Lookout will have a significant effect on the product’s potential to reach market.

Morgan Hambling, head of digital engineering at Winvic Construction, says, “The key to getting it to market is having something that’s easy to set up.”

The Innovate UK Fellowship, which covers two years of research, ends in May. Until then, the parties want to demonstrate that they have a functional product that brings value to users.

Development and testing

The development of AIVR Lookout requires a high degree of collaboration between Winvic, One Big Circle and Big-Deal.

Winvic provides field expertise and software and hardware test sites.

One Big Circle installs the multi-sensor cameras, as well as powerful servers on site. It also provides graphics cards and graphics processing units to run deep learning models – files trained using labeled data to recognize patterns – for object detection and recognition.

The videos captured by the camera system are sent to Big-Deal which carries out the labeling using frames – images in the videos.

Akinade explains the labeling process: “You have to draw bounding boxes around certain objects and assign them labels – ‘This is a construction worker, this is a bulldozer’. They also identify relationships and introduce safety contexts based on observations.

The tagged video is then used to train the machine learning model to understand these images in terms of the objects being represented and their relationships. Once this process is complete, One Big Circle integrates the model into the system.

AIVR Lookout testing took place at Mercia Park, a job park in North West Leicestershire, where Winvic has been involved in the construction of different facilities. CCTV cameras are mounted on tall towers and record activity on site.

Initially, they were recording the construction of a facility for the logistics company DSV. Since the completion of this project, data has been collected from the construction of 270,000 m2 global parts logistics center for IM Properties.

Big-Deal has already tagged 30,000 frames. He developed the machine-learning model for two stages of the construction process – excavating the land and pouring the foundation concrete – and sent it to One Big Circle for level-of-level testing and feedback. accuracy and ways to improve the predictive ability of the model.

Every day, data is collected by the system, but it is not yet at the stage where it can send alerts.

Do you like what you read? To receive daily and weekly New Civil Engineer newsletters, click here.

Previous UChicago researchers set a record by preserving quantum states for more than five seconds
Next “This is how they see the world”: Mille Lacs launches Rosetta Stone program to teach Ojibwe