The largest impact of drones over the past three to five years has been in GPS-denied environments,
particularly underground. (Photo: Richard Bishop/Exyn Technologies)
Mine Mapping Goes Mobile
We explore the future of mine mapping through drones, robots and autonomous
mining equipment
By Carly Leonida, European Editor
3GSM is dedicated to the detailed analysis of rock and terrain using 3D images or models generated using drone-collected data, explained Markus Pötsch, customer support and sales manager. “The use of drones has led to the collection of more comprehensive data set,” he said. “Data from highwalls and bench faces can now be gathered at the same time as from the floor and top of benches. Drones have the ability to change their flight path, altitude and camera angles easily, which is difficult or even impossible with pure terrestrial imaging.”
The 3GSM team has noticed that mapping data gathered by drones usually shows a higher level of detail compared to data gathered terrestrially. The coverage is more complete, and data sets have a high level of redundancy, which is important for 3D model generation.
“The amount of data collected by drones for mining projects can be overwhelming,” Pötsch said. “It’s important that software products can handle this without compromising performance and accuracy. Mining operations, particularly those in remote areas, may suffer from suboptimal network connections, so it’s also important to have software running on local computers as well as in the cloud. The software should be able to process the data and display the results on a standard computer and it should be easy to learn and easy to use in order to provide real value to people and operations.”
“Another advantage is that drones can be deployed during normal operations (except for blasting) without having to stop any mining activities,” said Delta Drone International CEO Christopher Clark. “Terrestrial scanners require multiple setups, yield ing digital terrain model data sets with large gaps making them inferior to drone-captured data. These are then not suitable for producing complete pit models on a daily basis.” With GPS or total station surveys, mining operations need to stop while the surveyor gathers data for safety reasons. This is undesirable and can result in stoppages, which means loss of production.
LiDAR Above and Below
Ground
Drone-mounted light detection and ranging
(LiDAR) technology has evolved significantly over the past five years, and
COVID-19 further accelerated its adoption
due to the fact that manned aircraft
could not easily traverse borders for nearly
two years. Above ground, drone-mounted
LiDAR scanners can be used to quickly
capture entire and accurate digital twins
of stockpiles and infrastructure for volume
calculations, condition monitoring, change
detection, reverse engineering, and space
management, among other things.
“The current generation of LiDAR sensors are significantly lighter (around 750 g) and better integrated, which allows us to operate fixed-wing vertical take-off and landing (VTOL) drones. These models can cover over 350 ha in a single 90-minute flight.” The primary trade-off between drone and manned LiDAR is time. Areas that may take a week to capture on a dronebased LiDAR solution, manned LiDAR can achieve in a single day. However, COVID restrictions have had an impact on manned aircraft availability with lead times in Africa extending into months. Drone-based LiDAR is significantly more cost-effective for smaller projects under 10 km2 daily, which caters to customers that are currently not being serviced by manned LiDAR providers.
While surface mining operations have seen many benefits, the largest impact of drones over the past three to five years has been in GPS-denied environments, particularly underground. Drones provide a safe and efficient method to gather excavation data, including where rock breakage has been achieved through gravity, or where the excavation is so large that the hanging wall is out of reach of conventional roof support methods and is left unsupported. They can also access failed headings, areas that have experienced a fall-of-ground, regularly blasted stopes and cave openings.
“Providing safe access for people into these excavations to measure dimensions or to gain information on draw points is not always possible,” Clark said. “However, an autonomous drone equipped with LiDAR technology and a camera can enter excavations without endangering personnel as the drone operator resides in a safe position underground.” As drone technology becomes widely accepted, new technologies are continually coming to the fore. With each advancement comes better artificial intelligence (AI) capabilities, more stable flights, and smaller airframes that have integrated or modular cameras.
Carlos Fraga, director of research and development, added: “Because drones are customizable to the needs of an organization through the use of specific payloads, the accuracy of data collection can be enhanced. Some drones can support up to three different payloads at a time, capturing different types of data simultaneously. “UAVs that are equipped with the right tools and payloads will be able to ‘see’ and capture data much faster than a team performing the same activities on foot.” Data captured from stopes can be used to assess overbreak and the underlying geological structures, calculate instope volumes and capture the final void shape. The data provides insights that allow mine operators to maximize orebody extraction and improve material flow.
Cavity monitoring systems — laser scanners placed at the end of a long pole — are traditionally used to measure underground cavities after blasting. To achieve this, operators need to approach the brow and insert the pole into the stope. RaffiJabrayan, vice president of business development and commercial sales at Exyn Technologies, explained: “Aside from being dangerous, the scan is captured from a stationary point and can often miss sections of the underground cavity. These areas are known as ‘shadows.’ “Using a system like the ExynAero underground, survey teams can stand safely away from the brow and task the robot with a mission to explore the interior of the cavity and safely return home with a complete map.” Dr. Farid Kendoul, CTO and cofounder of Emesent, said, “Drone-mounted rapid scanning methods enable data collection to occur at regular intervals, improving recognition of convergence trends and closure rates and enabling full autonomy of convergence monitoring.”
SLAM uses range and sensor data to estimate the motion of the range sensor itself. Once this motion has been estimated, the range data can be projected into a common coordinate frame to produce a 3D map. The estimated motion can also be used to control the robotic platform carrying the sensors. In the case of Emesent’s Hovermap system, LiDAR and internal measuring unit data are used to create maps and control the drone’s flight.
Handling Big Data
Jamie van Schoor, CEO and founder of
Dwyka Mining Services, added: “We view
drones as a platform to provide repeatable
remote sensing and visualization capabilities
across mine sites. The blended sensors
that are attached to drones are extending
the information we used to see based solely
on photogrammetry into LiDAR data.”
One of the challenges that van Schoor and many of his peers mentioned is the processing of data post-survey. “Edge processing means that companies are now able to access their scan data much more quickly,” he said. “But a big challenge that many mines face is how to manage the growing data sets with existing ICT infrastructure, because the two aren’t necessarily keeping pace. Cloud processing and storage for very large point cloud data sets is the obvious answer, but it does bring other challenges around ensuring connectivity and security.”
RAMJACK Technology Solutions acquired underground robotics specialist Inkonova, developer of the Batonoumous drone platform, earlier this year. The Inkonova system uses edge processing on the drone platform to allow surveyors to view the scan data in real-time while underground, which helps ensure complete coverage of the survey, reducing the need to revisit the area. Rupert Birch, Inkonova’s managing director, said, “Post-processing is also important, because if you can’t take the data and easily manipulate it into an existing mine planning package then that leaves the process open to error and delay.”
The data set sizes generated by dronebased surveys are both an opportunity and a challenge. While these systems offer more complete coverage and greater detail than manual surveys captured through a plethora of data points, the data set size can significantly lengthen processing times and storage requirements.
Ben Vagg, global product manager for surveying at Datamine, said, “With a standard type of survey instrument, you might collect tens of thousands of data points. Whereas with drone-based instruments, we’re talking tens of millions, if not hundreds of millions of data points. So, there is a discussion to be had around how much data mines realistically need. Are they getting any more benefit from an analysis point of view, versus how long it takes to process that data?”
Cloud-based processing and decimation can be of help but again, they depend upon the mine’s connectivity as well as upload and download speeds. There is an opportunity for mines to look at smarter ways of handling very large datasets. “In open-pit applications where there’s more photogrammetry used rather than LiDAR data capture, the turnaround time is significantly longer, but it does generate a visual representation of the mine,” Vagg said. “A mixture of photogrammetry and LiDAR capture for surface operations seems to be the most appropriate way to go today. Underground, doesn’t require as much colorization, so LiDAR is adequate.”
The Value of Autonomy
Different drones have different autonomous
capabilities; some have none at all,
while others offer a basic return-to-home
feature in which the drone automatically
returns to the point of origin if it loses
connection with the remote controller.
“Obstacle sensing is another popular feature,” said Drone Nerds’ Fraga. “During a flight, the pilot may have limited control over the drone when flying through areas where a visual line of sight is limited. With obstacle avoidance, a drone is set to fly an autonomous mission and can recognize potential obstacles, thus adjusting speed, altitude or direction independently.”
Some autonomous robots or drones can be given high-level objectives, oftentimes called waypoints, from a human operator to plan a mission. In a mining environment this usually consists of “takeoff and fly 40 meters forward into an open cavity” then “fly up 60 meters” to map the rest of the cavity before finally returning home. Autonomy level 4A, as set out by Exyn in its 2021 whitepaper “Defining Levels of Aerial Autonomy,” simplifies this by allowing the operator to define an area of interest for the robot to explore. For example, a survey team can draw a 3D box around a cavity of unknown size and the robot will choose points of interest itself to capture a complete map of its environment. This will enable use cases such as search and rescue, rapid infrastructure inspection, and fast digital twin creation for construction environments.
“We are currently integrating our autonomous technology onto a Spot robot from Boston Dynamics,” Jabrayan said. “We’ve also recently created a proof of concept with Sandvik to demonstrate what a mine of the future could look like without people underground. The idea is that Sandvik’s autonomous loader can be programmed with an autonomous mission or driven by an operator on the surface who tasks the robot with a stope or cavity that they want to be explored and mapped. After the loader reaches the mouth of the stope or cavity, the ExynAero will take off to explore it, returning back to the loader so both machines can continue onto the next mission. In the future, a fleet of robots would work underground with human operators sending missions and tracking progress from the surface.”
Emesent’s Hovermap system has an Autonomous Waypoints mode which enables it to autonomously fly beyond line of sight and communication range, even in GPS-denied environments using waypoints provided by the operator on a tablet and the Guided Exploration feature. Hovermap takes care of the rest, navigating to the waypoints, mapping the area, and keeping itself and the drone safe from obstacles as small as 2 mm.
Dr. Kendoul explained: “Hovermap also has a Pilot Assist mode that provides omnidirectional collision avoidance, line of sight GPS-denied flight capability, and stability control. Basically, it acts as a safety bubble for the drone. “We’re also commercializing the extended autonomous capabilities of Hovermap that were developed as part of our participation in the DARPA Subterranean Challenge. This includes Hovermap-enabled drones autonomously exploring beyond line of sight and standard communication range, defining their own missions, and navigating into unknown underground environments with the objective of exploring and searching the environment.”
What Will the Future
Look Like?
Today, we are just scratching the surface
of how drones and robotics can be deployed
in mining.
“There is a growing interest in more
integrated and autonomous systems and
operations,” Dr. Kendoul said. “Ultimately,
the application of robotics, machine
learning and AI has the potential to enable
full workflow automation of a mine,
joining up multiple autonomous processes
so that the mine itself becomes the
autonomous system, with key decisions
made by an integrated AI layer optimizing
the entire process.
“Multi-agent, fully autonomous drones and robots could form a pivotal part of the core infrastructure, acting as mobile sensors and constantly collecting and live streaming 4D data to a data management platform on the surface, with real-time automated decision making and actions enabled by machine learning and AI.
“Digital twins would run thousands of ‘what if’ simulations per second analyzing and validating data from multiple sources to identify insights, anomalies and optimizations. Those outputs would feedback 24/7 to autonomous robots that set to work to make the changes in the real world, with the whole cycle continuing in perpetual motion.”
Achieving complete workflow automation is a challenge, and a vital step in this journey is the ability to integrate data captured from every process, unmanned vehicle and data collection platform in the mine onto a single analytical platform.
Emesent’s Hovermap has been designed as a versatile plug and play payload that can be attached to multiple mobile data collection devices such as drones, vehicles and robots, and it can also be used as a static sensor. This enables the same framework to be used across any device, ultimately fusing data from different sources and applications and creating a unified, standard framework for autonomy and sensing.
Dr. Kendoul added: “We are currently ling a world-first solution using a network of mobile and static Hovermap systems to monitor an underground mining environment with the objective of ultimately enabling the remote operations of a complex system.”