PRECO Electronics PreView Sentry sensor mounted on a hauler. (Photo: PRECO Electronics)
Level 1 Automation Gains Ground
Onboard and centralized sensor-and-computer systems are said to be saving lives,
simplifying jobs and preventing costly disruptions
By Jesse Morton, Technical Writer
Those technologies typically are part of bigger systems that offer more than just obstacle detection and collision avoidance. No doubt, the biggest selling point for all of them is lives saved, followed by damage and disruption reduction and increased efficiency. Current headlines reveal the growing popularity of the technologies and systems, which are ever-evolving into component parts of broader, more comprehensive, equipment-brand-agnostic autonomous mining solutions.
Weaving a Cocoon of Safety
PRECO Electronics announced in February
it received a record number of nominations
for the annual Excellence in Safety
award. The award honors individuals
“for their initiative and commitment to
the advancement of safety.”
Advancing safety is at the core of
the company’s mission and offerings, and
the upswell in nominees points to a market
trend that bodes well for the company. “As
people recognize active warning systems
save lives, integration and implementation
of this technology is becoming commonplace,”
said Tom Loutzenheiser, vice president,
business development, PRECO.
Already, the trend is toward incremental adoption of automation solutions targeting safety, he said. “What seems to be occurring is rather than having mines with no automation or mines that are fully autonomous, mines are emerging more often with Level 1 automation,” Loutzenheiser said. With Level 1 automation solutions, the operator is in control of the rig, but is assisted when sensors detect obstacles or other dangers. At Level 2, when an obstacle is detected, a control system intervenes and takes action, such as evasive steering.
Level 1 capabilities are currently the most widely adopted, Loutzenheiser said. “The most relevant safety feature to the industry is active braking,” he said. “This feature is used mainly for backing.” Similar solutions in this space are often pitched as both machine awareness and situational awareness offerings. PRECO offers several either within or complementing its flagship Automotive Driver Assistance Systems or ADAS.
ADAS systems mainly consist of multiple sensors, which includes cameras, radar and LIDAR; computers; and associated software. “The majority of automotive ADAS systems implemented have sensors on the front and back of the vehicle to offer lane change assist, lane departure warning, rear cross traffic alert, and adaptive cruise control,” Loutzenheiser said. “All of these sensors communicate with one or more Electronic Control Units in order to make the proper decisions.”
Many ADAS systems include active braking capabilities that leverage radar and LIDAR, an onboard processor, control software, and obstacle detection and avoidance software. It offers two modes, auto-braking mitigation and collision avoidance. “The system alerts the driver to an imminent crash, and if the operator does not react in time, the system brakes on its own to avoid the collision,” Loutzenheiser said. “The auto-braking mitigation systems are designed to slow the vehicles to lessen impact and damage, while the avoidance systems attempt to avoid a crash altogether by entirely stopping a moving machine before hitting an object.”
The adaptive speed control capability included with some ADAS systems seeks to enable a machine to maintain a certain speed while keeping a specific distance from the leading vehicle. It is also called autonomous cruise control, adaptive cruise control, or radar cruise control, depending on the technology leveraged. “If the vehicle in front brakes, the adaptive speed control system initiates the braking system, just as the vehicle will speed up within the pre-selected range when the system determines the vehicle in front has increased its rate of movement,” Loutzenheiser said. “Adaptive speed control is almost always paired with forms of automatic braking systems and forward collision warning functions, which work even when adaptive speed control is not engaged.”
One example of a solution designed to be part of an ADAS system, but capable of stand-alone operation, is PreView Sentry. It can be deployed to a shovel to prevent it from dinging haul trucks. Preview Sentry leverages a radar and requires no additional software or hardware. “The system includes sensors, cable, and the operator interface, all configured to work together as a system,” Loutzenheiser said. “They could build a primary awareness system for a shovel using multiple radars and our new display.” The idea is to put “the operator in a near 360° cocoon of safety,” he said.
The radar detects both moving and stationary objects and reports the distance visually, on the display, and audibly, by a buzzer. “The way this works is the sensor transmits and receives low-power 24 GHz radar signals,” Loutzenheiser said. “It then processes the returned signals to determine if an object has reflected any energy to the sensor and reports this to the operator display. The sensor is designed to process and report detections within 240 milliseconds, allowing the operator to respond to any object within the detection zone quickly.”
On a shovel, Sentry alerts the operator if the swing of the shovel will hit an object. “This functionality would not be different on an autonomous hauler because the radar still detects objects and sends a signal to the vehicle from the loader,” Loutzenheiser said. “Integrated with ADAS, the system can prevent, for example, an operator from turning an excavator if there is a risk of collision.” Many drivers quickly adapt to operating rigs equipped with ADAS and other PRECO safety solutions, the company reported, and come to appreciate the added convenience and safety. They see it as protecting them from other drivers’ mistakes, not their own. “Although the ADAS features that take over operation of the vehicle to avoid accidents are still quite controversial, its adaptive speed control, night vision and blind spot monitoring capabilities are more widely accepted,” Loutzenheiser said.
ADAS systems are used in mine sites around the world and have more recently seen an uptick in interest from sites in South Africa, where new legislation mandates automated safety solutions on large equipment. For example, “we have done a lot of ADAS with operational machinery in the Sishen mine located in central South Africa in Kathu, Northern Cape,” Loutzenheiser said. Sishen, with some of the largest iron ore reserves in the country, originally sought an active braking system for Kumba’s haul trucks. It ultimately adopted some PRECO solutions as part of a bigger strategy to improve mine site safety in general.
The move could prove to be part of a trend for which PRECO is well-positioned. “With our many international dealers, such as APS and Trysome in South Africa, PRECO is able to integrate our systems with machines already in different stages of automation and ADAS,” Loutzenheiser said. “We have also partnered with BEML in India where we work our systems into their haul trucks,” he said. “In the mines in India they require collision avoidance systems on haul trucks, so they have a high level of interest in ADAS and automation.”
Such interest is backstopped by the company’s long history in the space. “We estimate they have more than 930 haul trucks set up with our systems,” Loutzenheiser said. “PRECO has been developing this technology for more than 10 years, and has been a part of the safety technology industry for more than 70 years.”
Giving Situational Awareness
Hexagon Mining reported two capabilities
within MineOperate Pro OP help coordinate
the tasks of shovels, trucks and dozers
to increase productivity and reduce
safety risks.
High- and low-precision shovels and
dozers can leverage the onboard software
and that of a centralized server, Hexagon’s
Fleet Management System (FMS), to coordinate
cleanup at the loading site. The
flipside of the coin is a capability that coordinates
trucks and dozers at the dump site.
Contrast that with the more traditional methods. “Cleaning up is a common task and relies on radio communication,” Romero said. “Usually, the radio channels are busy and don’t give enough time for people to coordinate.” The onboard software, MineOperate OP Pro, which runs the cleanup functionality, “is configurable,” Romero said, and it “communicates with FMS, which “facilitates the communication between equipment and keeps records of the communications.” Assignments are queried and accepted using onboard FMS panels. If the loading area needs to be cleaned, either the shovel or the dozer operator can query the other for cleanup task execution. “The operator uses the panel for this request,” Romero said.
Either operator starts by selecting the shovel. “For instance, shovel SH01,” Romero said. “The system creates this relation and provides information on the dozer panel when the next truck will arrive at SH01.” The recipient of the request, which can be either the dozer or the shovel, can either accept or reject the request. “Additional to the workflow, the dozer operator can select which shovel side, left or right, to clean up,” Romero said. With the impending release of new hardware, “the communication will pass directly between equipment without the need to rely on Wi-Fi infrastructure,” Romero said. “It will help the equipment interaction in areas where the mine doesn’t have network wireless infrastructure.”
Coordinating dozer and truck tasks at the dump site is made possible and benefitted by the same systems. “Because the areas can be huge, the truck operators are not always clear where they need to dump the material,” Romero said. “Additionally, as a normal process, the truck dumps the material and a dozer pushes the material. When the operators don’t know or remember where they need to dump, the material is spread out all over the place, which makes dozer work inefficient and unsafe.” The solution offers both guidance to truck operators and the next truck arrival time to the dozer. “Heavy equipment have a lot of blind spots,” Romero said. “Because all equipment positions are on the equipment screens, it gives situational awareness to avoid potential risks.”
MineOperate OP Pro and the FMS provide “all real-time information, equipment positions, ETA and dump directions, which are updated automatically by the system,” Romero said. “If, for any reason, during the shift the trucks need to dump in another place, but in the same dump, the controller needs to update the new dump guidance and the system sends automatically the new information to all equipment.” Assignments are relayed using the mine map. “This map is active and the operators can see FMS equipment locations,” Romero said. The map shows the area where haulers are to dump the material. “The information is the same for both equipment types,” Romero said. “Additionally, on the dozer screen, the dozer operator has what trucks are coming to dump and when they are going to arrive,” he said. “With this real-time information, the dozer operator knows when he can push the material.”
Both the onboard software and FMS are equipment brand agnostic. “Hexagon is committed to offering integrated solutions,” Romero said. “We know there are a lot of isolated onboard solutions that struggle to share data between equipment.” Integrated solutions further two company missions, he said. “What we are doing here makes the process safe and productive,” Romero said. Separately, Hexagon Mining reported some of the research it is conducting in partnership with the University of Arizona into hyperspectral imaging-based solutions could have machine vision and situational awareness applications.
Hyperspectral imaging captures the electromagnetic signatures of targets, such as leach stacks, rock faces or stockpiles. The typical scanner captures a continuous measurement for a range of bands. For example, it generates a reflectance count for the 10-nanometer band, one for the 20-nanometer band, and so on. A hyperspectral scan of a muck pile could give instant information on its grade. Last summer, the research team trialed stationary and airborne scanners at an open pit mine in Utah. The aerial drone-based scanner “worked well,” John Lyons-Baral, senior application engineer, Hexagon Mining, said. “What is great about the UAV is if you combine it with LIDAR, you can create the topography and the geologic mapping at the same time,” Lyons-Baral said. “The workflow on that, however, is a little harder than with the tripod-mounted scanner data.”
Geospatial, Hexagon’s GIS-leveraging open-pit mine planning platform, can assimilate hyperspectral data. In the future, it could process data from scanners mounted on heavy equipment or suspended over haul roads. “These sensors can be processed really quickly on board,” Lyons-Baral said. “Some of these are used in plants already,” he said. “It is actually being run on belts.” It could also be mounted to a shovel, he said. “It can have an alerting system and a routing system put in.” One possible application would be to have “FMS and our autonomous systems use it actively, saying, oops, that is not what you thought it was,” Lyons-Baral said. “There are different places it could be positioned,” he said. “It is going to depend on the mines, the methods, and what makes sense for them, where they need it to measure.” More research willl be done soon.
The system guides haul truck operators without input from the shovel operator. Instead, it uses advanced guidance technologies, high-precision GNSS positioning and proprietary onboard software “to provide centimeter-level accuracy for truck and shovel operators,” Modular reported. It “automatically tracks, collects, captures, stores and utilizes, in real-time, all GPS information on the shovel, including the spotting location,” Weisheit said. “The truck system also collects and utilizes real-time positional information, dynamically providing truck operators with guidance to the optimum load location.”
Two in-cab LED displays provide the information to the operator. When the hauler approaches the shovel, the displays present detailed guidance. The guidance is based on the specs of a “load zone polygon” at the shovel, Weisheit said during a presentation at SME. When the truck breaches the polygon, instructions on maneuvering into the optimal “spot” are generated, he said.
“The shovel operator has to set the spot point on each side when they are setting up their position,” Weisheit said. “The spot will stay with the shovel for a short amount of time in terms of movement, so there is some leeway given to the shovel to move within that space around some predefined characteristics,” he said. “Today, from a safety standpoint, we eliminate a spot automatically if the shovel was to breach either one of those loading zones as it moves, in which case it prompts the operator to reset that spot on either side.” The primary tangible deliverable of the system is exact “continuous guidance and navigation information,” Weisheit said.
The benefits are numerous and “include reducing shovel hang time, eliminating re-spotting of a truck, and facilitating double-side loading,” he said. The system allows the miner to optimize shovel swing angle and load cycle, and to maximize trucking capacity by minimizing total truck wait time. Results include improved compliance to plans along with potentially big gains in productivity, the company reported. In some cases, it can contribute up to eight additional loads per hour. “Dynamically providing truck operators with guidance to the optimum load location without the need for shovel bucket spotting support can increase shovel productivity by up to 34%,” Weisheit said. In deployments this year, the system could be used to guide hauler operators dumping at the crusher site. “Because it is a static, fixed dumping point, it is just a matter of surveying in that dump spot,” he said. “Once the truck breaches that polygon around the dump point, they receive guidance to get to the exact designated spot.” Beyond 2019, the system could be deployed in other applications, Weisheit said. “When we look down the road a little bit further, we’re seeing dozer spotting and getting into some of the autonomous technologies, running a manned and an unmanned fleet together, with paddock dumping and planned dumping, and lane guidance and keeping, with that high precision data being captured on the truck throughout the mine wherever they are at,” he said.
At the time, the partnership was said to “build upon Saab’s technical platforms and on the working methods and experience that Combitech gained in the course of digitalizing the Gripen E fighter aircraft, an effort that cut development time in half while radically reducing costs,” Atlas Copco reported. One of the expected deliveries was a project to develop TMS, which included loaders and trucks, traffic management and safety systems. And the goal was “to create the reliable algorithm in TMS that can be used in real mine conditions and provide the customers with mass mining applications with productive tools.” Last month, Epiroc told E&MJ that TMS is “being developed for mass mining applications, such as block caving,” and “is a pure software algorithm that integrates the data received from each autonomous loader, mine planning system and mine traffic control” system. “With TMS, we are integrating the data, received from the loader,” so each “can talk” to the others and “detect if another autonomous loader is at tipping point, for instance,” the company reported. “Even more, we have reached the integration with mine planning systems” that generate and dispatch assignments, and with mine traffic systems “to avoid dumping into the crusher if it is full.”
TMS is currently being trialed at a customer site, according to Mattias Pettersson, global product line manager automation, Epiroc. “We have other customers that will have this installed during the year as well,” he said. “With each new case, and each new application, it will obviously become a more complete product and a more complete offering.” Such is the evolution of the system, which, Pettersson said, tops a long line of predecessor solutions. “This whole solution started from our capabilities to automatically tram with our materials handling products, our loaders and trucks,” he said. “We have had the capability for a little more than 10 years for a machine to go from Point A to Point B in an autonomous fashion.”
Autonomous tramming leverages a number of sensors and onboard processing technology that enables a rig to know “where it is, relatively, in a mine,” Pettersson said. One of the goals of developing TMS was to prevent the rigs, while tramming in autonomous mode, from colliding, Epiroc reported. “The loaders can now work together in the same area in a coordinated fashion,” the company reported. “That was a main constraint in using the multi-machine fleet in underground previously.”
Trial results reveal TMS solves the problem, Pettersson said. “Knowing each position for each machine, knowing where they are heading, knowing what speed, we have an estimated time of arrival at certain points,” he said. “With that information, we have some logic, and get some optimization that prevents those machines from colliding if they are sharing a task or are sharing a part of the mine.”
Much of the computing is done onboard, so the rig is “not too reliant” on mine communications infrastructure, Pettersson said. “Of course, when we need to coordinate several machines then you need to have some sort of communication in between them and that is handled by a central node,” he said. “All machines communicate through the central node, where they are, where they are heading, what their speeds are and so on, and the central node decides who gets to go and who gets to stop.” The miner decides the rules, such as which machine has priority over the others. “Typically, we take input from an external system, something that gives us the priorities,” Pettersson said. “That usually is some sort of production planning system,” he said. “We try to optimize the output in the clearest possible way.”
One way is by managing traffic. TMS can process data coming in from a constellation of sources to arrive at rig assignments that streamline the flow of an entire fleet. “There might be information about one of the loading points being blocked,” Pettersson said. “It might be at the dumping station, or the crusher might signal it is full. A number of signals impact how we run, what missions we run, where to stop, and so on.”
The net result, Epiroc reported, is possible production optimization. “Normally you load from the system into a crusher or onto a conveyor,” Pettersson said. “Those type processes like to have a steady flow of material,” he said. “If we can have this optimized and really focus on that to make sure you discharge the material at a constant pace, that process will be even more effective.” TMS could enter the market as “one of our big upgrades,” enabling the company to “market it a little bit more widely,” Pettersson said.
Currently, it is being marketed to select clients. It is best suited for a mine running loaders or trucks on repetitive tasks. It is also best suited for a miner willing to partner on a project, adopt the requisite communications infrastructure, and follow the required adoptive process. Pettersson said he expects TMS to deliver “big benefits” for customers. He said he also expects its release will sync with the other headlines coming out of Stockholm. “We are stepping up one level from being just a machine and assistance supplier to being something a little bit more than that,” he said.
The goal is to develop solutions on “open architecture with an open interface so we can actually work with machines” and systems developed by other suppliers, Pettersson said. “It is part of our strategy in everything that we do that we think that the future mine is a good ecosystem of different systems with different organs,” Pettersson said. “We are ready to take a leading role by delivering open systems that are easy to be integrated, but also making sure that we can deliver that while looking at the overall process.”
The package was developed to satisfy a handful of needs. The long-term trend for both miners and contractors is to pursue “ways to keep their assets running reliably, independently and continuously,” Välivaara said. The idea, he said, is to grow productivity by keeping equipment in operation where possible during shift changes, blasting and ventilation times. “On the other hand, the deeper mines are seeing more geotechnical challenges,” Välivaara said. “So, to improve the safety of their workforce, the mining industry is constantly looking for ways to reduce operator exposure to these conditions where possible.”
The capabilities were engineered “to work around these challenges and other known bottlenecks within underground development drilling,” Välivaara said. The package is the result of the evolution of a number of predecessor solutions featuring varying degrees of automation. The major developmental hurdle was to define the algorithms that would guide the boom movements within the collision avoidance system, Välivaara said. “Several inventions were discovered during the development work, prompting one doctoral thesis that was successfully finalized during the project.”
Another hurdle was building an onboard system powerful enough to field the required computing and processing tasks. “The dynamic route calculation algorithms are fairly complex to run and they require quite a bit of power from the system hardware,” Välivaara said. “It’s all about measuring continuously and on a high-speed frequency the readings from the sensors to determine the current location of the boom, rock drill and drill bit, and to utilize the data to make higher-level strategic decisions.” The boom collision-avoidance system is based on a dynamic route recalculation model that, among other things, predicts potential collisions. It then acts to avoid those collisions by re-routing the boom movement from hole to hole. As a last line of defense, if the booms get too close to each other, the movement will be stopped.
“It prevents the drill, when being operated unattended, from shutting down the movements when the boom encounters a possible collision, thereby ensuring that not only is the collision avoided, but also that production is maintained without any interruptions,” Välivaara said. “This is particularly important to keep up the production when the unit is operating unmanned,” he said. “During manual boom control, the system also protects the unit from careless operation by first slowing and then stopping boom movements if they get too close to the carrier or another boom.”
The system leverages the sensors in the boom joints to measure position and calculate the location of the drill bit. “As the location of all of the booms are known, the smart software algorithms can then determine the right positioning strategy when moving them from hole to hole without interrupting each other,” Välivaara said. The system manages the boom movements automatically, he said. “The system only needs the desired sequence, meaning in which order the holes will be drilled by the machine.”
One important result is a net reduction in the amount of manual work in the drilling process. The semiautomated drill bit changer can contribute to that reduction. “As well as enhancing productivity, it reduces the risk of personnel exposure to weak ground conditions, such as at the face itself, and also the need to get in and out of the operator cab, which is one of the main causes for various ankle and knee injuries within the underground mining industry,” Välivaara said. The system allows the task to be executed in a safe location. “It consists of racks of drill bits, six or nine standard bits plus a reaming bit, for both booms,” Välivaara said. “The principle is simply to remove the old bit and then retrieve a new one,” he said. “The system positions the boom automatically next to the rack from where the operator can then manually release the worn bit and retrieve a new one.” The system uses the same sensors as those leveraged by the boom collision- avoidance system, meaning it requires no hardware in addition to what is already built into the drills.
The main benefit provided is increased operator safety, Välivaara said. “Not having to get out from the cabin to the front of the drill is a great improvement from both safety and ergonomics point of view.” Safety is also one of the main benefits offered by the teleremote-control capability, which Välivaara described as the “key part” of the package. It “includes a drilling control panel that enables the drill to be operated remotely, for example, from the surface of the mine,” he said. “This allows the operator to help the unit drill more holes during shift change, blasting and ventilations times, while reducing the risk of equipment damage.”
Onboard componentry consists of cameras, a safety-rated laser, scanners for area isolation, wireless antenna, and a control logic. Off-board components include the remote-control station, viewing screen for cameras located on the unit, and some auxiliary controls for both lights and cameras. Adoption requires a Wi-Fi network with a minimum of 10 megabits-per-second bandwidth in the mine, Välivaara said. “The package is light and compact, allowing the control station to be easily moved from one place to another,” he said. The system allows remote supervision of the drilling automation, and, where needed, operator assistance in drilling holes in challenging locations. “Also, the drill bit change can be done by remote by utilizing the teleremote drilling controls combined with the semiautomatic drill bit changer,” Välivaara said.
The ideal customer for the package is a miner with robust experience operating underground development drills that is interested in systematic automation, he said. Nonetheless, “anyone interested in reducing operator exposure to the headings, and particularly in the front area of the drill, would be able to see the benefits of what this package can deliver,” Välivaara said.
The package is the first of its kind within the industry, he said. “Sandvik is really setting the new standard,” Välivaara said. “The digitalization strategy for Sandvik Mining and Rock Technology is based on three cornerstones: autonomous equipment, connected fleet and data-driven productivity,” he said. “The automation upgrade package will strongly enhance our offering in the area of autonomous equipment.”