Mission OS | construction management solution

1. From Overload to Insight Artificial intelligence offers exciting opportunities to improve the efficiency and safety of many aspects of tunnelling. Here’s TJ’s peak into a few early ideas. For a few years now, we’ve been complaining about ‘information overload’ in tunnelling. All this data spewing from machines, surveying equipment and other sources, that could offer so much more insight if it were to be mined and analysed. Data management systems with their summarising dashboards and the ability to overlay information from different sources were the first step. Now various organisations are looking at deploying artificial intelligence (AI) to a wide range of applications across different types of tunnel. Here are a handful of examples, most of them in the early stages of development. These range from using machine learning to help spot defects in surveys to grouting control to predicting the ground ahead of the tunnel face. These new tools can arm human operators with more information ahead of time and, in some cases, can react quicker and automatically take over the controls. Perhaps the most interesting idea is Dynamic Infrastructure’s review of existing photos to spot defects, because it comes from outside the immediate tunnelling industry. It is a reminder that disruptive innovation could has the potential to cause the biggest leaps forward. One recurring theme from all the companies we spoke to was that the old adage of ‘garbage in, garbage’ out very much applies to machine learning and AI. The better the quality of information, the faster the machine will learn and the more accurate the end result will be. 10 Tunnelling Journal 10 Tunnelling Journala

2. FROM OVERLOAD TO INSIGHT Speedier survey reviews boost productivity by 60 percent surveys,” says Prokopova. Recent developments in image processing made the use of images – both photos and laser scans – the best solution. “Earlier there were attempts to use image processing to automate, but the results were never good enough,” says Prokopova. “Now the technology has reached a level where we can use artificial intelligence more easily.” In selecting the right network to learn how to spot defects, Amberg looked at a number of different options. “We needed something that could work with less data,” says Prokopova. Amberg was advised on AI by LeanBi, a technology company that advises on AI applications. The software only takes a few hours to train and needs a few hundred metres of tunnel. Amberg has used historic survey images from a variety of different tunnels so that users of the software didn’t have to train it themselves. One challenge for Amberg was the switch to working in the cloud. This is challenging for some of its customers too, says Prokopova, because they are generally large government departments or construction or engineering companies who need to be confident that their data is secure. A late hurdle in getting the new software to market was how to work with a web-based system inside a tunnel which has no internet. Amberg developed an offline version which can be used on a tablet and which then synchronises with the cloud once connected to the internet again. In theory the data could be transferred into BIM software and models. The constraining factor is that there is not yet a standard format which allows transfer between programmes. “Amberg Inspection Cloud can also be used to compare consecutive surveys and identify whether the condition of the tunnel lining is deteriorating,” says Prokopova. Eventually tunnel inspections may not have to be conducted on a regular basis – say every four years – but can be driven by what the state of the tunnel is. software to process the inspection of a 7km segmentally-lined road tunnel. The software was able to spot cracks and wet spots – which could then be examined by an engineer to determine if any action was required. “The brain work is still done by people,” says Prokopova. “The data shows only the tunnel surface, not the tunnel history or the geology around. There are many factors that have to be considered, so making those decisions cannot be automated.” At the beginning of the development period, Amberg considered using laser scan data points, but this would have required much more processing power. “And it also would have excluded customers who prefer to use photogrammetry for their Last year, Amberg Engineering launched AI-driven software which automatically identifies problem areas from tunnel inspection images. According to Amberg, using the new software can increase productivity by 60 percent because it removes the need for engineers to pore over hundreds of inspection photos. “Amberg Inspection Cloud works by bringing the data to the web and using AI to speed up the data processing, so the user does not have to go through all the data for the whole tunnel,” says Alzbeta Prokopova, application and support engineer, tunnel, at Amberg Technologies. “We want to deliver the data to the tunnel owner in an intuitive form so that he can maintain the tunnel.” Amberg has already used the Amberg’s software uses AI to spot anomalies in tunnel survey images and to make comparisons over time. Tunnelling Journal11

3. FROM OVERLOAD TO INSIGHT Feast of data to feed TBM’s ‘brain’ The team is also working on a predictive model that lets the operator know if settlement above the allowed amount is expected so that they can make adjustments to the face pressure or backfill grouting to minimise final settlements. “We want to be able to notify the operator well in advance of any potential exceedance,” says Grasmick. One barrier that the research project has run up against is availability of data from past projects in the region or in similar geology. Because data from previous projects is privately owned, it’s often not possible to gain access to that data for training the machine learning. “The UK is pushing towards open source borehole data which is really exciting,” says Grasmick. “Currently we are forced to do machine learning on projects that are 50% or close to 100% done. The question is how useful will that be for the next project.” “Really it comes down to data; the quality of it, the consistency and the format of the data. You need all the data to be entered digitally and in a common format and language set at the project or even business level,” says Grasmick. “We have been working to digitalise shift reports so we can put time stamps and georeferenced reports to shift activities that could be tied into the machine learning.” “Having so much data in the MissionOS platform gives us a great opportunity to widen the dataset available to the AI,” says Gamuda tunnelling lead Justin Chin. “We are also keen to use statistical methods to evaluate ground characteristics and relate this to machine performance, particularly for predicting future tunnelling performance” Although the research project only began in September 2019, the team can already report some success. “We have been able to optimise the TBM operation parameters to achieve maximum advance rates for different geologies,” says Mooney. Data management specialist Maxwell GeoSystems, working with contractor Gamuda and the Colorado School of Mines, is investigating how to use AI to improve tunnel process control. “We are working to assist Gamuda to come up with more intelligent ways of tunnelling, not only to speed up current projects but more importantly to help future projects,” says Jacob Grasmick, a geotechnical engineer at Maxwell GeoSystems. “It’s an exciting collaboration.” Gamuda has already developed its Autonomous TBM (A-TBM), which uses embedded logic to analyse data from the TBM in real time and make changes to operations including steering the machine, support pressures and screw speed. The A-TBM system has already been deployed on 10 of the 12 Herreknecht TBMs that are mining Line 2 of the Klang Valley Mass Rapid Transport project in Kuala Lumpur for MMC-Gamuda JV and won an innovation award at the last International Tunnelling and Underground Space Awards. Colorado School of Mines Center for Underground researchers, led by Professor Mike Mooney, have developed AI algorithms to characterize the ground that the TBM is mining through using the TBM operating data. They have also developed algorithms to show which operating parameters influence advance rate and how this varies across geology types. The goal is to use a wider variety of information sources which the AI will then process and feed into the TBM’s AI ‘brain’ to make it even more efficient. To date, Grasmick and his colleagues have developed an application programming interface (API) which can retrieve conventional data such as feeds from the TBM, monitoring instruments and boreholes. More challenging is working out how to incorporate data such as shift reports and associated construction records. Tunnelling Journal13

4. Comparing old photos could revolutionise tunnel maintenance Here’s a neat idea: gather up all the images taken of your tunnel over the years and use AI to look for signs of deterioration and map them. This is the basis of a new business called Dynamic Infrastructure, set up in 2018 by Saar Dickman, an entrepreneur and technology specialist, and Amichay Cohen, who has headed up major toll roads and concessions. They realised there was a technology void when it came to the maintenance of bridges and tunnels. “Everything is concentrated on the construction period, which is very short and expensive, but you don’t see too many tools supporting the 75-year lifetime,” says Dickman. Dickman and Cohen realised that for every tunnel and bridge there are thousands of photos, taken for various reasons, spread around different people and reports. They created a tool that can look at all the photos, find ones that match up in terms of location and create a story of what is happening to that patch of tunnel wall, or bridge girder. Three years ago, they started working on an algorithm that could work with any type of image, including photos from phones, cameras, drones or Lidar surveys and somehow align those images to be ‘looking’ at the same spot from the same direction. “If you can spot defects or deterioration ahead of time, you can invest now to save a lot more money in the future,” says Dickman. The level of detail that the machine learning can reach does depend on the quality of the images, says Dickman: “If you provide us with good photos, we will provide you with good results.” But the biggest challenge in getting their idea up and running have not been technical. “The key challenge is proving to a very professional partner that computers can support them in finding the right details. It’s something cultural,” says Dickman. “It takes some mind shifting to AI looks through historic photos and flags up abnormalities for the maintenance manager’s attention “The ‘per asset’ subscription allows flexibility and unlimited use per asset.” For some clients, Dynamic Infrastructure is using the images to create a 3D model of the structure, which allows virtual inspections so that consultants or specialist contractors can view issues and propose solutions. This isn’t detailed enough to be called a comprehensive digital twin, says Dickman. Having talked to infrastructure managers, they realised that an accurate working 3D model to aid communication and locating the defects was all that was needed. Maintenance managers start to understand the usefulness of the idea as soon as they see all the images merged and in one place, says Dickman. Early users are already finding ways to benefit from the service. For instance, one tunnel owner had a problem with bricks falling from the lining. Local contractors proposed an expensive solution, but by showing the photo story of the deterioration to specialists in other parts of the world, a more cost-effective fix was found. In the longer term, Dynamic Infrastructure’s software will be able to inform the operation and maintenance managers of future tunnels and bridges faults. “To get the advantage of AI, you want to train it so that it’s getting smarter and smarter. The aggregated knowledge of the machine will be far higher than that of any single researcher or owner,” says Dickman. “That’s a meaningful industry disruption.” understand that computers can scan thousands and thousands of defects.” The software has been trained to look for differences that indicate deterioration or problems, and alert the user, but not to suggest what could be done to tackle them. “Our job is to look for meaningful abnormalities, but we never decide whether something is a large maintenance problem or a minor one,” says Dickman. “It’s up to the owner or maintenance manager to decide what to do.” Dynamic Infrastructure is already being deployed on structures in the US, Germany, Switzerland, Greece and Israel, for both public and private sector clients. Mostly these organisations are trialling the system – which can be accessed anywhere – for one bridge or tunnel. “In some places we are moving from an initial pilot project to a bigger adoption,” says Dickman. “We start with one or two bridges or tunnels and then we can fine tune the rules and alerts and then expand to a larger project.” Each piece of infrastructure has its own ‘medical records’ which can be accessed from the Cloud anywhere through a mobile device. The price model involves an annual payment, scaled depending on the number of structures involved, which allows access to the medical records for as many users as the owner requires. “We have set the pricing to be affordable for any operator, be it a small Private-Public Partnership or a large Department of Transportion,” says Dickman. 14 Tunnelling Journal

5. Predicting shear zones increases production and safety lower accuracy. “We took the model that we had trained for our Peruvian project and used it in a tunnel in Chile and got over 65% accuracy. That meant that we needed smaller data sets to train it for new tunnels, say 50m of tunnel. The patterns the model detects are really similar from one tunnel to another.” From a technical perspective, one of the biggest challenges was getting the data into a compatible format for the machine learning. “We had to move the data sets to a new format – software engineers like zeros and ones,” says Merello. Otherwise, the challenges were cultural. The first was switching the geologists from paper to tablet records. The second was convincing clients that there would be any benefit from using AI. “This was something really new that they didn’t think they needed because face mapping with pictures and paper was doing a perfect job,” says Merello. “But once we started to show them that we could predict the shear zones in advance, they began to rely on this model.” With this initial success under his belt, Merello wants to go further with AI. He thinks it may be possible to predict the rock mass ahead without using probe holes, just information from the face mapping and the behaviour of the tunnel. The other thing he would like to develop is a way of predicting water inflow into the tunnel, which would be a huge step forward. Skava is committed to the ongoing developments. Merello now has two colleagues who are working on the development projects with him, and they hope to secure more funding from the Government. engineer from a sister company. Merello himself spent more and more time on the project, working on it full time latterly. Skava put the idea to the test for the first time on a 7.7km river diversion tunnel in Peru. Skava was employed on that project to provide face mapping records for every advance. The face mapping and probe hole information was used to train the AI model. “You train the model with existing data sets,” explains Merello. “The model detects patterns and then matches those patterns against ones it has already seen to predict what is coming up. It learns with every advance and every bit of information you provide.” The model was trained with 1692 data sets, with a further 423 data sets used to test its accuracy. The number of records required for the training depends on the variation in the data, says Merello. For homogeneous rock, you might require fewer data sets than if the ground is heterogeneous. For a drill-and- blast tunnel in rock, you might need 500 to 700m of tunnel or 200 to 300 data sets, he says. One surprising discovery for Merello and his colleagues was that the model, having been trained in one type of ground, could predict ground in quite different ground, although with Skava Consulting gives each of its employees two hours a week to look into new ideas. Project manager Juan Pablo Merello used his time to investigate how AI could make better use of probe hole data. Four years on, the company has developed a machine learning tool that can predict the rock ahead for a drill and blast tunnel to an accuracy of 85 percent. One of the biggest benefits, says Merello, is that overbreaks or collapses can be avoided because the tool – provided through an app on a tablet PC – flags up when there are shear zones ahead. “The shift manager really appreciated the forecast because he knew immediately what his shift would look like in a couple of hours’ in terms of rock quality,” says Merello. “That meant that short term logistics were improved. He knew if he had to install lattice girders, for example, or if the rock quality was going to get worse, he needed to call for the shotcrete to be ready.” Skava’s innovation programme, which is supported by funding from the Chilean Government, sees initial ideas for innovations assessed by a panel who decide whether it is worth devoting more resource to. Merello’s idea got the green light and the panel linked him up with a software Q Index Forecast 16 Tunnelling Journal 16 Tunnelling Journal

6. FROM OVERLOAD TO INSIGHT Patterns lead the way to more efficient grouting compare the analyses towards their geotechnical meaning. “In three to six months I think we can come up with a box which drives our grout pumps,” he says. “I think we can hit that schedule.” When Wannenmacher talked to operators about his idea, they were initially worried. They said ‘you are taking away our jobs’. Wannenmacher is convinced that the expertise of operators is still needed, but system will support and ease their work. There is an additional element on the horizon. Implenia has started combining polyurethane and cement-based grouting on a few trial projects. “At the moment, the grout pump operator decides when to add the polyurethane and how much. Hopefully, we can train the computer to do that.” Finding the right proportion of polyurethane – which varies between 0 and 40 per cent – has been a matter of trial and error so far. Once Implenia and eguana have optimised its AI control for the grout pumps, they may decide to market its ‘black box’ to other contractors. “We all have the same problem that we need to solve,” says Wannenmacher, “everybody needs a helping hand, we may provide it as a service.” eguana is already using AI in some other areas, for instance, to produce shift reports for tunnelling and grouting operations. E.g. data from the pump goes to the cloud, with the computer learning things such as how long a packer change takes and automatically slotting them in. “The engineers love it, among other things, because it saves them time otherwise spent producing reports at the end of a shift when everybody wants to go home,” says Wannenmacher. to oversee fracturing the rock.” To prevent this from happening, Wannenmacher and data management specialist eguana, are using AI to recognise pressure and flow patterns. “The biggest challenge has been how to analyse and process the data efficiently,” says Philipp Maroschek, CEO at eguana. “For AI, one of the most important things is to understand the structure of the data, how to process them and later to combine with geotechnical processes.” Their common goal is to link the AI to the controller of the pump to either turn it off or adjust flow. The system will spot that a fracture is about to happen a few seconds ahead of time. “I don’t think that many operators can recognise that within such a short period,” says Wannenmacher. To develop the system, Wannenmacher collected grouting data from various construction sites and analysed the grouting curves. “We analysed hundreds of thousands of pressure-volume curves to find similar patterns and work out what was normal and what wasn’t normal,” he says. The data sets are now used to train the machine learning algorithm and Pre-excavation grouting in rock tunnels can be a delicate business and requires people with experience to operate the grouting equipment. Pumping above a tolerable pressure can lead to fracturing (creating new cracks) or jacking (widening existing cracks) which results in uneconomical grouting with losses of time and material. “There are always time pressures related to grouting works, as there are for all tunnelling works,” says Helmut Wannenmacher, a geotechnical expert at Implenia who has around 20 years’ experience in rock tunnels, mainly for hydropower. “We expect our pump drivers to operate and oversee four pumps at the same time. From time to time, it’s very challenging for them to fulfil all tasks according to the design. “The overall grouting process is based on the observational method, implying in case of deviations of prediction, modifications of design”, says Wannenmacher. The actual process of fracturing of rock is rather short, so less-experienced workers keep pushing, regardless of whether it is needed and tend Typical grouting for widening of joint Typical grouting patterns for jacking/fracturing operation Tunnelling Journal17