Operating a large crane requires focusing on several different things simultaneously. To ensure smooth and safe navigation, the crane’s ability to read its environment, especially its own motion in relation to other moving things, is crucially important to help avoid collisions with obstacles, let alone people.
Konecranes has been collaborating with the University of Helsinki since 2020 to apply computer vision and deep learning to crane navigation, and the company has recently made a new donation to fund the university’s research on developing better computer vision methods. But how does this help with crane operations in practice?
Automation-assisted crane navigation needs to provide efficient and safe motion while addressing the increasing demand for data and computing power that advanced computer vision technology requires. Consequently, one of the key objectives of the University of Helsinki research project is to develop more efficient computer vision methods that use less energy, for example, by being able to function with a one-lens camera, which is more affordable than the sensors traditionally used for 3D imaging, such as stereo cameras.
When the cranes’ operating environment becomes increasingly complex due to obstacles, the cranes need a growing number of sensors attached to them to analyze their surroundings. Smarter and leaner computer vision solutions, such as the ones developed by the research project, can provide more cost-efficient sensors.
Navigating research challenges
Perfecting computer vision still faces many challenges. In certain tasks, such as recognizing objects and finding small details, computer vision already performs much better than human vision. However, understanding context and calculating distances continue to cause significant issues.
The research project at the University of Helsinki is making strides in enabling the use of simpler, one-lens cameras for measuring distance and motion in computer vision. “In the past six months, we have made breakthroughs in making cranes and other machinery measure distances better with a one-lens camera,” says Laura Ruotsalainen, Professor of Spatiotemporal Data Analysis for Sustainability Science at the Department of Computer Science at the University of Helsinki, who leads the research project.
“The issue of calculating distances has now been partially resolved, although computer vision technology still has challenges in more complex operating environments that have, for example, clear or reflective surfaces with few visible shapes,” Ruotsalainen says.
Collaboration brings benefits to all parties
The research project has also benefited from the test crane ‘Ilmatar’ which Konecranes has donated to Aalto University for research and teaching purposes. Earlier in the project, a lot of data on crane navigation was gathered using Ilmatar. Currently, the focus is on analyzing the acquired data, utilizing open data sets, and conducting image analysis to provide research with usable synthetic data.
The field of computer vision also attracts considerable interest from students. With the cutting-edge knowledge and competence the research program provides, graduates can usually find employment quickly in the private sector, perhaps also at Konecranes in the future. The University of Helsinki and Konecranes share a common goal – to strengthen Finland’s expertise in the field of computer vision.
“Our permanent team is rather small, so we need a constant flow of students to keep the research going. Konecranes’ donation has directly benefited the project as it has enabled us to employ a new doctoral student,” Ruotsalainen notes.
The research project has already produced one doctoral thesis focusing on developing methodology for any machine to better understand its surroundings, as well as several master’s theses on various related topics.
The University of Helsinki conducts academic research around computer vision, developing new mathematical models that form the foundation of Konecranes’ more practical research on the topic. Both the research conducted by the university and Konecranes’ own research are indispensable for future applications to be as safe and efficient to use in real life as possible.
“For Konecranes, at the heart of this collaboration is sharing information about technical and methodological advancements. We are the first to learn about the latest scientific breakthroughs related to methods, acquiring deeper insight into what is happening at the forefront of computer vision development. Our donation to the research project is part of Konecranes’ commitment to contribute to Finnish research that is meaningful from our perspective,” says Sami Terho, Senior Specialist, Crane Intelligence at Konecranes.
New innovations ahead
Although the short-term goal of the research project is to better understand crane navigation in increasingly challenging environments, the project also has larger ambitions.
“In the future, the scope of the project could be broadened from computer vision methods to researching the optimal balance between efficiency, safety, and sustainability in crane operations,” Ruotsalainen says.