Reducing large data for efficient perception in driverless vehicles

‘Tensor Decomposition for Efficient Robotic Perception’ is a Cohesion project, bringing together Dr Kim Batselier of ME’s Delft Center for System and Control (DCSC) and Dr Julian Kooij of the Intelligent Vehicles group at the department of Cognitive Robotics (CoR). Their aim is to use the mathematical concept of Tensor Decomposition to compress the huge amounts of sensory data perceived by driverless vehicles to improve their overall efficiency.

So what is Tensor Decomposition? “It’s a way to reduce large data to small data,’ says Kooij, “It’s been applied to various types of task but not really to this kind of problem where you take a lot of sensor data and compress it. So the project’s about applying these mathematical techniques to the perception side of robotics to see what benefits we can gain by reducing the amount of data that needs to be processed.”

Lots of Sensor data

Self-driving vehicles need a lot of sensor data of many different types: “With vehicles you have cameras, you have remote-sensing - Lidar or Light Detection and Ranging - you also have laser scanning, and maybe even audio microphones so all kinds of sensor data comes streaming in and you need be able to process this all at once,” explains Kooij.

“And if you think about a self-driving car,” adds Batselier, “if someone crosses the road, then the car needs to detect that person - that’s where the whole robotic perception comes in - and then immediately process it and act on it to stop the car in time. You can’t stop a second after you’ve seen the pedestrian, you have to have very short reaction time so the process needs to move fast and efficiently.”

Tensor Decomposition is a way to reduce large data to small data.

Dr. Julian Kooij

Researcher

Energy efficiency

One solution of course is bigger and more powerful machines in the back of the car but the more perception tasks the vehicle needs to make – for monitoring the surroundings, or comparing incoming images with a database of known locations, or even making predictions – the greater the amount of data that needs to be processed. “And more processing means more expensive hardware, and more heat and more energy.”

So Batselier and Kooij’s approach is to take existing methods and then compress them “in a smart way.” In other words, remove the parameters that you don’t actually need - this is the compression part – so there’s less to process. “And you can have different trade-offs,” says Kooij. “Either you reduce the processing needs for the same latency i.e. the time delay between perception, processing and reaction, or you just need less hardware, or you have the same hardware but do more tasks.”

“But what it comes down to,” summarises Batselier, “if you have a full tank of gas in your self-driving car, and it has to do all these tasks, how many kilometres can it drive with a full tank of gas basically? How efficient can you make it?”

Cohesion triggers you to look out for people and for intersections, places where the research could converge.

Choices and connection points

In the first stages of a 4-year PhD project, the Cohesion team is currently focussed on the process of how to make the choices about where and how to compress. “There are many, many choices possible,” says Kooij, “but how do you make these choices when some things might be compressible and it’s actually a really good representation because the data itself was not inherently very complex, but in other cases, there be maybe a lot of details in the data that you’re trying to compress and then you lose a lot of that detail.”

“So Tensor Decomposition formulae can be used as a compression technique to help make these choices,” adds Batselier.

Language, research questions and inspiration

In the meantime, it’s clear that working with someone from a completely different discipline has challenges but also benefits. Batselier: “Cohesion triggers you to look out for people and for intersections, places where the research could converge. Julian and I had some meetings and things started to converge more and more which led to the project proposal - but it took some time because we first had to find some common language between us!”

“Yes, language is an interesting point. And I’ve had this more often with people from Kim’s department, Systems and Control,” laughs Kooij. “But I think it’s also that the type of research question you’re asking is very different. So you might be working on similar concepts but one says, “we want to work on this because clearly we want to get our mathematical proof” and you’re like “But wait – you use these things to do this right?” And then it’s “No we use these things to do that!” So it is language but it’s also different conferences, different journals, different ways of writing papers, different research questions and different directions, and then figuring out: does it mean the same or something different?”

So does this mean they have to change the way they think? “Not really,” says Batselier, “it’s more a case of extending the way you think.”