Interview with Niels Van Damme About How to Make a Self-Driving Tram

Transcript of the interview:

Peter: I’m sitting here with Niels from the company that is doing some pretty interesting stuff. So can you, please, introduce your company a little bit?

Niels: Yeah. So my name is Niels. I’m a co-founder together with Sam Sander Smith. And our company is actually in the business of increasing safety and efficiency of railway vehicles in complex environments by teaching them to drive autonomously. So that’s our core business. How do we do that? Basically, we add sensors to current vehicles and we have software that gives audible and visual notifications to a driver at first. And then we gather data and we train our algorithms to go to a full self-driving mode.

Peter: So why have you decided to go for particularly railway vehicles?

Niels: During my university years, I got into deep learning and more specifically, computer vision.

And then one part of that was obviously self-driving cars, which is still a big business. And then after the university, I went to Silicon Valley to work there for six months.

Actually there I saw these self-driving cars. I was like how can we do this in cities of Europe? Because in fact self-driving cars are obviously coming, but it’ll be maybe 5, maybe 10, maybe 15 years ahead of time.

So I was like, how we can turn this into a business that actually can give autonomous transport solutions right now or in the near future? Maybe three, four, five years.

And then I started thinking, like, how Europe is different from other continents and countries.

And then obviously, if you look at a tram scheme, public transport, this is a big transportation industry in Europe and not so much in the USA, obviously.

And then if you look at public transport in general, obviously in Asia you have a lot of public transport.

But there it’s either a metro under the ground or it’s a monorail above cities. So these types of public transport don’t really benefit from computer vision technologies or from self-driving technologies.

Then we came back to Europe and we saw that actually the very old cities still have these trams driving through city centers where there are people walking there.

There are cars that are parked wrongly, cars that do crazy, crazy things. Bicycles that are driving across the street. So that’s where we saw that actually this could be a viable business.

And more specifically in Europe, because of the fact that we still use a lot of public transport.

And there are a lot of trams. And obviously, one big factor is the fact that the difficulty of making a vehicle on rails which would be self-driving is a lot less than making a self-driving car because there are a lot less variables in there.

So that’s the main reason why we went there. We went through the tram industry.

Peter: Do you cooperate with some tram producers?

Niels: Basically we deliver our product to three types of clients: obviously RTD tram operators, the people who operate trams in cities.

Then you have these private operators on industrial sites: there are big ports, big companies that have trains and trams driving on their production sites.

And then obviously that is the last one is manufacturers, although we see that, yeah, they rather invent something themselves or they want to buy something or they want to buy the full software.

So they’re not really into us delivering the hardware, the sensors suites, that stuff. Probably when we are a bit further in our roadmap, there might be the corporate cooperations with manufacturers.

But for now, we’re focusing on actually retrofitting. So it’s putting sensors on current vehicles, working with tram operators and private industries.

That’s what we’re focusing on right now.

Peter: So some sensors like optical sensors and some hardware that is computing this…

Niels: That’s the thing. So we’re working with a sensor suite that consists of cameras and radar. We think that LADAR is essential in this kind of setting.

We also use GPS, IMU and accelerometers. And obviously we work together with video for the hardware unit to fuse all the algorithms and compute our cooperation within video.

They deliver to hardware. And on top of that, it’s already autonomous straight or a top automotive grade, which allows us to put it in vehicles.

Actually, it’s redundant. There’s a lot of computing power. There are cybersecurity measurements that’s been taken to into the software of that video. So we build actually on top of that.

Peter: So you basically take the tram, one of the guys goes there just to place all these sensors.

Niels: Yeah. We have a specific arrangement of the cameras and the radar that we put on top of the vehicle.

And then we put the computing unit in there where there is a user interface for the driver to give assistance. And that’s basically it. So we actually put it into a current vehicle. That`s how it works.

Peter: Obviously one of the challenges of building this self-driving vehicle is enough data so that you can test the models. How do you do that?

Niels: So basically the fact that we do it in two stages, the fact that we first give an assistant system and then we work towards a full self-driving module is basically the fact that for now we don’t have enough data.

So the assistant system will allow us to gather all the data, even though it’s not full self-driving, we still put on all the sensors, all the cameras, all the radars.

And then whenever we sell an assistant system, it gathers data to train our neural nets, work for full software.

Obviously, we have a vehicle now driving around the city, which is our own, which is not a tram, which is a road legal vehicle we use. But we can still use it to gather data, essential data and obviously mimic or simulate…

Peter: Did you buy some old tram?

Niels: We bought an electrical vehicle where we basically simulated a tram kind of environment. And then we drive around in the city on top of the rails because these rails go through the city where cars also drive.

We can basically mimic a tram driving. This is only for the assistance system. But this allows us to go and check edge cases, for example, people lying on the floor.

Do we detect that? Do we stop that? Very special cases we can test with these people. Obviously not on public roads. That’s why we have our test site.

If we want to test some really awkward and special test cases or edge cases, we do that on a private test site here.

Peter: Sometimes people use unity or some gaming.

Niels: Yeah, for now we don’t really use simulation techniques because we do think that whenever our assistance system is commercialised, we will get enough data to train our neural network for the full self-driving.

But obviously, when there is not enough data, we could be looking at simulations within a unity, for example, to basically make a twin world and test edge cases there.

Because in a digital twin, you can basically test one case a million times. And that case might be something so special that it might only come forward like once in a lifetime for a tram driver.

But still, we can test this a million times in the digital twin or in the simulation. So it might be worth looking into this.

But we do still think that real world data is always better than simulated data. That’s the reason why we’re first going for the real world data. And if that’s not enough, we can extend that to simulation data.

Peter: What technology in terms of software do you use?

Niels: So basically, it’s all on a Linux based system. Everything is Python or C++. And for the neural networks, it’s a tensor flow thing.

On top it is the NVIDIA hardware and software, which allows us to get a better inference time to calculate all these objects and things like that a lot faster. And so there is computer vision technology. There’s object detection and object tracking.

There’s a sensor fusion algorithm between the cameras and the radar point clouds. So, yeah, that’s basically how we work. We’re really getting all the technology from the self-driving car industry. On top of this is very old industry, which is the tram industry.

Peter: How many people are working on that?

Niels: Currently we have a core team of four, which we obviously extend with some freelancers. But like I said, we are recruiting, we are hiring. We’re still looking for software engineers, machine learning engineers all the time.

But you need to be cut out of the right wood to join a startup and really go for it. This is fun but this is hard.

And sometimes you work a lot of hours to get things done to meet the deadlines, which is obviously in all industries, but still a startup is something different. And it could go well and we could get the next financing round next year or it could go wrong.

And we could stop the business or we couldn’t even be here anymore. So that’s basically what’s tricky: to get people into our business and hire them.

But still there are some people who really enjoy the startup community. And I really want to take the leap to go into a startup.

Peter: And do you train them somehow or you are just looking for trained ones?

Niels: Whenever they join and they do not have the expertise in their field or they want to extend their expertise in another field, we offer workshops throughout several online platforms.

You have Udacity, you have Coursera, but also and NVIDIA gives a lot of one-to-one trainings, workshops that we can have for free or we can get for free because we’re in their program. And that’s basically the cutting edge technology.

These are the technologies, these are the strategies and the techniques that they use in large, big corporations who work on self-driving cars.

So, I mean, yeah, that’s how we train them. And it’s mostly online because it’s self-paced. And then they can work on it whenever they want.

And obviously, when we have expertise in something or one of our team members has expertise in something, they can transfer that knowledge to other people within the business.

We’re now looking to try once a month to do the following: everyone picks a random topic and a thing that they know something about and others don’t. And they just give a ten-minute presentation once a month.

They give a ten minute presentation about something that probably we don’t know about. It could be something totally different than self-driving vehicles, but something that they think is worth for us to know about and that is sometimes pretty interesting.

Peter: And you are situated in Leuven, aren`t you?

Niels: No, we’re not in Leuven, we’re in Ghent.

Peter: Oh, sorry.

Niels: No problem. We’re in I think one of the smallest cities in Belgium, which is Ghent, which is very entrepreneurial-minded.

But for now we’re staying in Ghent, or staying in Belgium because it’s very centralized in Europe. And like I said, we’re focusing on this European market. I mean, the Netherlands is only one and a half hour drive from here.

France is a one hour and a half drive from here. I mean, we’re very close to everything. So that’s the reason why we’re for now staying in Belgium.

Peter: And you somehow cooperated with the University, right?

Niels: Yes. Basically, we’re in the number one University linked Technology Accelerator, which is called imec.istart. We’ve been chosen out of tons of startups who are in the tech industry.

And they thought that we had a very unique business proposition, a very scalable business, a very special thing, and especially in this time of day with all those self-driving vehicles, self-driving cars. It’s a hot topic.

There are a lot of buzz words, there’s a lot of buzz going on. So, yeah, that’s probably one of the reasons that they chose us. They actually helped a lot.

They gave us an office space for free basically. And they help us with business-related things, but obviously also with technology-related things.

They have a lot of technology in-house. If you consider working together with them, you could, because they basically have imec. istart which is for startups and then they have imec. for bigger companies which holds all the patents and all the technology.

And you’re obviously allowed to work with them and then you have the opportunity to work with them if it’s beneficial. So that’s really great about the accelerator program.

Peter: And you have already talked a little about that. But what are the plans for the company? What is the strategy?

Niels: Yeah, so we talked about it in the beginning. We’re now looking for pilot projects to really showcase that our technology is there, that we can show companies that it’s actually an added value to what they’re doing now.

And then hopefully we can start commercializing next year for the assistance system and heavily invest in H.R. and expand our team to 10, maybe 15 people to work on that self-driving software because there is a long road ahead of us.

And if you want to be on top of the game, you should accelerate really fast. So hopefully we can start working on the actual full self-driving software early next year, and then to get it with the data that we’re gathering from commercializing our assistance system.

We can start learning or training our algorithms to go to full-time driving and hopefully at the end of 2021, maybe 2022, we can actually do a pilot project, full, self-driving and hopefully in public roads if the framework allows it. That’s our goal.

Peter: And you will be focusing on the cities. And you will go to the City Hall and you will talk to all the transportation companies.

Niels: Yeah. So in other countries, the government and the cities work together really tightly with the tram operators.

So basically we will go throughout Europe, go to every city, talk to every operator, see where there are 30 innovation players, and then check if they want to cooperate with what we’re doing. And they basically increase safety.

That’s what we’re here for. We want to increase safety and eventually go to full self-driving, which is completely safe. Costs a lot less and can solve a lot of problems.

For example, the shortage of drivers that they have at the moment in Europe, the shortage of drivers for trams and buses, it’s a big thing and we hope to solve the coming use.

Peter: Not to mention that the cost of a driver for a tram is like 65 percent of the price.

Niels: Yeah. Theoretically, yeah. That’s because you need four people to get a travel up and running almost 24/7. I think they drive from five or six o’clock in the morning until twelve o’clock at night. So to keep it up and running, you need four people a year.

And then if you obviously can get that person out of there and switch it up with the software, that’s a cost reduction. I’m not saying that we need to fire all these people, but they can obviously get more into the service business, help people on the tram, maybe check for tickets and just get more services to do the clientele of the tram operators, which then again, will get more revenue and will increase business.

Peter: Niels, when did you start with deep learning and artificial intelligence?

Niels: Personally, I come from engineering, electro-mechanical engineering, which is more design of machines and stuff like that.

I did my major in automation technologies, which is more programming, and then I think it was 2016 during my my university years that I was actually a friend of mine who recommended me to to start looking into this machine learning and more specifically, deep learning industry.

I started doing some courses and just reading all about it, reading some papers and starting to get to know the ins and outs. And then I realized that this actually makes sense to start a business in something like that. I’m not saying that everyone should start a business in self-driving vehicles, because that’s really the end of the spectrum.

But there are a lot of things that you can do with machine learning and deep learning, even smaller projects. But also, like we see, there are a lot of companies eager to look for a project and to optimize what they’re doing right now.

And that’s what we’ve seen, too. But obviously, self-driving vehicles are all the way at the end of the spectrum of deep learning and machine learning.

But, yeah, I think you should start really early on. That’s what I did during my college years. After the college hours, after university hours, I did courses. I read a lot of books.

And that’s when I started and I kept on improving. And then I kept updated through posts on several websites, courses that come out. Yeah. It’s an interesting field and you can do a lot with it. So I highly recommend it.

Peter: How do you relax? What’s your hobby?

Niels: So basically for now this season has stopped but I play ice hockey.

Peter: Ice hockey?! Cool!

Niels: In a local ice hockey team which is actually on Thursday night and Monday night.

And it’s really relaxing because you can just let go of everything. Go for it and just go all the way. Go red and then just relax.

Exercising in general helps out a lot. It feels great. Especially the day after. You feel good. Yeah.

In these times I exercise a lot at home or I go for a ride or something like that. But I do think that exercise in general is what I do the most to relax.

Peter: As a Czech guy I really appreciate your liking ice hockey! We are an ice hockey nation (laughing).

Niels: I know. There are actually a lot of people in our team who go for training, training camps or something like that in Czechia. They go there for two weeks or something and they have there a dry training and an ice training.

Peter: We have a big tradition of ice hockey.

Niels: Yeah, I know.

Peter: And have you read any interesting book recently?

Niels: Yeah. I don’t really read a lot of, let’s say, non-technical books. I’m not really that kind of guy who can sit back in my couch and read a novel or something.

So I mean, the last book that I read, which I thought was pretty interesting, is actually a Dutch book. So it’s in Dutch from a Dutch guy.

Let me just check. I think it’s called Algorithmization, Get Used to It. It’s translation. It’s a book from Jim Stolze. He’s basically explaining how algorithms are already in our lives and are here to stay. And then he gives a lot of examples.

And even I was surprised by some examples like, oh, wow, are they really doing this? Is this real? What’s going on? It was a pretty interesting book, actually.

My father read it and he’s like, wow, is this happening already? And I’m like, yeah, they have the data to do it. So why not? And it’s a very thin book.

But you notice that algorithms are actually really here to stay. And every app you open, there is an algorithm behind it. Everything you do that’s tech related, there’s probably an algorithm behind it.

So, yeah, I was pretty amazed by what they’re actually already doing. The bigger companies like Facebook, Amazon.

Peter: Oh, yeah. And I wanted an extra question on singularity.

Niels: Yes, singularity. As a programmer myself, I’m more into transfer learning business and in the supervised learning business, especially for what we’re doing now.

But there are some self-supervised things going on, especially if you look at OpenEye and stuff like that. These algorithms can do a lot. But going from that to singularity is a big, big step away. And yet the thing is, you always program an algorithm to a certain task or a certain kind of thing.

And obviously, we need to take care of what we’re learning. The algorithm or what we’re teaching the algorithm and especially the end point. But still, I think singularity is… I don’t think I will ever see that in my life.

That there is an actual algorithm that can do as much stuff as we can, as humans can. I don’t think so, because the tasks that we can perform with our brain are so diverse.

We could drive a car and then stop, get outside, open a door with a key and then make dinner. I mean, it’s just so diverse that I think it’s probably not for us.

There might be the future. I’m obviously not scared. I think it’s fantastic that people are working on that. I think it’s fantastic that people are out doing stuff like OpenEye.

It’s pretty interesting. But it’s not for me. And I don’t know the nitty-gritty of it.

Peter: I was just curious what you think about it. Recently, I`ve been thinking a lot about artificial intelligence and let’s say full employment. I know there are two sides of the story. Some people say like there will be new jobs. And some people have this, let’s say, demolition theory or deflation theory that it’s not gonna happen.

Niels: Yeah, there are different kinds of ways. But like we saw in the past, every time there is a technology leap, obviously there is some kind of time that it takes for the economy to refactor and to change.

But still, obviously, a lot of people lost their jobs, but then they found something else to do. And I think in the future, people will be more creative than ever.

For example, this is actually a really good case with that Covid19. Basically that has nothing to do with technology but has the same kind of impact.

A lot of people lost their jobs. A lot of industries are now back to zero. And people need to be creative and people need to work around it and people need to refactor their skills and need to do other things.

And that’s the same with technology and it’s here to come whether we want it or not. I even think that my job I mean, programming, artificial intelligence will probably be done by an algorithm and better than a human can do.

So, I think we just need to keep focusing on improving ourselves, trying to learn fast. It’s one of the things if you can learn fast, if you can adapt fast, then you’re probably futureproof, in my opinion.

Peter: Well, it’s certainly true that there will be more creative jobs. Let’s say, we don’t even foresee what kind of jobs there will be.

Niels: Yes. Yeah, that’s the thing.

Peter: I’m just kind of afraid that it’s going to be so fast, you know, with all these kinds of things that it will take some time to adjust to our civilization. Let’s see.

Niels: Yeah. That’s the thing. That’s what excites me as well. Let’s see what happens. And maybe we are just fine. Maybe it’s all good.

Peter: We will be certainly fine. Yeah.

Niels: So I don’t really worry about this.

Peter: All right. Okay. Cool. Thank you very much Niels. It was nice.

Niels: It was a pleasure talking to you guys.