YAWN! 🥱 You’re behind the wheel of your self-driving car, and you’re getting a bit bored. Luckily, your car recognizes your facial emotion and warns you to keep paying attention. And just in time: the automatic brake system is malfunctioning, and you need to brake manually to prevent an accident. “An amazing example, that could happen in real life”, explains Computer Science & Engineering student Mana Mahmoudi. However, this is only possible when AI systems are able to recognize facial emotions. But that’s not quite that easy…
AI systems need input from facial emotion databases to be able to recognize emotions. But in order to know what a certain emotion looks like, humans have to rate images of facial emotions. But they don’t always agree: One person might rate an emotion as ‘surprise’ when in reality that person is scared. That’s why Mana’s bachelor research project focused on the different methods used to determine interrater agreement when it comes to recognition of facial emotions.
Which bachelor's programme are you doing, and why did you choose this one?
I am doing the Computer Science & Engineering bachelor's. And I chose this bachelor's degree because of the future prospects I have once I finish my studies at TU Delft. I will be able to go into the field of artificial intelligence and help improve the world through technology. Right now I’m doing a research project that will last for around 10 weeks. It gives us a glimpse into the future of what it’s like to work on a master's or PhD thesis. So that when we have to write a big thesis, we have some hands-on experience of how to manage it.
What is your research project focused on?
My research focuses on facial emotion recognition. I analyse databases to see how they account for subjective assessments of emotion recognition and raters' biases. Because it can be difficult for different raters to recognize a facial emotion. Contempt can be confused with anger, and surprise can be interpreted as fear. Those databases will be used for machine learning algorithms and AI and systems that will be used in the real world. That’s why it’s important that the data is correct. My research question is: What are the differences in interrater agreement methodologies in the different published data sets about facial emotion recognition?
I analyse databases to see how they account for subjective assessments of emotion recognition and raters' biases. Because it can be difficult for different raters to recognize a facial emotion. Contempt can, for example, be confused with anger.
And what's the societal impact of the project?
These databases are used to shape the world as we know it. They are really important to be able to move forward in any field. Emotion recognition can be used for self-driving cars; the example that's given in the introduction just shows how important good working emotion recognition features can be. And emotion recognition can also be used to check up on elderly people or detect mental stress in university students. And by doing research you also contribute to society’s knowledge and technological advancements.
Were there any challenges within this research project?
Yes, I think that every project has its challenges. And we learn the most from challenges. The challenge here was that it was my first “real” research in an academic environment. I had some challenges when it came to the systematic literature review methodology process. I had gathered 12,000 papers at some point, and I was not able to read them all of course. It was important to get the literature review correct because my research has to be reproducible. The aim for the literature review was to learn to search in the right way and to go in the right direction.
Were there any other challenges involved in the study?
Something that I found challenging when it came to the study, was the application process. The application process is quite strict. There are many applicants, and only around 550 students are accepted for the Computer Science & Engineering bachelor. As part of the process you have to do a Cognitive Skills Test and a team assignment. It was quite stressful to be able to prepare correctly. But I am glad to say that I passed with flying colours! I learned that everything is doable if you have a plan.
And something that was also challenging, was the housing situation. There’s not a lot of student housing available. And Dutch students sometimes post about available spots on Facebook, for example. But because I was an international it was more difficult to get housing because Dutch people prefer to live with other Dutch people. I really didn’t like that. Fortunately, I did find a place to live after a while.
What do you like about studying at TU Delft?
I enjoy the variety of courses we have because I feel like, no matter which domain of computer science I want to go into, I've got a solid knowledge base. And besides the education, I also really like the atmosphere on campus. There’s a lot of green areas and nature. When you’re tired of studying, you can just go outside, and sit and relax in the grass, or sit on top of the library. I also really enjoy being a student assistant and sharing about this amazing bachelor programme during the Bachelor Open Days.
What are your plans for the future?
I would like to do a master's and also a PhD if possible. That’s why I really like to gain experience with research. I think my bachelor's has already given me a good base to be able to build upon. I want to continue within the field of Computer Science & Engineering. I would like to work on artificial intelligence, and specifically health-related AI.
I got some hands-on experience with health-related work during the second year of my bachelor's. I worked on a project for Integrating the Healthcare Enterprise (IHE) Belgium, which coordinates the technical development activities of healthcare systems within the country. They wanted an application with which they could send medical images from one hospital to another hospital in Belgium. And they wanted an application that could be used on a national level and that everyone could use. Transferring medical images is not as easy as sharing an image on WhatsApp. There’s a lot of data involved with it. Together with four other teammates, I developed this application. We did a lot of coding and building the application. A LOT of debugging. This project didn't involve any AI, but I still really liked it. We went to a meeting in Brussels with some big companies, like Phillips, that were also providing systems used in IHE. The project was really interesting and allowed me to understand how big companies manage IT issues!
I think it would be amazing to be able to use artificial intelligence to improve health-related issues. I want to use AI to find new treatments and new solutions to treat patients. I think it’s really important and rewarding to make an impact on society and in that way be able to help others.
