Emma M. van Zoelen
Design and AI Research, Human-AI Collaboration,Human-Centered AI
I believe we should design a future world, instead of designing merely a future product. A future world in which technology is not added to our society, but in which it creates and cooperates with our society.
Many innovations and technologies advance in such a way that they don’t bring real value to our lives or society. They help us in all sorts of practical ways, but no one seems to stand still to truly think about the cultural and societal effects on a bigger scale.
Most people either just accept that these technologies are there, losing themselves in the chaos of fast developments, or they become afraid that technology will take over the Earth and become a danger to the human race.
This creates a challenge. The challenge to make technology that enables people and intelligent technology such as artificially intelligent systems to work together on the same level, to empower people in using their qualities such as imagination, to encourage people to live life to the fullest and create their own stories of life.
To achieve this goal, designers with a vision should cooperate across disciplines to be able to match (inter)national policymaking with technological development.
I think. A lot. About the world, society, life, death… And about technology and the role it has to play in the story of the human race.
I often wonder what the world would look like if certain technologies or ways of dealing with life would be different. I wonder what the changes would do to us, the people, what it would do to our feelings and perception of the world.
Stories, especially in the genres of Fantasy and Science-Fiction, help in challenging the views we have on the world. They create scenarios of how society might be different when some aspects would change.
Writers and storytellers are designers of worlds, whereas many people are designers of little aspects. I want to design systems that take a vision of a future world into account, by using narratives and stories.
I want to research the interaction between intelligent systems and people, and how they can empower each other.
I want to find out how the presence of intelligent systems influences people’s perception and experience of the world and how I can support the processes of changing values and cultures.
A design and AI researcher. I like to research fundamental issues around the social system of the world and how intelligent and autonomous systems can play a role in that, while using creative and pragmatic methods from design. I focus on the unique qualities of people and even though I might conduct more theoretical and technical work, experiments with humans are essential to me. In this, I highly value a combination of quantitative, qualitative and constructive methods to get a broad view on a research topic. I thrive in interdisciplinary teams where team members can provide me with different viewpoints. One of the most enjoyable challenges in interdisciplinary research for me is to try to find the common ground between those viewpoints, a direction that makes most out of all expertise present.
A few years ago, just before my Final Bachelor Project, I became interested in Artificial Intelligence as a technology, and its role in the social fabric of the world.
My FBP was a first exploration of the social role that AI can have. Many questions came out of this, which made me realize I needed more knowledge on AI.
Combined with a wish for learning more mathematical and theoretical knowledge and skills, this made me decide to pursue an AI Master’s program next to Industrial Design.
My goal was to focus on the Math, Data and Computing expertise area in the AI program, and to focus on the User and Society expertise are within ID.
Ultimately, I wanted to combine the two by working on complex technologies and their role in society. Moreover, I wanted to learn more about research and research methods.
With this, I wanted to become a design researcher into the topic of human- and society-centered AI.
Over the years, however, I first slowly became more of an AI researcher and less of a designer. Since the two programs were so separate, it was sometimes difficult to combine the two in one clear identity or profession.
While working on my AI thesis though, I realized that part of me is still definitely a designer. Although the topics in AI research highly interest me, I lean towards the topics that are more aimed at interaction with humans.
Also, I missed being creative throughout the research process and being able to speculate about future societies, something that has always been a great part of my process here at Industrial Design.
Following this, my FMP was a search for a balance between AI topics of research and viewpoints and methods from design.
Coming right out of my AI thesis, it was a challenge to get back to the design mindset, and I think that it took me the complete M2.1 semester to get comfortable in the methods I was choosing.
Using design to communicate my ideas at the Dutch Design Week helped me to gain back confidence in my design and storytelling skills, as I was able to paint a clear picture of the future-oriented ideas I had.
View some of the highlights of my story below.
Doing the Agents track in the AI Master program provided me with a basis in general logic, game theory and learning in intelligent agents and multi-agent systems, as well as an overview of the academic literature.
One course in the AI Master program required me to write AI research proposals based on literature from psychology and linguistics. This taught me to be critical on existing research and to find new ways to apply it to my own work.
Participating in an interdisciplinary research honors program made me work with peers and teachers from a wide variety of backgrounds (e.g. philosophy, business, psychology, chemistry). This inspired me and made me value the power of interdisciplinarity.
For my AI thesis I interned at TNO, where I discovered other people with a strong interest in human-agent collaboration. I worked on reinforcement learning agents that learned to communicate proactively by training in simulation first and learning more socially desirable behavior from collaborating with humans.
Conducting my first full research project helped me to confirm that I really enjoy research. I learned how to do a systematic analysis of qualitative data obtained in a longitudinal study.
After having worked on several academic papers, the first that was accepted was my M1.1 work. I went to AutoUI to present the work and was very inspired by the community and the work presented there.
I was invited to talk about my vision on AI & Design and my work at Mind the Step 2017, at an Art + Technology Society event and at the Design United exhibition of 2019. Talking about my ideas was very motivating and helped me start a discussion with a more general audience.
During my Master, it was sometimes difficult to see myself as an AI or design expert, and especially during my AI thesis I was often feeling like I wasn't good enough. The last semester I have worked on regaining my confidence, and other people's trust in me helped me to believe in myself again.
My FMP project is a research study on evolving and emerging leader-follower behavior between humans and intelligent machines. This topic originated from an interest in creating mutual understanding between humans and intelligent machines through emergent interactions. Through a final experiment, I studied how people shift leadership in situations where there are seemingly conflicting intentions between themselves and the machine.
M2.1 Paper M2.2 Paper + Report
During my M1.1 project I worked in a team to design a shared driving experience in an autonomous vehicle.
Our final design was a way to democratize driving by giving every passenger influence on the driving style of the vehicle, as well as giving the vehicle agency to mediate between the different preferred driving styles.
Within this project we focused on creating speculative, future oriented concepts and explored the value in challenging the status quo by going beyond traditional roles and hierarchies in car driving.
Furthermore, I personally worked a lot on creating a prototype with sensing and acting using electronics.
Last, working in a group with designers from outside of ID and even different cultures helped me to review and improve my own communication and collaboration behavior.
The work done in this project was published as a Work-In-Progress paper at the 2019 Automotive UI conference.
Paper AutoUI
Full Reflection
The M1.2 project was my first full research project, in which I created a sixth sense for the presence of digital devices.
The designed object collected the MAC-addresses of nearby devices and translated those into sound signals. Participants wore this prototype several hours a day for a period of fifteen consecutive days without knowing what the signals were.
I analyzed how the different participants attempted to make sense of the data through for example pattern recognition.
The writing of a research proposal based on literature in a relatively short amount of time was a great exercise.
Furthermore, conducting a longitudinal study using diaries and interviews gave me interesting insights in the requirements of longer testing.
I analyzed the data using Grounded Theory, a systematic approach to qualitative data that I liked.
Last, creating the prototype using a Raspberry Pi and Python programming improved my technical skills.
Paper M1.2
Full Reflection
For the Constructive Design Research course my group chose to do a Showroom type of research.
We created a vending machine that promised to give people a free muffin, but only did so after they had answered several very personal questions.
These answers were then published on a screen on top of the machine, and only removed after participants paid a chosen amount of money.
In this provocative type of research, it is very important to carefully plan the storyline and experience, to make sure people are surprised by the interactions at the right moment.
What I considered the most interesting aspect of this kind of research, is that it puts both the participants as well as the researchers in specific roles related to the scenario that is played.
This allows both to reflect on their experiences and behaviors in relation to a situation that otherwise does not exist yet.
CDR Report
Full Reflection
Poster
In this course I worked in a group to create data visualizations from sleep data that could be used as tools by different users.
We focused on creating a dual visualization for both a patient and a nurse, to enable a nurse to give personalized sleeping advice to a patient while encouraging the patient to understand where the advice comes from.
I worked on translating a large dataset into useful insights using visuals, by giving the user the ability to explore the data in the way they desired.
In developing the tool, my process was very user-centered as for every design and programming decision I reflected on the steps that the user had to go through.
Final Report
Full Reflection
Poster
During the Research Methods elective I learned about many different qualitative methods and tested out one earlier unknown method in practice.
My group worked on a card sorting study of design card sets, to get a better insight into how design students view different design card sets related to whether they would like to use them.
Exploring the method of card sorting was interesting to see how we can get access to people’s mental models in different ways.
The card sorts that came out of the research were analyzed using thorough clustering in a group to define new categories of cards.
Doing this with several people revealed the different interpretations that different people had and helped us to come to a consensus that felt true to the participants.
We submitted our work to CHI, where it was unfortunately rejected, but we are still in the process of getting it published at another venue.
Final Paper
In a group, I worked on creating a ritual for two people who miss each other because they live abroad.
Our final prototype consisted of two boxes with coasters that left traces of which coasters the other person had used, to emphasize the subtle ways in which we read each other’s behavior when we are physically close.
I worked on very subtle interactions, where details were key.
This relates a lot to creating mindful and unconscious interactions; interactions which happen around us all day but which we are unaware of.
In such interactions, it is very important to create a balance between providing a user with freedom to personalize and interpret the interaction, and controlling the interaction.
Control can trigger creativity in the ways in which users can express themselves when applied to the right extent.
Annotated Storyboard
Reflection
Foundations of Data Mining was a computer science course on different types of data mining and machine learning.
I worked with Python and the SK-learn and Keras libraries, which are very common tools used in both academia and industry for Machine Learning.
I completed assignments on supervised and unsupervised learning, using for example different types of regression, Support Vector Machines and k-means clustering.
When working with more complex data, I learned to use Principal Component Analysis and other data preprocessing methods.
Last, I got more familiar with the mathematical background of Neural Networks and built several NN models using Keras for both classification and regression.
Assignment 1
Assignment 2
Assignment 3
Assignment 4
The Ethics and the Risk Society course was about how we can deal with risk presented to society because of technological developments in an ethical manner.
It contained a lot of theory on ethics, philosophy of science and risk, but also helped me to build better arguments because of the many discussions and peer review sessions.
I wrote an essay that explored how automation will influence the way we work, in which I argued that developing machines in such a way that they can collaborate with humans is a middle ground between full automation and full human labor.
Reading work from different ethicists and philosophers and applying this to my question was an interesting challenge in creating a well-constructed thesis.
Final Essay
My chosen areas of expertise are User & Society and Math, Data and Computing. Click below to read about my skillset for each expertise area.
Next month, I will be starting a PhD in the Interactive Intelligence group at the TU Delft, combined with the Perceptual and Cognitive Systems group at TNO.
The topic of my PhD will be personalized learning in Human-Agent teaming context: I will be working on how we can build agents that support the learning of humans by learning themselves in collaboration.
This relates a lot to the topic of my FMP, as it is also about how both human and machine learn and adapt to each other.
Before I found this specific position, I set up a few requirements for a job.
Looking across disciplines and envisioning what role (AI) technology might play in a future world is what I would love to do. There are four aspects that I feel I need in a future job:
High-Level Human-Agent Interaction such as human-agent collaboration or teaming
Future Society Oriented, taking into account future societal developments and culture changes, which makes the work rather exploratory
Mixed Methods: also creative: it should be possible to use various methods, and there should be openness to unconventional and creative methods such as Research through Design
Multi/Inter-Disciplinary Team: it would be especially great if there are other people working on the same or a similar project, preferably people with different backgrounds
Currently, I think that the topic of my PhD meets these four requirements, and while working on it I will make sure that they are continuously met.
I do not yet know whether I want to stay in academia after my PhD, but I do know that I want to keep doing some kind of research, that involves the above requirements. I am looking forward to a future with lots of new human-AI collaboration knowledge!
