Note:

At present, I write here infrequently. You can find my regular, weekly blogging is over at The Deliberate Owl.

alt text

Of genius, PhDs, babies, and social robots

Although I haven't written a lot for my personal blog while in grad school, I have written a few posts for both the MIT Graduate Student Blog and the MIT Media Blog. I talk about having babies while in grad school doing research about robots and kids! You can find my writing at the links below. (Most of these are also cross-posted to my blog here!)

2017:

2018:

2019:

0 comments

a red and blue robot sits on a table

Tega sits at a school, ready to begin a storytelling activity with kids!

Last spring, you could find me every morning alternately sitting in a storage closet, a multipurpose meeting room, and a book nook beside our fluffy, red and blue striped robot Tega. Forty-nine different kids came to play storytelling and conversation games with Tega every week, eight times each over the course of the spring semester. I also administered pre- and post-assessments to find out what kids thought about the robot, what they had learned, and what their relationships with the robot were like.

Suffice to say, I spent a lot of time in that storage closet.

a child sits at a table that has a fluffy robot sitting on it

A child talks with the Tega robot.

Studying how kids learn with robots

The experiment I was running was, ostensibly, straightforward. I was exploring a theorized link between the relationship children formed with the robot and children's engagement and learning during the activities they did with the robot. This was the big final piece of my dissertation in the Personal Robots Group. My advisor, Cynthia Breazeal, and my committee, Rosalind Picard (also of the MIT Media Lab) and Paul Harris (Harvard Graduate School of Education), were excited to see how the experiment turned out, as were some of our other collaborators, like Dave DeSteno (Northeastern University), who have worked with us on quite a few social robot studies.

In some of those earlier studies, as I've talked about before, we've seen that the robot's social behaviors—like its nonverbal cues (such as gaze and posture), its social contingency (e.g., using appropriate social cues at the right times), and its expressivity (such using an expressive voice versus a flat and boring one)—can affect how much kids learn, how engaged they are in learning activities, and their perception of the robot's credibility. Kids frequently treat the robot as something kind of like a friend and use a lot of social behaviors themselves—like hugging and talking; sharing stories; showing affection; taking turns; mirroring the robot's behaviors, emotions, and language; and learning from the robot like they learn from human peers.

Five years of looking at the impact of the robot's social behaviors hinted to me that there was probably more going on. Kids weren't just responding to the robot using appropriate social cues or being expressive and cute. They were responding to more stuff—relational stuff. Relational stuff is all the social behavior plus more stuff that contributes to building and maintaining a relationship, interacting multiple times, changing in response to those interactions, referencing experiences shared together, being responsive, showing rapport (e.g., with mirroring and entrainment), and reciprocating behaviors (e.g., helping, sharing personal information or stories, providing companionship).

While the robots didn't do most of these things, whenever they used some (like being responsive or personalizing behavior), it often increased kids' learning, mirroring, and engagement.

So... what if the robot did use all those relational behaviors? Would that increase children's engagement and learning? Would children feel closer to the robot and perceive it as a more social, relational agent?

I created two versions of the robot. Half the kids played with the relational robot: the version that used all the social and relational behaviors listed above. For example, it mirrored kids' pitch and speaking rate. It mirrored some emotions. It tracked activities done together, like stories told, and referred to them in conversation later. It told personalized stories.

The other half of the kids played with the not-relational robot—it was just as friendly and expressive, but didn't do any of the special relational stuff.

Kids played with the robot every week. I measured their vocabulary learning and their relationships, looked at their language and mirroring of the robot, examined their emotions during the sessions, and more. From all this data, I got a decent sense of what kids thought about the two versions of the robot, and what kind of effects the relational stuff had.

In short: The relational stuff mattered.

Relationships and learning

Kids who played with the relational robot rated it as more human-like. They said they felt closer to it than kids who played with the not-relational robot, and disclosed more information (we tend to share more with people we're closer to). They were more likely to say goodbye to the robot (when we leave, we say goodbye to people, but not to things). They showed more positive emotions. They were more likely to say that playing with the robot was like playing with another child. They also were more confident that the robot remembered them, frequently referencing relational behaviors to explain their confidence.

All of this was evidence that the robot's relational behaviors affected kids' perceptions of it and kids' behavior with it in the expected ways. If a robot acted more in more social and relational ways, kids viewed it as more social and relational.

Then I looked at kids' learning.

I found that kids who felt closer to the robot, rated it as more human-like, or treated it more socially (like saying goodbye) learned more words. They mirrored the robot's language more during their own storytelling. They told longer stories. All these correlations were stronger for kids who played with the relational robot—meaning, in effect, that kids who had a stronger relationship with the robot learned more and demonstrated more behaviors related to learning and rapport (like mirroring language). This was evidence for my hypotheses that the relationships kids form with peers contribute to their learning.

graph showing on the left, that kids in the not-relational condition didn't have as strong a correlation while in the relational condition, there was a stronger correlation - but that this varied by gender

Children who rated the robot as more of a social-relational agent also scored higher on the vocabulary posttest.

This was an exciting finding. There are plenty of theories about how kids learn from peers and how peers are really important to kids' learning (famous names in the subject include Piaget, Vygotsky, and Bandura), but there's not as much research looking at the mechanisms that influence peer learning. For example, I'd found research showing that kids' peers can positively affect their language learning... but not why they could. Digging into the literature further, I'd found one recent study linking learning to rapport, and several more showing links between an agent's social behavior and various learning-related emotions (like increased engagement or decreased frustration), but not learning specifically. I'd seen some work showing that social bonds between teachers and kids could predict academic performance—but that said nothing about peers.

In exploring my hypotheses about kids' relationships and learning, I also dug into some previously-collected data to see if there were any of the same connections. Long story short, there were. I found similar correlations between kids' vocabulary learning, emulation of the robot's language, and relationship measures (such as ratings of the robot as a social-relational agent and self-disclosure to the robot).

All in all, I found some pretty good evidence for my hypothesized links between kids' relationships and learning.

I also found some fascinating nuances in the data involving kids' gender and their perception of the robot, which I'll talk about in a later post. And, of course, whenever we talk about technology, ethical concerns abound, so I'll talk more about that in a later post, too.

—

This article originally appeared on the MIT Media Lab website, February, 2019

0 comments

my slow cooker

How a special appliance has saved me both time and sanity

I want to thank one special appliance
Whose dedication and trusty alliance
Have been a time saver for a busy grad mom.
You snuck into my kitchen with quiet aplomb,
Arriving, in a box, some years ago—
Black and sleek. How was I to know
That you would save me countless hours?
Minimizing meal prep with your heating powers.
And you save me, too, from decision fatigue!
Other kitchen gadgets just aren't in your league.
So, on Sunday mornings, that was our routine!
Chopping veggies, carrots, and sometimes green beans.
Toss in some lentils, barley, or peas!
We varied by week: soup or Chinese?
Chili, orange chicken, sometimes a stew,
Rice with beans; often barbecue.
By evening, the apartment always smelled great.
My spouse and I filled up our plates.
And leftovers! Man, were those our goal!
We dished them straight into jars and bowls.
Dinners for a week—for two, no less!
No need to prep or make a mess.
Your 6-quart volume held just enough
To keep us fed when nights were rough.
'Cause let's face it. Grad school's no joke.
You're stressed and tired and sometimes you're broke.
Between classes, field studies, and paper writing;
Managing undergrads, coding, and citing...
A grad student's work never feels done...
(Even if I think some of mine is fun!)
So when I'm at home at the end of the day,
When I want to sleep and my kid wants to play,
Finding that dinner is ready! Already! It's nice.
Microwave a bowl and eat in a trice.
So as I reflect on what helped me through grad school,
I'd say you, dear slow cooker, were a most useful tool.
Food fuels the brain and the body too...
So I wanted to say: Dear slow cooker, thank you.

—

This article originally appeared on the MIT Graduate Student Blog, May 2019

0 comments

Why is having kids, moving out of the city, and following an unusual path a waste?

Randy, Elian at 8 months (sporting his lab t-shirt!, and I

"She's worried you'll waste your degree."

My friend (let's call her Anna) relays this message to me as coming from another friend, but I can tell from her tone of voice that she's clearly worrying about the same potential waste. That makes the question doubly irritating. As if pretending to be merely the messenger could disguise the passive-aggressive way of questioning my life decisions. Decisions which, I might add, I'm pretty darn happy with.

The primary decisions in question are these:

First, I had a baby in grad school. I'm growing another tiny human now, in fact—I gave my defense talk while 6 month pregnant! Evidently, instead of seeing this as a badass feat of time management and life balance, Anna took it as ultra-clear proof that childbearing, not science, is my ultimate goal in life, since the two clearly aren't compatible. As if there aren't amazing examples to the contrary, like two of my committee members, who are inspiring women with three kids apiece.

Second, while finishing my last semester of writing, I moved to a town that Anna has frequently referred to as "the middle of nowhere," despite it having a regional population in the 200,000's, as well as a branch of a state university. Maybe she thinks "middle of nowhere" really refers to how far you are from a large number of appropriately ethnic restaurants? Being out west, up in the skinny part of Idaho with the abundance of beautiful clear lakes, pine-filled mountainsides, and a peaceful pace of life has been wonderful. Less stressful. It's a nice place for writing, and a nice place for families.

And then, there's the somewhat non-traditional plan for my post-MIT life. It's not perfectly mapped out, but it will certainly involve my husband and I homeschooling/unschooling our kids, coming up with flexible work arrangements so we can travel more and spend more time with family, and having a high degree of independence. My husband's current software-as-a-service company is a good start. We have some other ideas, too—after all, leaving MIT and Boston doesn't mean I'm leaving research or a creative, intellectual life.

Given those decisions, well, of course! Getting a degree is a waste! If my life plan does not follow the norm, if it does not include seeking out a high-paying industry job in a big city or a prestigious professorship at an R1 school while placing my kids in daycare and coercive schooling for upwards of 14000 hours, then of course, I'm wasting my degree.

But isn't a big part of the point of grad school learning? Learning about project management. Developing writing skills. Doing independent research. Asking interesting questions. Pursuing ideas. Managing time, balancing multiple commitments, and being involved in many activities I care about. Whether or not I then use those skills to pursue any of the most common paths out of grad school isn't the point. What I learned will still serve me well in future endeavors—writing papers and essays, consulting, hiking in the mountains, self-funding our startups, blogging, gardening, reading philosophy, advocating for self-directed education, or spending time with the people who really matter to me.

The implicit assumption Anna had that "wasting my degree" is even possible is, frankly, an insult. She identifies as a feminist. Isn't feminism supposed to be about empowering and supporting women in making life choices that are right for them?

Grad school was one step that was right for me. Having kids I actually spent time with, moving out of the city, pursuing whatever creative, intellectual, maternal, or domestic activities I happen to want to do next...? Also right for me. Sorry to disappoint, Anna.

—

This article originally appeared on the MIT Graduate Student Blog, March 2019

0 comments

A girl grins at a red and blue fluffy robot and puts her arm around it

Relational AI: Creating long-term interpersonal interaction, rapport, and relationships with social robots

Children today are growing up with a wide range of Internet of Things devices, digital assistants, personal home robots for education, health, and security, and more. With so many AI-enabled socially interactive technologies entering everyday life, we need to deeply understand how these technologies affect us—such as how we respond to them, how we conceptualize them, what kinds of relationships we form with them, the long-term consequences of use, and how to mitigate ethical concerns (of which there are many).

In my dissertation, I explored some of these questions through the lens of children's interacts and relationships with social robots that acted as language learning companions.

Many of the other projects I worked on at the MIT Media Lab explored how we could use social robots as a technology to support young children's early language development. When I turned to relational AI, instead of focusing simply on how to make social robots effective as an educational tools, I delved into why they are effective—as well as the ethical, social, and societal implications of bringing social-relational technology into children's lives.

Here is a précis of my dissertation. (Or read the whole thing!)

a girl looks at the dragonbot robot as it tells a story

Exploring children's relationships with peer-like social robots

In earlier projects in the Personal Robots Group, we had found evidence that children can learn language skills with social robots—and the robot's social behaviors seemed to be a key piece of why children responded so well! One key strategy children used to learn with the robots was social emulation—i.e., copying or mirroring the behaviors used by the robot, such as speech patterns, words, even curiosity and a growth mindset.

My hunch, and my key hypothesis, was this: Social robots can benefit children because they can be social and relational. They can tap into our human capacity to build and respond to relationships. Relational technology, thus, is technology that can build long-term, social-emotional relationships with users.

I took a new look at data I'd collected during my master's thesis to see if there was any evidence for my hypothesis. Spoiler: There was. Children's emulation of the robot's language during the storytelling activity appeared to be related both to children's rapport with the robot and their learning.

Assessing children's relationships

Because I wanted to measure children's relationships with the robot and gain an understanding of how children treated it relative to other characters in their lives, I created a bunch of assessments. Here's a summary of a few of them.

We used some of these in another longitudinal learning study where kids listened to and retold stories with a social robot. I found correlations between measures of engagement, learning, and relationships. For example, children who reported a stronger relationship or rated the robot as a greater social-relational agent showed higher vocabulary posttest scores. These were promising results...

So, armed with my assessments and hypotheses, I ran some more experimental studies.

a boy sits across a table from a red and blue robot

Evaluating relational AI: Entrainment and Backstory

First, I performed a one-session experiment that explored whether enabling a social robot to perform several rapport- and relationship-building behaviors would increase children's engagement and learning: entrainment and self-disclosure (backstory).

In positive human-human relationships, people frequently mirror or mimic each other's behavior. This mimicry (also called entrainment) is associated with rapport and smoother social interaction. I gave the robot a speech entrainment module, which matched vocal features of the robot's speech, such as speaking rate and volume, to the user's.

I also had the robot disclose personal information, about its poor speech and hearing abilities, in the form of a backstory.

86 kids played with the robot in a 2x2 study (entrainment vs. no entrainment and backstory vs. no backstory). The robot engaged the children one-on-one in conversation, told a story embedded with key vocabulary words, and asked children to retell the story.

I measured children's recall of the key words and their emotions during the interaction, examined their story retellings, and asked children questions about their relationship with the robot.

I found that the robot's entrainment led children to show more positive emotions and fewer negative emotions. Children who heard the robot's backstory were more likely to accept the robot's poor hearing abilities. Entrainment paired with backstory led children to emulate more of the robot's speech in their stories; these children were also more likely to comply with one of the robot's requests.

In short, the robot's speech entrainment and backstory appeared to increase children's engagement and enjoyment in the interaction, improve their perception of the relationship, and contributed to children's success at retelling the story.

A girl smiles at a red and blue fluffy robot

Evaluating relational AI: Relationships through time

My goals in the final study were twofold. First, I wanted to understand how children think about social robots as relational agents in learning contexts, especially over multiple encounters. Second, I wanted to see how adding relational capabilities to a social robot would impact children's learning, engagement, and relationship with the robot.

Long-term study

Would children who played with a relational robot show greater rapport, a closer relationship, increased learning, greater engagement, more positive affect, more peer mirroring, and treat the robot as more of a social other than children who played with a non-relational robot? Would children who reported feeling closer to the robot (regardless of condition) more learning and peer mirroring?

In this study, 50 kids played with either a relational or not relational robot. The relational robot was situated as a social contingent agent, using entrainment and affect mirroring; it referenced shared experiences such as past activities performed together and used the child's name; it took specific actions with regards to relationship management; it told stories that personalized both level (i.e., syntactic difficulty) and content (i.e., similarity of the robot's stories to the child's).

The not relational robot did not use these features. It simply followed its script. It did personalize stories based on level, since this is beneficial but not specifically related to the relationship.

Each child participated in a pretest session; 8 sessions with the robot that each included a pretest, the robot interaction with greeting, conversation, story activity, and closing, and posttest; and a final posttest session.

graph showing that children who rated robot as more social and relational also showed more learning

Results: Relationships, learning, and ... gender?

I collected a unique dataset about children's relationships with a social robot over time, which enabled me to look beyond whether children liked the robot or not or whether they learned new words or not. The main findings include:

  • Children in the \textit{Relational} condition reported that the robot was a more human-like, social, relational agent and responded to it in more social and relational ways. They often showed more positive affect, disclosed more information over time, and reported becoming more accepting of both the robot and other children with disabilities.

  • Children in the \textit{Relational} condition showed stronger correlations between their scores on the relationships assessments and their learning and behavior, such as their vocabulary posttest scores, emulation of the robot's language during storytelling, and use of target vocabulary words.

  • Regardless of condition, children who rated the robot as a more social and relational agent were more likely to treat it as such, as well as showing more learning.

  • Children's behavior showed that they thought of the robot and their relationship with it differently than their relationships with their parents, friends, and pets. They appeared to understand that the robot was an "in between" entity that had some properties of both alive, animate beings and inanimate machines.

The results of the study provide evidence for links between children's imitation of the robot during storytelling, their affect and valence, and their construal of the robot as a social-relational other. A large part of the power of social robots seems to come from their social presence.

In addition, children's behavior depended on both the robot's behavior and their own personalities and inclinations. Girls and boys seemed to imitate, interact, and respond differently to the relational and non-relational robots. Gender may be something to pay attention to in future work!

Ethics, design, and implications

I include several chapters in my dissertation discussing the design implications, ethical implications, and theoretical implications of my work.

Because of the power social and relational interaction has for humans, relational AI has the potential to engage and empower not only children across many domains—such as education, in therapy, and pediatrics for long-term health support—but also other populations: older children, adults, and the elderly. We can and should use relational AI to help all people flourish, to augment and support human relationships, and to enable people to be happier, healthier, more educated, and more able to lead the lives they want to live.

Further reading

Links

Publications

  • Kory-Westlund, J. M. (2019). Relational AI: Creating Long-Term Interpersonal Interaction, Rapport, and Relationships with Social Robots. PhD Thesis, Media Arts and Sciences, Massachusetts Institute of Technology, Cambridge, MA. [PDF]

  • Kory-Westlund, J. M., & Breazeal, C. (2019). A Long-Term Study of Young Children's Rapport, Social Emulation, and Language Learning With a Peer-Like Robot Playmate in Preschool Frontiers in Robotics and AI, 6. [PDF] [online]

  • Kory-Westlund, J. M., & Breazeal, C. (2019). Exploring the effects of a social robot's speech entrainment and backstory on young children's emotion, rapport, relationships, and learning. Frontiers in Robotics and AI, 6. [PDF] [online]

  • Kory-Westlund, J. M., & Breazeal, C. (2019). Assessing Children's Perception and Acceptance of a Social Robot. Proceedings of the 18th ACM Interaction Design and Children Conference (IDC) (pp. 38-50). ACM: New York, NY. [PDF]

  • Kory-Westlund, J. M., Park, H., Williams, R., & Breazeal, C. (2018). Measuring Young Children's Long-term Relationships with Social Robots. Proceedings of the 17th ACM Interaction Design and Children Conference (IDC) (pp. 207-218). ACM: New York, NY. [talk] [PDF]

  • Kory-Westlund, J. M., Park, H. W., Williams, R., & Breazeal, C. (2017). Measuring children's long-term relationships with social robots Workshop on Perception and Interaction dynamics in Child-Robot Interaction, held in conjunction with the Robotics: Science and Systems XIII. (pp. 625-626). Workshop website [PDF]

0 comments