Brown CS Blog

Why Choreorobotics

Click the links that follow for more news about Sydney Skybetter, Stefanie Tellex, and other recent accomplishments by our faculty and students

by Megan Gessner

My name is Megan and I belong to a weird cohort of highly-specific nerds who are artists but also techies but also care about humanity but also think about the philosophical and ethical implications of their work. I am a professional dancer/choreographer who is finishing up a Master's in computer science. Over the past few years, I’ve been trying to bridge the gap between dance and computer science in a variety of projects – using a Kinect sensor and Unity to make a really hacky version of Just Dance, building a 3D convolutional neural network to recognize tap dance steps that have been segmented with audio (on the way to automatic transcription and notation of tap dance), immersing myself in the research literature landscape of movement computation, playing with motion capture systems, and now being part of the Choreorobotics team. This team was assembled by Professors Sydney Skybetter and Stefanie Tellex in service of developing the curriculum for the emergent Choreorobotics 101 course, cross-listed in the CS and TAPS departments. 

Honestly, I have been doing all this because I think it’s Rad™. And because I refuse to give up on any of the things I’m passionate about, so I’m synthesizing them. What really has captured my imagination is the creative potential of incorporating computation into the choreographic process, and allowing dancerly notions of embodiment inform how machines can represent, generate, analyze, and visualize movement. I dream about an immersive feedback system that can activate my improvisatory and creative practices – this could take the form of abstract visualizations of movement that react to what I’m doing (and which I then respond to, ad infinitum), or suggestions of new movement possibilities when I’m stuck trying to compose. I want to learn what patterns are ingrained in my body and how I might break out of those patterns to find new ways of moving. Maybe these suggestions could come from a robotic dance partner, who works with me in the studio :) 

But in order to achieve these goals (which I’ve now learned are actually not at all trivial), lots of really neat computational problems emerge. How should we encode movement? How can we generate new movement? How do we compare and analyze new movement to a database of other movement, and how do we efficiently retrieve this data? What is the best way to visualize such high dimensional data so that a dancer/choreographer may learn and use that information? Beyond all this, can a computer really capture the human essence and emotional vigor of a dance performance? When can we call something *dance* (I argue it’s all movement, the act of living in an embodied way itself means we always dance, we always plan and then execute a choreography of motions that takes us through space and time). 

These complicated and sort of subjectively/ambiguously-defined research challenges must all follow task number one, which has been to simply “make a robot dance”. To learn how to do this, I’ve been engaging with the Choreorobotics 101 syllabus readings and poring through documentation and tutorials for ROS (robotics operating system) and its available packages for motion planning, studying dynamic movement primitives, as well as the software stack for the Kinova MOVO. Making a robot dance requires knowledge of the hardware underlying the robot, control theory, motion capture technology, skeletal and kinematic representation, and a healthy dose of systems programming and software design, but it also requires the more fleshy skills of movement notations and improvisational technologies (digital and embodied) like those of William Forsythe and Rudolf Laban, and reflecting on how to design choreographic interfaces to be inclusive of all people so that they may all dance with and create movement for robots, à la Data Feminism and Design Justice. This is all to say, I’ve needed to reach across disciplines to make this research feasible, thinking critically about design, computational creativity, engineering, software, computer vision, visualization, dance aesthetics and creative practices, and performance theory, and this is exactly what this new Choreorobotics discipline, and the new course’s syllabus, brings together.

Maybe you agree with me now that this stuff is Rad™. This is what Choreorobotics is. It’s studying bodies moving through the lens of how to artificially recreate these complex processes, and allowing art and technology to intermingle to produce new questions and new answers.

For more information, click the link that follows to contact Brown CS Communication and Outreach Specialist Jesse C. Polhemus.