Consilience Project Teaches Technology by Capturing Students' Motion...and Attention

Wiley 5th grader engaged during the Consilience team's presentation about motion capture.
A Wiley 5th grader is engaged during the Consilience team's presentation about motion capture..

May 21, 2014

Kids today are already familiar with motion capture technology...they just might not know it.

So when Wiley School 5th graders recently visited Krannert, Bruno Azeredo and Sandra Lopez from Illinois’ Consilience Project explained how it works. First students discovered that it's what tracks their movements when they play Wii and Kinect. Then they saw a clip of Avatar filmmakers using it to capture actors’ movements and facial expressions to animate the “avatars.” Already familiar with these technologies, the students immediately connected the dots…and were ready to experience it for themselves.

Strapping on motion sensors, the students then participated in some activities which took hands-on learning to a whole new level. Just like the actors in Avatar, students’ motions activated the sensors. As Lopez had them “dance,” the computer program translated their movements into patterns of colored light on a monitor, as well as sound. And based on how slow or how fast students moved, they could change the speed of the “music.” Another fun activity was a software program that played different musical instruments triggered by both the speed of students’ movements and their location within the circle of motion detector equipment.

The fact that many of the kids had already experienced a similar technology when playing Wii and Kinect was evidently a plus.

Lopez works with a small group of 5th graders during their visit to Krannert.
Lopez works with a small group of 5th graders during their visit to Krannert. Students are equipped with black velcro wrist bands to which the motion sensors have been attached.

Says Lopez, “I definitely think they have an affinity for the technological aspects of it, because it’s something that they can relate to.”

In addition, Yankee Ridge students who had earlier visited Beckman’s Illinois Simulator Laboratory (ISL) also got to play with Kinect. Lopez say this hands-on experience helped students understand that the game console was “kind of a watered down version for your living room” of what they were getting to experience in the lab.

Lopez describes how one of the highlights of the lecture explaining the engineering and technological aspects of motion capture was when presenters showed a youtube clip about how the movie Avatar was made.

“When they realize this is the same thing that was used for that, then that’s a source of excitement.”

Bruno Azeredo
Bruno Azeredo explains the principles of motion capture during an instructional session to a group of Wiley 5th graders.

The idea of using dance movements to help teach STEM wasn’t in the forefront of Bruno Azeredo’s mind when he arrived in Champaign back in 2011 to begin his graduate program. The Mechanical Science and Engineering (MechSE) Ph.D. student got involved with dance recreationally. (According to Lopez, he’s really good at the tango.) But he soon discovered that dancing isn’t just for recreation anymore, but that it can also be useful in STEM education.

Upon learning about Professor Kirstie Simson's work in dance, and that she and MechSE professor Armand Beaudoin taught a class in engineering and dance, Azeredo says they “began a conversation about dance and its relevance to engineering education.” Says Azeredo, "We converged on two very important points:
(1) STEM education needs to include the arts, and
(2) dancers are a phenomenal example through which we can explore a new integrated educational approach at the undergraduate level."

The fourth member of the team, Camille Goudeseune  of Beckman’s Illinois Simulator Laboratory (ISL), is a kindred spirit who uses technology as art. Goudeseune’s research is key to the project, as it involves producing sound by means of computer software, focusing on the interface between performer, instrument-controller, and sound synthesizer.

Thus, funded by a Focal Point grant (Illinois Graduate College awards to promote interdisciplinary research), the Consilience Project was born in Fall of 2012. Not only has the team developed educational materials, but they have taught classes to undergraduate and K-12 students, including the students from Wiley and Yankee Ridge schools, who participated in the project’s outreach efforts.

Sandra Lopez appreciates the Wiley students' antics as they experiment with motion capture
Sandra Lopez appreciates Wiley students' antics as they experiment with motion capture.

Sandra Lopez, from the Dance Department, was brought on board for the project’s second year to help the team reach their education goals. Responsible for developing the outreach program and working with schools, Lopez says she was interested in Consilience because it would expose kids to something that they had never seen or experienced.

“Part of what I try to do through my art is to give people new experiences, and this kind of equipment is super expensive and is not something that you run into, especially in a school. So I was very eager to find a way to incorporate movement elements and theatrical elements in an activity that is going to allow kids to be part of something that they usually don’t get to be a part of.”

Could a project like this influence kids to choose STEM careers? Lopez believes the exposure makes STEM more accessible. “It gives them that idea like, ‘Oh, this is actually made by people like me or could be me later on.’ So I think that it is a big component—especially having the actual programmers talk to them about how they did it. I think that’s very neat.”

What part of the program engaged kids the most? “Actually putting on the markers, seeing what they are doing and hearing how they are changing—that’s what would get them the most excited.”


A Wiley student enjoying the experience.

The students who participated in Consilience thought it was awesome too.

“I would ask them at the end, ‘What do you think? How was this as a learning experience?’ Lopez reports, "and they would say, ‘Oh, this was awesome. It was so much fun.’ They get to do something more experiential; that makes a difference.”

What did Lopez get out of it personally?

“For me,” she admits, “the most rewarding part was working with the kids and seeing their reactions to being a part of something like this. I think they often don’t get to do things like this with expensive equipment or going to theaters and things like that. There’s this stigma—'Because they’re kids, they’re going to mess something up.' So having them in places where they are not usually allowed and being able to move differently—to me was the best part of the project.”

Regarding the schools chosen to participate in the project, it’s no accident that Wiley School students got involved.

“We have sort of a push in our school district to infuse arts into everything," admits Wiley music, dance, and drama teacher, Cara Maurizi, "So this was sort of on a grand scale of infusing the arts.”

Plus, Wiley has a history with Krannert. According to Maurizi, a few years ago, the school partnered with Krannert’s Art Museum piloting a week-long project for 5th graders integrating art, reading, science, math, drama, and music. “And we thought, ‘What if we could take that to the performing arts center and do a similar project?’”

Cara Maurizi, Wiley art teacher
Cara Maurizi, Wiley arts teacher.

Maurizi indicates that one of the school’s core philosophies is interdisciplinary learning, especially when it involves the arts.

“We have found that learning increases. They make connections. So when things are separated, it is harder to see how it relates to them in their everyday lives. But when we infuse the arts into things that they are already doing in the classroom, it seems like that retention is longer; they can make better connections; they can think in a more higher-order way versus just these separate units. So it had been working really well for us.”

So what does Maurizi believe her students learned about technology through the Consilience project? “I would say they’re learning a lot about how computers are capable of doing so much,” says Maurizi.

The teacher also agreed that her students’ familiarity with Wii and Kinect helped them relate to the project.

“We talked at the end of our dance session today that the goal for these engineers is to produce music without instruments and with dancers, you could actually produce sound. They were able to make connections right away between the video games that they know like Wii and Kinect, so it was cool for them to see how that can be taken to another level.”

Story and photographs by Elizabeth Innes, Communications Specialist, I-STEM Education Initiative.
More: K-6 Outreach, Wiley School, 2014

Wiley 5th graders experiment with motion capture technology on a stage at the Krannert Center.
Wiley 5th graders experiment with motion capture technology on a stage at the Krannert Center.




Nobel Project’s End-of-Year Zoom Bash Recaps Learning

February 1, 2022
The STEM Illinois Nobel Project held a special, end-of-the-year Zoom event celebrating its participating students’ achievements.
Full Story

It’s not magic, it’s physics

January 26, 2022
In Franklin STEAM Academy, Musical Magnetism program makes STEM fun, approachable.
Full Story

Program prepares STEM educators to teach all students

November 30, 2021
This summer, a group of educators gathered to learn about engaging STEM activities they can do with their students.
Full Story

Research Experiences for Undergraduates (REU) program

November 11, 2021
Undergrads get a taste of research through I-MRSEC’s REU program.
Full Story

Goldstein’s Renaissance Engineering Summer Camp

November 1, 2021
Goldstein’s Renaissance Engineering Summer Camp Incorporates Art, Design, Mechatronics, and Mentoring
Full Story

TechTogether Chicago to Redefine the Hacker Stereotype

July 10, 2021
New workshops that can help inspire students to pursue careers in technology..
Full Story

Aerospace Engineering Launches Virtual Summer Camps to Pique Students’ Interest in Aero.

July 2, 2021
Design an aircraft then watch it soar after launching it with a huge rubber band. Build a Mars lander to safely transport a real egg, then test the contraption by dropping it from a second story window.
Full Story

Undergrads Experience Materials Science Research Courtesy of the I-MRSEC REU

June 16, 2021
Ten undergraduate students are spending the summer of 2021 discovering what research is like.
Full Story

MatSE Afterschool Academy

MatSE Afterschool Academy

June 14, 2021
MatSE Afterschool Academy to Introduce Students to Materials Science and Beyond.
Full Story

Taylor Tucker Embraces Multidisciplinary Interest

Taylor Tucker Embraces Multidisciplinary Interest

June 14, 2021
Taylor Tucker Embraces Multidisciplinary Interest While Researching Task Collaboration.
Full Story

Exposes Franklin Middle Schoolers to Science, CS

What Studying Engineering at Illinois is Like?

May 25, 2021
NSBE’s Michaela Horn Exposes Franklin Middle Schoolers to Science, CS, and What Studying Engineering at Illinois is Like.
Full Story

Jenny Saves a Convertible.

Children’s-Book-Writing Duo/

May 19, 2021
Convertibles and Thunderstorms—Children’s-Book-Writing Duo on Their Way Thanks to Illinois Training and Encouragement from Mentors.
Full Story

Improve Learning in Engineering

Improve Learning in Engineering

May 17, 2021
Liebenberg Espouses Mini-Projects to Engage Students Emotionally, Improve Learning in Engineering.
Full Story

Joshua Whitely makes an adjustment to the 3D Bioprinter during the demo.

BIOE435 Capstone Projects

May 12, 2021
BIOE435 Capstone Projects - BIOE Seniors Use Knowledge/Skills to Problem Solve.
Full Story

Elani and Gonzalo shine a UV light on a rose that has absorbed a solution that has made it fluorescent.

Illinois Scientists Shine a (UV) Light on Fluorescence

May 7, 2021
What is fluorescence? What causes it?
Full Story

Joshua Whitely makes an adjustment to the 3D Bioprinter during the demo.

HackIllinois 2021 “Rekindled Connections” With The Tech Community

May 5, 2021
Annual student hackathon HackIllinois with the aim of developing projects on current problems facing society.
Full Story

A Shane Mayer-Gawlik image of the Bridger Aurora, part of his Night Skies photography collection exhibited at the Art-Science Festival.

The Art-Science Festival

April 26, 2021
Illinois Art-Science Festival: Illuminating the Universe...from the Quantum World to the Cosmos.
Full Story

Joshua Whitely makes an adjustment to the 3D Bioprinter during the demo.

Illinois Engineering Seniors Prepared to Change the World

April 22, 2021
Ready. Set. Go! Illinois Engineering Seniors Prepared to Change the World.
Full Story

HML 2021 Virtual Health

HML 2021 Virtual Health

April 19, 2021
Make-a-Thon Gives Citizen Scientists a Shot at Making Their Health-Related Innovations a Reality.
Full Story

I-MRSEC’s Music Video

I-MRSEC’s Music Video

April 7, 2021
I-MRSEC’s Music Video for EOH ’21 Plugs Graphene, 2D Materials
Full Story

Health Make-a-Thon Orientation

HML 2021 Health Orientation

March 30, 2021
HML 2021 Health Make-a-Thon Orientation Prepares Finalists for Competition.
Full Story

Andrea Perry shows Franklin students how to take apart the magnetic drawing board they received in their kit

Musical Magnetism

March 25, 2021
Musical Magnetism: Encouraging Franklin Middle Schoolers to Express Science Via the Arts.
Full Story

Carmen Paquette street performing.

Love of Science

March 9, 2021
Paquette Conveys Her Love of Science, Dance to Franklin STEAM Students Via Musical Magnetism.
Full Stroy

An Engineering Exploration participant exhibits the tower they built as part of the engineering challenge related to Civil Engineering

Engineering Exploration

March 2, 2021
SWE’s Engineering Exploration Outreach Lives Up to Its Name.

ChiS&E’s Family STEM Day

ChiS&E’s Family STEM Day

February 23, 2021
Helps Chicago Youngsters Progress Along the STEM Pipeline Toward Engineering.

Kathny Walsh

Kathy Walsh

February 17, 2021
On Her First Foray into STEAM, Kathy Walsh Acquaints Franklin Students with Microscopy, Haiku.

ChiS&E student

ChiS&E CPS Students

January 19, 2021
Illinois Undergrads Encourage ChiS&E CPS Students Toward Possible Careers in Engineering.

I-MRSEC’s Music Video

CISTEME365 Provides Year-Round PD/Community

January 4, 2021
to Illinois Teachers in Support of Informal STEM Education Efforts to Underserved Students.