CS & Media Arts combines artistic expression with emerging technologies

July 16, 2020

By Hannah Bernstein

Everyone has felt wonder and awe while watching an animated movie, enjoying the attention to detail and beautiful artistry. In Pixar’s 2017 hit Coco, for example, the guitar-playing characters’ fingers match the real chords, and in Disney’s 2012 movie Brave, Merida’s fiery red hair is composed of more than 1,500 individual curls.

Fun facts like that are exactly why second-year Jackie Allex loves her major. As a combined computer science and media arts student, she has the opportunity to use cutting-edge technologies to design and develop beautiful works of art in animation, virtual reality, and more.

“Finding out how those amazing graphics are designed as an animation student, and computing algorithms and programming CGI as a computer science student, has opened up so many doors for me,” Allex says.

The CS & Media Arts combined-major program is a partnership between Khoury and the College of Arts, Media, and Design, and allows students to explore new and ever-expanding computer science technology while receiving a foundational education in artistic expression. But the program provides more than what can be said on a brochure.

Arielle Bishop, a third-year student in the combined program, was very clear: Although many people may think she’s earning a degree in design, she’s in the program because she wants to be a digital artist, rather than a designer.

“There is a distinction between the idea of artistic expression through a more technical medium, which is media arts, and creating a specifically purposeful experience tailored to a certain audience, which is design,” Bishop explains.

Right now, Bishop is completing an internship at Cuseum, a company designed to help museums increase their engagement. She’s involved in their augmented reality project, developing tools for visitor experiences that complement the physical act of being in the building. For her, the program gives her the unique ability to exist in both the tech and art worlds at the same time, as technology’s power as an artistic medium begins to rise.

“We are uniquely positioned at a time where VR/AR and machine learning/AI are on the rise,” Bishop said. “Being able to study this area at this point in history leaves us uniquely positioned to catch the start of a new wave in art.”

Associate Dean of Students Benjamin Hescott said computer science and art share common goals in creativity and innovation, making them an ideal combination.

“In the information age, it’s critical to consider the technical and the artistic, and the combined major in computer science and media arts does both,” Hescott said. “Our students are combining art with technology in ways never done before, amplifying both disciplines and creating innovative solutions in the process.”

Andrea Raynor, the associate dean for undergraduate programs and professor in the College of Arts, Media and Design, agreed.

“It is important to understand that creativity and the sciences work in concert with each other,” Raynor said. “While students are engaged in the combined major of computer science and media arts, they have the opportunity to delve into the creative and expressive aspects of media arts while also integrating their knowledge of computer science.”

Applying computer science in the art world

Allex is working at the Human Movement Neuroscience Lab for her co-op. The lab specializes in exploring the neural bases of human movement using data collected with different techniques, such as motion capture, eye tracking, virtual reality, and computer vision. Allex has been creating 3D animations of human movement using motion capture data — and the researchers even brought circus performers into the lab to record their movements for the project.

“We use high-quality hardware and software, and look to improve problems that people often encounter with these tools,” Allex said, highlighting common technical problems like camera distortion. “[We want to] collect the best data to come to conclusions about the way human brains work to allow us to perform tasks, how our vision is related to our body movements, and more.”

Jennifer Der, a fourth-year student, has focused her computer science and media arts combined major on the intersections of photography and computer science. She came to Northeastern with no computer science or technology experience, and says it took a lot of courage to ask for help from professors and peers. But now, she sees an innate connection between the artistic workflow and the software development process.

“The steps in curating a collection of photographs are paralleled in the steps of planning, designing, and implementing a piece of software,” Der said. “The combined major allows me to make these connections because we are not pigeon-holed.”

In her three co-ops, Der has explored all sides of computer science. She first worked in quality assurance with the Massachusetts Medical Society, then middleware and API at a startup called Kyruus, and finally now as a full stack developer at Certain Lending.

“I feel equipped to either pursue careers as an artist, or a developer, or something completely different that combines both parts of my degree,” Der said. “Pursuing this combined degree has provided me with more opportunities than if I pursued either part of my degree separately.”

Experiential opportunities

Students in this program have also explored opportunities beyond coursework and co-op. Der and fourth-year student Raymond Huang have both participated in the student volunteer program at ACM’s annual computer graphics conference, SIGGRAPH.

Student volunteers are provided full access to the conference’s research presentations, industry talks, and technical seminars when they aren’t working. And, Huang has taken on leadership roles within the volunteer program to build his project and people management skills, too.

“I’m hoping my involvement [with the conference] will keep getting more intense,” Huang said. “I really love what it stands for and everything that goes on there.”

In 2018, Der had the opportunity to attend a talk with Pixar animators, where they talked about how they handled the large numbers of animated crowds required for the movie Coco. Instead of animating each crowd to perfect resolution but eventually reducing the pixels and image quality as they added perspective, they determined how to only generate the pixels they needed to make the crowd look realistic, speeding up development and rendering time. The project required knowledge of art, computer science, and mathematics to complete.

It’s that kind of innovation and experimentation that makes the intersection of technology and art such a dynamic place to be.

“For me, the power and joy of computing is in the tangible things you can create that would otherwise be impossible,” Huang said.