Cozmo is a table-top robot released last October by Anki. Recently, DesigningMusicNow got to interview composers Brian Trifon and Brian Lee White of Finishing Move Inc. and award winning composer Gordy Haab about the music behind Cozmo. Gordy Haab is a GANG award winning composer whom has also scored such game titles as Star Wars: Battlefront, Star Wars: The Old Republic, Kinect Star Wars & The Walking Dead: Survival Instinct. Brian Trifon and Brian Lee White of Finishing Move Inc. have worked on many of the award winning games including Halo Wars 1 & 2.
The following interview questions for the Cozmo music team were prepared by Michael Sweet’s advanced interactive video game scoring course at Berklee College of Music. The student composers included Ross Warren Alexander, Dominic Delore, Kaela Fanelli,Timothy Przybylinski, and Ian Silver.
As an introduction to Cozmo, and to help acquaint you with some of the audio aspects of Cozmo, we’ve included the music and sound diary post from the Cozmo development team below:
1. Can you describe the individual roles of the sound team?
The music team was made up of Gordy Haab and Finishing Move, Gordy handled the composition and orchestration while Finishing Move did implementation and acted as music supervisors, this was all headed up on Anki’s side by their audio director Brian Min. Anki handled all of the sound design and vocalizations internally, with Ben Gabaldon driving Cozmo’s sound design and voice.
2. In the past you had quite a bit of experience working on game music, how did you go from game music to robot music? What was the connection to Anki?
Finishing Move has been working with Anki on their first product, Overdrive, for a couple years now. Overdrive has a totally different musical style, much more electronic and EDM driven. When they approached us about Cozmo and told us about the music direction they wanted to take, we knew we had to get Gordy involved because of his stellar orchestral chops. We had already been working together on Halo Wars 2, so our two teams were spooled up and comfortable collaborating, so we told Anki about bringing on Gordy and everything just clicked into place.
3. What are the biggest challenges of providing sounds for a robot, instead of a game?
With normal console games, or even films for that matter, years of tried methods have rendered a somewhat dependable formula – or at least a set of expectations. With films you’re dealing with a finite timeline, and with games there are certain parameters we’ve all become accustomed to working within. But Cozmo is alive, ever-changing, and has a mind of his own! So it was challenge to fully understand how the music would work with his behavior without first writing the music completely, and then play testing it. And since Cozmo was constantly evolving for everyone involved, even our best musical ideas may or may not work out as planned once it became a part of Cozmo. Composing for a robot is really uncharted territory, so we all had to be creative and open minded – and honest with ourselves if something was not working. There was quite a bit of trial and error to get Cozmo’s sound just right.
4. What kinds of things are you trying to achieve with the music underscore during the gameplay, emotionally, or from a story perspective?
We wanted the music to be fun, playful, and a bit snarky in order to fit Cozmo’s personality. Rather than compose music that solely accompanied the players experience, we wanted to compose music that accompanied Cozmo’s experience. We all thought scoring from Cozmo’s perspective would add to his human quality. So if the player wins a game against Cozmo, you may not necessarily hear victorious music for your win – but rather, music that fits Cozmo’s aggravation.
5. How many assets does Cozmo come with in terms of dialog assets, music assets, and sfx? How many are on the device versus the phone?
We just handled the music side of Cozmo, so we can’t speak to the other assets, but there is roughly 25 minutes of music, mostly horizontally implemented looping cues with transitional elements. The unique thing about having 2 speakers, one on the phone and one on the bot is that you can get some cool separation of the music versus his vocalizations. So the music and UI sound design plays from the phone speaker while his vocalizations come directly from the speaker on the bot. If you think of it in terms of a live play performance, Cozmo is the actor on stage and the phone acts as the orchestra pit, scoring his actions.
6. What was the development like? Can you describe the process of getting the sounds on the device for testing? Did you have a virtual Cozmo for a testing environment?
The whole music team did a few onsites to get a sense of Cozmo in person, and Brian White did most of the music implementation on site at Anki. Unfortunately, beyond testing things in Wwise with Soundcaster, there was no virtual way to know exactly how it would feel on the bot in a 3D physical space without pushing a build and actually testing it on device. Early on, the device wasn’t as stable as it is now, so being on site with engineering support was critical, because often an engineer would have to help work through device issues that weren’t related to Wwise or the music system.
7. Can you tell us a little about the Wwise states you used in Wwise for the robot? e.g. bored, busy, chill – which we saw in your video diary. Approximately how many variations per emotion state does Cozmo have to choose from when speaking?
Can’t really say on the voice or Sound design component, again we just focused on music. We used a combination of states and switches to drive the Wwise interactive music system. Everything is built off a single play/stop event and lives in a single music switch container, which holds all the cue/playlist associations and transition rules based on state/switch status. There are roughly 50 or so states/switch combinations at the moment that drive both things like mood as well as the minigames you can get into.
8. Are you running Wwise on just the phone, or is Wwise also running on the robot?
Wwise runs on the phone and streams audio to the bot, but not without some challenges along the way, AudioKinetic did some custom work to get the audio to buffer correctly from the phone to Cozmo’s speaker.
9. Can Cozmo run without the phone app running, or are some of the assets dependent on the phone connectivity?
At the moment Cozmo always requires a connection the phone to operate.
10. What limitations did you have on the sonic palette due to the playback device?
Because music playback is always going to happen on a smartphone or tablet speaker, and not even via headphones because you want to be able to hear Cozmo’s speaker, you run into your typical mobile game low bandwidth scenarios such as little to no low end, limited stereo or mono only playback, and the inherent frequency peaks in small speakers that can cause certain instruments to pop out too much on the device (think bell sounds, etc). Most of this stuff can be overcome and balanced out by just previewing the mixes on a mobile device. Of course there are some tools that let you stream your DAW output to a smartphone, but usually it’s just a matter of dropping an asset on dropbox and streaming it back off your phone real quick to check the translation.
11. What kinds of synthesis and effects plugins were used to create the sounds?
Well with the music, we actually chose to not synthesize the music, but rather use a small ensemble of live instruments in order to further add to the human-like quality. Cozmo’s “band” is 13 musicians, each adding their own personality to a somewhat ragtag overall sound. And the music was recorded live with some of the greatest musicians in Hollywood.
12. Did you just compose using the full orchestral palette and let the mastering do the rest?
Yes, it just happened that the type of ensemble we decided would work well with Cozmo’s personality also works well on small speakers. Cozmo uses a very small orchestra recorded in an intimate environment, not a blown out epic film orchestra, which helps with clarity and translation on the phone. Traditional small ensemble instruments actually translate surprisingly well on a mobile device, so it’s more about the composition and arrangement than any production or mix trick that you might find used in a more pop music context. For example, a properly recorded acoustic bass has rich harmonic overtones that allow it to speak in a low bandwidth context with little additional processing. Ultimately, we mainly focused on high quality source recordings and performances to carry the score, nothing inherently fancy with regards to the mix or mastering process.
13. How many iterations of sfx and music did you go through before you arrived at the final version? How long did you work on the project?
We worked on the music on and off for about 4 months. And the music went through many changes as Cozmo changed prior to release.
14. How is Cozmo different on day 2 versus day 1? Does his personality change over time? Did this impact the sounds that you created in terms of repetition?
We don’t know how he’ll differ, because each user’s experience will be different, since Cozmo adapts to his surroundings and people he meets. He may react differently to us than to you based on how his experience playing with you was!
15. Is this a new career path for sound designers and composers? What are the specific unique skills that are different for a project like this verses a traditional game?
We hope so! As to a set of unique skills, time will tell. First and foremost I’d say adaptability, open-mindedness, and a willingness to completely let go and just have fun!
16. What skills should students be learning at college to work on a job like this?
I suppose that would vary depending on if they wanted to move more into the composition side of things or into sound design and implementation. But even you just want to compose for games, you definitely want to be familiar with middleware like wwise, how interactive music gets pieced together and the differences between composing linear music versus music that will be used interactively. The more you understand the entire pipeline the more you can adjust the specifics aspects of your job to the bigger picture of game development.
If you’d like more information about Anki’s Cozmo, please visit the following websites:
Bio: Brian Lee White of Finishing Move, Inc.
Reformed guitarist turned composer and producer, Brian Lee White spent his earlier years working at his father’s small music shop in the mountains of central California. Between stringing guitars and repairing dented band instruments, he developed a passion for computers and recording, sparking a lifelong obsession with music technology.
After graduating from UCSC, and despite receiving a perfectly respectable degree in business management, Brian somehow managed to avoid any kind of “real job” and spent the next decade producing records, composing for ad campaigns and authoring educational content and online courses for Lynda.com. In 2012, having had his fill of the freelancer’s life, Brian formed Finishing Move, Inc. with longtime friend and collaborator Brian Trifon, where the duo began lending their unique sound and talents to games and interactive music. Now with several high profile projects under their belt, including a 2014 GANG award for Best Instrumental, Finishing Move is excited to share their latest work for the highly anticipated Halo Wars sequel being released by Microsoft Studios in February 2017 along with the album HYPECINE Vol. 1, for which they just released on all digital stores and streaming outlets. The album, which is geared towards music editors and trailer companies for licensing, consists of cinematic bass music and hybrid scores.
Brian lives in California with his wife Emma and their two mischievous cats (Liam and Leeloo.) When he’s not crunching on a deadline with Finishing Move or herding cats around the studio, he occasionally writes music for fun under the artist alias Gasmilk, having released a string of singles as well as a debut EP, Beautiful Things.
Finishing Move’s credits include, Halo: The Master Chief Collection, Halo 2: Anniversary, The Halo Channel (Microsoft/343i), Cozmo (Anki), Massive Chalice (Double Fine), Overdrive (Anki), Tilt Brush (Google), Drop Chord (Double Fine).
Bio: Brian Trifon of Finishing Move, Inc.
With dreams of becoming a heavy metal guitar god, Brian spent his childhood making a racket with his electric guitar and small collection of FX pedals. Brian’s interest in distortion pedals, reverb units and recording gear eventually led to a deep love and curiosity for electronic music production and musical sound design. After graduating from USC’s prestigious Thornton School of Music with a degree in Guitar Performance and working alongside Grammy nominated electronic artist BT, Brian spent the next decade composing music for film, TV, games, and ad campaigns, as well as producing several influential electronic albums under the name Trifonic. Some of the most notable work being featured on mix albums alongside tracks by Massive Attack, Trentemøller, Hybrid and Sasha and being licensed for tv shows such as CSI: Crime Scene Investigation, and BBC’s Top Gear.
In 2012, Brian formed Finishing Move, Inc with longtime friend and collaborator Brian Lee White, after realizing that by combining their complementary skills, they could tackle demanding, large scale projects. Under Finishing Move the duo began lending their niche sound and talents to games and interactive music. With several high profile projects including Halo: The Master Chief Collection, Halo 2: Anniversary, The Halo Channel (Microsoft/343i), Cozmo (Anki), Massive Chalice (Double Fine), Overdrive (Anki), Tilt Brush (Google), Drop Chord (Double Fine) and a GANG award for Best Instrumental under their belt, they are currently working on the much anticipated Halo Wars sequel for Microsoft Studios, due to be released February 2017.
Bio: Gordy Haab
Gordy Haab is a multi award-winning film, video game and television composer who has written music for many well-known titles, including most recently: Electronic Arts’ best-selling “Star Wars: Battlefront” which received seven nominations and four wins at the 2016 G.A.N.G. Awards—earning, “Music Of The Year,” “Best Interactive Score,” “Best Audio Mix” and “Sound Design Of The Year.” Activision/AMC’s “The Walking Dead: Survival Instinct”, based on the #1 hit TV series, Microsoft’s “Kinect: Star Wars”, which won “Best Music” at the 2012 Hollywood Music In Media Awards, and EA/Bioware/LucasArts’ highly popular “Star Wars: The Old Republic,” for which he was awarded “Best Original Soundtrack” and “Best Instrumental Music” at the 10th Annual G.A.N.G. Awards, presented at the Game Developers Conference in San Francisco. In October 2016, it was announced Gordy would be scoring Microsoft Studios “Halo Wars 2” alongside Brian Trifon & Brian Lee White of Finishing Move Inc.
Known for his unsurpassed understanding of the orchestra and his creative ability to blend contemporary and traditional styles, Haab recently combined the “scarier” side of the orchestra with Appalachian bluegrass instruments to create a frightening and backwoods-ey score for “The Walking Dead”. Just prior, Haab paid homage to composer John Williams’ iconic score for George Lucas’ “Star Wars” by composing and recording more than two hours of new music for “Kinect: Star Wars” at the famed Abbey Road Studios with the London Symphony Orchestra and Choir. Haab’s goal was to make the game’s music both unique and familiar to “Star Wars” fans, in order to create a seamless listening experience from the movie to the game, while still taking fans to new musical places. To accomplish this he used the same orchestra, choir, studio, microphones, equipment and studio layout as used for Williams’ original recording sessions more than 30 years ago.
A few of Haab’s other film, television and video game credits include: Anchor Bay’s “Behind the Mask: The Rise of Leslie Vernon”; MTV’s “The Truth Below”; Dave Barry’s “Guide to Guys”; Lionsgate’s “War”; Dream Entertainment’s “3 Below”; Roadside Attractions’ “Shrink”; TLC’s “Little People Big World”; ABC’s “Greek”; NBC’s “Kath and Kim”; VH-1’s “Scream Queens”; Endeavor’s “At the Edge of the World”; LucasArts’ “Indiana Jones and the Staff of Kings”, and many others.
In 2007, Haab scored the Internet phenomenon, “Ryan vs. Dorkman II” – recently featured in the Huffington Post and named the “Top Star Wars Derivative Film Of All Time” by TIME Magazine. The five-minute film, scored with a full orchestra at Capitol Records, received more than 7 million views on YouTube, including views from LucasArts, which is how they first learned of Haab’s work.
Haab is currently the co-head of orchestral music and original scoring for “DP Music,” an exclusive music content provider for Lionsgate Pictures, and is published by Cherry Lane Music Publishing. He also co-founded the Novo Philharmonic Orchestra for which he currently acts as composer and artistic director. Novo is a 95-member rock/pop/film orchestra in Los Angeles, comprised of some of the world’s most cutting-edge musicians.
Haab is a graduate of the University of Southern California, where he received his master’s in Scoring For Motion Pictures, Television and Other Media. Prior, he received a bachelor’s degree in Music Composition at Virginia Commonwealth University. While he learned from many composers in university and conservatory settings, Haab says that most of his music education comes simply from, “Playing in 100’s of rock bands and being an avid fan of Hitchcock, Kubrick, Lucas and Spielberg films . . . “B” Horror Movies of the 70’s and 80’s . . . and all of their great scores.”