Scoring to Picture!
Game Composing’s Final Frontier:
Part 1: The Influence of Film Scores
“Are we there yet?”
Over the last two decades -since the advent of CD-ROMs- game composers have pursued orchestral instrumentation and filmic stylings, seeking the nuanced musicianship and epic nature of contemporary film scores. The ability to include recorded music in games also allowed a wider range of musical genres in game scores, with many development teams willing to experiment with musical aesthetics, perhaps more so than many contemporary film scores. In parallel, a stylistic resurgence of game music’s earliest roots, based on 80s arcade and classic chip-based music, has spawned its own music genre. Considering all this, writing game music has become a vastly rich and exceedingly vibrant environment in which to play!
The influence of film scores on game music continues now and will continue into the future (and there’s evidence of influence working in the other direction as well.) As game composers we draw from the same range of classical, romantic, and 20th century orchestral concert repertoire. In turn we’re also stylistically influenced by the great lineage of iconic film composers. We now record with many of the same A-list orchestras and musicians world-wide. Our orchestral recording techniques, hardware, software, and mixing are often on par with film. Many scores are mixed in surround. Our ancillary soundtrack recordings are compelling in and of themselves. Live concerts of game music are now a regular occurrence. Legions of game music fans are more dedicated than ever. And perhaps most notably, the caliber of raw composition skill among our ranks is simply staggering.
So what’s left? To be sure, our growth as composers never ends, but there’s one important lesson from film music that game composers have yet to truly apply to games: and that is scoring to picture!
Yes we score cinematic cut-scenes in great detail, because those are essentially short films embedded within the larger game narrative. Like film they’re linear, so the correlation to film is direct. But gameplay itself, the central aspect of this medium, is typically scored quite loosely to picture and gameplay. In essence, films are scored moment to moment, while games are scored minute to minute. Now, there have been many important advances in music design and implementation in recent years. The best games take great care with the spotting of music cues, and how the music is balanced with the rest of the audio. Adding/subtracting layers is common in order to parallel the games’ intensity/mood, and state based transitions are often employed to help music follow the narrative flow. We should be extremely proud of how far our discipline has come (and this year’s GDC audio track lineup continues to push on these boundaries.) But is that it, or is there more to learn from our older more experienced sibling, film music? Is there untapped potential for our scores to be even more integral to our games, the way film music is inextricably tied to film, moment to moment?
If you’ve seen, or been part of, a spotting session for a film, you know that while some moments are more important than others, the detail is often spelled out in terms of seconds (and even frames). Beyond that, film composers, in both overt and subtle ways, are scoring with an awareness of what’s on screen at each and every point along the way. Game composers, for reasons we’ll be exploring, haven’t had that luxury. More frequently than not, composers score gameplay blindfolded; seeing only screenshots or short game-captures, often never hearing their music in game until the game’s released!
You’ll notice I haven’t used the words interactive or adaptive music in this post (until now,) and that’s on purpose. I want to emphasize the creative goal, ‘scoring to picture,’ rather than the logistical techniques. Adaptive and interactive methods are simply tools; a means by which composers are able to score music to a dynamic narrative and nonlinear picture. Also, those terms tend to be over-interpreted as ’special’ concepts only for certain games and not others. The truth is that all game music is adaptive to one degree or another; including basic designs that feature 3 minute music loops that cross-fade between each other.
In our discipline, there’s often an unnecessary divide between those who compose, vs. those who design and integrate music. (Of course I’m speaking generally. There are many exceptions to this currently and historically.) Having a music team with various roles and functions is one thing, but in today’s environment composers have little influence on how their music will work against picture in games. Imagine if this were the case with film:
The implications of these questions bring up notable challenges and even deeper questions, which is what the subsequent parts of this writing will unpack:
Part 2: Rethinking the Culture and Marketplace of Game Composing
“Do sound designers like games more than composers?”
(coming soon)
Part 3: Applying Film Scoring Concepts to Dynamic Game Scores
“How do I get there from here, …and from here, and from way over there?”
(coming soon)
Hey Guy! This is very thoughtful work! Can’t wait to see Part 2 and Part 3.
Thanks Rudy! I’m looking forward to seeing parts 2 and 3 as well 🙂
In fact, the discussion and response to part 1 is influencing them.
Fantastic article, I look forward to parts 2 and 3.
I’m going to be touching on this on my Ori and the Blind Forest panel at GDC. It’s my belief that composers should spend as much time as possible (and as early as possible) playing the game and getting involved in the process early on in order to find the answers to the questions posed above in Guy’s post. It would be almost unheard of for a film composer to not watch the film in playback (usually with the director), why should it be different in games? This is especially important with single player narrative games.
Other reasons for getting involved early is you can find out how far you really want to go with adaptive/interactive music and thus you’re not constrained by a system that has already been designed with a billion layers/stingers/stems in mind. For some games, it’s just not necessary, others it is. But if the composer isn’t involved early on and doesn’t have a creative input on how the music will be integrated into the game, I’m not sure one’s best effort can be presented.
That’s not to say it can’t be done. I’m sure there are tons of great examples where the composer just handed off a lot of files and the score was well received, but I wonder how much better we could do if more composers were playing the game back constantly and constantly be in touch with designers to get those little ebbs/flows/arcs that are so prevalent in film, to be more readily felt by the game player. Composers should have a great sense/instinct for drama, and more could probably be done to harness that. Currently though, I think it is the exception, not the norm. It’s very obvious to see in a game where the composer has had quite a hands-on feel in the production/integration of music into the game.
I could go on for hours – anyway, look forward to seeing the rest of this. Thanks Guy!
Thanks Gareth! I couldn’t agree more with your ideas here. Yes, spending time with the game throughout the process is key as a composer, both for design ideas, but also to take the perspective of the player/participant (which is different than an observer). I perceive my music differently as a player than as an observer, though both are valid and needed POVs.
Even if composers simply consult on the music design (early) with the audio team, that would go a long way to making composition and design coherent, without a great deal of technical knowledge needed by the composer. It would bring the composers’ music sensibilities into play.
In part 3 I’ll be talking about the ‘spectrum of adaptibility’ (loose to tight scoring) and how, as you pointed out, different games will call for different scoring approaches.
and btw Gareth, well deserved props you’ve been getting for your score to Ori! may it continue.
Ideally, as a community, would we want an “industry standard” fine-line between the game mechanics and the audio it drives and a specific tool set that lives right on that line? Or would we want more of a gray zone where it would be established what roles would be cast to each department or individuals within those departments for a particular project? Should it start becoming expected for interactive composers to be at least somewhat familiar with an engine’s API to script their own audio cues? Or do we just want an interface (middleware) for working with basic cues set up by someone else on the development team?
In the book “Audio Programming for Interactive Games” published in 2004 by Martin Wilde, he states:
“There are two distinct disciples required to achieve great audio in a game. The first is composition – creating and assembling all the content that will be used in the game. The second is programming – designing and building a system to realize the audition of the composer in a game context” … “There are very few people who can perform both of these responsibilities well” … “We should let people do the jobs they’re good at, and not set them or our game up for failure by asking them to do more than they know how to do. Thus there should be (at least) two complementary audio professionals hired for each game: a composer and sound artist, and an audio programmer.”
This is essentially how we’re working on our own game, there are two of us. I’m building the integration tools that work with the game mechanics, she writes music that fits into the cues and various things I’m setting up for her. When she delivers assets, we sit together for a proper integration and mixing session while playing the game (in a sense, we are mixing, but not exactly “scoring” to picture), take notes on what did and didn’t work, what we would like to be able to do moving forward, and go back and update our workflow to fit the specific technical demands for the project. Even if we were using middleware, everything is so project specific that it would be impossible to do what we’re doing without someone constantly updating and tweaking audio-specific scripts. And it would be completely unreasonable to expect her, with no prior programming experience, to know how to do that.
Having not worked on any fancy-pants AAA teams, I’m curious to know if there is still this two-discipline outlook for audio teams, or if it’s becoming more common, due to more modern tools, for these lines to be blurred and the responsibility of seamless integration to rest more heavily on the composer/sound designer with whatever tools the rest of the team “allows” them to use.
Hi Brett, Really good points.
For standards, middleware (Wwise, FMOD, Elias, Fabric, et al) and 3rd party game engines (Unity, Unreal, etc) are becoming those de facto standards. Not surprisingly most of these tools didn’t even exist a decade ago, so the pipeline has evolved significantly, and will continue to evolve (thankfully). The standards will not be static 🙂
In between the composer and programmer, thanks to graphic interfaces and high level scripting, audio integration can be and is often done by sound designers and composers. This lessens the load and reliance on a traditional programmer; who is still very much needed, but can focus on lower level tasks and functionality. A two person team of programmer / composer is still a great option, but they can now more easily meet in the middle(ware) if desired.
…more on this specific topic in parts 2 and 3
The important point is that the composer at least has some creative input into how the music will work in game, moment to moment, even if someone else is doing the technical implementation. A lot of great designs come out of open collaboration, and we have to leverage people’s strengths and experience.
Yes, absolutely!