What I learned about Sound Design from Visual Novels
Visual novels are a medium that exists in the space between books, comics, TV shows, and games. Depending on the budget and ambition of the developers, you get titles that veer close to other mediums, blurring genre lines. Almost every step from a bare-bones wall of text to a full-on animated TV show exists as a separate title. Not only that, but they also blend multiple genres of game into them, from point and click adventure, to strategy, fighting games, and so on. Visual novels are so stripped of distraction that any element added or missing stands out clearly. It's a medium where the audience has a low expectation of production value, so the creators are allowed to cut out elements that would in a video game be an unacceptable compromise. Thanks to this, visual novels taught me just how powerful even the most basic level of sound design can be.
In a book, the text has to do all the work to paint a picture in your mind, not just of what a scene looks like, but the sound and smell of being there. A visual novel simply strips these words out by having a picture show you what it's supposed to look like. It's the same during dialog. Characters changing expressions, pausing for emphasis, and so on can all be precisely managed by the creator as they script how each line and mouse click trigger changes on screen. It's a different kind of writing, that leaves less to the reader's imagination. But that doesn't mean your imagination isn't active. Because you as a reader is still filling in the blanks and painting an entire world around the characters that isn't explicitly shown by the creators. And the sound is a powerful piece of that puzzle.
You might not realize it, but a big part of skillful play of any game is using sound to tell what's happening off-screen. If you cross the street in an open world game, you don't need to look left and right before you cross, because once you step onto the road you can hear cars slowing down and coming to a stop. You don't need triple monitors or VR to get a wider view of what's happening in the game, just pay attention to the sound and your perception extends beyond the frames of the display.
Unfortunately, there are more examples of poor sound design in visual novels than there are of good. It's not unexpected, given how large many of these projects can be. Writers in Japan are paid by the kilobyte size of the text script. This has resulted in many visual novels that can take 30-50+ hours to finish. This hasn't stopped some titles from being ambitious with their sound design, but it's not surprising most would decide it's simply too much work for something the audience and reviewers will only notice on a subconscious level and never give them credit for.
As an example, the clip below compares Cinders (Moa Cube, 2012) and Muv-Luv Alternative (Age, 2006). Cinders has no ambient sound whatsoever, which is especially harmful to its presentation because it has animated visual elements in the background. When I played it, the absence of sound jumped out at me, because there was a mental mismatch between what I was seeing and hearing. In Muv-Luv Alternative, on the other hand, there are no animated backgrounds, but it does have ambient and incidental sound effects, which breathes life into the still images. The sound tells me what the scene would have looked like if there had been animation. This makes my brain fill in the blanks in a more natural manner and doesn't create the dissonance I felt in Cinders. Note how the text consists of only an internal monolog, and how the visual and audio elements are used to tell the story of the world around him.
Another good example of how sound can be used to amplify storytelling is in Fate/Stay Night (Type-Moon 2004). This is a story that has been adapted into multiple TV series, movies, and spin-offs. But I think the original visual novel is the most interesting and effective story told in its universe. In a late part of the story, the protagonist has to use a weapon that gradually destroys his memories. To convey the effects of memory loss the game makes sharp cuts with static noise to indicate where gaps in his memory exist. Once you're familiar with this technique, it's used without any additional explanation of what's happening. If you hear static and the scene suddenly changed, you know that part of memory was lost, and because it doesn't explain or linger on this, the technique becomes more powerful. And as memory continues to degenerate, gaps appear in the script to indicate how the character is forgetting names of people and places.
In the next clip, I'll highlight a couple more good examples of sound design. The first is Chaos;Head (5pb, 2008). The character is sitting in front of a computer, with fans and hard drives crackling. When the camera spins around in a smooth motion, you hear that he's spinning on a chair. This serves as a nice excuse to why the camera motion is so even. These are all simple little sounds that add up to create a coherent and believable scene. The second example is Quartett (Littlewitch, 2004). It uses a visual style reminiscent of comic books and is of note because it cuts between multiple characters in different locations. Thanks to the sound design, you can tell it takes place in a city during Christmas, and how people are gathering outside the church. Inside the church, they didn't draw any people, but you can hear their low murmurs. It's easy to overlook how important the ambient sound is during a sequence of images like this, as they just wash over you while you're looking at the characters and reading the dialog. But they're critical in expanding the experience of the world of the story, without having to stop and spell things out.
What I learned from visual novels is that our ears are always open, waiting to be fed information. When we're thrown even the most basic scraps of sound, it has a dramatic effect on how three dimensional and believable the world of that story feels. Ideally, all the tools used by games and movies should be interweaved, so that music, writing, visuals, and sound design are crafted and tuned with the intent to tell a specific story. Where each moves in and out of center stage for maximum effect. What visual novels did for me, is zoom in on how important the basics of sound are, as they provided examples where it's easy to focus on what's missing, without being distracted by everything else. It trained me to be a better listener, as I'm now better at noticing lackluster sound design in other games, where I otherwise might have been too distracted by the game happening on screen.