Create a sound design for a children’s book. The design involves not only the audio content but also the playback and control system that allows the reader to trigger the sounds in a simple, asynchronous way.
Activity Objectives
- The sound content is designed to have flexible in timing and asynchronous execution. In other words, the reader should be able to stay on a given page for as long as they want and the sound content that goes with that page will keep playing. The reader should also be able to skip ahead or back to any page and be able to hear the appropriate sound without having to start from the beginning.
- The method for triggering the sound should be simple and clear (a child should be able to do it without messing it up).
- The sound content helps to tell the story from the book in a way that enhances the experience and makes it more engaging for the reader.
Activity Goals
Ideally they would use technology that is well-suited to the task. A digital sampler, for example, is very good at looping audio and playing it in a way that it is randomly accessible. Accomplishing these same goals with a CD player would be challenging and not an appropriate application of the technology.
Hopefully they can also learn something about control protocols such as MIDI or OSC that can be used to trigger the sounds using a control surface that will be easy for the reader to understand. It is possible to do this by triggering the sounds using hot keys in QLab but then you’re showing the reader a lot more buttons than you need to.
Description
For this project you will create a sound design for a children’s book. Choose a book and create a sound design for it in Logic using sound effects and music. Use sound to help tell the story and enhance parts of the story that are not coming through clearly in the text. From Logic you will export the different elements of the sound design and program them into an EXS24 software sampler for playback. The sounds will be triggered with a PC1600x MIDI controller. The soundscape for the book should be continuous and any changes or new events must be triggered using a button on the PC1600x. Put a sticker or label on each page of the book indicating the button number that needs to be pressed.
- The sound should be programmed in the sampler in a way that allows the reader to stay on a single page for as long as they want without the sound moving ahead without them.
- The reader should also be able to jump to a page in the book out of sequence and still have the right sound happen when they push the button for that page.
- There should be a constant soundscape for the entire book. Use silence as a design tool, not as an excuse to not have to come up with something else.
You need to turn in a folder containing all the EXS Sampler Patch and Sampler files that need to be loaded into the sampler along with the .SYX file containing all the MIDI programming for the PC1600x controller.
You will present your project to the class.
Time Length
The project falls into three stages:
- The design of the actual sound content. I usually recommend that the students just lay out the whole thing in Logic using a pre-defined time for each page. Essentially you hit play and you hear the soundscape for the whole book from start to finish.
- Converting and programming the sound content into the sampler. This involves creating some loops, exporting individual effects to separate samples, and loading all the sounds into the sampler.
- Programming the controller. In this stage, the students sort out how to program the buttons on the control surface in a way that triggers the sounds they need.
So ideally you would devote three class periods to this. One for each of these stages. The students should show up to each class with an attempt already completed so you can spend the class period helping them kill the bugs.
Required Materials
When I do this project they use the Logic EXS24 sampler and a Peavey PC1600x controller. I think any hardware or software sampler would be up to the task. You could also do it in QLab if you really wanted to and just use MIDI triggers to recall the various sounds out of sequence.
The controller is tricky. The PC1600x is no longer available and there really isn’t anything else available that is as flexible. If you use a sampler you need to use a controller capable of sending multiple MIDI note on/note off messages with a single button. Each sound will be triggered with a different note and you may have multiple sounds per page. You could also use Max/Msp to collect the single note on/off events and send out the more complex messages to the sampler.
I think the absolute simplest, low-tech way to pull this off would be to set up the various sounds in QLab and trigger them using hot keys on the computer typing keyboard. You sacrifice some of the desired learning outcomes this way but the basic concepts are still there.
If you wanted to go really high-tech, you could embed sensors into the pages of the book so that the sound is automatically triggered when they turn the page.
Adaptation
You might be able to pair people up on this. Essentially have a designer and a programmer. This might allow you to play to the strengths of various students in the class.
Evaluation
Each student will perform their project in front of the class. Essentially they read their book and trigger their sounds. Then we have a discussion about the design choices and how effective they were in telling the story. Then I try to break it. I’ll open up a random page of the book and trigger that sound to see if the correct thing happens. Skip back to the beginning and see if the book will start correctly. See how long I can stay on a single page (their sound should keep going). Then as a group we look at their programming and see if there’s anything unique they did that the class can learn from.
Jason Romney, UNC School of the Arts; inspired by ideas from Maribeth Back and Elisheba Ittoop