Shiffman Daniel. Learning processing
Подождите немного. Документ загружается.
382 Learning Processing
used to play a WA V or AIFF fi le. WA V is an audio fi le format that stands for “ Waveform audio format ”
and is the standard format for PCs. AIFF stands for “ Audio Interchange File Format ” and is commonly
used on Macintosh computers.
All of the functions we looked at in Chapter 16 are available for a sound fi le. e main diff erence, of
course, is that we cannot use image( ) to display the Movie object since there is no image.
Example 20-1: Simple sound with video library
import processing.video.*;
Movie movie;
void setup() {
size(200, 200);
movie = new Movie (this, " dingdong.wav " );
}
void draw() {
background(0);
noLoop();
}
void mousePressed() {
movie.stop();
movie.play();
}
Exercise 20-1: Using the bouncing ball sketch from Example 5–6, play a sound eff ect every
time the ball bounces off a window’s edge.
20.2 Getting Started with Sonia and Minim
e Movie class will do for playing simple sound eff ects, but for more sophisticated playback
functionality, we will need to look at libraries designed for use with sound. We will start with Sonia
( http://sonia.pitaru.com/ ). First things fi rst, as with any contributed library, you fi rst need the import
statement at the top of your code.
import pitaru.sonia_v2_9.*;
Sonia is an interface to a Java sound library, entitled Jsyn by Phil Burk. In order for it to work, Sonia has
to communicate with the audio playback and input hardware on your computer. is communication is
achieved via Jsyn. Before any sounds can be played, therefore, that link needs to be established, that is, the
sound engine needs to be started. is is done with the following line of code, which you would place in
setup( ) .
Construct a Movie object with a WAV
(or AIFF) fi le.
The sound is played whenever the mouse is
pressed. Including the function stop() before
play() ensures that the sound fi le will play
from the beginning.
Sound 383
void setup() {
Sonia.start(this);
}
Now, it would be rather irresponsible of us to make this connection to the sound hardware, but never
bother to close it down when we are fi nished. By now, the fl ow of a Processing sketch is all too familiar,
start with setup( ) , loop with draw( ) , and stop whenever you quit the sketch. e thing is, we want to
terminate the sound engine during that last step, when the sketch quits. Fortunately, there is an extra
hidden function we can implement entitled stop( ) that does precisely that. When a sketch quits, any
last minute instructions can be executed inside the stop( ) function. is is where we will shut down the
Sonia engine.
void stop() {
Sonia.stop();
super.stop();
}
Sonia.stop( ) is the call to stop Sonia. ere is also, however, the line super.stop( ) . e meaning of “ super ”
will be explored further in Chapter 22. For now, we can understand that line of code as saying “ Oh,
whatever else you would normally do in stop( ) , do that stuff , too . ”
All of the above applies to Minim as well. Here is the corresponding code:
import ddf.minim.*;
void setup() {
size(200, 200);
Minim.start(this);
}
void stop() {
super.stop();
}
20.3 Basic Sound Playback
Once this call to the library has been made, you have the whole world (of sound) in your hands. We
will start by demonstrating how to play sound fi les. Before a sound can be played, it must be loaded into
memory, much in the same way we loaded image fi les before displaying them. With Sonia , a Sample
object is used to store a reference to a sound sample.
Sample dingdong;
e object is initialized by passing the sound fi lename to the constructor.
dingdong = new Sample( "dingdong.wav");
Start the Minim sound engine!
With Minim, sounds will be closed
individually here. We will see this
in a moment.
The fi le “dingdong.wav” should be placed in
the data directory.
384 Learning Processing
Just as with images, loading the sound fi le from the hard drive is a slow process so the previous line of
code should be placed in setup( ) so as to not hamper the speed of draw( ) .
e type of sound fi le compatible with Sonia is somewhat limited. Only sounds that are formatted as WA V
or AIFF fi les (16-bit mono or stereo) are allowed. If you want to use a sound fi le that is not stored in a
compatible format, you could download a free audio editor, such as Audacity ( http://audacity.sourceforge.net/ )
and convert the fi le. Most audio fi les can be saved raw or with compression. Sonia only works with raw fi les.
However, in the next section, we will demonstrate how compressed (MP3) fi les can be played with the
ESS library.
Once the sound is loaded, playing is easy.
dingdong.play();
e following example plays a doorbell sound whenever the mouse is clicked on the graphical doorbell.
is Doorbell class implements simple button functionality (rollover and click) and just happens to
be a solution for Exercise 9–8 . e code for the new concepts (related to sound playback) is shown in
bold type .
Example 20-2: Doorbell with Sonia
// Import the Sonia library
import pitaru.sonia_v2_9.*;
// A sample object (for a sound)
Sample dingdong;
// A doorbell object (that will trigger the sound)
Doorbell doorbell;
void setup() {
size(200,200);
Sonia.start(this); // Start Sonia engine.
// Create a new sample object.
dingdong = new Sample( " dingdong.wav " );
// Create a new doorbell
doorbell = new Doorbell(150,100,32);
smooth();
}
void draw() {
background(255);
// Show the doorbell
doorbell.display(mouseX,mouseY);
}
fi g. 20.1
The play() function plays the sound sample once.
Sound 385
void mousePressed() {
// If the user clicks on the doorbell, play the sound!
if (doorbell.contains(mouseX,mouseY)) {
dingdong.play();
}
}
// Close the sound engine
public void stop() {
Sonia.stop();
super.stop();
}
// A Class to describe a "doorbell" (really a button)
class Doorbell {
// Location and size
float x;
float y;
float r;
// Create the doorbell
Doorbell (float x_, float y_, float r_) {
x = x_;
y = y_;
r =
r_;
}
// Is a point inside the doorbell (used for mouse rollover, etc.)
boolean contains(float mx, float my) {
if (dist(mx,my,x,y) < r) {
return true;
} else {
return false;
}
}
// Show the doorbell (hardcoded colors, could be improved)
void display(float mx, float my) {
if (contains(mx,my)) {
fill(100);
} else {
fill(175);
}
stroke(0);
ellipse(x,y,r,r);
}
}
386 Learning Processing
// A Class to describe a " doorbell " (really a button)
class Doorbell {
// Location and size
float x;
float y;
float r;
// A sample object (for a sound)
_________ ____________;
// Create the doorbell
Doorbell (float x_, float y_, float r_, _______ filename) {
x = x_;
y = y_;
r = r_;
_________ = new _________(_____________);
}
void ring() {
____________________;
}
boolean contains(float mx, float my) {
// same as original
}
void display(float mx, float my) {
// same as original
}
}
Exercise 20-2: Rewrite the Doorbell class to include a reference to the Sonia sample. is
will allow you to easily make multiple Doorbell objects that each play a diff erent sound.
Here is some code to get you started.
If you run the Doorbell example and click on the doorbell many times in rapid succession, you will notice
that the sound restarts every time you click. It is not given a chance to fi rst fi nish playing. While this
is not much of an issue for this straightforward example, stopping a sound from restarting can be very
important in other, more complex, sound sketches. e simplest way to achieve such a result is to always
Sound 387
check and see if a sound is playing before you call the play( ) function. Sonia has a nice function called
isPlaying( ) which does exactly this, returning true or false. We want to play the sound if it is not already
playing, that is,
if (!dingdong.isPlaying()) {
dingdong.play();
}
Exercise 20-3: Expand the Doorbell class to animate (perhaps moving its location and
changing its size randomly) only while its sound is playing. Create an array of fi ve Doorbell
objects. Here is part of a function to add to the Doorbell class:
void jiggle() {
if (_________________)) {
x += ____________;
y += ____________;
_________________;
}
}
With Minim instead of Sonia, very little changes. Instead of a Sample object, we have an AudioPlayer
object. e following example includes the solutions for Exercises 20-2 and 20-3 (but uses the Minim
code, rather than Sonia). It does not implement an array, however.
Example 20-3: Doorbell with Minim
import ddf.minim.*;
// A doorbell object (that will trigger the sound)
Doorbell doorbell;
void setup() {
size(200,200);
// start up Minim
Minim.start(this);
// Create a new doorbell
doorbell = new Doorbell(150,100,32, "dingdong.wav");
smooth();
}
Remember, “!” means not!
388 Learning Processing
void draw() {
background(100,100,126);
// Show the doorbell
doorbell.display(mouseX,mouseY);
doorbell.jiggle();
}
void mousePressed() {
// If the user clicks on the doorbell, play the sound!
if (doorbell.contains(mouseX,mouseY)) {
doorbell.ring();
}
}
// Close the sound files
public void stop() {
doorbell.close();
super.stop();
}
class Doorbell {
// Location and size
float x;
float y;
float r;
// An AudioPlayer object
AudioPlayer dingdong;
// Create the doorbell
Doorbell (float x_, float y_, float r_, String filename) {
x = x_;
y = y_;
r = r_;
// load " dingdong.wav " into a new AudioPlayer
dingdong = Minim.loadFile(filename);
}
// If the " doorbell " is ringing, the shape jiggles
void jiggle() {
if (dingdong.isPlaying()) {
x + = random( – 1,1);
y + = random( – 1,1);
r = constrain(r + random( – 2,2),10,100);
}
}
// The doorbell rings!
void ring() {
if (!dingdong.isPlaying()) {
dingdong.rewind();
dingdong.play();
}
}
The doorbell object must close its sound.
An AudioPlayer object is used to store
the sound.
The doorbell only jiggles if the
sound is playing.
The ring() function plays the
sound, as long as it is not
already playing. rewind()
ensures the sound starts from
the beginning.
Sound 389
// Is a point inside the doorbell (used for mouse rollover, etc.)
boolean contains(float mx, float my) {
if (dist(mx,my,x,y) < r) {
return true;
} else {
return false;
}
}
// Show the doorbell (hardcoded colors, could be improved)
void display(float mx, float my) {
if (contains(mx,my)) {
fill( 126,114,100);
} else {
fill(119,152,202);
}
stroke(202,175,142);
ellipse(x,y,r,r);
}
void close() {
dingdong.close();
}
}
One of the advantages of using Minim over Sonia is that Minim supports MP3 fi les. MP3 (or “ MPEG-1
Audio Layer 3 ” ) fi les are compressed and therefore take up much less hard drive space than raw WAV or
AIFF sound fi les.
AudioPlayer dingdong = Minim.loadFile( "dingdong.mp3");
20.4 A Bit Fancier Sound Playback
During playback, a sound sample can be manipulated in real time. Volume, pitch, and pan can all be
controlled using Sonia and Minim .
Let’s start with volume in Sonia . A Sample object’s volume can be set with the setVolume( ) function,
which takes a fl oating point value between 0.0 and 1.0 (0.0 being silent, 1.0 being the loudest). e
following snippet assumes a sample named “ tone ” and sets the volume based on mouseX position (by
normalizing it to a range between 0 and 1).
float ratio = (float) mouseX /width;
tone.setVolume(ratio);
Pan works the same way, only the range is between 1.0 (for left) and 1.0 (for right).
float ratio = (float) mouseX /width;
tone.setPan(ratio*2 – 1);
e pitch is altered by changing the rate of playback (i.e., faster playback is higher pitch, slower
playback is lower pitch) using setRate( ) . You can set the sample playback rate to anything you like, a
The doorbell has a close() function
to close the AudioPlayer object.
Volume ranges from 0.0 to 1.0.
Pan ranges from –1.0 to 1.0.
390 Learning Processing
fairly comprehensive range is from 0 (where you would not hear it at all) to 88,200 (a relatively fast
playback rate).
float ratio = (float) mouseX / width;
tone.setRate(ratio*88200);
e following example adjusts the volume and pitch of a sound according to mouse movements. Note the
use of repeat( ) instead of play( ) , which loops the sound over and over rather than playing it one time .
Example 20-4: Manipulating sound (with Sonia)
// Import the Sonia library
import pitaru.sonia_v2_9.*;
// A Sample object (for a sound)
Sample tone;
void setup() {
size(200,200);
Sonia.start(this); // Start Sonia engine.
// Create a new sample object.
tone = new Sample( " tone.wav " );
// Loop the sound forever
// (well, at least until stop() is called)
tone.repeat();
smooth();
}
void draw() {
if (tone.isPlaying()) {
background(255);
} else {
background(100);
}
// Set the volume to a range between 0 and 1.0
float ratio = (float) mouseX /
width;
tone.setVolume(ratio);
// Set the rate to a range between 0 and 88,200
// Changing the rate alters the pitch
ratio = (float) mouseY / height;
tone.setRate(ratio*88200);
// Draw some rectangles to show what is going on
stroke(0);
fill(175);
rect(0,160,mouseX,20);
stroke(0);
fill(175);
rect(160,0,20,mouseY);
}
fi g. 20.2
A reasonable range for rate (i.e., pitch)
is between 0 and 88,200.
The volume is set according
to the mouseX position.
The rate is set according to
the mouseY position.
Sound 391
// Pressing the mouse stops and starts the sound
void mousePressed() {
if (tone.isPlaying()) {
tone.stop();
} else {
tone.repeat();
}
}
// Close the sound engine
public void stop() {
Sonia.stop();
super.stop();
}
Exercise 20-4: In Example 20-4, fl ip the Y-axis so that the lower sound plays when the
mouse is down rather than up.
An AudioPlayer object can also be manipulated with Minim using the functions: setVolume( ) ,
setPan( ) , and addEff ect( ) , among others documented on the Minim site ( http://code.compartmental.
net/tools/minim/ ).
20.5 Live input
In Chapter 16, we looked at how serial communication allows a Processing sketch to respond to input
from an external hardware device connected to a sensor. Reading input from a microphone is a similar
pursuit. In essence, the microphone acts as a sensor. Not only can a microphone record sound, but it can
determine if the sound is loud, quiet, high-pitched, low-pitched, and so on. For example, a Processing
sketch could determine if it is living in a crowded room based on sound levels, or whether it is listening
to a soprano or bass singer based on pitch levels. is section will cover how to retrieve and use sound
volume data using the Sonia library. For analyzing sound pitch levels from a microphone, visit the Sonia
web site ( http://sonia.pitaru.com ) for further examples.
e previous sections used a Sample object to play a sound. Sound input from a microphone is retrieved
with a LiveInput object. ere is a somewhat odd distinction here in the way we will use these two
classes that we have yet to encounter over the course of this book.
Consider a scenario where we have three sound fi les. We would create three Sample objects.
Sample sample1 = new Sample( "file1.wav");
Sample sample2 = new Sample( "file2.wav");
Sample sample3 = new Sample( "file3.wav");
Technically speaking, we have made three instances of Sample objects, born via the Sample class. If we
want to play a sound, we have to refer to a specifi c Sample object.
sample1.play();
The sound can be stopped
with the function stop().