Skip to content

Synthetizer Class

spessasus edited this page Sep 16, 2024 · 67 revisions

Synthetizer Class

This is the main module that generates the sound.

Yes, it's spelled "Synthetizer" and not "Synthesizer" here.

MIDI implementation chart

Tip

If you encounter any errors in this documentation, please open an issue!

Table of Contents

Importing

// normal install
import { Synthetizer } from "./spessasynth_lib/synthetizer/synthetizer.js";
// npm package
import { Synthetizer } from "spessasynth_lib";

Tip

Using the npm package? Make sure you've read this

Initialization

Caution

Note that you need to add the worklet processor for the synthesizer to work! See Importing the worklet

const synth = new Synthetizer(tagetNode, soundFontBuffer, enableEventSystem(optional), startRenderingData(optional), effectsConfig(optional));
  • targetNode - the AudioNode the synth should play to. Usually AudioContext.destination.
  • soundFontBuffer - the ArrayBuffer to your soundFont.
  • enableEventSystem - boolean, disables the event system. Useful when rendering audio to file
  • startRenderingData - object, used for rendering to file. It's formatted as follows:
    • parsedMIDI: a MIDI class instance. The synthesizer will immediately start rendering it if specified
    • snapshot: a SynthesizerSnapshot object, a copy of controllers from another synthesizer instance. If specified, synth will copy this configuration.
    • oneOutput: a boolean - indicates the One output mode
    • loopCount: the number of loops to play. Defaults to 0. Make sure your OfflineAudioContext's length accounts for these!
  • effectsConfig - optional, the configuration for audio effects. See below.

Effects configuration object

  • chorusEnabled - boolean - indicates if the chorus effect is enabled.
  • chorusConfig - object - the configuration for chorus. Pass undefined to use defaults. Described below.
    • nodesAmount - number - the amount of delay nodes (for each channel) and the corresponding oscillators.
    • defaultDelay - number - the initial delay, in seconds.
    • delayVariation - number - the difference between delays in the delay nodes.
    • stereoDifference - number - the difference of delays between two channels (added to the left channel and subtracted from the right).
    • oscillatorFrequency - number - the initial delay oscillator frequency, in Hz.
    • oscillatorFrequencyVariation - number - the difference between frequencies of oscillators, in Hz.
    • oscillatorGain - number - how much the oscillator will alter the delay in delay nodes, in seconds.
  • reverbEnabled - boolean - indicates if the reverb effect is enabled.
  • reverbImpulseResponse - AudioBuffer - the impulse response for the reverb. Pass undefined to use defaults.

Tip

Pass undefined to chorusConfig or reverbImpulseResponse to use the defaults.

Important

If you're rendering the audio to a file, it is highly recommended to supply the synthesizer with the reverb buffer (even the stock one) through effects config, because it internally fetches the file which will result in reverb enabling after a second or so.

isReady

A promise that gets resolved when the worklet synthesizer gets fully initialized (including sf3 support)

await synth.isReady;

Tip

It is recommended (but not always required) to wait for this promise.

Methods

Important

The synthesizer internally sends commands to the AudioWorklet where all the processing happens. Keep that in mind as not all methods will immediately report values! (E.g. noteOn won't instantly increase the voice count in channelProperties)

sendMessage

Sends a raw MIDI message to the synthesizer. Calls noteOn, noteOff, etc. internally.

synth.sendMessage(message);
  • message - an array of bytes (numbers from 0 to 255). The MIDI message to process.

Note

This only allows sending messages to the first 16 channels.

noteOn

Plays the given note.

synth.noteOn(channel, midiNote, velocity, enableDebugging);
  • channel - the MIDI channel to use. Usually ranges from 0 to 15, but it depends on the channel count.
  • midiNote - the note to play. Ranges from 0 to 127.
  • velocity - controls how loud the note is. 127 is normal loudness and 1 is the quietest. Note that velocity of 0 has the same effect as using noteOff. Ranges from 0 to 127.
  • enableDebugging - boolean, used only for debugging. When true, the console will print out tables of the soundfont generator data used to play the note.

noteOff

Stops the given note.

synth.noteOff(channel, midiNote);
  • channel - the MIDI channel to use. Usually ranges from 0 to 15, but it depends on the channel count.
  • midiNote - the note to play. Ranges from 0 to 127.

Note that when highPerformanceMode is set to true, the note will always have a release time of 50ms.

stopAll

Stops all notes. Equivalent of MIDI "panic".

synth.stopAll();

programChange

Changes the preset for the given channel.

synth.programChange(channel, programNumber);
  • channel - the MIDI channel to change. Usually ranges from 0 to 15, but it depends on the channel count.
  • programNumber - the MIDI program number to use. Ranges from 0 to 127. To use other banks, go to controllerChange.

pitchWheel

Changes the channel's pitch, including the currently playing notes.

synth.pitchWheel(channel, MSB, LSB);
  • channel - the MIDI channel to use. Usually ranges from 0 to 15, but it depends on the channel count.
  • MSB and LSB. 7-bit numbers that form a 14-bit pitch bend value.

setPitchBendRange

Changes the channel's pitch bend range, in semitones. Uses Registered Parameter Number internally.

synth.setPitchBendRange(channel, pitchBendRangeSemitones);
  • channel - the MIDI channel to use. Usually ranges from 0 to 15, but it depends on the channel count.
  • pitchBendRangeSemitones - the pitch bend range, in full semitones.

systemExclusive

Handles a MIDI System Exclusive message.

synth.systemExclusive(messageData);
  • message data - Uint8Array, the message byte data Excluding the 0xF0 byte!

Tip

Refer to this table for the list of supported System Exclusives.

controllerChange

Sets a given MIDI controller to a given value.

synth.controllerChange(channel, controllerNumber, controllerValue);
  • channel - the MIDI channel to use. Usually ranges from 0 to 15, but it depends on the channel count.
  • controllerNumber - the MIDI CC number of the controller to change. Refer to this table for the list of controllers supported by default.
  • controllerValue - the value to set the given controller to. Ranges from 0 to 127.

Note

Note that theoreticallly all controllers are supported as it depends on the SoundFont's modulators.

resetControllers

Resets all controllers to their default values. (for every channel)

synth.resetControllers();

lockController

Causes the given midi channel to ignore controller messages for the given controller number.

synth.lockController(channel, controllerNumber, isLocked);
  • channel - the channel to lock. Usually ranges from 0 to 15, but it depends on the channel count.
  • controllerNumber - the MIDI CC to lock. Ranges from 0 to 146. See the tip below to see why.
  • isLocked - boolean, if true then locked, if false then unlocked.

Tip

To lock other modulator sources add 128 to the Source Enumerator (Soundfont 2.04 Specification section 8.2.1) For example to lock pitch wheel, use synth.lockController(channel, 142, true). (128 + 14 = 142)

channelPressure

Applies pressure to the given channel. Usually this controls the vibrato amount.

synth.channelPressure(channel, pressure);
  • channel - the channel to use. Usually ranges from 0 to 15, but it depends on the channel count.
  • pressure - the pressure to apply. Ranges from 0 to 127.

polyPressure

Applies pressure to the given note on a given channel. Usually this controls the vibrato amount.

synth.polyPressure(channel, midiNote, pressure);
  • channel - the channel to use. Usually ranges from 0 to 15, but it depends on the channel count.
  • midiNote - the note to apply pressure to. Ranges from 0 to 127.
  • pressure - the pressure to apply. Ranges from 0 to 127.

muteChannel

Mutes or unmutes a given channel

synth.muteChannel(channel, isMuted);
  • channel - number- the channel to mute/unmute. Usually Usually ranges from 0 to 15, but it depends on the channel count.
  • isMuted - boolean - if the channel should be muted. boolean.

transpose

Transposes the synth up or down in semitones. Floating point values can be used for more precise tuning.

synth.transpose(semitones);
  • semitones - number - the amount of semitones to transpose the synth by. Can be positive or negative or zero. Zero resets the pitch.

setMainVolume

Sets the synth's main volume.

synth.setMainVolume(volume);
  • volume - number - the synth's volume. Ranges from 0 to anything, but 1 is the recommended maximum.

Note

Raising the gain above 1 can lead to unexpected results.

setInterpolationType

Sets the synth's interpolation method.

synth.setInterpolationType(type);
  • type - number - the interpolation type. Currently defined types:

  • 0 - linear interpolation (default)

  • 1 - no interpolation (nearest neighbor). Useful for songs like chiptunes.

addNewChannel

Adds a new channel. Invokes a newchannel event.

synth.addNewChannel();

reloadSoundfont

Caution

This function is deprecated. It may be removed without further notice. Use soundfontManager instead.

Changes the soundfont of a Synthesizer's instance.

await synth.reloadSoundFont(soundFontBuffer);

Important

This function is asynchronous.

  • soundFont - the soundfont to change to, an ArrayBuffer instance of the file.

getSynthesizerSnapshot

gets a current snapshot of the Worklet synthesizer.

Important

This function is asynchronous.

const snapshot = await synth.getSynthesizerSnapshot();

the returned value is formatted like this. It is essentially an object of the entire synthesizer instance from the audioWorklet side.

debugMessage

Prints out the synth class instance, both from the main thread and the AudioWorklet thread.

synth.debugMessage();

Properties

eventHandler

The synthesizer's event handler. Refer to Event handling for more.

soundfontManager

The synthesizer's soundfont manager. Refer to The soundfont manager for more.

voicesAmount

The current amount of voices (notes) playing or during their release phase.

console.log(`This synthetizer is currently playing ${synth.voicesAmount} notes!`);

voiceCap

The maximum allowed voices at once. If new voices are added, the voices considered unimportand are killed. Defaults to 450.

synth.voiceCap = 100; // max 100 voices at once

currentTime

The connected AudioContext's time.

console.log(`The current AudioContext's time is ${synth.currentTime}!`); // example usage

system

Indicates the current system the synth is in. Currently, there are: GM, GM2, GS, XG. Defaults to GS

console.log(synth.system); // "gm"

highPerformanceMode

Boolean, if the high performance mode is enabled. High performance mode currently overrides release time to be almost instant. Intended for "Black MIDIs".

synth.highPerformanceMode = true; // we can now play black MIDIs! >:)

channelProperties

The current channel properties. An array of objects formatted like this:

/**
 * @typedef {Object} ChannelProperty
 * @property {number} voicesAmount - the channel's current voice amount
 * @property {number} pitchBend - the channel's current pitch bend from -8192 do 8192
 * @property {number} pitchBendRangeSemitones - the pitch bend's range, in semitones
 * @property {boolean} isMuted - indicates whether the channel is muted
 * @property {boolean} isDrum - indicates whether the channel is a drum channel
 */
console.log(synth.channelProperties[0]); // {voicesAmount: 0, pitchBend: 0, pitchBendRangeSemitones: 2, isMuted: false, isDrum: false }

Event handling

The synthesizer supports event handling. For example the MidiKeyboard uses handling to visualize keypresses.

Use the property .eventHandler to access the system.

Adding event listener

eventHandler.addEvent(name, id, callback);
  • name - the type of the event. refer to the table below.
  • id - unique id for the event listener. Can be anything, as long as it's unique.
  • callback. a fuction that gets called on the event. Callback takes an object argument. The properties depend on the event type. Refer to the table below.

Removing event listener

eventHandler.removeEvent(name, id);
  • name - the type of the event.
  • id - the unique id of the event you wish to remove.

Event types

Name Description Callback Properties
noteoff Key has been released midiNote: number - the note that was released,
channel: number - the channel number which got the note released
noteon Key has been pressed midiNote: number - the note that was pressed,
channel: number - the channel that the note was played on,
velocty: the velocity of the note
pitchwheel Pitch wheel has been altered channel: number - the the channel that was pitch bent,
MSB: number - Most Significant byte of the message,
LSB: number - least significant byte of the message
controllerchange Controller has been changed channel: number - the channel that CC was changed on,
controllerNumber: number - the number of the MIDI controller list.
controllerValue: number - the new value of the controller
programchange Program has been changed channel: number - the channel that had its program changed
preset: Preset - The preset that was set
channelpressure Channel's pressure has been changed channel: number - the channel affected
pressure: number - the new pressure
polypressure Note's pressure has been changed midiNote: number - the note that was affected
channel: number - the channel affected
pressure: number - the new pressure
drumchange Channel's drum mode was changed channel: number - the channel,
isDrumChannel: boolean - if the channel is now a drum channel or not
stopall All voices were stopped None
newchannel A new channel was added to the synth None
mutechannel A channel has been muted/unmuted channel: number - the channel that was altered
isMuted: boolean - if the channel is muted or unmuted
presetlistchange The preset list has been changed/initialized The soundfont preset list. As follows: an array of objects: {presetName: the name of the preset, program: the preset's program number, bank: the preset's bank number}
allcontrollerreset All controllers have been reset (and programs!) None. Note: if there were any locked controllers, they will be restored via controllerchange event after. For example allcontrollersreset will be called and then controllerchange for the locked controller.
soundfonterror The loaded soundfont was invalid The error message from the SoundFont2 class

One output mode

This is a special synth mode, which causes the synth to have one output (instead of 18), but 32 channels.

Every midi channel has 2 audio channels. So it looks like this

  • MIDI channel 0:
    • audio output 0
    • audio output 1
  • MIDI channel 1:
    • audio output 2
    • audio output 3
  • MIDI channel 2:
    • audio output 4
    • audio output 5

etc.

This allows for many things, such as exporting files of individual channels. Example:

const midi = YOUR_MIDI_HERE;
const soundfont = YOUR_SOUNDFONT_HERE;
const sampleRate = 44100;
const offline = new OfflineAudioContext({
    numberOfChannels: 32,
    length: sampleRate * midi.duration,
    sampleRate: sampleRate
});
const synth = new Synthetizer(
    offline.destination, 
    soundfont,
    false, // disable event system for faster rendering
    {
        parsedMIDI: midi,
        oneOutput: true
    }
);
// render
const renderedData = await offline.startRendering();
for (let i = 0; i < 16; i++) 
{
    const audioOut = audioBufferToWav(buf, false, i * 2);
    // do whatever you want with the channel buffer
    console.log('Rendered channel', i, 'data:', audioOut);
}

Important

Chorus and reverb are disabled when the one output mode is on.

Caution

The OfflineAudioContext must be initialized with 32 channels! Otherwise there will be an error!

The Soundfont Manager

The soundfont manager allows for handling multiple soundfonts.

Every operation sends a new presetlist event.

Accessing it

Use the soundfontManager property.

synth.soundfontManager;

Accessing the list of soundfonts

synth.soundfontManager.soundfontList;

Which is a list of objects defined as follows:

  • id - string - the unique soundfont identifier.
  • bankOffset - number - the bank offset for the soundfont.

The list is ordered from the most important soundfont to the least (e.g. first soundfont is used as a base and other soundfonts get added on top (not override))

Important

When first creating the synthesizer, soundfontList contains one soundfont with the identifier main and bank offset of 0.

The behavior is defined as follows:

  • The program looks for the first soundfont that has the requested program:bank combo and uses it.
  • If not found, program looks for the first soundfont that has the requested program number and uses it.
  • If not found, program uses the first preset of the first soundfont.

Adding a new Soundfont

This function adds a new soundfont at the top of the soundfont stack.

await synth.soundfontManager.addNewSoundFont(soundfontBuffer, id, bankOffset = 0);
  • soundfontBuffer - ArrayBuffer - the soundfont binary data.
  • id - string - unique ID for the soundfont. Any string as long as it's unique.
  • bankOffset - number, optional - the bank offset for the soundfont.

Important

This function is asynchronous.

Removing a Soundfont

This function removes a specified soundfont.

synth.soundfontManager.deleteSoundFont(id);
  • id - string - unique ID for the soundfont to delete.

Changing the order of Soundfonts

This function reorders the soundfonts.

synth.soundfontManager.rearrangeSoundFonts(newOrderedList);
  • newOrderedList - array of string - The new list of the soundfont identifiers, in the desired order.

Clearing the Soundfonts

This function removes all soundfonts and adds a new one with id main and bank offset of 0.

await synth.soundfontManager.reloadManager(soundfontBuffer);
  • soundfontBuffer - ArrayBuffer - the new soundfont to reload the synth with.

Important

This function is asynchronous.

Clone this wiki locally