soundit

Make audio

This module uses iterators to represent audio streams. These iterators return float values between [-1.0, 1.0) and can be chained, averaged, and precomputed to your heart’s content to create music. We call these iterators “sounds”.

Note that all sounds are in 48kHz.

There is also a kinda sus music parser which can aid in creating longer music. More info on that can be found in the music_to_notes’s docstring.

Sound generators:

sine square sawtooth triangle silence piano (requires init_piano to be called)

Sound creation utilities:

passed

Sound effects:

fade volume cut pad exact delay

Frequency utilities:

make_frequencies_dict make_indices_dict

Music functions:

split_music music_to_notes notes_to_sine _notes_to_sound (unfinalized API)

Audio source utilities:

chunked unchunked

We also provide some utility functions with other tools such as converting chunks into discord.py AudioSources or decompressing audio on the fly with FFmpeg. These only work when their required library is installed.

discord.py utilities:

wrap_discord_source unwrap_discord_source play_discord_source

FFmpeg utilities:

file_chunks make_ffmpeg_section_args create_ffmpeg_process chunked_ffmpeg_process

sounddevice utilities:

play_output_chunks create_input_chunks

There is also some built-in music that are prefixed with MUSIC_, such as MUSIC_DIGITIZED, provided for testing purposes.

Classes

LRUCache	An LRU cache
LRUIterableCache	An LRU cache for iterables
DiscordIteratorSource	Internal subclass of discord.py's AudioSource for iterators

Functions

silence	Returns silence
sine	Returns a sine wave at freq
square	Returns a square wave at freq
sawtooth	Returns a sawtooth wave at freq
triangle	Returns a triangle wave at freq
init_piano	Loads the piano sound for use
piano	Returns a piano sound at index
file_chunks	Returns a sound from an audio file using FFmpeg
lru_iter_cache	Decorator to wrap a function returning iterables
create_ffmpeg_process	Creates a process that run FFmpeg with the given arguments
chunked_ffmpeg_process	Returns an iterator of chunks from the given process
make_ffmpeg_section_args	Returns a list of arguments to FFmpeg
loop_stream	Consumes a stream of buffers and loops them forever
equal_chunk_stream	Normalizes a stream of buffers into ones of length buffer_len
passed	Returns a sound lasting the specified time yielding the seconds passed
fade	Fades in and out of the sound
both	Deprecated. sound.chunked accepts floats
single	Merge stereo sounds into mono
volume	Multiplies each point by the specified factor
cut	Ends the sound after the specified time
pad	Pads the sound with silence if shorter than the specified time
_cut_cross	End sound at the first sign flip after the specified time
exact	Cuts or pads the sound to make it exactly the specified time
delay	Add silence before the sound
_resample_linear	Resample the sound using linear interpolation
play_discord_source	Plays and waits until the source finishes playing
wrap_discord_source	Wraps an iterator of bytes into an audio source
chunked	Converts a stream of floats or two-tuples of floats in [-1, 1) to bytes
unwrap_discord_source	Converts an audio source into a stream of bytes
unchunked	Converts a stream of bytes to two-tuples of floats in [-1, 1)
make_frequencies_dict	Makes a dictionary containing frequencies for each note
make_indices_dict	Makes a dictionary containing note indices of common note names
music_to_notes	Converts music into notes (two tuples of note name and length)
split_music	Splits music into individual sequences
notes_to_sine	Converts notes into sine waves
_notes_to_sound	Converts notes to a sound using the provided func
play_output_chunks	Plays chunks to the default audio output device
create_input_chunks	Returns chunks from the default audio input device
reload	Reloads this module. Helper function

Attributes

has_discord
has_sounddevice
has_av
has_numpy
RATE
A4_FREQUENCY
A4_INDEX
NOTE_NAMES
piano_data
MUSIC_DIGITIZED
MUSIC_MEGALOVANIA
MUSIC_DIGITIZED_DUAL

Documentation

has_discord = False

has_sounddevice = False

has_av = False

has_numpy = False

RATE = 48000

A4_FREQUENCY = 440

A4_INDEX = 57

NOTE_NAMES

silence()

Returns silence

sine(freq=A4_FREQUENCY)

Returns a sine wave at freq

square(freq=A4_FREQUENCY)

Returns a square wave at freq

sawtooth(freq=A4_FREQUENCY)

Returns a sawtooth wave at freq

triangle(freq=A4_FREQUENCY)

Returns a triangle wave at freq

piano_data

init_piano()

Loads the piano sound for use

The raw file 0.raw was generated from Online Sequencer’s Electric Piano instrument (from https://onlinesequencer.net/app/instruments/0.ogg?v=12) and FFmpeg was then used to convert it into a raw mono 48kHz signed 16-bit little endian file (using ffmpeg -i 0.ogg -f s16le -acodec pcm_s16le -ac 1 -ar 48000 0.raw).

piano(index=A4_INDEX)

Returns a piano sound at index

file_chunks(filename: str, start: float = 0)

Returns a sound from an audio file using FFmpeg

Parameters:

• filename – path to the audio file

• start – seconds into the audio to start at

Returns:

stream of two-tuples of floats decoded from the audio file

class LRUCache(*, maxsize: int | None = 128)

An LRU cache

Parameters:

maxsize – the maximum size of the cache (None means unbounded)

maxsize

The maximum size of the cache

0 means that the cache will remain empty. Specifying None means the cache will grow without bound.

Note that changes to maxsize won’t take effect until the next get() call with a key not in the cache. It is not recommended, but you can call ._ensure_size() to force it to resize the cache.

hits = 0

The number of cache hits

misses = 0

The number of cache misses

results: dict

The dictionary between keys and values

Checking and modifying the cache manually isn’t recommended.

get(key, value_func)

Return the value for this key, calling value_func if needed

Parameters:

• key – unique key for a value

• value_func – a zero-arg function returning a value for this key

clear()

Clears the cache and the hits / misses counters

class LRUIterableCache(*, maxsize: int | None = 128)

Bases: LRUCache

An LRU cache for iterables

This class internally stores itertools.tee objects wrapped around the original values and returns a copy of the tees in the cache.

We use tee objects for a few reasons:

• They can be iterated at different speeds.

• They are iterators (lazily evaluated).

• They can be copied (major orz for this one).

• They are fast (implemented in C).

See LRUCache for more info on caching. See itertools.tee for more info on tee objects.

get(key, iterable_func)

Return the iterator for this key, calling iterable_func if needed

If a matching tee is cached, return a copy. If no tee is found, create a new one using the iterable_func and make a copy of it.

lru_iter_cache(

func=None,

*,

maxsize: int | None = 128,

cache: LRUIterableCache | None = None,

)

Decorator to wrap a function returning iterables

Parameters:

• maxsize – the maximum size of the cache (None means unbounded)

• cache – the cache to use

See LRUIterableCache for more info.

create_ffmpeg_process(

*args,

executable='ffmpeg',

pipe_stdin=False,

pipe_stdout=True,

pipe_stderr=False,

**kwargs,

)

Creates a process that run FFmpeg with the given arguments

This assumes that ffmpeg.exe is on your PATH environment variable. If not, you can specify its location using the executable argument.

For the pipe_* arguments, if it is True, subprocess.PIPE will be passed to subprocess.Popen’s constructor. Otherwise, None will be passed.

All other keyword arguments are passed directly to subprocess.Popen.

chunked_ffmpeg_process(

process: Popen,

*,

close: bool | None = True,

) → Iterator[bytes]

Returns an iterator of chunks from the given process

Parameters:

• process – the subprocess to stream stdout from

• close – whether to terminate the process when finished

This function is hardcoded to take PCM 16-bit stereo audio, same as the chunked function. See that function for more info.

make_ffmpeg_section_args(

filename,

start,

length,

*,

before_options=(),

options=(),

)

Returns a list of arguments to FFmpeg

It will take the required amount of audio starting from the specified start time and convert them into PCM 16-bit stereo audio to be piped to stdout.

The before_options argument will be passed after -ss and before -i, and the options argument will be passed after -t and before pipe:1.

If length is None, the audio will play to the end of the file.

The returned args are of this form:

-ss {start}
-t {length}
{before_options}
-i {filename}
-f s16le
-ar 48000
-ac 2
-loglevel warning
-nostdin
{options}
pipe:1

loop_stream(

data_iterable: Iterable[bytes],

*,

copy: bool | None = True,

when_empty: Literal['ignore', 'error'] | None = 'error',

) → Iterator[bytes]

Consumes a stream of buffers and loops them forever

Parameters:

• data_iterable – the iterable of buffers

• copy – whether or not to copy the buffers

• when_empty – what to do when data is empty (ignore or error)

Returns:

stream of buffers

The buffers are reused upon looping. If the buffers are known to be unused after being yielded, you can set copy to False to save some time copying.

When sum(len(b) for b in buffers) == 0, a RuntimeError will be raised. Otherwise, this function can end up in an infinite loop, or it can cause other functions to never yield (such as equal_chunk_stream). This behaviour is almost never useful, though if necessary, pass when_empty=”ignore” to suppress the error.

Example

>>> from itertools import islice
>>> parts = [b"abc", b"def", b"ghi"]
>>> looped = list(islice(loop_stream(parts), 9))
>>> looped[::3]
[b'abc', b'abc', b'abc']
>>> looped[1::3]
[b'def', b'def', b'def']
>>> looped[2::3]
[b'ghi', b'ghi', b'ghi']

equal_chunk_stream(

data_iterable: Iterable[bytes],

buffer_len: int,

*,

copy: bool | None = True,

) → Iterator[bytes]

Normalizes a stream of buffers into ones of length buffer_len

Parameters:

• data_iterable – the iterable of buffers

• buffer_len – the size to normalize buffers to

• copy – return copies of the internal buffer. If False, the yielded buffer may be reused to reduce object creation and collection.

Returns:

stream of buffers with len(buffer) == buffer_len except the last one

The last buffer yielded is always smaller than buffer_len. Other code can fill it with zeros, drop it, or execute clean up code.

Example

>>> list(equal_chunk_stream([b"abcd", b"efghi"], 3))
[b'abc', b'def', b'ghi', b'']
>>> list(equal_chunk_stream([b"abcd", b"efghijk"], 3))
[b'abc', b'def', b'ghi', b'jk']
>>> list(equal_chunk_stream([b"a", b"b", b"c", b"d"], 3))
[b'abc', b'd']
>>> list(equal_chunk_stream([], 3))
[b'']
>>> list(equal_chunk_stream([b"", b""], 3))
[b'']
>>> list(equal_chunk_stream([b"", b"", b"a", b""], 3))
[b'a']

passed(seconds=1)

Returns a sound lasting the specified time yielding the seconds passed

This abstracts away the use of RATE to calculate the number of points.

If seconds is None, the returned sound will be unbounded.

Example

>>> x = list(passed(0.25))
>>> x[0] * RATE
0.0
>>> x[1] * RATE
1.0
>>> len(x) / RATE
0.25

fade(iterator, *, fadein=0.005, fadeout=0.005)

Fades in and out of the sound

If the sound is less than fadein + fadeout seconds, the time between fading in and fading out is split proportionally.

both(iterator)

Deprecated. sound.chunked accepts floats

single(iterator)

Merge stereo sounds into mono

volume(factor, sound)

Multiplies each point by the specified factor

cut(seconds, sound)

Ends the sound after the specified time

pad(seconds, sound)

Pads the sound with silence if shorter than the specified time

_cut_cross(seconds: float, sound)

End sound at the first sign flip after the specified time

exact(seconds, sound)

Cuts or pads the sound to make it exactly the specified time

delay(seconds: float, sound)

Add silence before the sound

_resample_linear(factor: float, sound)

Resample the sound using linear interpolation

The returned sound will be shorter/longer by 1 over the specified factor. Resampling a sound created by cut(1, sine(440)) would give a sound similar to one created by cut(1/factor, 440*factor).

async play_discord_source(voice_client, source)

Plays and waits until the source finishes playing

class DiscordIteratorSource(iterator, *, is_opus=False)

Bases: discord.AudioSource

Internal subclass of discord.py’s AudioSource for iterators

See wrap_discord_source for more info.

is_opus()

cleanup()

read()

wrap_discord_source(iterator, *, is_opus=False)

Wraps an iterator of bytes into an audio source

If is_opus is False (the default), the iterator must yield 20ms of signed 16-bit little endian stereo 48kHz audio each iteration. If is_opus is True, the iterator should yield 20ms of Opus encoded audio each iteration.

Example

# source implements discord.AudioSource
source = wrap_discord_source(chunked(cut(1, sine(440))))
ctx.voice_client.play(source, after=lambda _: print("finished"))

chunked(sound)

Converts a stream of floats or two-tuples of floats in [-1, 1) to bytes

This is hardcoded to return 20ms chunks of signed 16-bit little endian stereo 48kHz audio.

If the sound yield float instead of two-tuples, it will have both sides play the same point.

If the sound doesn’t complete on a chunk border, null bytes will be added until it reaches the required length, which should be 3840 bytes.

Note that floats not in the range [-1, 1) will be silently truncated to fall inside the range. For example, 1.5 will be processed as 1 and -1.5 will be processed as -1.

unwrap_discord_source(source)

Converts an audio source into a stream of bytes

This basically does the opposite of wrap_discord_source. See that function’s documentation for more info.

unchunked(chunks)

Converts a stream of bytes to two-tuples of floats in [-1, 1)

This basically does the opposite of chunked. See that function’s documentation for more info.

make_frequencies_dict(*, a4=A4_FREQUENCY, offset=0)

Makes a dictionary containing frequencies for each note

• a4 is the frequency for the A above middle C

• offset is the number of semitones to offset each note by

make_indices_dict(names=NOTE_NAMES, *, a4=57, offset=0)

Makes a dictionary containing note indices of common note names

• a4 is the note index for the A above middle C

• names is a list of note names

• offset is the number of semitones to offset each note by

music_to_notes(music, *, line_length=1)

Converts music into notes (two tuples of note name and length)

This function returns a list of two-tuples of a string/None and a float. The first item is the note name (or a break if it is a None). The second item is its length.

Note that there is a break between notes by default.

A music string is first divided into lines with one line being the specified length, defaulting to 1. Each line is then split by whitespace into parts with the length divided evenly between them. Each part is then split by commas “,” into notes with the length again divided evenly between them.

Empty lines or lines starting with a hash “#” are skipped.

Note names can be almost anything. A note name of a dash “-” continues the previous note without a break between them. A suffix of a tilde “~” removes the break after the note, whereas an exclamation point “!” adds one.

split_music(music)

Splits music into individual sequences

Lines starting with a slash “/” will be added to a new sequence. All other lines (including blanks and comments) will be part of the main sequence.

>>> assert split_music("1\n1") == ["1\n1"]
>>> assert split_music("1\n/2\n1") == ["1\n1", "2"]
>>> assert split_music("1\n/2\n/3\n1\n/2") == ["1\n1", "2\n2", "3"]

notes_to_sine(notes, frequencies, *, line_length=1)

Converts notes into sine waves

• notes is an iterator of two-tuples of note names/None and lengths

• frequencies is a dict to look up the frequency for each note name

• line_length is how much to scale the note by

_notes_to_sound(notes, func)

Converts notes to a sound using the provided func

The provided func is called with the note to get its sound. When there are no more notes to add nor sounds to play, this stops.

play_output_chunks(chunks: Iterable[bytes], **kwargs: Any)

Plays chunks to the default audio output device

This is hardcoded to take PCM 16-bit 48kHz stereo audio, preferably in 20ms blocks.

Keyword arguments are passed to sounddevice.RawOutputStream.

Note that the sounddevice library is required for this function.

create_input_chunks(**kwargs)

Returns chunks from the default audio input device

This is hardcoded to yield 20ms blocks of PCM 16-bit 48kHz stereo audio.

Keyword arguments are passed to sounddevice.RawInputStream.

Note that the sounddevice library is required for this function.

reload()

Reloads this module. Helper function

MUSIC_DIGITIZED = Multiline-String

Show Value

# names="do di re ri mi fa fi so si la li ti".split()
# offset=1
# line_length=1.15

. mi mi mi
fa do . do
. so mi do
re mi,re - mi

la3 mi mi mi
fa do . do
. so mi do
re mi,re - mi

do mi mi mi
fa do . do
. so mi do
re mi,re - mi

la3 la3 la3 fa3
fa3 fa3 fa3 fa3
do do do so3
so3 so3 si3 si3

la3 la3 la3 fa3
fa3 fa3 fa3 fa3
do do do so3
la3,do,mi,so la,do5,mi5,so5 la5 .

do so mi do
re re,mi so mi
do so mi do
re re,mi so re

do so mi do
re re,mi so mi
do so mi do
re re,mi so re

- do . la3
. la3 do re
so3 do mi do
re do,re - do

- do . la3
. la3 do re
so3 do mi do
re do,re - do

do la3 - so3
do re,mi - re
. . do so3
re mi re do

. . do so3
fa mi,re - do
- so3 re do
mi so re do

la3 mi mi mi
fa do . do
. so mi do
re mi,re - mi

do mi mi mi
fa do . do
. so mi do
re mi,re - mi

la3 la3 la3 fa3
fa3 fa3 fa3 fa3
do do do so3
so3 so3 si3 si3

la3 la3 la3 fa3
fa3 fa3 fa3 fa3
do do do so3
la3,do,mi,so la,do5,mi5,so5 la5 .

do so mi do
re re,mi so mi
do so mi do
re re,mi so re

do so mi do
re re,mi so mi
do so mi do
re re,mi so re

- do . la3
. la3 do re
so3 do mi do
re do,re - do

- do . la3
. la3 do re
so3 do mi do
re do,re - -

do

MUSIC_MEGALOVANIA = Multiline-String

Show Value

# names="do di re ri mi fa fi so si la li ti".split()
# offset=6
# line_length=2.2

la3 la3 la - mi - - ri - re - do - la3 do re
so3 so3 la - mi - - ri - re - do - la3 do re
fi3 fi3 la - mi - - ri - re - do - la3 do re
fa3 fa3 la - mi - - ri - re - do - la3 do re

la3 la3 la - mi - - ri - re - do - la3 do re
so3 so3 la - mi - - ri - re - do - la3 do re
fi3 fi3 la - mi - - ri - re - do - la3 do re
fa3 fa3 la - mi - - ri - re - do - la3 do re

do - do do - do - do - la3 - la3 - - - -
do - do do - re - ri - re do la3 do re - -
do - do do - re - ri - mi - so - mi - -
la - la - la mi la so - - - - - - - -

mi - mi mi - mi - mi - re - re - - - -
mi - mi mi - mi - re - mi - so - mi re -
la do mi do so do mi do re do re mi so mi re do
la3 - ti3 - do la3 do so - - - - - - - -

la3 - - - - - - - do la3 do re ri re do la3
do la3 do - re - - - - - - - - - re mi
la - re mi re do ti3 la3 do - re - mi - so -
la - la - la mi la so - - - - - - - -

do - re - mi - do5 - ti - - - si - - -
ti - - - do5 - - - re5 - - - ti - - -
mi5 - - - - - - - mi5 ti so re do ti3 la3 si3
so3 - - - - - - - si3 - - - - - - -

mi3 - - - - - - - - - - - do - - -
ti3 - - - - - - - si3 - - - - - - -
la3
-

MUSIC_DIGITIZED_DUAL = Multiline-String

Show Value

    # names="do di re ri mi fa fi so si la li ti".split()
    # line_length=4.36
    # offset=13
/   # offset=1

    . mi mi mi fa       do . do .           so mi do re         mi,re - mi
/   .

    la3 mi mi mi        fa do . do          . so mi do          re mi,re - mi
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    do mi mi mi         fa do . do          . so mi do          re mi,re - mi
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    do mi mi mi         fa do . do          . so mi do          re mi,re - mi
/   la3 la3 la3 la3     la3 la3 la3 la3     la3 la3 la3 la3     la3 la3 la3 la3

    do mi mi mi         fa do . do          . so mi do          . . . .
/   la3 la3 la3 la3     la3 la3 la3 la3     la3 la3 la3 la3     la3,do,mi,so la,do5,mi5,so5 la5 .

    do so mi do         re re,mi so mi      do so mi do         re re,mi so re
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    do so mi do         re re,mi so mi      do so mi do         re re,mi so re
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    - do . la3          . la3 do re         so3 do mi do        re do,re - do
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    - do . la3          . la3 do re         so3 do mi do        re do,re - do
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    do la3 - so3        do re,mi - re       . . do so3          re mi re do
/   .

    . . do so3          fa mi,re - do       - so3 re do         mi so re do
/   .

    la3 mi mi mi        fa do . do          . so mi do          re mi,re - mi
/   la3 . la3 .         fa3 . fa3 .         do . do .           so3 . so3 .

    do mi mi mi         fa do . do          . so mi do          re mi,re - mi
/   la3 . la3 .         fa3 . fa3 .         do . do .           so3 . so3 .

    do mi mi mi         fa do . do          . so mi do          re mi,re - mi
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    do mi mi mi         fa do . do          . so mi do          . . . .
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        la3,do,mi,so la,do5,mi5,so5 la5 .

    do so mi do         re re,mi so mi      do so mi do         re re,mi so re
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    do so mi do         re re,mi so mi      do so mi do         re re,mi so re
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    - do . la3          . la3 do re         so3 do mi do        re do,re - do
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    - do . la3          . la3 do re         so3 do mi do        re do,re - -
/   la3 la3 la3 fa3     fa3 fa3 fa3 fa3     do do do so3        so3 so3 si3 si3

    do . . .
/   la3 . . .