GeneSynth

1. GeneSynth Sound Synthesis by Genetic Algorithm

2. Introduction • Genetic Algorithms (GA) • Search Algorithm • Uses Darwin-inspired “survival of the fittest” heuristic. • Used In a wide range of fields • TSP, Financing, Music • But not for sound synthesis • Search space is too large

3. GeneSynth • Experiment in using Gas to synthesize useful sounds • Uses an input sound as a target to model as a heuristic • Motivation • Musical • Analysis

4. Goal 1: Sound Fitting • Take an audio file and fit other user-specified sounds using the audio file as a guide • Example: • Beethoven’s Fifth symphony, 1st movement as a target • Coin-jingling sounds as specified sounds • Result: Coin samples placed to sound as close as possible to the symphonic version.

5. Goal 2: Family Tree Exploration • Many search algorithms have a history associated with each solution • GAs are no exception • A GA solution’s history resembles a family tree • Because GAs create solutions by pairing two other solutions together • The ancestors and siblings of a good solution may yield interesting sounds for composers.

6. Goal 3: Modelling • If a perfect or near-optimal solution is reached, it can be used as a model • The model can then be modified in an interesting way and resynthesized to create a new useful result • But how easily will a perfect model be reached? • It depends on the sounds we select for use in the algorithm, as well as the efficiency of the GA. • If we use a sinusoidal unit generator, the problem begins to resemble the phase vocoder.

7. The classical GA • Origins in the mid 80s • Many variants have since been created. • GeneSynth is one of these variants • The classical GA must be understood to appreciate the variants

8. ＧＡ Overview and Terms • Chromosome: a string of data which represents a solution. Example: • 00001011 (a set of 8 bits that represents the number 11) • Individual: the solution represented by a chromosome • ‘11’ in the above solution. • Population: a set of chromosomes. Example: • {00001011,00100011,10110000} (three chromosomes)

9. More GA Terms.. • Mutation function: a function that changes sections of the chromosome based on some probability. Example: • M(01001011) -> 00001010 • (Two bits are flipped, changing the chromosome) • Crossover function: a function that takes two chromosomes and combines them together to result in two new chromosomes • C(10001010, 00001111)-> 10001111,00001010 • (The original chromosomes swap halves)

10. Fitness Function: a function that assigns a score to an Individual, based on how good a solution it is. If the problem is to find the number close to 77, a fitness function could be: F(x) = | 77 – x | Psuedocode: Create Initial Population Repeat Score Each Individual Select pairs of good-ranking individuals Mutate the selected individuals Crossover the individuals Until goal score is met GA terms and overview

11. A simple example • Suppose our problem is to find the number 77. • We will use the encoding AND functions in the above examples. • We randomly create a set of 8 bit strings to start off the population of size four. • {00100010, 11001010, 00010000,11110000} • We then score each individual and create the next four via mutation and then crossover. • This process Is repeated until we have found the number 77.

12. Fixed Length Chromosome Independent chromosome structure Fixed Mutation and crossover probabilities Variable Length chromosomes Self-referential chromosome structure Mutation and Crossover Probabilities vary over evolution Classical GA vs GeneSynth

13. Chromosome Structure • A bottom up example • A SoundRoot is a unit generator, which takes parameters and creates audio by some method • Ex. Sin(freq,phase), AudioFiles(FileName) • A Transformation takes existing audio and parameters and processes it • Ex. Distortion, Normalization, Amplitude Modulation • SoundRoots and Transformations are the basis of sound generation for GeneSynth

14. Chromosome Structure • A SoundCell is a unit that contains one SoundRoot, and any number of SoundCells and Transformation • Note the recursive definition. • A SoundCell specifies a sound entity that will be used some number of times in the synthesis SoundCell Transformations SoundRoot Child SoundCells

15. A CellDefGene is a set of SoundCells. A PlaceGene is a set of several pieces of information A reference to a Cell In the CellDefGene, A time and volume to place the sound Chromosome Structure: Genes CellDefGene SoundCells Two PlaceGenes 1.4 secs -6db 3.12 secs -8 db

16. The chromosome structure • So, a chromosome is a CellDefGene and some number of PlaceGenes. Chromosome PlaceGenes CellDefGene 1.4 s -6db 1.1 s -8db 0.4 s -2db 1.4 s -8db 3.1 s -6db 2.8 s -6db

17. Chromosome to Individual • If we realize every SoundCell at the times and levels specified by the PlaceGenes, we have generated our individual, an audio file. Chromosome PlaceGenes CellDefGene 1.4 s -6db 1.1 s -8db 0.4 s -2db 1.4 s -8db 3.1 s -6db 2.8 s -6db

18. Fitness Function • The fitness function uses a target soundfile, and compares the individual soundfile against it. This is done by • Taking FFTs of both, and comparing them • Sums the decibel rating of the ratio of each bucket in the FFT. • Taking the RMS value of both and comparing them

19. Mutation • A chromosome mutates by randomizing some part of it. In GeneSynth, • a new SoundCell can be created or deleted, Its parameters randomized, or one SoundCell might attach another to it. • A PlaceGene will mutate by changing the SoundCell it refers to, or by changing its amplitude or time. PlaceGenes CellDefGene 1.4 s -6db 1.1 s -8db 0.4 s -2db 1.4 s -8db 3.1 s -6db 2.8 s -6db PlaceGenes CellDefGene 1.4 s -6db 1.1 s -8db 0.4 s -2db 1.4 s -8db 3.1 s -6db 2.8 s -12db

20. Crossover • Pick two genes and taking a random number of SoundCells and the PlaceGenes that refer to them to make one new Chromosome, and leaving the remainder to make another. 1.4 s -6db 1.1 s -8db 0.4 s -2db 1.4 s -8db 3.1 s -6db 2.8 s -6db 3.1 s -8db 2.1 s -9db 2.5 s -7db 0.7 s -2db 1.1 s -7db 1.6 s -4db 1.4 s -6db 0.7 s -2db 1.1 s -7db 1.6 s -4db 3.1 s -6db 2.8 s -6db 0.4 s -2db 2.1 s -9db 2.5 s -7db 0.7 s -2db 1.1 s -8db 0.4 s -2db

21. score: 0.110832, Adam Joaquin, 9 cells, 22 genes score: 0.220700, Jill Hayashi, 2 cells, 42 genes score: 0.012515, Paul Berio, 16 cells, 3 genes score: 0.150242, Nimrod Palestrina, 8 cells, 46 genes score: 0.303568, Rebecca Silverstone, 15 cells, 111 genes score: 0.252139, Alicia Ligeti, 9 cells, 95 genes score: 0.384353, John Shakesphere, 15 cells, 93 genes score: 0.245086, Kyoko Joyce, 2 cells, 107 genes score: 0.073234, Rebecca Palestrina, 2 cells, 40 genes score: 0.363259, Leonardo Joyce, 13 cells, 105 genes Best in generation 1 is 0.384353 score: 0.384353, John Shakesphere, 15 cells, 93 genes score: 0.328594, Samuel Terentino, 2 cells, 125 genes score: 0.295409, William Shakesphere, 15 cells, 94 genes score: 0.407776, Eve Silverstone, 15 cells, 110 genes score: 0.344386, Jack Terentino, 5 cells, 62 genes score: 0.168303, Albert Athene, 7 cells, 74 genes score: 0.274905, Akita Terentino, 5 cells, 112 genes score: 0.366032, Jill Shakesphere, 12 cells, 78 genes score: 0.387343, Ikroop Shakesphere, 18 cells, 108 genes score: 0.332004, Rebecca Shakesphere, 17 cells, 110 genes Best in generation 2 is 0.407776 score: 0.407776, Eve Silverstone, 15 cells, 110 gene score: 0.062087, Akira Shakesphere, 3 cells, 13 genes score: 0.448460, Vincent Shakesphere, 19 cells, 208 genes score: 0.291499, Jack Terentino, 5 cells, 76 genes score: 0.287326, Alicia Joyce, 7 cells, 127 genes score: 0.458743, Akira Palestrina, 7 cells, 86 genes score: 0.395988, Samuel Silverstone, 24 cells, 158 genes score: 0.349573, Molly Silverstone, 12 cells, 82 genes score: 0.369854, Ikroop Dickens, 14 cells, 97 genes score: 0.438446, Leonardo Shakesphere, 19 cells, 144 genes Best in generation 3 is 0.458743 score: 0.458743, Akira Palestrina, 7 cells, 86 genes score: 0.314851, Quentin Shakesphere, 8 cells, 61 genes score: 0.295788 , Ella Joyce, 6 cells, 41 genes score: 0.476386, Jane Palestrina, 8 cells, 53 genes score: 0.133919, Ludwig Van Shakesphere, 3 cells, 62 genes score: 0.321841, Jean Shakesphere, 11 cells, 97 genes score: 0.337629, Rebecca Dickens, 9 cells, 63 genes score: 0.405638, Amadeus Shakesphere, 24 cells, 151 genes Best in generation 4 is 0.476386 Example of evolution

22. Sound Examples (early results) Target Generation 1, best Generation 57, best

23. More to do.. • The algorithm is not yet complete • Logging not implemented • Evolution parameters need to auto adjusted • More SoundRoots and Transformations needed.

24. Early Conclusions on GeneSynth • The GA can produce meaningful results • If a target is supplied it “borrows” some of Its meaning and uses It to search. • The GA is useful for sound fitting and variation. • Because there is no mathematically defined optimum for this, there will be many ways to search for it. • Gas often find interesting niches and hot spots in the search space (local maxima,) some of whichh may be interesting. • Generating a near perfect model has not been done yet • It will be a while before we can tell this, because the algorithm does not run fast enough yet.