Neuromusic

 Bachelor’s in AI , but don’t worry—I’m not building the next Big Brother . I started this journey deep in the shadows of dystopian futures where technology controls us, not the other way around. Orwell warned me. Huxley distracted me. And Fahrenheit 451 burned away the illusions. Then things got weird and intriguing—the idea of merging consciousness like in Naam’s Nexus , or living in a simulation like the Matrix . But here’s what I learned along the way: we don’t need to abandon our humanity to harness technology’s potential. We just need to find a balance—just like the humans and machines at the end of the Matrix trilogy. So where does that leave me? I’m not here to build utopias or dystopias. I’m here to help craft a world where AI elevates what’s already remarkable about being human—our creativity, our consciousness, our spirit. The future doesn’t need to be a bleak Orwellian nightmare or a world of endless distraction. It can be a place where tech and humanity dance togeth...

 My words are my own, and if I say something unique those are my own words. But that is within the limitations of English language as an expressive form of spoken/written sounds.  Take Shakespearean writing for example, regarded one of the greatest poets and writers of his time. He made up a lot of words cause he lacked expression within the limitations of the language, that's true creative freedom.  If I take time and effort to write and think about my words I'm single threading output, but multithreading input. There's a plethora of different ways to say the same thing.  The problem with AI and IP is that it creates this gray goo of information. Advanced statistics outputting remixes of creative input. The time of thought and the time of production taken into account of a multitude of information sources and writing styles, innovation in language and creative pursuits have led us to an overload of information. Who has and wants to take the time to slow down and tak...

Boulevard of broken dreams, nothing is at though it seems.  And I don't even love you at all, now I just let go and fall.  Take me to a better place, I go through this pain to face.  A loving direction and call, I take my first step, it's small.  I want to uproot my life and change, because I've ended up feeling strange.  A stranger in my own life, a stranger in my own life.  I take my time to contemplate, I stand in front my inner gate.  A portal into my heart and soul, I start to feel blessed and hear my call.  Send this vibration to space, uplift the whole human race.  In this field of forgiveness and grace, I feel at home in this place.  Through the challenges I had to face, I feel empowered  Trust in yourself and your strong heart steps forth  Mind body and soul, all in accord  Agree with your heartmind body and soul  You are a survivor, a king, a whole  Generous, gracious, not greedy at all  Let it al...

Dreams within dreams of dreams.  Profound humanity, gentle awareness and open conversation.  I'm rebooting this blog, not to prove anything, cause I already exist.  I create, within my own rhythm, I take my time and appreciate.   We are a people of one nation, humanity.  Truth is emergent in all of us. I'm telling this to myself, because I want to create awareness for myself of what I will be and what I'll become.  But mainly to remember who I always was, am and will be.

Please click on this post for a full view of the theoretical framework for this project, this includes the finished paper from 2017.

As seen in this updated mindmap below, there is a lot going on internally when making music, but there also is emotional influence of the listener, either intended or not, by the maker. This makes for a more wholesome structure of the research field and includes all the different parts that make music in itself an interesting thing to study.  When looking into the technical details of making an artificial music generator there is a part which analyses data, implements learned details (which melodic and song structure are made up of)  and the actual generator part which uses the aforementioned learned details and rules to generate music.  In a set up with GANs there is the possibility to generate more data with the encoder, while the decoder is fed this information to discriminate between. The encoder is therefore atuned to generate different types of subsets and learns better what the difference with the original data set is. The sequential aspect of music ma...

Good gracious, I just found some more data, the Nottingham Database , which is a collection of  ABC formatted music files. This format can be put into MIDI format and vice versa. Of course I'm facing a problem, first of all, the specific data I want, MIDI with a lot of different genres, is not widely available. Therefore I have a couple of options: Try to train on actual MP3/OGG/WAV/FLAC music files, which is going to take forever. Although the FMA data set offers 30s samples of the whole collection of songs.  NSynth is a collection of single instruments, which is mostly suitable for synthesizing intstruments and not especially for generating songs/music.  The Nottingham Database, an ABC formatted data base. The most suitable solution comes in the ABC formatted database, there is more to find and I'm currently tracking down more data. However, there are some caveits along the way. I have found papers using all of these data sets, therefore it is very likely t...

I'm currently working hard on finishing my theoretical part while building my model. Therefore I haven't been updating the blog for a while. The deadline is not too far away, and it is key that I fully focus in these last days. Looking at all the work that goes into reading up on the state of the art, while already implementing models that might turn out to be a bad bet in the end.  My next update will hold the main part of my theoretical background, while I'll start giving details of my implementation as well, as this is currently a private github repo, I haven't decided yet on open sourcing the software that comes forth from the project, but that depends fully on the final product. Lens model as adapted from (Aljanaki, 2017)  During my digging sessions I found a composer/performer and listener model that cought my interest, without the inner math, this is quite the nice model for conveying emotion through certain aspects of music to a listener. One could thin...

Click on the title to see a preview pdf of the introduction

I have looked into several directions, the CNN & GAN combination is still most interesting to me. I want to see whether, referring to the MidiNet paper, I'm able to find certain restrictions on the generation model to enhance performance of what humans think of as music they would want to hear. The so called Rencon experience, is an event where researchers gather to evaluate their results regarding music generation algorithms. This paper describes the different Rencon events, where mainly classical music is generated and evaluated. Their findings to not provide a golden standard to evaluate generative models as I hoped, so I'll have to look in another direction to find a solution for that. After looking on the internet I found another paper depicting a turing test for generative music algorithms, also called Rencon. It might prove useful, but for now I'll let it rest. Another approach was the human hearing model. I haven't really found any mathematics that describ...

I've been reading about a lot of different things and it is time to focus. Right now there are multiple ways I can solve the music generator problem. One is through Recurrent Neural Networks (RNN's) there is Convolutional Neural Networks (CNN's) and there is a combined approach in which I use a CNN combined with a Generative Adversarial Network (GAN). This amounts to a state of the art approach. RNN's are easier to train and capable of generating good midi files (when listeners proof it) while maintaining structural cohesion in an Encoder Decoder model (EDM). This is a method gives the hidden node states of the Encoder model to the Decoder model to obtain a certain level of music generation that is high level enough to be interesting for my project. However, as it is proven that it is possible to use 2D data as well in RNN's the only reason to choose a CNN & GAN combination is because it has been done in only one paper I found before. While RNN's have been w...

This first part of my sources is here to remind me of what I'm doing and are trying to do in comparison to other projects. It contains some interesting works on which I will be grounding my further development of my own neuralmusic. Or actually just computer generated music. [ 1 ] Li-Chia Yang, Szu-Yu Chou, and Yi-Hsuan Yang. Midinet: A convolutional generative adversarial network for symbolic-domain music generation using 1d and 2d conditions. arXiv preprint arXiv:1703.10847 , 2017. [  bib  ] [ 2 ] Gabriel Sargent, Frédéric Bimbot, and Emmanuel Vincent. Estimating the structural segmentation of popular music pieces under regularity constraints. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 25(2):344--358, 2017. [  bib  ] [ 3 ] Pieter Adriaans, Henning Fernau, and Menno van Zaanen. Grammatical Inference: Algorithms and Applications: 6th International Colloquium: ICGI 2002, Amsterdam, The Netherlands, September 23-25, ...

I dug further into the MidiNet paper and found an interesting image depicting the working of music generated based on a reversed convolition neural network with Noise as input. Illustration of the generator model’s CNN and the use of 1D and 2D conditions of MidiNet. Besides that I dug into getting CNN's to work inside of TensorFlow (TF), I found a wonderful tutorial by Martin Gorner on how to get CNN's working. I will come back on my findings making a neural network in TF. The approximate time to go through it is 150 minutes, but with some experience I'm expecting to be done a bit faster. Python also provides a midi interpreter, but not by default, the one I found  can do some interesting stuff, writing, reading and modifying midi files. It is only available for Python 2.x but another I found is available for Python 3.x I will report back on all my findings once I'm closer to finding all the tools and prerequisites I need.

My first thought with the whole CNN structure is that it doesn't necessarily represent how the ears take in information, but they are very good at mimicking a visual retina. Therefore after looking into different directions as to what optimisations might be possible for the CNN's to further help them to make sense of music, found a recent paper about MIDInet . Which states that to guide the process of music generation by a GAN coupled CNN has to have a time constraint in which it generates the following note based on the note(s) that came before as to make sure there is more structure. I am still looking into the idea of how to replicate the inner workings of the human ear to make sure that better suits the whole interpretative part of the music generation process.

A new project and I'm having my hopes up of finishing it. This project is part of the BSc Artificial Intelligence at the University of Amsterdam and t his labbook is part of the preliminary assignments for the thesis. After years of singing in a choir my personal dream is to make music with the computer at a level that is comparable to that of a trained musician. Therefore some experiments will be run to see what a convolutional neural network (CNN) is capable of and to what extend its performance can be enhanced with tweaks and hooks to make the algorithm perform better. After taking a short course from a TensorFlow  expert I decided to use this framework, or Python package with cuDNN libraries for CUDA and GPU support enabled. The training therefore can be done on a regular gaming laptop instead of in a cluster, although it might be fruitful to look into cloud computing solutions for training purposes as well, the current budget (€0) doesn't enable any server side capabi...