I was inspired to make this, because I cannot play by ear and am more comfortable with sheet music, but I wanted to learn random songs that I found on YouTube that weren't in any database of songs (e.g., MuseScore).
Music For Dummies takes a Youtube URL, generates a Music XML file, and then renders the Music XML file as sheet music in the browser. This allows you take advantage of browser features such as printing the music and saving the music as a pdf for future reference.
The backend is written in Python. We used PyTube to convert a Youtube URL to an mp3 file. Then we used Basic Pitch, an open-source AI model made by Spotify in 2022, to predict the MIDI for the mp3 file generated by the Youtube audio (sidenote, a MIDI is just a digital representation of the musical characteristics in the file). To render this in browser, we converted the MIDI to a MusicXML file (which is a more visual representation than a MIDI) using a Python library called Music 21 .
Our frontend is written in Vanilla Javascript and uses Pico.css (which is a minimal CSS framework for HTML). We also use Open Sheet Music Display to render the MusicXML file in browser.
Finally, we use Flask to connect our frontend and backend.
There are dozens of file formats for music and most of them are terrible, except for certain proprietary ones (looking at you, MuseScore -> .mscz and Guitar Pro -> .gp) that you basically have to pay to access their software with.
Even the format we ended up using, MusicXML, is pretty bad (it's the best of the worst) and MuseScore (a website that hosts millions of sheet music files, most of which you have to pay for) itself recommends editing a MusicXML file after importing it into their suite. Also, there are dozens of subcategories of MusicXML files, so our original plan of using AlphaTab to embed sheet music in browser with playing, metronome, looping, and printing did not work, because pretty much only Open Sheet Music Display works with MusicXML files (there is a service called SoundSlice that can also do this, but that is paid). Also also, Open Sheet Music Display does have a playing option but only for their sponsors (sadly).
Learning the many, many ways music can be represented digitally and navigating the lack of any good, free standards for digital music.
Representing audio as sheet music is surprisingly difficult and not in the ways you would expect. The challenge is almost entirely in rendering your prediction of the sheet music, not generating the sheet music itself. To be honest, this would work better as a command line tool to generate a pdf, because you could Lilypond (LATEX for sheet music) to do this (but making the frontend was an enlightening experience).
One of us learning React to make our website look good. Hopefully, getting access to the new version of OpenSheetMusicDisplay when it comes out, to make it so you can listen to the sheet music. Implementing a better machine learning algorithm that can handle a lot of instruments playing at the same time (which Music For Dummies is pretty bad at currently).