Close
0%
0%

Autogenerated light shows from music

Everyone knows music is fun to listen to. Why don't we make it fun to watch as well?

Similar projects worth following
Music visualisation is something I've wanted to do for a long time. I recently discovered that there's a massive community of people who make stepfiles - files for DDR-like games containing tons of useful information synchronised with thousands of different songs. Unfortunately, not so many play DDR and Stepmania these days. I decided to recycle all the hard work people put into these files and use them for a different purpose: music visualisation!

I plan to expand this to more kinds of displays, and more kinds of files (Karaoke lyric files, A better Synthesia for MIDIs, and eventually something to visualise any raw song data) and more kinds of displays (POV fan displays, LED panels, or big professional lighting setups through DMX). I want to make new ways to experience music.

See it in action here: https://www.youtube.com/watch?v=fc1rPT-fTfw

Here's a demonstration of what it can do so far:

What's next for this project?

Well, there's a lot more emphasis patterns that could be implemented and perfected. Sometimes it doesn't pick up something interesting, and sometimes it thinks too much is interesting (as shown above in the "Brain Power" song). There's also often patterns in easier difficulties that are excluded in the higher difficulties; perhaps I can find a clever way to combine them?

I also don't want to limit this project to just stepfiles and lightstrips: I want to gather more information from many different song files and display them in all sorts of interesting ways, from the cheap LED stip I got from Ebay all the way up to professional DMX laser light shows.

How it works, or what can we gather from a Stepfile

A stepfile is a file used by a DDR Machine or other rhythm games like Stepmania, BeatX etc. which holds information about a song and what keys it has, keys being the arrows needed to be pressed in a "level".

Read more »

  • Game vs visualiser comparison (and other updates)

    laurens.weyn06/13/2016 at 19:31 0 comments

    I once again have free time to work on this! Unfortunately I missed the deadline for the anything goes part of the hackaday prize. Oh well.

    Anyhow, here's some notable changes:

    • Data sent to the light strip no longer gets corrupted! It used to sometimes go off sync somewhat and display random gibberish until it figured out where packets start again, but this is now fixed.
    • This also allows for higher framerates, though I haven't tested anything beyond 60fps (and I honestly don't have much of a reason to yet. Perhaps just to see how far it will go...)
    • Stop parsing is less "violent", and will no longer burn out your retinas when playing songs like Brain Power.
    • The camera setup has been upgraded! There's a paper diffuser and the exposure has been turned way down. Unfortunately, this means the camera can pretty much only pick up the light strip and not much of the room it lights up behind it, making videos pretty 1 dimensional. I guess you can't have everything.

    I've tried to explain how patterns are read, but I think seeing it visually might make it easier, so I've made this video with a stepfile in both a rhythm game (as intended) and visualised on my light strip side by side:


  • How to determine when each note happens in a stepfile

    laurens.weyn04/27/2016 at 09:51 0 comments

    This was probably the hardest component of the software to get working. In order to read a step file, you need to know when each key happens with respect to time. (as a side note, it seems the Stepmania wiki has recently taken down the only resource on the internet on the format of .sm files. It's available on the wayback machine here: https://web.archive.org/web/20150506065854/http://www.stepmania.com/wiki/file-formats/sm )

    The beats of keys

    steps are stored in sets of 4 characters (one for each arrow), which are held in measures separated by commas. Each measure is passed at 4 beats per second, so if there are 4 sets of 4 characters, each one lasts a beat. If there are 8 sets, there's half a beat between each, etc.

    Alright, so we know the time in beats for each key, but we can only measure seconds since the start of the song, so we'll obviously need to convert that to the current beat number based on the BPM. Sounds simple enough.

    If only it were that simple.

    Seconds to beats

    As I've mentioned in the description, the BPM can change all over the place. These changes are given in a list, in the format <beat>=<BPM>. So, 0=200 means that the song starts off with a BPM of 200.

    But things get difficult when you encounter something like 24=100. So after beat 24, the BPM changes to 100. Note that this is beat 24, not the 24th second. This gets confusing.

    Your first instinct to solve this problem is probably an "as you go" approach. Increment a BPM counter proportional to the BPM of 200 as time passes (updated every 60th of a second or so) and when the counter hits 24, halve the speed to 100. This... might work, but it's very imprecise as these BPM changes won't all conveniently happen at exact multiples of a 60th of a second, so you'll begin to drift offbeat. What's worse, some gimmick/dubstep songs stack lots of these changes very closely together, which is sure to throw this off completely.

    Clearly, we need to make a function that takes in the seconds (as a double) since the start of the song, and returns the current beat (also a double). This is not straightforward. Eventually though, you might come up with a solution, at which point you'll read on to BPM stops and realise just how that solution won't work at all.

    Stops and negative BPM

    Stepfiles also let you stop the BPM. Of course, this can't be done with BPM changes, because if you have, say, 10=60, 20=0, 21=60, then at beat 20 the BPM will become 0 and beat 21 will never be reached. Therefore, stops are seperate from BPMs, and are stored in the format <beat>=<seconds>, where beat is on which beat the stop happens, and seconds is how long it stops before the BPM returns to normal.

    Now we're throwing seconds into this mess, and suddenly all that code that relies on everything being measured in beats falls apart.

    Then you learn that there's also "negative" BPM. When a negative number BPM is given, it will look for the next positive BPM change and jump forward the difference in beats when the beat becomes positive again.

    How the heck do I make this work...

    Clearly, this isn't an easy problem to solve. I tried 3 different solutions, and all of them failed on anything beyond a very simple or conveniently laid out song. Debugging this wasn't straightforward either, so I took a song's BPM changes and stops and I drew a graph on paper of beat vs time, so I could figure out if my functions were giving the right output. They weren't. Nothing was working.

    And then it hit me. "Hold on a minute", I thought, "Didn't I just solve this problem in drawing the graph of beat vs time?"

    The solution that was literally right in front of me the whole time

    At this point, I didn't know whether I should feel like an idiot or a genius. Nonetheless, I wrote code that converted each BPM change and stop to a BPMLine object, which is basically a set of line segments in the classic form of y=mx+c, and stored them in an array. Each BPMLine is valid for a range of times, and the array as a whole makes up the graph. To find the...

    Read more »

View all 2 project logs

Enjoy this project?

Share

Discussions

adam vendra wrote 05/09/2017 at 20:01 point

Great stuff, maybe you could ad a plugin to use it with stepmania. Still amazing work though

  Are you sure? yes | no

Frederik wrote 11/06/2016 at 03:02 point

This is really great. 

Where is the code. I will build this assap.


  Are you sure? yes | no

benedikt966 wrote 09/28/2019 at 10:53 point

same here =)

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates