addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Hoodlums Message Board › Music Visualization

Music Visualization

Neil B
user 13347289
London, GB
Post #: 6
We're investigating the idea of writing music visualizations as a fun way to learn openGL in haskell.

We want a source of audio data that will be available on mac, linux and windows. Ideally, we integrate (however loosely) with a popular music player, so that people can test and use the visualizations easily with their music collections.

My initial thoughts were to find or write a plugin for itunes or VLC which would serve (via tcp) (or perhaps broadcast via udp) the audio analysis data passed to it by the player's visualization framework. Then the haskell visualizations can run as separate processes which just consume these events.

Abandoned approaches:

* I had a look at the various interfaces to VLC media player, most of which build around libVLC, but none seem to provide raw audio out or, say, spectrum analysis. On the #videolan irc channel I was told I'd need to patch and compile vlc to do what I wanted.
* I had a shorter look at itunes plugins to see if any of them published events regarding the audio (rather than track id). Perhaps their visualization sdk would allow arbitrary code, and we could load a vis that just published the data it was handed. This is worth looking at, but as there's no itunes on my platform (linux) I'm not so motivated.

One proposed solution:

Stream the music from the VLC player, have the visualizer be a consumer of that stream, and have another consumer of the stream actually play the music (or the server can do this, while it streams).

We could adapt or write a shoutcast / icecast client which receives and decodes the audio, analyzes it, and hands that data off to the visualization (either over tcp or within the same process, perhaps via a Chan).

In linux, you can start vlc streaming an mp3 with the following incantantion:

cvlc /tmp/sample.mp3 --sout '#std{access=http,mux=ts,dst=­0}'

you can test this works by running another vlc instance and connecting to the stream

vlc http://localhost:8080...­

If you want to investigate the many streaming options, open vlc proper and go to "media > streaming..."

The protocol to consume seems simple. It was enough for me to just open a socket, GET / HTTP/1.1\n\n and start reading the stream.

VLC can also transcode into a number of formats as it streams, so we could choose the easiest format to decode. There seems to be just one mp3 decoder on hackage:­.

The biggest problem with this approach is likely to be delay between the audio and the visualization (coming as they would from either two separate consumers of the stream, or from the server and a visualizing consumer, respectively).

Ideas welcome!
Powered by mvnForum

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy