The Naked Scientists

The Naked Scientists Forum

Author Topic: What is the proper algorithm for visualizing music?  (Read 1564 times)

Offline Atomic-S

  • Hero Member
  • *****
  • Posts: 928
  • Thanked: 18 times
    • View Profile
While listening to music off of the Internet, it was visualized on the screen by a squggly green line. It occcurred to me that if I turned off the speakers and looked only at the green line, I would have almost no idea what was being played. There are other, fancier visualization schemes that make fancy patterns, but these I found were just different ways of displaying the same wave. They too, while nice, do not seem to display the music in any meaningful way. When I hear Beethoven's 9th symphony, what I see is not squiggly lines but roiling thunderclouds, great igneous buttes, and occasionally a shaft of sunlight piercing the gloom and lighting up lovely flowers upon a meadow. Off to the side someplace stands a Gothic cathedral, which is visited. None of that appears, however, on the screen. So it would appear that what the brain picks out of music is not some kind of a squiggly line nor even a simple spectrogram. There must be a different codec involved. What would that be, and is it the same for all people? (The fact that movie makers choose certain types of music for certain types of scenes suggest that it is, but the fact that people do not all prefer the same music suggest that it isn't.)


 

Offline techmind

  • Hero Member
  • *****
  • Posts: 934
  • Un-obfuscated
    • View Profile
    • techmind.org
What is the proper algorithm for visualizing music?
« Reply #1 on: 11/12/2010 15:24:06 »
While the ear certainly does interpret the sound as a spectrogram at one level, the brain makes all sorts of higher-level interpretation, based -I guess- on a combination of 1) innate synesthesia, and audio mimicry and 2a) learnt associations and 2b) personal experience.

1) would cover things like tone, pace, regularity or randomness ('style' of the music) conjouring up images of storms/lazy summer days/playfulness/water etc/marching/battle/strife and changes from one mode to another.

2a) would cover things like music becoming associated with particular events/products/films etc through the media/advertising
2b) would cover the more personal experience that music may conjour up images or feelings of an earlier time in your personal life when you first heard or were regularly exposed to that music. These associations are likely to be very individual, and not necessarily connected to the style of the music.


In principle I'm sure you could write software to emulate much of 1) and 2a) and summon appropriate images from a database and/or computer-generated environments - although there's probably quite a lot of ambiguity (and context) in the interpretation...


I'd guess the advertising and movie industries already have extensive databases of music (and musical passages) to convey different moods.
 

The Naked Scientists Forum

What is the proper algorithm for visualizing music?
« Reply #1 on: 11/12/2010 15:24:06 »

 

SMF 2.0.10 | SMF © 2015, Simple Machines
SMFAds for Free Forums