COMPUTING SYNESTHESIA
By on November 13, 2007 | Permalink | Comments (9)
Speaking of visual and conceptual relationships, I’ve been enjoying some strange trips from using a unique software called RGB MusicLab. It was cleverly designed by Kenji Kojima who explains it quite well by saying,
RGB MusicLab converts RGB (Red, Green and Blue) value of an image to chromatic scale sounds. The program reads RGB value of pixels from the top left to the bottom right of an image. One pixel makes a harmony of three note of RGB value, and the length of note is determined by brightness of the pixel. RGB value 120 or 121 is the center C, and RGB value 122 or 123 is added a half steps of the scale that is C#. Pure black that is R=0, G=0, B=0 is no sounds.It is not an impression of paintings or photographs of a composer. It reads a score from an image data directly.
So, as l recall one of the most influential exhibits I’ve ever experienced, here a few pieces I’ve come up with.
Duration: 0:30
Tempo: 183
Instrumentation: 2 electronic kits (right and left channels), 1 saw wave (center)
hmm…an interesting start
Duration: 2:09
Tempo: 440
Instrumentation: Choir (left), atmosphere (center), orchestral harp (right)
notice how dark the image is in relation to the pitch
Duration: 1:05
Tempo: 470
Instrumentation: celesta (left), pizzicato strings (center), slap bass (right)
this one moves quite quick but is full of color
Duration: 1:15
Tempo: 190
Instrumentation: music box (left), guitar harmonics (center), woodblock (right)
similar color throughout, crazy ending(?)
Duration: 2:02
Tempo: 600
Instrumentation: harpsichord
Keep in mind, this is Bach’s seal in which he signed his music
Duration: 3:51
Tempo: 137
Instrumentation: Electric Piano (left), Car Engine (center), Electric Piano (right)
Spastic, yet rich and full
Duration: 0:29
Tempo: 80
Instrumentation: Tubular Bells (left), Rain (center), Tubular Bells (right)
The fluidity of this picture seems to bring out an actual “song”
Duration: 0:31
Tempo: 600
Instrumentation: Celesta (left), Dulcimer (center), Celesta (right)
I absolutely love this interpretation, it is mad and fits the mood completely
Duration: 1:27
Tempo: 309
Instrumentation: Timpani (left), Reverse Cymbal (center), Electric Piano (right)
Totally rad and conveniently weird - a true stare.
Duration: 1:20
Tempo: 151
Instrumentation: Percussive Organ (left), Contrabass (center), Bottle Blow (right)
Reminds me of the moth and it’s journey
Categories: Musical Endeavors, Present, Visual Music
Share
TrackBack
TrackBack URL for this entry:
http://existentialmedia.org/mt/mt-tb.cgi/406
Comments (9)
Writing through color is something I thought about, but didn’t get deep enough into (yet) to organize anything.
But yeah, the reverse picture was intriguing as well. I want to input a picture that is already a mosaic leading to even smaller readable pixels with less work. If it was colorful, it could lead to some insane rainbow crystals type of shit.
Posted by Jordan 
| November 14, 2007 @ 9:49 AM
this is so cool. this was made for you. you could arrange lots of monochromatic images and play them simultaneously, like members of an orchestra (sort of)…i’m just thinking of how this could be utilized in an installation…you should take it big. could you score a film this way? i mean, using the same sort of idea? also, look up an event called unsilent night, i bet they’re doing it in portland. god, this is so cool, i can’t get over how much it fits your personality. thank you!
Posted by alisha | November 14, 2007 @ 3:59 PM
sweet jesus that was sick. I really REALLY love the idea of orchestrating a piece completely by creating images. So good man, so good.
Posted by jenna | November 14, 2007 @ 6:38 PM
This is way cool. I downloaded it and have been tinkering around with it.
Posted by Mike O. | November 16, 2007 @ 3:02 PM
yeah, jordan, try it with a circular fractal.
Posted by ariel | November 20, 2007 @ 5:47 PM
Jordan! This is amazing. Can this work backwards? I haven’t listened to any yet because I’m at school, but I’m very interested. Also, how detailed an image can you derive sound from? What are the variables there?
Posted by Mulder | December 4, 2007 @ 5:18 PM
backwards, yes, i really want to try that, but i have a feeling it could end up sounding very dry, if it is too planned. you could always use the mosiac generator in the program, but that takes away most reasons for planning it out.
recently, my comp teacher lectured about a very similar program that her husband is developing. she has been one of the first people to really utilize it and has done some pretty incredible things with it. recorded breathing, stacked chords from library recordings, overtone harmonics, etc. it’s sort of like a grandfather to this one. more on that later, or sooner than later, i suppose. last final is tonight, then it’s on!
Posted by Jordan | December 5, 2007 @ 11:52 AM
Woah, did you take a look at Kenji Kojima’s paintings?
Posted by Amanda | May 10, 2008 @ 5:20 PM
WHOA! This is perhaps the coolest thing I have ever seen / heard.
Weird ass imaginary science fiction music!
Thank you for sharing this dude!
I am interested in how this works. Can you do it in reverse? Also, can you create a picture with the colors/notes in mind. Fuck music notation. Let’s paint in photoshop. OH Dang. What if you took one of those colorful pictures of a quasar or something?
Posted by matthew
| November 14, 2007 @ 9:38 AM