Here is screen recording and a few images from my latest coding experiment - ‘Muse 001’. - A study combining interactive 3D with generative audio, created with #threejs, #animejs, #tonejs, #blender, and #mixamo.
This is really an interactive project that is best viewed via a web browser window and with headphones or decent speakers. This screen capture doesn’t really do it justice and the audio is also pretty crap in this recording.
Here is the link to full interactive version:
https://henryegloff.com/works/muse-001/
In a nutshell, in the interactive version the audio is composed with code and generated on the fly. I also included some probability in the code so the audio has no set form as well as a very low file size. I then used the timing of the audio events to trigger effects in the 3D environment.
Unfortunately, one thing that didn’t work so well is that the motion capture sequence that I used doesn’t loop in time with the rhythm of the dancing. - I’m looking forward to trying this again at some point and getting the dancing sequence in perfect time with the audio.
Otherwise, this is the first example I have published along these lines, and I feel like I have only just touched the surface of what is possible here. One of the next things I would like to try is making the audio ‘spatial’, - as in emitted from objects and positions within the 3D environment.
This is really an interactive project that is best viewed via a web browser window and with headphones or decent speakers. This screen capture doesn’t really do it justice and the audio is also pretty crap in this recording.
Here is the link to full interactive version:
https://henryegloff.com/works/muse-001/
In a nutshell, in the interactive version the audio is composed with code and generated on the fly. I also included some probability in the code so the audio has no set form as well as a very low file size. I then used the timing of the audio events to trigger effects in the 3D environment.
Unfortunately, one thing that didn’t work so well is that the motion capture sequence that I used doesn’t loop in time with the rhythm of the dancing. - I’m looking forward to trying this again at some point and getting the dancing sequence in perfect time with the audio.
Otherwise, this is the first example I have published along these lines, and I feel like I have only just touched the surface of what is possible here. One of the next things I would like to try is making the audio ‘spatial’, - as in emitted from objects and positions within the 3D environment.
I’m also planning to develop tutorials and resources for things like combining Tone.js with Three.js and Anime.js, and integrating motion captures with Three.js. So if you’re interested in this kind of stuff please stay tuned. 🙂
