Music In Motion
Music In MotionTrailer
Music in Motion is researching the relationship of physical movement to sound creation in three-dimensional space. Guests will explore sound and create music around them through kinesthetic activity by interacting with the virtual environment. Our team is discovering more intuitive forms of musical control and nuanced performance through interactive sound environments and haptic music devices.
Music In Motion Demo
In this project, I have implemented several functionalities including the progress state machine, the uniform length segmentation of the tracing bezier curve. In addition, in the production process, I have experimented applying the Fast Fourier Transform on movement data and the fluid experience in virtual reality.
The major contribution of mine in this project is flocking trail system which is our essential interaction motivator.
Throughout the whole project, we were looking for a way to encourage a performative and gentle movement and and meanwhile generate harmonious to create a complete music experience. After the rapid prototyping process experimenting different ways of interaction, we found the hard trail and the flocking Boid system worked the best with our vision.
As the interaction programmer I started with the traditional flocking system. I set the controller position as the target and made it possible to lerp between different profiles with different velocity, different rotation radius and change the weights of cohesion, separation and alignment according their distance to the controller. However, the outcome was not so good as we expected. The individual often moved too far away with the influence of those flocking characteristics. Therefore, I started to design a brand-new system. Since it was inspired by Boid system, we called it Fake Boid system. Inherited from Boid system implemented by Keijiro, Fake Boid system had the same structure as the controller-behavior structure where the whole group of flocking had one controller containing all parameters, responsible for the spawning individuals and other overall functionalities while the behavior controls the actual movement of each individual.
At the beginning of developing the new system I aimed to eliminate the essential problem of the old one, so I started with a simple tracing behavior. I set a buffer to hold the previous position of the controller and put the objects into those places. Fake Boid system controller held the whole buffer data for the individual to acquire. Each individual was randomly assigned a index of the buffer. To enable the change of the individual density distribution, I implemented a functionality that can set an index as the most density point and the density decreased on both sides by a segmented parabolic cumulative distribution function. And to add those random movement to the objects when they are following the controller, I came up with the idea of using Perlin Noise. Each individual would have a point of attraction and it could move around it randomly with Perlin Noise. Those points of attraction which were named as the refer points would trace the controllers and the actual individuals randomly moved and were contained by a equation according to how far they were from their own refer points.
Fake Boid
However, in the static state, the individual behavior was too chaotic. An orbiting behavior was expected. To force each individual to orbit in a same pattern, the random rotation function was applied onto the buffer refer points in the system controller. And the rotation function was kept updated and modified to avoid rotate around a small radius on the high-latitude area on the sphere surface. The final implementation is descripted as below.
With the positions of the previous frame A and the current frame B, assuming the movement was a rotation on the equator, the rotation axis through the sphere origin O can be acquired by cross(OA, OB). And to generate a different direction rotation I rotated the axis with another rotation. The angle of the rotation of the rotation axis was clamped to a small value. The rotation axis of the rotation axis is random generated. Then rotate the refer point to C according to the new rotation axis. As a result, we got a random rotation on the sphere surface always along the equator.
Equator Rotation Axis
Acquire New Axis And Rotate
Fake Boid Orbiting
Later, as we moved forward to brainstormed the progress of our experience. We came up with the idea of collecting the flocking groups. Therefore, the system required the capability of merging one Fake Boid group into another. Different groups of Fake Boid were controlled by different system controllers. As long as the merging process completed the individual would lost reference to the previous system controller, and meanwhile the new one would replace it. With the most naïve way of implementation done. The merging process was instantly triggered and completed by the moment of collision.
Fake Boid Instant Merging
After tracing implementation finished, the process happened after several seconds. To avoid the flashing of the individuals when merging happened, Bezier Curve interpolation from the previous refer point position to the new refer point was implemented.
Fake Boid Delayed Merging
For the finale of the experience, we decided to make fun of our Fake Boid system. I implemented a profile interpolation mechanism which enabled the system controller parameters to interpolate among different profile files. By changing the constraints parameters, the behavior was tremendously changed. I changed the distance constraint function a little bit making it possible to have the individual move within a sphere shell.
Finale
