Hello all, this is Jorge with some developer updates about Hands! I am super glad right now to have finally fixed some of the hands issues in Salsa Virtual.
Why hands are complicated
I remember a variety TV show in which a prankster would walking down the sidewalk carrying two huge garbage bags which were dripping! He would approach the victims head on and they would start that little dance of “wait, is he avoiding me or am I avoiding him?” To make the encounter funnier, the prankster would say “Watch out! It’s dripping! You go left, I go right! Oh no! OK, I go right you go left! Wait, you go this way and I go this way!” And of course the two of them would stay in limbo for 30 seconds.
It’s hard to put things in perspective. Especially if you’re teaching dancing, and you have to describe which hand you’re using, going which direction, what hand and direction your partner is using… and all of this while facing a mirror and seeing the mirror opposite for your students!
Transforming Hands
So each of the hands can be positioned somewhat independently by locating the wrists in different places, like moving your computer mouse. They can also rotate in place, like when you are fanning yourself on a hot day (and that’s just rotating on one axis). They also have general orientation. Can you point at the same building while facing it. and while facing away from it. And finally they have weight. Unless you have spaghetti arms (a common mistake for beginner dancers), moving your hand will probably slightly move your body through your muscles.
There are all local transforms.
Global transforms include positioning your whole body at a different location, bringing your hands with you. Global rotation can be a rotation from a parent element, like the forearms. “Wax-on Wax-off” will certainly move your wrists at an angle, event though you are just rotating your forearms. And if you’re facing North, the orientation of your hands is usually north as well.
The Code
I’m still having trouble deciphering my code because there are so many conditions. Mocap animations run in the background until the point where the hand connect when you ask someone to dance. In a nutshell, this is the pseudo code (for open hand hold):
Save the default orientation in case everything burns to the ground
Set points of reference (elbows, shoulders)
If hands connected
Am I a non-player character (NPC)?
If so,
Set the connection point to the middle point between the Lead’s Left hand (L.LH) and Follow’s Right Hand (F.RH), and also, L.RH with F.LH
Also, set the rotation of the connection point to the direction of the lead’s elbow to the follow’s elbow
Pivot point will take care of constant rotation around its position
Modify the animations with hand holds
If not then I’m a Player Character (PC), am I the Left hand?
Yes, I’m the left hand. Am I, the player character (PC), acting as the lead?
Yes, I’m the PC’s left hand acting as a lead
Connection point (L.LH|F.RH) follows the left controller, NPC follower’s right hand F.RH only rotates once to match the controller
No, I’m the PC’s left hand acting as a follow
Connection point (L.LH|F.RH) follows the right controller, NPC lead’s left hand L.LH only rotates once to match the controller
No, I’m the right hand. Am I, the player character (PC), acting as the lead?
Yes, I’m the PC’s right hand acting as a lead
Connection point (L.RH|F.LH) follows the right controller, NPC follower’s left hand F.LH only rotates once to match the controller
No, I’m the PC’s right hand acting as a follow
Connection point (L.RH|F.LH) follows the left controller, NPC lead’s right hand L.RH only rotates once to match the controller
If at any point we disconnect, we revert to the original motion captured animation.
Did that cause you a headache? It did for us. Yup, a several weeks-long headache!
A bug becoming a feature?
To test out the hands connection, we decided to make visible our selected avatar. Normally we keep it hidden because it gets on the way and there’s clipping due to watching from inside a 3D model’s head… In the screenshot we’re looking down because otherwise there would be a tongue, teeth, mouths and eyeballs on the screen! Sharp viewers of the screenshot below might notice the clothing clipping on the left side of the screen.
However it was awesome to see our selected avatar as a virtual body, instead of just looking at it through the reflection of a virtual mirror. The alternative and seeing the VR headset’s controllers (for one-to-one reference with the real world), or disembodied hands.
What do you think? Do you prefer:
- VR headset controllers: I need to know where the buttons are
- Disembodied hands: I don’t think I’ll look at my forearms that much
- Full Avatar inside-out view: I don’t mind clothes clipping, but I want to see all my limbs in VR
Let us know in the comments!