Rokoko

To start

We’re going to explore the UI of Rokoko Studio , understand  how to properly  fit the MoCap suit, and connect it to the software. Additionally we will learn how to import our custom avatars, perform self-made animations, and export them. Through the process, we will also identify any potential errors and learn how to resolve them.

Introducing the VRM

First, we import our VipeHero into Blender, and we need to make a couple of quick adjustments to ensure everything works correctly in Rokoko. The first and simplest one is renaming the bones of our VIPE. But don't worry, we've created a simple script to automate this process with just two clicks. First, open the Python console in Blender.

Second, we need to open the script that we've provided right here for you ——————<3

VIPE_Bones_Rename.py

Once the script is imported, you need to select the VIPE's skeleton in object mode, and then click the play button.

Once you've done that, you could use your VIPE Hero in Rokoko. But before that, we need to make a small adjustment to the weighting.

We select our VipeHero's mesh, enter edit mode, and select all the vertices with the 'A' key.

Now we switch to weight paint mode, reselect the vertex option, and go to the 'Data' option where we can adjust the bones and their weighting to the mesh (don't worry, we only need to modify two bones).

We're looking for the "leftThumbMetacarpal" bone in the bone list, reducing its "Weight" to 0.0, and then clicking on "Assign" to remove all influence of that bone from the mesh. Afterward, we apply the same process to the "rightThumbMetacarpal," and we'll have our VipeHero ready for action!

Using it

1_EXPLORE THE INTERFACE

The first thing you'll encounter when opening the application is its main menu.

(In the top left corner, we can see the different windows. We will enter the "Projects" one to start creating our animations.

We need to create a project in order to create a new scene where we will develop our custom animations. Naming them in a clear and simple way will allow your colleagues to pick up your work without any problems.

Once inside the scene, we will begin to explore the interface options:

On the left side of the screen:

• Clips: This is where your animations will be saved.
• Actors: It represents the people connected to the suit.
• Character: All FBX models in the scene.
• Input Devices: Displays connected hardware (this option is available in multiple areas of the interface).
• Face Capture:

On the right side, you'll find:

• Devices: Refers to connected hardware again.
• Streaming: You can directly display the scene in various 3D software.
• Export: This is where you will save the clips created in the scene, in FBX or BVH format.

Finally, at the bottom, you can record the animations with the REC button.

2_ADD AN ACTOR

We need to add an actor to the scene. Click on the "Actors" option, and you will see this contextual menu.

You can add a name, color, and enter the measurements of the person who will be using the suit. Then, you will need to pair the actor with the suit to capture our movements. Right-click on the actor and select "Pair with Rokoko suit."

3_IMPORT AN AVATAR

We already have our actor on the scene; it's time to add the character that will be animated from the 'Characters' option

To import a model, there are a couple of things to consider. First, we need to prepare our model in FBX format. The second, and potentially the trickiest part, is the skeleton. Rokoko only recognizes 3 naming conventions (you can find a list below to check if your bones are named correctly). Finally, you'll need to place your character in a T-pose or an A-pose.

4_CREATE AN ANIMATION

Now, the fun part of the whole process, acting.

Before we start doing anything, we need to transfer our movements to the character or retarget them. Thanks to Rokoko, this process is incredibly simple. All we have to do is left-click on the 'Character' option, and a contextual menu will appear. From there, click on 'Retarget to Actor' and choose the actor you want if there are multiple people performing.

With everything set up correctly, we're ready to create our stories!

All that's left is to press the REC button at the bottom of the interface and enjoy your performances. Rokoko will do all the work for you.

5_EDIT AN ANIMATION

Once the animation is complete, it will be saved in 'Clips.' Double-clicking on it will open the animation player, where you can make minor adjustments to our animation. You can return to the scene by clicking the arrow that will appear next to 'Clips.

5_EXPORT YOUR ANIMATIONS

Once the animation is complete, it will be saved in 'Clips.' Double-clicking on it will open the animation player, where you can make minor adjustments to our animation. You can return to the scene by clicking the arrow that will appear next to 'Clips'.

6_ADD DYNAMIC BONES

To add dynamic bones in Blender, you'll need to have the 'Bonedynamics' addon installed. Fortunately, we have a free version available for this, and I'll provide a link to the Addon just for you <3

Bonedynamics Pro

"Once you have the addon installed, you need to select the bones you want to make dynamic, click on 'Add dynamic bones,' choose one of the available presets (the one that works best), and then bake the animation.