Is it possible to bake face Bone rig animation to shape key animation with diffeo?

wolf359

Hi

I found the option to bake the face rig morphs to shape keys.

works great!!

I know it is possible to create a facial/lipsync animation via the “ import action” using one of the daz options( mimic,Anilip-2 etc)

I really prefer to dump the entire Daz/rigify Rig and bind my Character meshes to an Auto rig pro rig due to superior retargeting& viewport control options.

 

I can manually animate the face with these transfered shape keys.

 

Is there any way to have Diffeo apply new actions to drive only the shapekey facial and lipysinc morphs after the original Daz body rig is removed/replaced?

Padone

As I understand it there are two ways. The FACS animations rely on a facs rig so if you change the rig it's not possible to transfer the animation. The papagayo animations rely on morph names and should work both on G1 G2 with facial shapekeys and G3 G8 with facial bones. So if you convert the bones to shapekeys I guess you should be able to import papagayo on shapekeys, if you use the same names.

Didn't try let me know if this works. Then we can ask to Thomas if something is needed.

wolf359

The papagayo animations rely on morph names and should work both on G1 G2 with facial shapekeys and G3 G8 with facial bones. So if you convert the bones to shapekeys I guess you should be able to import papagayo on shapekeys, if you use the same names.

 

 

 

I am not using papgayo which only exports. “MOHO” .dat files( like lipsyncontizer)

I am talking about effecively importing animated pose files base soley based on morphs as was done with G1,G2 and the ,millenials of the past

before  face bone rigs came with G3

 

Here is an example of a G8 male re-rigged with Auto rig pro

 

 

Except his face mesh contains ALL of the Daz phonemes and expressions as blend shapes.

 

I have even assigned Drivers to them.

They could be manually hand keyed

However I as hoping for a way to just import a pre animated.duf file, exported from Daz studio ,that was created with mimic basic or Anilip 2.

 

 

 

   

Padone

No I don't think it is possible. I did some tests and diffeomorphic recognizes the genesis figure so we can't import the daz studio animations on a custom figure, even if the custom figure is "compatible" with genesis. What you can do is to use a retargeting tool from the daz figure to your custom figure but this doesn't work for morphs.

We could ask Thomas to let diffeomorphic import the daz animations on a custom figure too, if compatible names are found for bones and morphs. But honestly don't know how much this makes sense. I mean if you change the rig and morphs from the standard daz figure then importing the daz animations may mostly fail.

That said, I was able to do something that may help. We could ask Thomas to automate step 3, if it isn't already done in some other way that I don't know of. Let me know.

steps:

1. load the daz animation on the genesis figure
2. convert morphs to shape keys
3. cut and paste the timeline from the morphs to the shapekeys
4. delete or disable the original rig

 

edit. I opened a issue at diffeomorphic of course feel free to participate.

https://bitbucket.org/Diffeomorphic/import_daz/issues/1176/

 

wolf359

@Padone

Thank you sir, for opening the issue on the blog.

I shall try what you suggested

 

FYI, this is NOT any kind of “show stopper” for my Daz Genesis to Blender Character animation pipeline.

 

It is frankly more of a case of me trying to “have my cake and eat it”  as well.

 

I really LOVE animating & retargeting to the ARP body rigs but the facial rigs it creates. have no automated facial mocap performance options, not even IPhone ARK kit ,AFAIK.

 

 

Thanks

catmaster

The option provided by Diffeomorphic to transfer face rig controls to shape keys is very useful, it's essential to make interactive expressions for G8 characters in Unreal games.

It would be desirable to transfer pre made facial rig based animations to pure shape key animations, if for some reason the G8 skeleton needs to be replaced.

I like to keep G8 original deform bones in game and retarget other animations including those made with AutoRig to original G8 deform bones, and bake the facial animations before importing to UE5. These animations can be used as animation layers for different bone groups in UE5.

1. load the daz animation on the genesis figure
2. convert morphs to shape keys
3. cut and paste the timeline from the morphs to the shapekeys
4. delete or disable the original rig

Transferring rig animations to shape key animations would be very complex if possible, and doubt that transferring keyframes from bone channels to shape key channels would be enough to transfer different types of animations. Baking animations should work for shape keys but the purpose is to keep the keyframes.

Found that you already test this on Bitbucket, so transferring tracks actually works and that it's possible to be automated.

 

Padone

Yes, you don't transfer from bones to shapekeys, but from morphs (driving bones) to shapekeys. This works fine but there are a lot of tracks to copy and paste by hand, one for every shapekey.

update. The new commit by Thomas can trasfer mimic, facecap and livelink facial animations to shapekeys and load them to custom rigs, so you can use any commercial rig not only mhx/rigify.

catmaster

The latest Diffeomorphic commits are awesome to convert bone animations to pure shapekey animations, that the Daz figures can be rigged with any other armature or even rigged with single bone and still have the original morphs and animations.

There are other uses of this new feature of Diffeomorphic, which makes Daz figures even more useful with game engines. These figures if rigged with custom skeleton can use their skeleton based animations, plus the pure morph based animations converted by Diffeomorphic. That means they can use two sets of animations designed for completely different skeletons without retargeting. For example some beasts I purchased from Daz store can use beast animations from unreal when attacking players and use the genesis 8 sex animations when players are defeated.

Also character physics simulation preset and character animations can be based on different skeletons. For example the physics simulations are based on unreal mannequin skeleton, but its animations use the morph converted Daz animations based on genesis skeletons.

wolf359

catmaster said:

The latest Diffeomorphic commits are awesome to convert bone animations to pure shapekey animations, that the Daz figures can be rigged with any other armature or even rigged with single bone and still have the original morphs and animations.

I should have  My tutorial using ARP body rig  with, mimic animated face, up by this weekend.

benniewoodell

Hey Wolf359, just in case you ever decide to try and rig the face with autorig pro or someone else asks you randomly, remember my problem with the eyes not moving? I figured it out, you have to separate them, which I did, but you have to bind the eyes each separately to the rig in ARP. Now the next problem I found was the teeth distorted to stay part of the jaw so I'm guessing those have to be separated as well, but one problem down lol. I didn't realize I could still use the morphs in diffeo still like catmaster mentioned if I used a different rig. That's good to know. Looking forward to checking out your tutorial when it's done! 

wolf359

benniewoodell said:

Hey Wolf359, just in case you ever decide to try and rig the face with autorig pro or someone else asks you randomly, remember my problem with the eyes not moving? I figured it out, you have to separate them, which I did, but you have to bind the eyes each separately to the rig in ARP. Now the next problem I found was the teeth distorted to stay part of the jaw so I'm guessing those have to be separated as well, but one problem down lol. I didn't realize I could still use the morphs in diffeo still like catmaster mentioned if I used a different rig. That's good to know. Looking forward to checking out your tutorial when it's done! 

 

@benniewoodell

Just a brief 11 minute, overview

of how I use Daz studio facial animation with custom ARP rigs.

Padone

@wolf359 Again, if you convert the morphs to shapekeys using the morph names, then you can load the mimic animations directly on the ARP rig. That is, diffeo will load the daz animation on the custom rig, using the morph names to match the keyframes in the duf file.

wolf359

@Padone

Thanks

I will have to try that tomorrow, do we still import mimic etc. via the “import action” button.

I seem to remember it being “greyed out”after I unbound and detached/deleted the Daz rig

at any rate this is a very exciting time for Blender animators!!

particularly those of us who have invested in ARP.

wolf359

UPDATE:

I am still not able to load a mimic /face animation

to a character re-rigged with ARP, unless I Append it to the shapekey editor from a file where the animation was baked from a Daz rig first.

 

Not a show stopper for me at all because though  I had already planned a pipeline whereby I will be creating & importing my facial performances in advance in a bare bones scene with base G8 M/F  and saving them as browser Assets (with very specific naming conventions) to be appended to my ARP rigs during the animation stage.

 

ARP is so good at binding& weighting  I may actually start making my own custom clothing again inside blender and binding them to my Characters with ARP

Padone

@wolf359 You have to convert the shapekeys with the morph name, not with the label, otherwise the duf file doesn't find a match.

wolf359

@Padone

Awesome!!

Now I can switch out the lipsync at will by importing a new .duf from mimic

Thanks

Padone

@wolf359 Please note that the mimic animation also includes movements for eyes and head, since those are bones can't be converted to shapekeys, nor imported into a custom rig as ARP. But face units and visemes will transfer fine.

wolf359

@Padone

Perfectly fine! 
My head movement during speech will be hand keyed or driven by
mocap performances retargeted from external sources. 

I will be able to use any motion body created with iclone as ARP has  built in retargeting templates for my  Reallsuion CC3 rigs as well as Mixamo, unity, unreal skeleton Perception neuron& Rokkoko mocap suits.

Very exciting time for me, thanks to Thomas and yourself