7
Vote

Model needs support for skinned animations

description

Both CMO and SDKMESH support skinned animations. These are currently ignored by the Model implementation.

comments

walbourn wrote Jan 25, 2013 at 5:16 PM

Some challenges here include the fact that CMO relies on multiple streams for the skinning, while most SDKMESH's with skinning information is a single stream. Multiple streams complicates the ModelMeshPart class a fair amount, but may be needed to avoid having to do 'merging' of vertex buffers at load-time.

walbourn wrote Mar 11, 2013 at 7:13 PM

SkinnedEffect should use IEffectAnimation interface

pnt1614 wrote Jun 5, 2013 at 2:09 AM

@walbourn, do you have any example about using SkinnedEffect?

walbourn wrote Jun 5, 2013 at 3:43 AM

The use of SkinnedEffect is pretty simple. You need a Vertex layout like such:
struct Vertex
{
    XMFLOAT3 position;
    XMFLOAT3 normal;
    XMFLOAT2 textureCoordinate;
    XMUBYTE4 blendIndices;
    XMFLOAT4 blendWeight;};
};

const D3D11_INPUT_ELEMENT_DESC InputElements[] =
{
    { "SV_Position",  0, DXGI_FORMAT_R32G32B32_FLOAT,    0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "NORMAL",       0, DXGI_FORMAT_R32G32B32_FLOAT,    0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "TEXCOORD",     0, DXGI_FORMAT_R32G32_FLOAT,       0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "BLENDINDICES", 0, DXGI_FORMAT_R8G8B8A8_UINT,      0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
    { "BLENDWEIGHT",  0, DXGI_FORMAT_R32G32B32A32_FLOAT, 0, D3D11_APPEND_ALIGNED_ELEMENT, D3D11_INPUT_PER_VERTEX_DATA, 0 },
};
When drawing with SkinnedEffect, you need to set the bone transforms just before it gets Apply called
XMMATRIX bones[] = { ... };
skinnedEffect.SetBoneTransforms(bones, 4);
The bigger question is the management of animating the bones matrices themselves, which is the real heart of this work item.

wchao1115 wrote Aug 14, 2013 at 5:57 PM

Not sure if this is a known issue. I tried the SkinnedEffect and found that it uses float4x3 for the bone transform constants and that creates some artifacts on a model I'm trying it on. Updating the vertex shader and constants to use float4x4 along with the corresponding change in SkinnedEffect::SetBoneTransforms fixes the issue nicely. It appears that the CMO file records the bone transform as matrix 4x4. Here is the updated shader (I'm trying this with the May's drop).

void Skin(inout VSInputNmTxWeights vin, uniform int boneCount)
{
float4x4 skinning = 0;

[unroll]
for (int i = 0; i < boneCount; i++)
{
    skinning += Bones[vin.Indices[i]] * vin.Weights[i];
}

vin.Position = mul(vin.Position, skinning);
vin.Normal = mul(vin.Normal, (float3x3)skinning);
}

walbourn wrote Aug 14, 2013 at 6:48 PM

It is typical to use a 4x3 matrix where the final row/column is [ 0 0 0 1 ]. It should very much be the case that animated bones do not have projection transformations, so even if the CMO records a 4x4, it should always have a [ 0 0 0 1 ] in the final column.

The key issue is making sure you are in agreement on whether or not the data in your constant buffer in the shader is ROW MAJOR or COLUMN MAJOR. HLSL typically uses COLUMN_MAJOR as a performance optimization, but the applications almost always store them as ROW MAJOR which means you need to 'transpose' before storing the data into a constant buffer.

wchao1115 wrote Aug 14, 2013 at 10:50 PM

The matrix gets transposed correctly and transported correctly to the shader. This is all the code inside DirectXTK. It may be possible that the CMO is produced incorrectly. The artifact is scoped and minor but observable.

wchao1115 wrote Aug 14, 2013 at 11:09 PM

Just checked and CMO is correct. The 4th column is 0,0,0,1. However it could be that the POSITION element in the layout is defined as DXGI_FORMAT_R32G32B32_FLOAT (XMFLOAT3) but the shader declares it as float4. I'm not sure if there's such thing as a register being partially initialized when the content of mismatched size being read into it.

wrote Oct 22, 2013 at 3:57 AM

Associated with changeset 31658: DirectXTK: Skinned Model support
  • Updated DGSLEffect with support for skinning
  • IEffectSkinning interface added
  • Updated SkinnedEffect to use default texture if no texture set
  • Updated Model loading to use skinned effects and vertex types