DX10 Picking Demo Sample to DXUT+DXTK

Nov 14, 2015 at 9:55 PM
Wonder if could please some one of the experts can give me some a quick way of how to get the the vertice position or nay other element of a Vertex struct (local or World Space coordinate).

currently i'm using DXUT and DXTK Sample VS2013 with a SDK mesh 3d Data.
trying to replicate the sample of DX10 SDK Picking Demo with out success.

what i want is :

1-Click any area 2d area of the projection and get a 3d vector position, using on first instance boundingbox intersection and then mesh intesection. if any vertice is intersecting with a Ray then get the current position of that vertice (of Triangle Face).
  1. display the data on screen.
any help will be appreciated.!!

what i got currently:
HRESULT CALLBACK OnD3D11CreateDevice( ID3D11Device* pd3dDevice,
                                      const DXGI_SURFACE_DESC* pBackBufferSurfaceDesc,
                                      void* pUserContext )
    HRESULT hr;

    auto pd3dImmediateContext = DXUTGetD3D11DeviceContext();
    V_RETURN( g_DialogResourceManager.OnD3D11CreateDevice( pd3dDevice, pd3dImmediateContext ) );
    V_RETURN( g_SettingsDlg.OnD3D11CreateDevice( pd3dDevice ) );

    // Initialize the font
    g_pTxtHelper = new CDXUTTextHelper( pd3dDevice, pd3dImmediateContext, &g_DialogResourceManager, 15 );

    // Create other render resources here
    g_States.reset( new CommonStates( pd3dDevice ) );
    g_FXFactory.reset( new MyEffectFactory( pd3dDevice ) );

    //Draw bound box model
    t_effect.reset(new BasicEffect(pd3dDevice));
        void const*     t_shaderByteCode;
        size_t          t_byteCodeLength;

        t_effect->GetVertexShaderBytecode(&t_shaderByteCode, &t_byteCodeLength);

            t_shaderByteCode, t_byteCodeLength,

        if (FAILED(hr))
            return hr;
    t_batch.reset(new PrimitiveBatch<VertexPositionColor>(pd3dImmediateContext));

    // Load the mesh
    WCHAR str[MAX_PATH];
    V_RETURN(DXUTFindDXSDKMediaFileCch(str, MAX_PATH, strFilename));
    g_FXFactory->SetPath( L".\\Models\\" );
    //Create SDKMeshModel
    g_Model = Model::CreateFromSDKMESH( pd3dDevice, str, *g_FXFactory, true );

    // Update Buffer info Status Data.

    // Setup the camera's view parameters
    static const XMVECTORF32 s_vecEye = { 0.f, 0.f, -5.f, 0.f };
    g_Camera.SetViewParams( s_vecEye, g_XMZero );

    g_HUD.GetButton( IDC_TOGGLEWARP )->SetEnabled( true );

    return S_OK;

void CALLBACK OnD3D11FrameRender( ID3D11Device* pd3dDevice, 
                                  ID3D11DeviceContext* pd3dImmediateContext,
                                  double fTime,
                                  float fElapsedTime,
                                  void* pUserContext )
    // If the settings dialog is being shown,
    // then render it instead of rendering the app's scene
    if( g_SettingsDlg.IsActive() )
        g_SettingsDlg.OnRender( fElapsedTime );

    // Check for picked triangles

    // Clear the render target and the z buffer 
    ID3D11RenderTargetView* pRTV = DXUTGetD3D11RenderTargetView();
    pd3dImmediateContext->ClearRenderTargetView(pRTV, customColor);
    ID3D11DepthStencilView* pDSV = DXUTGetD3D11DepthStencilView();
    pd3dImmediateContext->ClearDepthStencilView( pDSV, D3D11_CLEAR_DEPTH, 1.0, 0 );

    // Get the projection & view matrix from the camera class
    mWorld = g_Camera.GetWorldMatrix();
    mView = g_Camera.GetViewMatrix();
    mProj = g_Camera.GetProjMatrix();

    // Bound Box Model model projection and view matrix

    //IA setup (context)

    // If a triangle is picked, draw it
    if (g_nNumIntersections > 0)
        // Draw the picked triangle
        UINT Strides[1];
        UINT Offsets[1];
        Strides[0] = sizeof(D3DVERTEX);
        Offsets[0] = 0;

        pd3dImmediateContext->IASetVertexBuffers(0, 1, &g_pVB, Strides, Offsets);
        pd3dImmediateContext->Draw(3 * g_nNumIntersections, 0);

        // Set render mode to wireframe
        bWireframe = true;

    // Draw 3D object
    XMVECTOR qid = XMQuaternionIdentity();
    const XMVECTORF32 scale = { 1.f, 1.f, 1.f};
    const XMVECTORF32 translate = { 0.f, 0.f, 0.f };
    const XMVECTORF32 rotate = { 0.f, 0.f, 0.f, 1.f };
    XMMATRIX local = XMMatrixMultiply(mWorld, XMMatrixTransformation(g_XMZero, qid, scale, g_XMZero, rotate, translate));
    auto pComboBox = g_SampleUI.GetComboBox( IDC_MESH_LIST );
    int meshDraw = -1;
    if ( pComboBox )
        meshDraw = PtrToInt( pComboBox->GetSelectedData() );

    auto pCheckBox = g_SampleUI.GetCheckBox( IDC_WIREFRAME );
    bWireframe = (pCheckBox) ? pCheckBox->GetChecked() : false;

    auto pBBCheckBox = g_SampleUI.GetCheckBox(IDC_BOUNDINGBOX);
    bCreateBB = (pBBCheckBox) ? pBBCheckBox->GetChecked() : false;

    //Render MODEL
    if ( g_Model )
        if ( meshDraw == -1 )
            g_Model->Draw(pd3dImmediateContext, *g_States, local, mView, mProj, bWireframe);
            int index = 0;
            for( auto lit = g_Model->meshes.cbegin(); lit != g_Model->meshes.cend(); ++lit, ++index )
                if ( index != meshDraw )

                auto mesh = lit->get();
                assert( mesh != 0 );

                modelBox = mesh->boundingBox;
                ModelCenter = modelBox.Center;

                //Prepare Mesh for Rendering
                mesh->PrepareForRendering(pd3dImmediateContext, *g_States, false, bWireframe);
                //mesh->Draw(pd3dImmediateContext, local, mView, mProj);

                for (auto fit = mesh->meshParts.cbegin(); fit != mesh->meshParts.cend(); ++fit)
                    auto t = fit->get();
                    assert(t != 0);

                    t->Draw(pd3dImmediateContext, t_effect.get(), t_inputLayout.Get());

                }   //for (auto it = mesh->meshParts.cbegin(); it != mesh->meshParts.cend(); ++it)
            }   //for (auto it = g_Model->meshes.cbegin(); it != g_Model->meshes.cend(); ++it, ++index)
        }   //if ( meshDraw == -1 ) else
    }   //if ( g_Model )

    // Render HUD
    DXUT_BeginPerfEvent( DXUT_PERFEVENTCOLOR, L"HUD / Stats" );
    g_HUD.OnRender( fElapsedTime );
    g_SampleUI.OnRender( fElapsedTime );

    static ULONGLONG timefirst = GetTickCount64();
    if ( GetTickCount64() - timefirst > 5000 )
        OutputDebugString( DXUTGetFrameStats( DXUTIsVsyncEnabled() ) );
        OutputDebugString( L"\n" );
        timefirst = GetTickCount64();

// Checks if mouse point hits geometry the scene.
    //HRESULT hr;
    XMFLOAT3 vPickRayDir, vPickRayOrig;
    XMVECTOR dir, orig;
    FLOAT fBary1, fBary2, fDist;
    const DXGI_SURFACE_DESC* pd3dsdBackBuffer = DXUTGetDXGIBackBufferSurfaceDesc();
    size_t* dwNumFaces, pIndices = 0;
    D3DVERTEX* pVertices;

    g_nNumIntersections = 0L;

    // Get the pick ray from the mouse position
    if (GetCapture())
        const XMMATRIX pmatProj = g_Camera.GetProjMatrix();
        XMFLOAT4X4 tmp;
        XMStoreFloat4x4(&tmp, pmatProj);

        POINT ptCursor;
        ScreenToClient(DXUTGetHWND(), &ptCursor);

        // Compute the vector of the pick ray in screen space
        XMFLOAT3 v;
        v.x = (((2.0f * ptCursor.x) / pd3dsdBackBuffer->Width) - 1) / tmp._11;
        v.y = -(((2.0f * ptCursor.y) / pd3dsdBackBuffer->Height) - 1) / tmp._22;
        v.z = 1.0f;

        // Get the inverse view matrix
        const XMMATRIX matView = g_Camera.GetViewMatrix();
        const XMMATRIX matWorld = g_Camera.GetWorldMatrix();
        XMMATRIX m1 = matWorld * matView;
        XMMatrixInverse(nullptr, m1);

        XMFLOAT4X4 m;
        XMStoreFloat4x4(&m, m1);

        //D3DXMatrixInverse(&m, NULL, &mWorldView);

        // Transform the screen space pick ray into 3D space
        vPickRayDir.x = v.x * m._11 + v.y * m._21 + v.z * m._31;
        vPickRayDir.y = v.x * m._12 + v.y * m._22 + v.z * m._32;
        vPickRayDir.z = v.x * m._13 + v.y * m._23 + v.z * m._33;
        vPickRayOrig.x = m._41;
        vPickRayOrig.y = m._42;
        vPickRayOrig.z = m._43;
        dir = XMLoadFloat3(&vPickRayDir);
        orig = XMLoadFloat3(&vPickRayOrig);
    // Get the picked triangle
    if (GetCapture())
        //mesh data
        for (auto Kit = g_Model->meshes.cbegin(); Kit != g_Model->meshes.cend(); ++Kit)
            for (auto Lit = (*Kit)->meshParts.cbegin(); Lit != (*Kit)->meshParts.cend(); ++Lit)
                dwNumFaces += ((*Lit)->indexCount - (*Lit)->startIndex) / 3;
                pIndices = (*Lit)->indexCount;
                ____pVertices__ = ???????????????????????????????????????__ <-- ??

                for (size_t i = 0; &i < dwNumFaces; ++i)
                    FXMVECTOR v0 = pVertices[pIndices[3 * i + 0]].p;
                    FXMVECTOR v1 = pVertices[pIndices[3 * i + 1]].p;
                    FXMVECTOR v2 = pVertices[pIndices[3 * i + 2]].p;

                    // Check if the pick ray passes through this point
                    if (IntersectTriangle(orig, dir, v0, v1, v2, &fDist, &fBary1, &fBary2))
                        if (g_bAllHits || g_nNumIntersections == 0 || fDist < g_IntersectionArray[0].fDist)
                            if (!g_bAllHits)
                                g_nNumIntersections = 0;
                                g_IntersectionArray[g_nNumIntersections].dwFace = i;
                                g_IntersectionArray[g_nNumIntersections].fBary1 = fBary1;
                                g_IntersectionArray[g_nNumIntersections].fBary2 = fBary2;
                                g_IntersectionArray[g_nNumIntersections].fDist = fDist;
                                if (g_nNumIntersections == MAX_INTERSECTIONS)
    return S_OK;
this code currently load the SDKmesh and then displays the #faces and #meshes loaded.
Nov 15, 2015 at 7:18 PM
in picking function, (grabbed from DX10 Pick Sample) dont know how to "translate" that line to Dx11.

orig line from DX10:
        DWORD* pIndices;
        D3DVERTEX* pVertices;
        pVertices = (D3DVERTEX*)g_Mesh.GetRawVerticesAt(0);
        pIndices = (DWORD*)g_Mesh.GetRawIndicesAt(0);
Nov 16, 2015 at 9:28 PM
Edited Nov 16, 2015 at 9:29 PM
The vertex and index data in Model is stored in IBs and VBs which have USAGE_DEFAULT so they are not readable by the CPU. In order to support face-by-face level picking, the Model loader needs to be modified to maintain a CPU memory array for the indices and the vertices (likely limited to only the position information) for this kind of collision testing. I have a work item open for this.

That said, face-by-face collision is extremely expensive and most real-time applications go through great pains to avoid doing it. This is usually a collision against a simple bounding volume (sphere, box, OOB, capsule, etc.) instead, or colliding against a simplified version of the original mesh (called a collision mesh). Other than as a learning exercise, what's your scenario for doing face-based picking?
Nov 17, 2015 at 5:49 AM
Thank you so much for your soon answer mr. Walbourn !

now i clearly understand the limitations of the current way that the models are loaded on CreateFromSDKMESH@Model method.

what i want to do is for academic purposes on the current laboratories on my faculty. As Ergonomic Assesments and Engineering Laboratories courses.
currently working on my PhD on Engineering, part of my research involves a solution like the Pick10 Demo from the Direct X SDK.

Basic flowline:
  1. load a stl, obj or ply model (solidworks or cad model exported), then
  2. convert to FBX/SDKMesh (using meshconvert) so we can "manipulate" the 3d data.
  3. display on screen basic model's stats as : #faces, #mesh, #faces/3 [vertices], bounding box, etc.
  4. run collision test on the current loaded model (rays from some offset outside the bound box directed to the model)
    [elaborate around 100 diferent ray test around the model] so we can retrieve their collision position of each one and calculate the lengths respect their origin.
  5. save that data.

hope i come out with some sort of solution for this. mean while ill keep on this and update this thread with my current progress.

thank you on advance !
Nov 17, 2015 at 4:27 PM
Edited Nov 17, 2015 at 4:27 PM
My plan for collision meshes is to add functionality to the Model loader that creates (in addition to everything else it does already) a std::vector<uint32_t> of the indices and a std::vector<XMFLOAT3> of positions for each ModelMeshPart for the CPU to access. This should be fairly easy to add to the existing infrastructure.
Nov 17, 2015 at 6:47 PM
that sounds perfect for what i want on my project. just grab vectors, and normals, obviously among other custom data, like the other ones structures on VertexTypes.h

ill be keep on this.

Cheers !
Nov 27, 2015 at 12:44 AM
in detail, AFAIK, DX10 interesects execute the next steps (as you stated in some MSDN topic, source below):
  1. Perform a ray-hit test against the bounding box of the mesh. If it missed, no hits. If it hit, further processing.
  2. Allocate an array of Booleans with the same number of entries as the maximum vertex index in the mesh.
  3. Generates a plane containing the test ray, and store if the vertex was 'in front' or 'behind' that test plane.
  4. For each triangle in the mesh, first check to see if all vertices in the triangle are on the same 'side' of the test plane by looking in that array. If they are, skip testing that triangle because it can't hit the ray.
  5. Compute triangle hit test. All hits are stored in an array in sorted order.
  6. All the tests in this algorithm can be performed with DirectXMath functions.
as Note: D3DX kept all the IB/VB data as both STAGING buffers and DEFAULT buffers so it could lock them for this test. This is not particularly memory efficient, as only the position information needs to be kept for the collision testing, and in many cases a simplified mesh can be used instead of the rendering mesh to speed up collision.


i understand how implement, step1 and 5 (DirectXMath and SimpleMath), the question is how can i implement steps 2,3 and 4 ?

Regards !