Revisiting basic landmark position printing

Hello, I’m trying to print the landmarks at each frame for the face mesh model just as a proof of concept that I can access these positions. I looked through https://github.com/homuler/MediaPipeUnityPlugin/issues/5, https://github.com/homuler/MediaPipeUnityPlugin/issues/23, https://github.com/homuler/MediaPipeUnityPlugin/issues/55, https://github.com/homuler/MediaPipeUnityPlugin/issues/43, and what I thought we do is put the below code in RenderOutput:

public override void RenderOutput(WebCamScreenController screenController, TextureFrame textureFrame) {
        graph.ObserveOutputStream("face_landmarks", LandmarksCallback).AssertOk();
        if (multiFaceLandmarksStreamPoller != null)
        {
            Debug.Log("Fetching landmarks");
            var landmarks = FetchNext(multiFaceLandmarksStreamPoller, multiFaceLandmarksPacket, "face_landmarks");
            Debug.Log(landmarks[0]);
        }
        else
        {
            Debug.Log("can't fetch landmarks");
        }
        var faceMeshValue = FetchNextFaceMeshValue();
    RenderAnnotation(screenController, faceMeshValue);

    screenController.DrawScreen(textureFrame);
  }

In this function I get error that LandmarksCallback cannot convert to CalculatorGraph.NativePacketCallback.

And I added the following method in the same file (FaceMeshGraphs.cs):

static IntPtr LandmarksCallback(IntPtr ptr)
    {
        using (var packet = new NormalizedLandmarkListVectorPacket(ptr, false)) //originally this was NormalizedLandmarksListVectorPacket (note the extra s in Landmark)
        {
            using (var landmarks = packet.Get())
            {
                // do something
            }
            return Status.Ok().mpPtr;
        }
    }

But I get that IntPtr cannot be found, and NormalizedLandmarkList must be implicitly convertilce to IDisposable. Please let me know if I’m on the right track or any fixes!

[Jatin-Pawar]

Hey @ksyao2002

Are you trying to see if we get landmarks for the face and what all data do we get?

I am also trying to do the same. I looking into this project to see what all data do we get from MediaPipe in this plugin. I am interested in 3D coordinates (x,y,z) of the face mesh 438 points so that I can use these to attach objects like glasses or something.

I would be happy to connect and discuss/share my findings and doubts.

Kind Regards ~ jk!ng

[homuler]

I’m sorry I can’t show you working code soon, but I’ll update sample graphs later. For now, please refer to InstantMotionTrackingGraph. This graph outputs debug logs and I hope you can get some ideas from it.

A callback is defined here… https://github.com/homuler/MediaPipeUnityPlugin/blob/2ba2cbc8813735ddfbed3abd2124ce9500efddf7/Assets/Mediapipe/Samples/Graphs/InstantMotionTracking/Scripts/InstantMotionTrackingGraph.cs#L90-L107

and it is passed to MediaPipe here. https://github.com/homuler/MediaPipeUnityPlugin/blob/2ba2cbc8813735ddfbed3abd2124ce9500efddf7/Assets/Mediapipe/Samples/Graphs/InstantMotionTracking/Scripts/InstantMotionTrackingGraph.cs#L57