Hands solution Side packet issue

Description

I’ve just done your tutorial (FaceMesh) and trying to make a Hand Tracking project, refering to the same tutorial. But in the middle of the phases, I got a novel error. (I am very very new to C#, so probably my code is ridiculous…)

This is the error

MediaPipeException: INVALID_ARGUMENT: ValidateRequiredSidePackets failed to validate: 
; Side packet "input_horizontally_flipped" is required but was not provided.
; Side packet "input_rotation" is required but was not provided.
; Side packet "input_vertically_flipped" is required but was not provided.
; Side packet "num_hands" is required but was not provided.
Mediapipe.Status.AssertOk () (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Framework/Port/Status.cs:149)
HandsPractice+<Start>d__10.MoveNext () (at Assets/HandLandmarkPractice/HandsPractice.cs:60)
UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) (at <d3b66f0ad4e34a55b6ef91ab84878193>:0)

This is my code so far

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using Mediapipe;
using Mediapipe.Unity;
using Mediapipe.Unity.CoordinateSystem;

using Stopwatch = System.Diagnostics.Stopwatch;

public class HandsPractice : MonoBehaviour
{
    [SerializeField] private TextAsset _configAsset;
    [SerializeField] private RawImage _screen;
    [SerializeField] private int _width;
    [SerializeField] private int _height;
    [SerializeField] private int _fps;

    private CalculatorGraph _graph;
    private ResourceManager _resourceManager;

    private WebCamTexture _webCamTexture;
    private Texture2D _inputTexture;
    private Color32[] _pixelData;

    private IEnumerator Start()
    {
        if (WebCamTexture.devices.Length == 0)
        {
            throw new System.Exception("Web Camera devices are not found");
        }

        // Turn on Webcam
        var webcamDevice = WebCamTexture.devices[0];
        _webCamTexture = new WebCamTexture(webcamDevice.name, _width, _height, _fps);
        _webCamTexture.Play();

        // return coroutine function
        yield return new WaitUntil(() => _webCamTexture.width > 16);

        _screen.rectTransform.sizeDelta = new Vector2(_width, _height);
        _screen.texture = _webCamTexture;

        // Bring Hands moedel
        _resourceManager = new LocalResourceManager();
        yield return _resourceManager.PrepareAssetAsync("hand_landmark_full.bytes");
        yield return _resourceManager.PrepareAssetAsync("hand_landmark_lite.bytes");
        yield return _resourceManager.PrepareAssetAsync("hand_recrop.bytes");
        yield return _resourceManager.PrepareAssetAsync("handedness.txt");

        var stopwatch = new Stopwatch();

        // image frame for input stream
        _inputTexture = new Texture2D(_width, _height, TextureFormat.RGBA32, false);
        _pixelData = new Color32[_width * _height];

        // CalculatorGraph initialize
        _graph = new CalculatorGraph(_configAsset.text);
      
        // Bring landmarks ourputstream
        var handLandmarksStream = new OutputStream<NormalizedLandmarkListVectorPacket, List<NormalizedLandmarkList>>(
            _graph, "hand_landmarks");

        // polling
        handLandmarksStream.StartPolling().AssertOk();

        // Run graph
        _graph.StartRun().AssertOk();

        stopwatch.Start();

        // Bring Rectransform of Screen
        var screenRect = _screen.GetComponent<RectTransform>().rect;

        while (true)
        {
            _inputTexture.SetPixels32(_webCamTexture.GetPixels32(_pixelData));

            // ImageFrame initialize
            var imageFrame = new ImageFrame(
                ImageFormat.Types.Format.Srgba,
                _width, _height,
                _width * 4,
                _inputTexture.GetRawTextureData<byte>());
            // Make Timestamp
            var currentTimestamp = stopwatch.ElapsedTicks / (System.TimeSpan.TicksPerMillisecond / 1000);

            // imageFrame을 packet에 싸서 inputstream으로 보내기
            _graph.AddPacketToInputStream("input_video", new ImageFramePacket(imageFrame, new Timestamp(currentTimestamp))).AssertOk();

            yield return new WaitForEndOfFrame();

            // Get landmarks values
            if (handLandmarksStream.TryGetNext(out var handLandmarks))
            {
                foreach (var landmarks in handLandmarks)
                {
                    var fingerTip = landmarks.Landmark[8];
                    Debug.Log(screenRect.GetPoint(fingerTip));
                }
            }
        }
    }

    private void OnDestroy()
    {
        if (_webCamTexture != null)
        {
            _webCamTexture.Stop();
        }
        if (_graph != null)
        {
            try
            {
                _graph.CloseInputStream("input_video").AssertOk();
                _graph.WaitUntilDone().AssertOk();
            }
            finally
            {
                _graph.Dispose();
            }
        }
    }
}

I tried to provide side packets by adding this code right below running graph.

var sidePacket = new SidePacket();
sidePacket.Emplace("input_horizontally_flipped", new BoolPacket(true));
sidePacket.Emplace("input_rotation", new IntPacket(0));
sidePacket.Emplace("input_vertically_flipped", new BoolPacket(true));
sidePacket.Emplace("num_hands", new IntPacket(2));

but error still exists.

This is how I set objects and Inspector. Exactly same with the tutorial.

I have no idea where to start T.T Please give me any advices you can give.

[G93-7]

I think you must create a side packet before you run the graph. After that you must pass the side packet as a parameter to the StartRun function like this: _graph.StartRun(sidePacket).AssertOk();

[azmath37]

Hi @Hyempire I am facing same issue, did you find any solution…?