[Windows Editor] Exported Package looks for .bytes files in local AppData Directory in new project, doesn't copy Streaming Assets folder

Plugin Version or Commit ID

v0.9.1

Unity Version

2021.2.19f1

Your Host OS

Windows 10

Target Platform

UnityEditor, Windows Standalone

Target Device

Windows 10 Desktop/Editor

[Windows Only] Visual Studio C++ and Windows SDK Version

Visual Studio C++ : 14.29.30133 Windows SDK : 10.0.18362.0

[Linux Only] GCC/G++ and GLIBC Version

No response

[Android Only] Android Build Tools and NDK Version

No response

[iOS Only] XCode Version

No response

Build Command

python build.py build --desktop cpu --opencv cmake -vv

Bug Description

Using this project cloned from this commit, I built Mediapipe for Windows Desktop using Docker, and edited the Mediapipe_api>BUILD file to exclude the things we didn’t need so it just had hand tracking for desktop.

We exported the package using the Tools > Export package command, and imported into a brand new project.

Exported package

Once imported, we had the errors due to hand_recrop.bytes not being in the package imported somehow.

HandTrackingGraph: System.NullReferenceException: Object reference not set to an instance of an object
  at Mediapipe.Unity.LocalResourceManager+<PrepareAssetAsync>d__11.MoveNext () [0x000b3] in C:\Unity_Projects\TestMedImport\Packages\com.github.homuler.mediapipe\Runtime\Scripts\Unity\ResourceManager\LocalResourceManager.cs:60 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.GraphRunner/<Initialize>d__39:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/GraphRunner.cs:132)
Mediapipe.Unity.WaitForResult/<Run>d__20:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/WaitForResult.cs:51)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

and

HandTrackingSolution: Mediapipe.InternalException: Failed to prepare dependent assets
  at Mediapipe.Unity.GraphRunner+<Initialize>d__39.MoveNext () [0x00180] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\GraphRunner.cs:134 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.ImageSourceSolution`1/<Run>d__12<Mediapipe.Unity.HandTracking.HandTrackingGraph>:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/ImageSourceSolution.cs:79)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

To fix this, I noticed that the Bootstrap Asset Loader Type was loading from ‘Local’ location by default which for us was this: C:\Users####USERNAME####\AppData\LocalLow\DefaultCompany\MediaPipeUnityPlugin

Of course in a new project when importing the package, the files from AppData don’t carry across to the new project directory so that rules out Local loading in the package by default unless users are told to move the files. I had t oexplicitly copy hand_recrop.bytes.

I then switched to ‘Streaming Assets’ Asset Loader Type then I noticed the Streaming Assets folder with the bytes file was not copied across in the exported package either, and for some reason the plugin built the Streaming Assets folder and placed it in the project directory > Packages folder (See in the example project.)

To finally get things working, I had to do this:

1. Create new Streaming Assets folder in Assets folder of new project
2. Copy files from the AppData folder from the Mediapipe project I used to build the package - in C:\Users####USERNAME####\AppData\LocalLow\DefaultCompany\MediaPipeUnityPlugin
3. Copy the files from Packages>Streaming Assets in the Mediapipe project I used to build the package to the new project’s Streaming Assets folder
4. In the Hand tracking scene on Solution > Bootstrap, I had to set the Asset Loader Type to ‘Streaming Assets’

If this behaviour is unique to Windows 10 editor/builds, can it be simplified or documented for users?

Steps to Reproduce the Bug

1. (Building not required as I already included Built mediapipe in this project) Using this project cloned from this commit, I built Mediapipe for Windows Desktop using Docker (BUILD file edited to just build hand tracking and a bit of Pose Tracking)
2. Make a new Unity project fresh
3. In the built mediapipe project, use Tools>Export Package
4. Open fresh project, import the package and try to run the sample scenes
5. See errors that hand_recrop.bytes is missing

Log

Errors from editor:

HandTrackingGraph: System.NullReferenceException: Object reference not set to an instance of an object
  at Mediapipe.Unity.LocalResourceManager+<PrepareAssetAsync>d__11.MoveNext () [0x000b3] in C:\Unity_Projects\TestMedImport\Packages\com.github.homuler.mediapipe\Runtime\Scripts\Unity\ResourceManager\LocalResourceManager.cs:60 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.GraphRunner/<Initialize>d__39:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/GraphRunner.cs:132)
Mediapipe.Unity.WaitForResult/<Run>d__20:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/WaitForResult.cs:51)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

and

HandTrackingSolution: Mediapipe.InternalException: Failed to prepare dependent assets
  at Mediapipe.Unity.GraphRunner+<Initialize>d__39.MoveNext () [0x00180] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\GraphRunner.cs:134 
  at Mediapipe.Unity.WaitForResult+<Run>d__20.MoveNext () [0x0007c] in C:\Unity_Projects\TestMedImport\Assets\MediaPipeUnity\Samples\Common\Scripts\WaitForResult.cs:51 
UnityEngine.Logger:LogError (string,object)
Mediapipe.Unity.MemoizedLogger:LogError (string,object) (at Assets/MediaPipeUnity/Samples/Common/Scripts/MemoizedLogger.cs:199)
Mediapipe.Unity.Logger:LogError (string,object) (at Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/Logger.cs:90)
Mediapipe.Unity.ImageSourceSolution`1/<Run>d__12<Mediapipe.Unity.HandTracking.HandTrackingGraph>:MoveNext () (at Assets/MediaPipeUnity/Samples/Common/Scripts/ImageSourceSolution.cs:79)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)

Screenshot/Video

No response

Additional Context

No response

[homuler]

Sorry, hand_recrop.bytes is actually not needed for HandTrackingGraph. #606 fixed it and also enables us to select required solutions at build time.

[homuler]

First, please forget about folders under C:\Users\USERNAME\AppDatal\LocalLow. This error occurs because hand_recrop.bytes is not included in the package (AssetDatabase fails to load the file and asset is null, so at least the Null Check is missing here). https://github.com/homuler/MediaPipeUnityPlugin/blob/77ba1a4d2249b8db09b3a1e975a01f6e50d21e33/Packages/com.github.homuler.mediapipe/Runtime/Scripts/Unity/ResourceManager/LocalResourceManager.cs#L52-L60

I didn’t include it as it should be included in the exported package file when I use Tools>Export Package

I mean MediaPipeUnity.0.9.1.unitypackage you uploaded actually does not include hand_recrop.bytes. I guess hand_recrop.bytes does not exist in the repository used to build the package (not the project folder used to test the package) in the first place.

I built Mediapipe for Windows Desktop using Docker, and edited the Mediapipe_api>BUILD file to exclude the things we didn’t need so it just had hand tracking for desktop.

Have you included hand_recrop.bytes? I recommend that you first check to see if the same error occurs without changing mediapipe_api/BUILD.