No trajectory returned and freeze | Hololens 2 Development

Describe the bug Hi. I’m trying to set up the pick and place tutorial in Hololens 2. When I try to change the position of the Robot along with the Table, Target, and TargetPlacement, I get the error ``No trajectory returned from MoverService.``` and the gameplay freezes.

I would like to move the whole setup away from the (0, 0, 0) position since the same position is occupied by the Hololens2 upon deployment.

I’m assuming I have to update the robot’s position on the ROS side when I change the robot’s position in Unity. Would someone kindly guide me how to resolve this?

To Reproduce Steps to reproduce the behavior:

1. Add MRTK to your pick and place project.
2. Move the Robot along with the Table, Target, and TargetPlacement to a location other than (0, 0, 0). (You can add all of them to an empty object as children).
3. Press on Publish.
4. See errors shown below.

Console logs / stack traces

Unity Side

No trajectory returned from MoverService.
UnityEngine.Debug:LogError (object)
TrajectoryPlanner:TrajectoryResponse (RosMessageTypes.NiryoMoveit.MoverServiceResponse) (at Assets/Scripts/TrajectoryPlanner.cs:156)
Unity.Robotics.ROSTCPConnector.ROSConnection/<SendServiceMessage>d__84`1<RosMessageTypes.NiryoMoveit.MoverServiceResponse>:MoveNext () (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@c27f00c6cf/Runtime/TcpConnector/ROSConnection.cs:289)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<RosMessageTypes.NiryoMoveit.MoverServiceResponse>:SetResult (RosMessageTypes.NiryoMoveit.MoverServiceResponse)
Unity.Robotics.ROSTCPConnector.ROSConnection/<SendServiceMessage>d__85`1<RosMessageTypes.NiryoMoveit.MoverServiceResponse>:MoveNext () (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@c27f00c6cf/Runtime/TcpConnector/ROSConnection.cs:320)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1<object>:SetResult (object)
Unity.Robotics.ROSTCPConnector.TaskPauser/<PauseUntilResumed>d__5:MoveNext () (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@c27f00c6cf/Runtime/TcpConnector/TaskPauser.cs:28)
System.Threading.CancellationTokenSource:Cancel ()
Unity.Robotics.ROSTCPConnector.TaskPauser:Resume (object) (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@c27f00c6cf/Runtime/TcpConnector/TaskPauser.cs:33)
Unity.Robotics.ROSTCPConnector.ROSConnection/<>c__DisplayClass112_1:<ReceiveSysCommand>b__1 (string,byte[]) (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@c27f00c6cf/Runtime/TcpConnector/ROSConnection.cs:723)
Unity.Robotics.ROSTCPConnector.ROSConnection:Update () (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@c27f00c6cf/Runtime/TcpConnector/ROSConnection.cs:583)

ROS Side

[ INFO] [1646094286.105217000]: Loading robot model 'niryo_one'...
[ INFO] [1646094286.105271900]: No root/virtual joint specified in SRDF. Assuming fixed joint
[ INFO] [1646094287.141101200]: Ready to take commands for planning group arm.
[ INFO] [1646094287.357652500]: Planning request received for MoveGroup action. Forwarding to planning pipeline.
[ INFO] [1646094287.357962800]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1646094287.358096100]: RRTConnect: Starting planning with 1 states already in datastructure
[ERROR] [1646094292.366276100]: RRTConnect: Unable to sample any valid states for goal tree
[ INFO] [1646094292.366361000]: RRTConnect: Created 1 states (1 start + 0 goal)
[ INFO] [1646094292.366395100]: No solution found after 5.008331 seconds
[ INFO] [1646094292.377677600]: Unable to solve the planning problem
[ WARN] [1646094292.378002900]: Fail: ABORTED: No motion plan found. No execution attempted.

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots or videos to help explain your problem.

Environment (please complete the following information, where applicable):

• Unity Version: Unity 2020.3.11f1
• Unity machine OS + version: Windows 10]
• ROS machine OS + version: Ubuntu 18.04, ROS Melodic]
• ROS–Unity communication: Docker
• Branch or version: v0.7.0

[peted70]

@saratrajput ok, let me know if it works ok and I will submit a PR for it. My initial quick test worked but am testing it further here.

[peted70]

I think this might be a case of changing the Target and TargetPlacement usage from transform.position to transform.localPosition in TrajectoryPlanner.cs. (I haven’t fully tested this but a quick test seemed to work).