Python script getting stuck in the env.step(action) call in older RaiSim version (protocol version 10014) when using Windows 11 and RaisimUnity

[manuelbirlo]

I'm not sure if the following issue is linked to this repo, but since I assume there is a connection to potential issue related to older RaiSim version I'll post it here anyway:

Reproduction steps:

1.) I installed the d-grasp repo that is based on this RaiSim repo: https://github.com/christsa/dgrasp

2.) I started RaiSimUnity in https://github.com/christsa/dgrasp/tree/main/raisimUnity/win32 and clicked "Auto-connect" on the left panel of the Unity GUI.

3.) I launched the runner_motion.py script https://github.com/christsa/dgrasp/tree/main/raisimGymTorch/raisimGymTorch/env/envs/dgrasp that is based on your runner scripts here but uses the MANO hand instead of a robot. I ran the script from my raisimGymTorch folder via python raisimGymTorch/env/envs/dgrasp/runner_motion.py -ao -o '004' -sd 'pretrained_policies'

Observed behavior:

The script enters the env.step(...) wrapper function but never returns from it: reward_ll, dones = env.step(action_ll.astype('float32')) (here is the link to the code line: https://github.com/christsa/dgrasp/blob/8816d1ba80fbd95ec93142cb91695719effd05e9/raisimGymTorch/raisimGymTorch/env/envs/dgrasp/runner_motion.py#L325

I've also tested the runner.py script of the d-grasp project and observed the same problem: the script always get stuck at the env.step(action) call. I can see a visualization in RaisimUnity but there is no motion of the visualized objects, because the code getting stuck.

I tested runner.py in this official RaiSim repo with the provided RaiSimUnity and it works!
Now, I believe it has something to do with the fact that the d-grasp Projekt uses a slightly older RaiSim version: d-grasp has RaiSim protocol version 10014 while your current RaiSim repo state has protocol version 10018.

If I run d-grasp's runner_motion.py script without RaiSimUnity being connected the scripts seems to run normally and the env.step(action) calls run successfully, but as soon as I connect to RaiSimUnity the env.step(action) call never returns.

Are you maybe aware of this problem? Was there a related fix in this current repo that is not part of the d-grasp project?