Targets not updated correctly in HandReach-v0 #2061
Description
It seems that the HandReach-v0 environment is not updated correctly using the latest versions of gym (0.17.3), mujoco-py (2.0.2.13), Mujoco 2.0 and Python 3.8.6 running on Ubuntu 18.04. The hand is rendered and updated correctly (see screenshot below) but the finger tip marker and target marker positions are not updated correctly.
The environment should update the positions (self.sim.model.site_pos[site_id]) in the self._render_callback function - however these changes seem to be ignored by mujoco and reset at the next timestep. I have observed this issue both when rendering directly to display (mode="human") and when rendering to image files (mode="rgb_array").
As this environment was developed for an older version of Mujoco, I assume this may have something to do with how Mujoco 2.0 handles site positions nested inside bodies (as can be seen in the default xml file. I already fixed this issue by moving the markers outside of the <body name="floor0" ... > tag. If anyone is able to replicate this problem, I will create a pull request with my fix.