Any help for multiple inputs observation space (Dict image and vector)

[shaoxiang]

I see that the demos in the Lab are all single_observation_space, but my experiment is multiple observation_space. Combine images and vectors into a multi-input observation space. I want to understand how to develop it. Can anyone help me? Also, I find it difficult to understand the meaning of num_states and there is no relevant explanation.
The following is the approach I found for Stable Baselines3:
Stable Baselines3 supports handling of multiple inputs by using Dict Gym space. This can be done using
MultiInputPolicy, which by default uses the CombinedExtractor features extractor to turn multiple
inputs into a single vector, handled by the net_arch network.

[FANG-Zhiwei]

Hi shaoxiang, have you found a solution? I also find it hard to use Dict observations.
BTW, as far as I know, the num_states/state_space are used for actor-critic policy.

[Toni-SM]

Hi @FANG-Zhiwei and @shaoxiang

These discussions may help:

• [Question] QuadCopter training with camera #1451
• Model design: IsaacLab application Toni-SM/skrl#230