proposing sensor message for markers/fiducials

[maxbader]

Hi

We enhanced the stage ros simulation to work with fiducials and I also wrote a ros package to detect visual markers (v4r_artoolkitplus, v4r_ellipses) in all this cases a common sensor message would be needed to represent a detected marker as pose with an id. In addition a second message is needed to represent the detection with a list of marker (pose+id), header min, max sensor range of the detection.

Marker.msg

# This expresses a marker/fiducial in a 3D space
uint32 id                    # id of the detected fiducial (0 if id could not be identified)
geometry_msgs/Pose pose      # pose of the detected marker/fiducial

MarkerDetection.msg

# This expresses a marker/fiducial detection
Header header                 # timestamp in the header is the acquisition time and frame

float32 distance_min          # distance minimum range value [m]
float32 distance_max          # distance maximum range value [m]
float32 distance_max_id       # distance maximum range value to detect id [m]

float32 angle_horizontal_min  # start horizontal angle of the detection [rad]
float32 angle_horizontal_max  # end horizontal angle of the detection [rad]
float32 angle_vertical_min    # start vertical angle of the detection [rad]
float32 angle_vertical_min    # end vertical angle of the detection [rad]

Marker[] markers              # detected markers

MarkerStamped.msg

# An marker with reference coordinate frame and timestamp
Header header
Marker marker

MarkerWithCovariance.msg

# This expresses a marker in free space with uncertainty.

Marker marker

# Row-major representation of the 6x6 covariance matrix
# The orientation parameters use a fixed-axis representation.
# In order, the parameters are:
# (x, y, z, rotation about X axis, rotation about Y axis, rotation about Z axis)
float64[36] covariance

MarkerWithCovarianceStamped.msg

# MarkerWithCovarianceStamped.msg
# This represents a marker with reference coordinate frame and timestamp and covariance matrix
Header header
MarkerWithCovariance marker

[gavanderhoorn]

Standardising marker detection msgs seems like a good idea.

Quick observation: I don't see a confidence field anywhere. Some trackers include this information, and especially in very noisy environments this can be important. I'm not sure if you intended the WithCovariance msg types to convey that information? Typically those are only used to encode uncertainty wrt measurements, right? I'm not sure whether something like confidence in the detection as a whole can be expressed that way?

[maxbader]

Very good point!
Three ideas:

1. We thought it would be good to add sigma values to the Detector message for spherical coordinates but that would mean that all marker have the same covariance.
2. to add an additional vector in the detection for the covariance of each marker on this was one can use it or not. Problem there is no covariance massage and no multidimensional array :-(
3. to add a noise vector to the Marker.msg which is empty or has six entries for a polar noise model.

I would go for the option 2 and to add a covariance message to the geometry_pkg to unify in general the covariance representation in 3D.

[jack-oquin]

The basic concept seems good. But, especially for new sensor_msgs entries, the exact proposal deserves considerable review. We can do that here, but some discussion on [email protected] seems appropriate.

Specifically, the first question that comes to mind is: does this need to go in sensor_msgs or could it have its own package? If so, does that package need to be in the common_msgs repository?

I am also concerned that the angle_{horizontal|vertical}_{min|max} fields need a clear definition. When dealing with angles, min and max can be problematic, especially in the region around Pi and -Pi radians. Should we consider using quaternions instead, as we do for poses?

[jack-oquin]

Opened a mailing list thread here: https://groups.google.com/forum/#!topic/ros-sig-drivers/QgCL__X5isw

[maxbader]

Hi

using angle_{horizontal|vertical}_{min|max} has the benefit to define a field of view and of course one hast to be careful not to go beyond 2pi. The information is needed if one likes to turn a robot to search for a marker or likes to fully explore an environment with the sensor.

[jack-oquin]

How do you distinguish a front semi-circle (-Pi/2 .. Pi/2) from a rear semi-circle with the same angles?

[maxbader]

you are right it is not that simple without additional agreements.
instead of angle_{horizontal|vertical}_{min|max} we could add a could add a quaternion for the view direction and two variables to describe the field of view.

geometry_msgs/Quaternion view_direction       # view direction
float32 fov_horizontal                        # field of view horizontal [rad]
float32 fov_vertical                          # field of view vertical [rad]

[jack-oquin]

That seems better. With the Velodyne driver we ended up using two parameters: view_direction and view_width (which is a 2D constraint).

Some did not agree with that approach, but 360 degree devices are tricky, even in two dimensions.

[maxbader]

We made a implementations using the MarkerDetection messages for stage_ros publishing fiducials.
ros-simulation/stage_ros#36 and soon we like to publish a node for doing slam using fiducial messages form stage and my artoolkitplus node.
For the moment we placed the messages description and generation in stage_ros but we also have a brunch in https://github.com/tuw-robotics/stage_ros without_stage_ros_msgs which is using a common_msgs fork located at https://github.com/tuw-robotics/common_msgs
The current solution placing the message into stage_ros is not nice, It would be much nicer to have the marker messages in sensor_msgs to allow also notes like artag detentions to use it.
Maybe we should also consider to name all proposed sensor_msgs/Marker*.msgmore general like sensor_msgs/Pose*.msg which distinguishes itself to the geometry_msgs/Pose*.msg messages by having an ID value.
We are also happy to provide a riz plug-in to visualize the proposed messages.

Should I place a pull request on my common_msgs fork with the new marker messages and yes to which brunch?

[jack-oquin]

Why is sensor_msgs a better location than some new package name, like marker_msgs?

[tfoote]

Two things to note. Please review the guidelines for submitting to common_msgs. And secondly Marker has both multiple meanings and a name collision with an existing message. visualization_msgs/Marker

With the above in mind. I'd suggest adapting the proposal to be a fiducial_msgs package and use Fiducial instead of Marker

Also there are already several messages for fiducials defined and in use. Off the top of my head I can think of ar_track_alvar_msgs Please make an effort to compare and contrast this approach to that one and see if there are other similar packages already defined in the community as well.

[maxbader]

I know there are other similar message out there but that's exactly a issue why I like to proposed a set of marker messages. ar_track_alvar_msgs are not supporting sensor parameters such as fov, min max range, ....

@Naming fiducial_msgs: Marker are not only vision based, eg. RFID
@collision with visualization_msgs/Marker: the namespace (pkg name) should solve the problem

Markers are used in many application and having a common marker message will reduce the number of dependencies to pkgs.

[jack-oquin]

I know there are other similar message out there but that's exactly a issue why I like to proposed a set of marker messages. ar_track_alvar_msgs are not supporting sensor parameters such as fov, min max range, ....

Have you asked @sniekum to add the fields or extra messages you need to ar_track_alvar_msgs?

If you don't like that name, maybe you and he and others could discuss the intended scope and agree on more inclusive terminology. The best way to get really effective common messages is leveraging the community's experience.

[tfoote]

@Naming fiducial_msgs: Marker are not only vision based, eg. RFID

Neither are fiducials. From google: "(especially of a point or line) assumed as a fixed basis of comparison." Merriam-Webster "taken as standard of reference "

@collision with visualization_msgs/Marker: the namespace (pkg name) should solve the problem

Yes, it's not a technical problem. But people like to use shorthand when communicating. And "Marker" out of context does not communicate the semantic meaning. And if anyone who's been using the system already would assume that you're referring to the already commonly used message for displaying markers in 3d space in rviz.

Markers are used in many application and having a common marker message will reduce the number of dependencies to pkgs.

I think everyone generally agrees with that statement. We just need to figure out clearly what a "common marker message" is. The first thing to do is to define the scope of what we're talking about including what use cases we want to support. From there we can determine what's generic and shared vs what's application or use case specific.

We both want things to be generic so as to be reusable. But the data also has to be self contained and semantically meaningful.

My suggestion to use fiducial is to bring in the semantic that these are reference positions. And not generic markers such as sign posts, or generic labels such as sign posts or an indicator of presence but not location as all of those would fit a "marker" defnition.

Before we get too far into the details about the data structures we need to agree on what information we want to capture, and how we plan to use it. For example things like the id field, why is that a uint32? What's the key, and how do you expect to differentiate it? Similarly the type of object was observed would be very valuable. If you're observing an ar tag, a qr code, an RFID, and a dot pattern. Which ones are captured and how does a data consumer know that? And should things like the header be replicated with the pose, as in a PoseStamped such that they can be easily used and iterated or at a higher level?

As a recent example of standardizing a common message see the new BatteryState message: #74 In this one we merged two existing messages and improved on them based on experience. An important part early was to define the scope and background to get everyone on the same page.

[maxbader]

What about that, I will make a new pkg marker_msg with the proposed messages types.
In addition we will offer a pkgs to view the marker in rviz marker_rviz_plugin, a ekf-slam marker_ekf_slam and the integration into tuw_artoolkitplus, tuw_ellipses as well as in stage_ros.
Hopefully the stage_ros community will accept our fiducial pull request with an additional dependency to marker_msg

[jack-oquin]

+1 for the package name fiducial_msgs.

I agree with Tully that "marker" already has other meanings within the ROS eco-system.

[maxbader]

A type field is a good idea but the name under which the message is published can serve as identifier for the type unless you like to mix in one detection multiple types.

we also have to be careful with the meaning of confidence

• - pose confidence: covariance matrix I would not place this one into the Marker.msg and MarkerDetection.msg because the algorithm using the detected markers such as an ekf-slam has normaly its own noise model for a detected maker in spherical coordinates. But the results of such an algorithm is typically a vector with MarkerWithCovarianceStamped in an euclidean system. Therefore with need a message Markers.msg with a vector of MarkerWithCovarianceStamped.
• - id confidence: confidence on the detected id. Two vectors per marker(ids and the id_confidences) could be used for that which also solves problem of an unknown id by having empty vectors

[peci1]

Interesting discussion on a much needed topic :)

Just to add my bit from a completely different point of view: for me as a non-native English speaker, the term fiducial is a very difficult and unusual word. More than 50% of EN-CS dictionaries do not know this word. And in those that do, I'm not sure the translation is good enough - it's possible Czech just doesn't have a word for this term. The words usually output by dictionaries can be translated back as default, but I feel that's not the right meaning (any native speakers please correct me).

Thus I'd be against the name fiducial_msgs, because there might be people in the world who can't tell just by the name what is in such a package. And common_msgs should IMO be easily understandable.

What about calibration_marker_msgs, spatial_marker_msgs or positional_marker_msgs?

[peci1]

Of fiducial_marker_msgs to at least give another word helping to track down the meaning of the package name...

[gavanderhoorn]

It is indeed not a well-known word (in general discourse I guess), but fiducial is the proper name for the concept. See Wikipedia fi: Fiducial_marker. Also: Merriam Webster: fiducial.