Orthophoto color correction before colorizing Clouds

[jistiak]

I have been trying Myria3D on some publicly available lidar data and it performs very well in most cases. One observation from my side is that, Myria performs better with data which are recorded during summer or are more colorful and vibrant in general. If the orthophotos are lacking color, specially for vegetation, forest, lakes, rivers etc objects, the model misclassifies a lot, because they all kind of look the same. The model also detects synthetic sports grounds or grassy fields as water. So I was thinking if there is a way to do color correction on the orthophotos before colorizing the point clouds so that the results are better/comparable regardless of the flight conditions. What about the infrared band then? How should I process that?

I don't know if all the training data for Myria were taken very homogeneously or you implement some kind of color correction too. Do you even think it's a good idea?

An example of two adjacent tiles where the conditions are very different.

[CharlesGaydon]

Hi @jistiak, thanks for your post and your insights! Indeed the differences between adjacent orthophotos can be huge due to different conditions and periods of acquisitions. Althougt the orthophotos go through a post-processing including cosmetic spectral correction, they may indeed lack contrast.

The current production model was trained on ~150km² of data in a relatively narrow area. A new model is pre-reployed on branch staging-20230930_60k_basic_targetted_epoch37_Myria3DV3.4.0, which was trained on the same volume of data but extracted from very diverse regions (10 x 25000km² areas in the south of France). So you may want to check if it is more robust to spectral variations and low contrasts. We would love to have some feedback if you do such tests, since we are in the process of evaluating if this new model generalizes better to new areas.

Here is how color data is currently integrated:
Before being fed to the model, the spectral information (R-G-B-I channels) is scaled to 0-1 interval (by the max theorerical value of the challen = 255*256). The points with echo number > 1 are considered to be occluded and their color is set to 0. It is not further normalized.
However, a "color intensity channel" is created on the fly as (R+G+B)/3. This channel is later standardized on a Lidar sample (50mx50m) basis (see this transform). This gives a color intensity channel that should have higher local contrast, although we lose the absolute values of the color intensity in the process.

Based on what you are observing, I think there would be some great benefit to preprocessing the orthophotos to have more systematic color values and contrast. There are probably methods out there for RGB channels, but I am not very familiar with that. I'll try to look into it.

[CharlesGaydon]

I got some additional information regarding the post-processing of IGN's orthophotos after acquisition:

• Date of aerial image acquisition (PVA): Aerial image acquisitions can take place from April to November, and multiple dates may be considered to cover a department. This introduces variability based on whether the PVA dates are suitable for distinguishing classes and not too heterogeneous in terms of intra-department acquisition dates.
• Radiometric equalization follows. Subsequently, there is an automatic algorithm known as "physical equalization" that aims to minimize radiometric differences between images. Typically, this process mitigates the intra-department radiometric variability dependent on PVA dates and times, as well as the positioning of the images relative to the sun (hot spot).
• Finally, the last phase, known as enhancement, involves choosing curves per channel, such as gamma curves, for each department. This ensures that grays are truly gray, whites are truly white, and the visual result is aesthetically pleasing.