Enhancement: Provide unmerged dump ordered by directory name

[amaltaro]

Enhancement Description

The WM MSUnmerged service consumes RSE Unmerged dump from the Rucio Consistency Monitoring system, an API like:
https://cmsweb.cern.ch/rucioconmon/unmerged/files?rse=T1_ES_PIC_Disk&format=json

It would be helpful if the list of files would be grouped/ordered by directory name, such that it's easier to iterate through the list of files to be considered for deletion, in addition to consuming less resources.

Use Case

To better deal with memory footprint in the WM MSUnmerged, where it can be many GBs of memory for large json dumps (common for FNAL_Disk and T2_CH_CERN).

Possible Solution

Perhaps consuming a format different than the JSON object, such that we can have a better control of what we load into memory. Still, if data is not ordered, it can be extremely inefficient both for the service and for the storage.

Related Issues

No response