Over segmentation of cells in samples with non-uniform cell geometry

[Riley-Grindle]

Hi Baysor Team,

I am a new-ish user to the baysor tool and have a series of Xenium samples our group is trying to re-segment. After multiple iterations of config settings it seems that I have yet to find an optimal segmentation profile that dynamically handles diverse cell shapes. With that said, is there a setting I may have missed, or a proper way to handle segmentation in cases of non-uniformity? I have attached my config.toml below. (Note: the prior segmentation being referenced was generated via XeniumRanger).

Additionally, it appears my z_location is being removed from associated output files.

Thank you for your consideration.

`[data]
x = "x_location"
y = "y_location"
z = "z_location"
gene = "feature_name"
min_molecules_per_cell = 150

[segmentation]
prior_segmentation_confidence = 0.2
scale = 15
scale_std = "100%"
n_cells_init = 6456`

[VPetukhov]

Hi @Riley-Grindle ,

That's a good question. scale_std, unfortunately, doesn't work as intended and in my tests its effect on the results was minor (if any). I'd try setting num. clusters to 10-15 and increase scale to be closer to the size of larger cells. Also, if your prior segmentation doesn't work well (which I assume because of confidence 0.2), you can switch to nuclei segmentation and set a higher confidence. It would help with undersegmentation.

Additionally, it appears my z_location is being removed from associated output files.

What about other columns? x/y_location and feature_name?

[Riley-Grindle]

Disregard the last part of that query. The z_location is preserved in the transcript coordinates, and the cell/nuclei boundaries were never 3 dimensional to begin with.

Thank you for the suggestions! I will initiate a test run with some of these in mind and return with feedback.

[VPetukhov]

Let me know how that works! If that helps, I'll add it to the documentation.

[Riley-Grindle]

So I set the scale_std param to 0.8 and my initial scale to the largest observed cell radius (15) which seemed to allow the algorithm to handle highly variable sizes. The only thing i am now noticing is that there are cell boundaries that overlap slightly. Is this to be expected?

[VPetukhov]

Does that mean that the solution helped? :)

The only thing i am now noticing is that there are cell boundaries that overlap slightly. Is this to be expected?

Do you have 3D data? If so, it's often impossible to create non-overlapping polygons in 2D.