ENH: Coreg GUI

[GuillaumeFavelier]

This PR adds a CoregistrationUI class which acts as the the user interface for coregistration that relies on pyvistaqt.

This is still a work in progress, the UI components are not connected to Coregistration yet.

GUI API

• add check box
• add radio button (incomplete: the connections are not tested)
• add input text field
• add WidgetList classes to handle connections to a group of widgets
• add file/folder button (incomplete: the "Load" button modifies the field but not the other way around)
• add multiple dock system (incomplete: only temporary solution for ipywidgets)

Application

• use traitlets
• change subjects_dir
• change subject
• load fiducial file
• make head skin surface transparent
• change & omit distance
• change scaling mode
• change icp number of iterations
• change fit point matching method
• fit fiducials
• fit icp
• save and load trans
• show high resolution head
• load info file and reset
• default unit to mm to avoid 0.00x
• change the default omit_hsp_distance to 10.0mm
• edit fiducials weights
• edit fitting parameters
• edit other weights (hsp, eeg, hpi)
• check ipywidgets.FileUpload for _dock_add_file_button (works for file path, not directory or for saving files)
• refactor plot_alignment()
• show hsp
• grow hair
• allow picking of fiducials
• omit distance
• edit fiducials coords
• restore the checkboxes when the fiducials are unlocked
• widget command forwarding
• add new app to mne/commands

Feedback

• selecting a fiducial changes the view (from ENH: Coreg GUI #9689)
• Update the scene after each iteration of fit_icp (ENH: Coreg GUI #9689 (comment))
• allow picking through the MRI markers (useful for micro-adjustments)
• still work when no measurement data is provided
• the widget to load the digitization data says 'Save' instead of 'Load' or 'Open'
• set fiducials when no measurement data is provided

Known bugs

• inconsistent behaviour of checkbox layout for ipywidgets
• hiding the EEG channels removes one sphere at a time (not a smooth UI feedback)
• the EEG channels' glyphs cannot be oriented and scaled by the distance (not yet)

It's an item from #8833

Current status
qt

ipywidgets

import os.path as op
import mne
from mne.viz._coreg import CoregistrationUI
from mne.io import read_info

data_path = mne.datasets.sample.data_path()
subjects_dir = op.join(data_path, 'subjects')
subject = 'sample'
fname_raw = op.join(data_path, 'MEG', subject, subject + '_audvis_raw.fif')
fname_trans = op.join(data_path, 'MEG', subject,
                      subject + 'audvis_raw_auto-trans.fif')
src = mne.read_source_spaces(op.join(subjects_dir, subject, 'bem',
                                     'sample-oct-6-src.fif'))

info = read_info(fname_raw)
# Set up coreg model
coreg = CoregistrationUI(info, subject, subjects_dir)

Closes #6995

[hoechenberger]

So IDK what exactly is the issue here but with the transparent skull, I get totally confused because everything seems to be … inverted? Something must be off with the shadows I suppose?

Screen.Recording.2021-10-26.at.14.48.59.mov

[hoechenberger]

Placing fiducials is very slow, is there any chance we can speed this up significantly?

Screen.Recording.2021-10-26.at.14.55.22.mov

[hoechenberger]

I couldn't get those MEG fiducials to appear until I realized I was using

1. the wrong field
2. the wrong file

I think 2. should be easy to resolve, no? At least I shouldn't be allowed to pick files with an unsuitable extension?

[GuillaumeFavelier]

Thank you for trying it out @hoechenberger and also for sharing your issues 👍

what exactly is the issue here but with the transparent skull

Do you have the same result with a "plot_alignment" plot with transparency? I'm trying to confirm that it is directly related to this PR. I personally think that it could be caused by backface culling tricks.

Placing fiducials is very slow

I checked and I do not see any obvious optimization that I can do easily, I have to spend the time to figure it out so I'll note this in #8833

I shouldn't be allowed to pick files with an unsuitable extension

In comparison, this is easily doable. I can use the endings parameter of the check_fname() function to take care of that.

[agramfort]

+1 to prevent loading something else than a .fif file for load dig points unless we already support more formats? I think addressing the speed issue can be done in a follow up PR

…

[hoechenberger]

Do you have the same result with a "plot_alignment" plot with transparency?

Yes:

Screen.Recording.2021-10-26.at.15.34.23.mov

# %%
from pathlib import Path
import mne


sample_dir = Path(mne.datasets.sample.data_path())
fs_subjects_dir = sample_dir / 'subjects'
fs_subject = 'sample'
inst_fname = sample_dir / 'MEG/sample/sample_audvis_1Hz_raw.fif'
trans_fname = sample_dir / 'MEG/sample/sample_audvis_raw-trans.fif'

info = mne.io.read_info(inst_fname)
trans = mne.read_trans(trans_fname)

# %%
mne.viz.plot_alignment(
    info=info,
    subject=fs_subject,
    subjects_dir=fs_subjects_dir,
    trans=trans,
    meg=False,
    eeg=False,
    surfaces={
        'head-dense': 0.5
    }
)

Confirmed this behavior with main.

With maint/0.23, I don't get a transparent skull.

[larsoner]

We should just get rid of the "inside head" surface in the new version, it never works/worked that well anyway. With depth peeling the opacity should be enough to make things look correct I think

[GuillaumeFavelier]

This failure at least is legitimate

Apparently it was caused by a difference with the coreg configuration. It's solved now.

[hoechenberger]

I think addressing the speed issue can be done in a follow up PR

+1

Thanks @GuillaumeFavelier for your work here!! ❤️

[agramfort]

works for me and good enough !

@larsoner I let you merge if happy

thx @GuillaumeFavelier !

[larsoner]

Running:

mne coreg -s sample -d ./subjects --fif MEG/sample/sample_audvis_raw.fif --trans MEG/sample/sample_audvis_raw-trans.fif

What I see is:

Notice that fiducials are unlocked but all dig points are shown. It should either be in unlocked state with just the MRI fids shown (i.e., if there is no -fiducials.fif in the bem directory), which would look like this:

Or it should be in the locked state with all dig points shown, like this:

[larsoner]

I'll add my gripe about lock/unlock separately, in the meantime let's merge this. Awesome work @GuillaumeFavelier !

[agramfort]

You made it @GuillaumeFavelier 🚀🙌🍾🍻

[drammock]

🍾 🎉 💯