Medicine motion data is in different format than other motion models (possibly timestamps)

[ryan-budde]

I am still working out exactly what's wrong here. I have been testing different motion correction settings and benchmarking against one another.

I'm starting with the new motion syntax (as opposed to the old syntax which returned motion, temporal_bins, and spatial_bins)

# NEW FORMAT
from spikeinterface.sortingcomponents.motion import estimate_motion
        motion, motion_info = estimate_motion(recording=rec,
            peaks=peaks, peak_locations=peak_locations, extra_outputs=True, **estimate_motion_kwargs)

However this "extra outputs" variable appears to be empty as of right now, so I fill it in myself.

I want to use the built in

# THE FUNCTION WE CALL, BOTH OLD AND NEW
si.plot_motion_info(motion_info, rec, color_amplitude=True,figure=fig,
            amplitude_cmap="inferno",
            scatter_decimate=10,)

Like is shown on the motion tutorials on the docs. However this function is still written for the old format of motion_info, which was structured as:

#  OLD FORMAT
motion_info = dict(
                    parameters=parameters,
                    peaks=peaks,
                    peak_locations=peak_locations,
                    temporal_bins=temporal_bins,
                    spatial_bins=spatial_bins,
                    motion=motion,
                )

I am attempting to preserve that structure, so in my code I add the following:

# MY NEW FORMAT
motion_info.update(
            parameters=parameters,
            peaks=peaks,
            peak_locations=peak_locations,
            motion=motion,
        )

to add it all into one place.

It appears that si.plot_motion_info does not extract temporal_bins or spatial_bins from the motion_info struct but rather from the motion object directly.

Here are my results when I use dredge:

This result is pretty bad, from a motion correction standpoint, but it demonstrates a few things going on in the code.

1. First, it's important to know that I'm using 10 minutes (600 seconds) of NPX data which start at 25 minutes into the file. That is, the rec recording object is only 600 seconds long. To be honest, I'm not sure how the rec object knows that the .bin file it loaded from doesn't start at t=0 (presumably reading the associated .meta file)
2. The top left plot is the original, uncorrected peaks. The time span is 0 to 600 seconds. This data has been time shifted.
3. The top right plot is the corrected peaks. The time span is 0 to 600 seconds. This data has been time shifted
4. The bottom plots show the motion vectors. The time span is 1500 to 2100 seconds. This data has not been time shifted. However it shows that the si.plot_motion_info is aware of both the motion vector and the original timeline.

This is true for dredge, nonrigid accurate, and kilosort-like. However things break with Medicine.

There are two mistakes here:

1. The motion vector is not applied. The top right graph is exactly the same as the top left, and there is no way for the user to determine if the Medicine motion vector is better / worse than the other methods.
2. Unlike the other methods, the motion plots on the bottom both reference the timescale 0 to 600. This is not an error per se, but it does indicate a difference in behavior, and may explain the underlying error of #⁠1 above.

I am still digging into things, but I anticipate that there is something different about the formatting of the motion object between medicine and the other motion correction methods. This may be a unique error for data pulled out of a snippet of the recording, or may exist more broadly.

I'd appreciate any insights into how these motion vectors are aligned to the different time vectors, either 0-600 or 1500-2100

UPDATE:
I can confirm that within motion, temporal_bins_s and temporal_bins_edges_s begin at 1500 for dredge and 0 for Medicine

[samuelgarcia]

Hi Ryan,
thanks a lot for this feedback.

@cwindolf could be interrested in the issue.

I'm starting with the new motion syntax (as opposed to the old syntax which returned motion, temporal_bins, and spatial_bins)

Yes it was part of the refactoring : create a Motion object that handle internal the temporal bin and spatialbins (and also multi segment story).

A main goal was to be able to estimate the motion only on some portion of a long recording, so time handling was important.

# NEW FORMAT
from spikeinterface.sortingcomponents.motion import estimate_motion
        motion, motion_info = estimate_motion(recording=rec,
            peaks=peaks, peak_locations=peak_locations, extra_outputs=True, **estimate_motion_kwargs)

However this "extra outputs" variable appears to be empty as of right now, so I fill it in myself.

I need to check but some method still return some outputs for debuging.
But typically temporal_bins and spatial_bins are not anymore part of this extra dict.

I want to use the built in

# THE FUNCTION WE CALL, BOTH OLD AND NEW
si.plot_motion_info(motion_info, rec, color_amplitude=True,figure=fig,
            amplitude_cmap="inferno",
            scatter_decimate=10,)

Like is shown on the motion tutorials on the docs. However this function is still written for the old format of motion_info, which was structured as:

#  OLD FORMAT
motion_info = dict(
                    parameters=parameters,
                    peaks=peaks,
                    peak_locations=peak_locations,
                    temporal_bins=temporal_bins,
                    spatial_bins=spatial_bins,
                    motion=motion,
                )

Oups, this is a bug we should change this and update the doc.
Thank you.

1. First, it's important to know that I'm using 10 minutes (600 seconds) of NPX data which start at 25 minutes into the file. That is, the rec recording object is only 600 seconds long. To be honest, I'm not sure how the rec object knows that the .bin file it loaded from doesn't start at t=0 (presumably reading the associated .meta file)

Yes by default the now spikeglx reader extract the true "t_tstart" wich come from the acuiqistion system, maybe we could remove it. So that the t_start could be still 0.

3. The top left plot is the original, uncorrected peaks. The time span is 0 to 600 seconds. This data has been time shifted.

This is a plotting bug, peak do not have time concept we should update the code to hack this.

5. The top right plot is the corrected peaks. The time span is 0 to 600 seconds. This data has been time shifted

Same.

7. The bottom plots show the motion vectors. The time span is 1500 to 2100 seconds. This data has not been time shifted. However it shows that the si.plot_motion_info is aware of both the motion vector and the original timeline.> > This is true for dredge, nonrigid accurate, and kilosort-like. However things break with Medicine.>

This is the expected behavior but for medecine this is a bug we need to correct it.
The timeshould be also handle in medicine.

There are two mistakes here:

1. The motion vector is not applied. The top right graph is exactly the same as the top left, and there is no way for the user to determine if the Medicine motion vector is better / worse than the other methods.

Because the motion is at wrong time. So it is not applied at the correct time.

[ryan-budde]

As I said, I found that within motion, temporal_bins_s and temporal_bins_edges_s begin at 1500 for dredge and 0 for Medicine

For now the following is working for me

tstart = 1500.
if loopMotionParams["preset"] == 'medicine':
            motion.temporal_bins_s = np.add(motion.temporal_bins_s, tstart)
            motion.temporal_bin_edges_s = np.add(motion.temporal_bin_edges_s, tstart)

And then plotting works. Obviously this is only a short term fix!
I am still processing and evaluating if this has any impact on re-interpolation to create a new raw .bin file