Jupyter Notebook: "Interrupt kernel" restarts kernel, all variables lost

[Ramkumar-R]

Issue Report Checklist

• Searched the issues page for similar reports
• Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
• Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
• Could not reproduce inside jupyter qtconsole (if console-related)
• Tried basic troubleshooting (if a bug/error)
  • Restarted Spyder
  • Reset preferences with spyder --reset
  • Reinstalled the latest version of Anaconda
  • Tried the other applicable steps from the Troubleshooting Guide
• Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description

With Jupyter notebooks, "Interrupt Kernel" does just that - stops the kernel and all variables are preserved. "Restart kernel" is a separate option. However upon using a fresh installation of Spyder with the Notebook plugin, clicking "Interrupt kernel" or the square box icon restarts the kernel and all variables are lost.

What steps reproduce the problem?

1. Open a Jupyter notebook.
2. Define some variable say j = 3
3. Click the square box icon for "Interrupt kernel" and enter "j" in a cell

What is the expected output? What do you see instead?

"The kernel appears to have died..."
and
Traceback... Name 'j' is not defined.
Neither of the above should not happen.

Paste Traceback/Error Below (if applicable)

PASTE TRACEBACK HERE

Versions

• Spyder version: * Spyder version: 6.0.1 (conda)
• Python version: 3.12.7 64-bit
• Qt version: 5.15.2
• PyQt5 version: 5.15.10
• Operating System: Linux-6.8.0-49-generic-x86_64-with-glibc2.35

Dependencies

PASTE DEPENDENCIES HERE

[ccordoba12]

Hey @Ramkumar-R, thanks for reporting. I have a couple of questions for you, according to what you said:

What steps reproduce the problem?

1. Open a Jupyter notebook.

Did you open a notebook in JupyterLab and then connect to its kernel from Spyder? Or are you using Spyder-notebook to work with notebooks?

2. Define some variable say j = 3
3. Click the square box icon for "Interrupt kernel" and enter "j" in a cell

Could you post a screenshot or a small video showing that? It'd help us a lot to better understand your problem.

[Ramkumar-R]

@ccordoba12 Thank you so much for looking at this! Here are some screenshots.
The first shows a Jupyter Notebook (opened standalone) and what happens when I do an "Interrupt Kernel". It's a keyboard interrupt and then the variable i is preserved.
The next two screenshots show what happens when I open the same notebook in Spyder (I installed Spyder and its Notebook plugin just yesterday).
Does this make it clearer? "Interrupt Kernel" should do just that - interrupt it, without losing variables.
I was looking to move from browser-based Jupyter to Spyder since Spyder is just "WOW" but this is a bit of a showstopper :-(.

[ccordoba12]

Does this make it clearer? "Interrupt Kernel" should do just that - interrupt it, without losing variables.

Yes, this is much clearer now, thanks!

I was looking to move from browser-based Jupyter to Spyder since Spyder is just "WOW" but this is a bit of a showstopper :-(.

Thanks for your kind words! And I totally understand this a serious bug that prevents to use our notebook plugin.

Since this is directly related to that plugin, I'm going to move this issue to its corresponding repository.

[ccordoba12]

@jitseniesen, are you able to reproduce this problem?

[jitseniesen]

@Ramkumar-R Thank you for the report. I have reproduced the issue, but I need to look further into it to see what is going wrong.

[Ramkumar-R]

Sorry for the delayed ack. Greatly appreciate your effort and time! Looking forward to the fix when it happens!

[jitseniesen]

It took me a while because I was looking in the wrong place, but I think I have now figured out what is going on.

When you open a notebook inside Spyder, the notebook server uses conda run to start a Spyder kernel in the correct conda environment. Then, when you press the button for interrupting the kernel, the notebook server sends SIGINT to the process group led by the conda run process (at least on Linux, I did not investigate what happens on other OSes). The Spyder kernel handles the SIGINT which gives the traceback you can see in the notebook. However, the conda run process also receives SIGINT, it terminates and this in turn kills the Spyder kernel process.

The issue does not appear with Jupyter kernels inside a notebook, because Jupyter kernels are not started with conda run but with python -m ipykernel_launcher. The issue also does not appear with Spyder kernels inside Spyder's IPython console because that uses a different mechanism to interrupt kernels. Finally, the issue did not appear in Spyder 5 because that version did not use conda run but it used a shell script to activate the correct conda environment. In that case, the process group leader is bash (or perhaps another shell) and that ignores SIGINT. The commit that changed the the activation script to conda run is spyder-ide/spyder@57432a0, part of PR spyder-ide/spyder#20827. Reverting that commit (which needs to be done by hand) does resolve the issue.

So, how to fix this? Here are some ideas:

• Somehow change the notebook server so that it does not send a SIGINT to the conda run process. I don't know how technically feasible this is and it may be rather fragile.
• Somehow get the conda run process to ignore SIGINT. I don't know exactly how to do this, but this looks easier than the previous idea. Perhaps we can replace the conda run command with a Python command that installs a signal handler and then calls the correct function in the conda Python library. Or perhaps we can get inspiration from how Jupyter does this.
• Go back to using an activation script instead of conda run. I don't know why this change was done so this may well break something.

@mrclary You authored the aforementioned commit, what do you think?

[mrclary]

conda run is extremely convenient, concise, cross-platform, and ensures that the conda environment is correctly and fully activated without having to initialize the shell.

However, given the issue with the notebook plugin, we should definitely investigate. I prefer your first two suggestions over reverting to activation scripts, but these are contingent on being whether they are possible to do.

Does this issue manifest for pip environments?

[mrclary]

@jitseniesen, I can't even get the notebook plugin to work at all. Tried with Spyder 6.0.1 and Spyder 6.0.3.

conda create -n test python=3.11 spyder=6.0.3 spyder-notebook
conda activate test
spyder --debug-info=minimal

I see the following:

Unsatisfied version 4.2.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/application (required ~4.0.3)
Unsatisfied version 2.0.1 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyter/ydoc (required ~1.0.2)
Unsatisfied version 6.2.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/coreutils (required ^6.3.2)
Unsatisfied version 4.2.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/translation (required ^4.3.2)
Unsatisfied version 4.2.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/application (required ^4.3.2)
Unsatisfied version 4.3.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/apputils (required ^4.4.2)
Unsatisfied version 4.2.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/ui-components (required ^4.3.2)
Unsatisfied version 4.2.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/notebook (required ^4.3.2)
No satisfying version (^4.3.2) of shared module @jupyterlab/docregistry found in shared scope default.
Available versions: 4.2.0 from _JUPYTERLAB.CORE_OUTPUT
Unsatisfied version 4.2.0 from _JUPYTERLAB.CORE_OUTPUT of shared singleton module @jupyterlab/rendermime (required ^4.3.2)
Language pack 'en_US' not installed!
Deprecated shortcut selectors: ".jp-Notebook:focus" was replaced with ".jp-Notebook.jp-mod-commandMode:not(.jp-mod-readWrite) :focus" in JupyterLab 4.1 (present in ".jp-Notebook:focus")

The selectors will be substituted transparently this time, but need to be updated at source before next major release.

and no kernel associated with the notebook, nor can I select a kernel.

For Spyder, the Python interpreter is set to Internal.