OSError: exception: access violation writing 0x0000000000000000

[beast331]

Description

I have a large training dataset (44 million rows, 9 columns). When I pass it into LightGBM and run the fit method, it throws this error.
The code I used is below. It's very short and fairly simple.

Reproducible example

import lightgbm as lbm
model = lbm.LGBMClassifier(learning_rate=0.2,max_depth=-5,random_state=42)
model.fit(X, y, verbose=1)

Environment info

Running a Windows 10 with 8GB ram. Using Anaconda and Jupyter notebook.

LightGBM version or commit hash:
LightGBM version 3.3.1

Command(s) you used to install LightGBM - I used a couple of different commands to try and fix the issue. All install commands were run through Jupyter notebook

!{sys.executable} -m pip install lightgbm
pip install --no-binary :all: lightgbm

[shiyu1994]

@beast331 Thanks for using LightGBM. Is it possible to share the data with us, so that we can reproduce the error?

[beast331]

@beast331 Thanks for using LightGBM. Is it possible to share the data with us, so that we can reproduce the error?

Hi thanks for the quick reply! Is there somehow I can upload the CSV file here?

Also, I decided to try the "train" function instead of "fit" and it seems to be working. What is the difference?

[shiyu1994]

Not sure whether GitHub has a limitation of the attached file size. You may send that to my email [email protected] as well. Thanks!

[jameslamb]

@beast331 , thanks for the report and for using LightGBM!

If you start a new Python session and run this code again, do you get the same error? We have seen similar errors very rarely in our tests and it's often resolved by just starting a new Python session.

for reference:

• OSError: exception: access violation reading 0x0000000000000058 #3047
• OSError: exception: access violation reading 0x0000000000000274 #2329
• MinGW fails with access violation writing error #1818
• OSError: exception: access violation writing #1475

[no-response]

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.