SNOW-2688693: The write_pandas function returns invalid value [?] for parameter 'ON_ERROR' instead of actual errors #2638
Description
Python version
Python 3.12.9 (main, Feb 4 2025, 14:38:38) [Clang 16.0.0 (clang-1600.0.26.6)]
Operating system and processor architecture
macOS-14.7.7-arm64-arm-64bit
Installed packages
annotated-types==0.7.0
anyio==4.11.0
asgiref==3.10.0
asn1crypto==1.5.1
astroid==3.3.11
backoff==2.2.1
boto3==1.40.47
botocore==1.40.47
cachetools==6.2.0
callee==0.3.1
certifi==2025.10.5
cffi==1.17.1
cfgv==3.4.0
charset-normalizer==3.4.3
click==8.3.0
colorama==0.4.6
coverage==7.10.7
cryptography==46.0.0
dataclasses-json==0.6.7
debugpy==1.8.17
deepdiff==8.6.1
defusedxml==0.7.1
Deprecated==1.2.18
dill==0.4.0
distlib==0.4.0
et_xmlfile==2.0.0
execnet==2.1.1
factory_boy==3.3.3
Faker==37.11.0
fastapi==0.118.2
filelock==3.19.1
frozendict==2.4.6
googleapis-common-protos==1.70.0
gql==3.5.3
graphene==2.0
graphql-core==3.2.6
graphql-relay==3.2.0
greenlet==3.2.4
grpcio==1.75.1
gunicorn==23.0.0
h11==0.16.0
httpcore==1.0.9
httptools==0.6.4
httpx==0.28.1
hvac==2.3.0
identify==2.6.15
idna==3.10
importlib_metadata==7.1.0
iniconfig==2.1.0
isort==6.1.0
Jinja2==3.1.6
jmespath==1.0.1
lia-web==0.2.3
libcst==1.8.5
markdown-it-py==4.0.0
MarkupSafe==3.0.3
marshmallow==3.26.1
marshmallow-enum==1.5.1
marshmallow_dataclass==8.7.1
mccabe==0.7.0
mdurl==0.1.2
multidict==6.7.0
mypy==1.18.2
mypy_extensions==1.1.0
nodeenv==1.9.1
numpy==2.3.3
oauthlib==3.3.1
odfpy==1.4.1
openpyxl==3.1.5
opentelemetry-api==1.25.0
opentelemetry-distro==0.46b0
opentelemetry-exporter-otlp==1.25.0
opentelemetry-exporter-otlp-proto-common==1.25.0
opentelemetry-exporter-otlp-proto-grpc==1.25.0
opentelemetry-exporter-otlp-proto-http==1.25.0
opentelemetry-instrumentation==0.46b0
opentelemetry-instrumentation-asgi==0.46b0
opentelemetry-instrumentation-boto3sqs==0.46b0
opentelemetry-instrumentation-botocore==0.46b0
opentelemetry-instrumentation-redis==0.46b0
opentelemetry-instrumentation-requests==0.46b0
opentelemetry-instrumentation-sqlalchemy==0.46b0
opentelemetry-instrumentation-starlette==0.46b0
opentelemetry-instrumentation-urllib==0.46b0
opentelemetry-propagator-aws-xray==1.0.1
opentelemetry-proto==1.25.0
opentelemetry-sdk==1.25.0
opentelemetry-semantic-conventions==0.46b0
opentelemetry-util-http==0.46b0
orderly-set==5.5.0
packaging==25.0
pandas==2.3.3
pandera==0.26.1
pathspec==0.12.1
pika==1.3.2
platformdirs==4.5.0
pluggy==1.6.0
polyfactory==2.22.2
pre_commit==4.3.0
promise==2.3
propcache==0.3.2
protobuf==4.25.8
psycopg2==2.9.10
pyarrow==21.0.0
pycparser==2.23
pydantic==2.12.0
pydantic_core==2.41.1
Pygments==2.19.2
PyJWT==2.8.0
pylint==3.3.9
pyodbc==5.2.0
pyOpenSSL==25.3.0
pytest==8.4.2
pytest-cov==6.3.0
pytest-django==4.11.1
pytest-mock==3.15.1
pytest-xdist==3.8.0
python-calamine==0.5.3
python-dateutil==2.8.2
python-dotenv==1.1.1
python-multipart==0.0.20
pytz==2025.2
pyxlsb==1.0.10
PyYAML==6.0.1
redis==6.4.0
requests==2.32.5
requests-mock==1.12.1
requests-oauthlib==2.0.0
rich==14.1.0
ruff==0.14.0
s3transfer==0.14.0
sentinel==1.0.0
setuptools==80.9.0
sgqlc==16.5
shellingham==1.5.4
six==1.17.0
slack_sdk==3.37.0
sniffio==1.3.1
snowflake-connector-python==3.18.0
snowflake-sqlalchemy==1.7.7
sortedcontainers==2.4.0
sqlakeyset==2.0.1746777265
SQLAlchemy==2.0.43
sqlparse==0.5.3
sseclient-py==1.8.0
starlette==0.48.0
starlette-context==0.3.6
strawberry-graphql==0.283.2
strawberry-sqlalchemy-mapper==0.7.0
stringcase==1.2.0
structlog==24.4.0
tenacity==8.2.3
tomlkit==0.13.3
typeguard==4.4.4
typer==0.19.2
types-cffi==1.17.0.20250915
types-pyOpenSSL==24.1.0.20240722
types-python-dateutil==2.9.0.20251008
types-redis==4.6.0.20241004
types-requests==2.32.4.20250913
types-setuptools==80.9.0.20250822
typing-inspect==0.9.0
typing-inspection==0.4.2
typing_extensions==4.15.0
tzdata==2025.2
urllib3==2.5.0
uvicorn==0.37.0
uvloop==0.21.0
virtualenv==20.34.0
watchfiles==1.1.0
websockets==15.0.1
Werkzeug==3.1.3
wrapt==1.17.3
xlrd==2.0.2
xlsxwriter==3.2.9
yarl==1.22.0
zipp==3.23.0What did you do?
Call `write_pandas` with a `null` value in a column that's non-nullable in the destination table (or many other types of errors).
write_pandas(
conn=conn,
df=some_df # DataFrame should have null is some column that doesn't allow null.
table_name="SOME_TABLE",
schema="SOME_SCHEMA",
chunk_size=chunk_size,
use_logical_type=True,
)What did you expect to see?
I expected to see an error that says
Cannot insert null value into non-nullable column.
Instead I get
SQL compilation error: invalid value [?] for parameter 'ON_ERROR'"
This error stems from this line. It seems that the parameter for ON_ERROR is not being passed correctly. I verified this by cloning the repo locally and modifying that line to use string interpolation instead of a parameter. After making that change, I get the actual underlying error.
Can you set logging to DEBUG and collect the logs?
import logging
import os
for logger_name in ('snowflake.connector',):
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
ch.setFormatter(logging.Formatter('%(asctime)s - %(threadName)s %(filename)s:%(lineno)d - %(funcName)s() - %(levelname)s - %(message)s'))
logger.addHandler(ch)