Support for Azure AD Service Principal with SQLAlchemy

[FaresBadrCA]

Problem Summary:

When using Azure AD Service Principal, a client-id and client-secret are used to obtain a short-lived token. The token may be used in the connection string:

uri = f"databricks://token:{DATABRICKS_TOKEN}@{DATABRICKS_HOST}:443?http_path={DATABRICKS_HTTP_PATH}"
engine = create_engine(uri)

However, there is no mechanism to ensure that connections do not stay in the connection pool past the token expiration.

Proposed Solution:

Allow client_id and client_secret as connection arguments, and ensure connections do not stay open past token expiration.

url = f"databricks://{DATABRICKS_HOST}:443?http_path={DATABRICKS_HTTP_PATH}"    
engine = create_engine(url, connect_args = {client_id: CLIENT_ID, client_secret: CLIENT_SECRET, scopes: SCOPES  } )

[amirhessam88]

any updates on this ?

[asiunov]

Wouldn't this work?

for azure auth

from databricks.sdk.core import Config, azure_service_principal

def credentials_provider():
  return azure_service_principal(Config(host="<...>.azuredatabricks.net", azure_tenant_id=..., azure_client_id=..., azure_client_secret=...))

create_engine(f"databricks://{host}:443?http_path={http_path}", connect_args={"credentials_provider": credentials_provider})

for dbx service principal

from databricks.sdk.core import oauth_service_principal

def credentials_provider():
  return oauth_service_principal(Config(host="<...>.databricks.com", client_id=..., client_secret=...))