Running CS-Tools in Azure Pipeline

[eckhami]

Hi,
am trying to run CS-Tools within Azure Pipeline and to write to Snowflake, according to https://thoughtspot.github.io/cs_tools/getting-started/#azure-pipelines

The connection to Thoughtspot seems to work ok, only when trying to configure the SnowFlake bit, I get below message:

╭──────────────── Your Snowflake Syncer encountered an error. ─────────────────╮
│                                                                              │
│ 2 arguments have errors.                                                     │
│                                                                              │
│ authentication                                                               │
│ > key-pair                                                                   │
│ x Input should be 'basic', 'key-pair', 'sso' or 'oauth'                      │
│                                                                              │
│ private_key_path                                                             │
│ >                                                                            │
│ https://ecolab-my.sharepoint.com/:u:/r/personal/...<my_username>.../         │
│ Documents/cstools/rsa_map_cstools_tables_refresh_key.p8                      │
│ x Path does not point to a file                                              │
│                                                                              │
│ Mitigation                                                                   │
│ Check the Syncer's documentation page for more information.                  │
│ https://thoughtspot.github.io/cs_tools/syncer/snowflake/                     │
╰──────────────────────────────────────────────────────────────────────────────╯

# COMMON PARAMETERS FOR THE SNOWFLAKE SYNCER
# https://thoughtspot.github.io/cs_tools/syncer/snowflake/
DECLARATIVE_SYNCER_SYNTAX: >-
  account_name= $(SNOWFLAKE_ACCOUNT)
  &username= $(SNOWFLAKE_USERNAME)
  &warehouse= $(SNOWFLAKE_WAREHOUSE)
  &role= $(SNOWFLAKE_ROLE)
  &database= $(SNOWFLAKE_DATABASE)
  &schema= $(SNOWFLAKE_SCHEMA)
  &authentication=key-pair
  &private_key_path=$(SNOWFLAKE_PRIVATE_KEY_PATH)

I tried &authentication=key-pair in all variations: without any quotation marks, with single and double quotation marks. I even put it in a variable - none of these seem to work.

As for the &private_key_path I used a sharepoint link (as in the screenshot above), and when I click on the link the p8 file will be downloaded - so it is in fact pointing to a file.

what am I missing here?

Thanks
Michael

[TS-sumeet-jain]

Hi @eckhami,

Use the authentication bit at the end. For private key, Please store the key as environment variable( SNOWFLAKE_PRIVATE_KEY_PATH ) and then at runtime dump in a file. Pass the file to snowflake syncer.

Under the env define the file as:

env:
KEY_PATH: private-key.p8

just before running the cstools script add:
echo "${{ secrets.SNOWFLAKE_PRIVATE_KEY_PATH }}" > $KEY_PATH

Once done add below to remove the private key file:
rm -f ${{ env.KEY_PATH }}