Searchable deploy - Tables do not exist. VW_TS_BI_SERVER_ADVANCED_AUTHOR

[333-Spur-Tmps]

Hi @boonhapus

I've ran the commands searchable bi-server and searchable metadata and synced successfully to databricks. All 16 metadata tables have been created and data loaded in databricks.

Now when I run the command searchable deploy command only the first 10 tables get loaded into my TS connection and then the scripts fails with an error stating missing table VW_TS_BI_SERVER_ADVANCED_AUTHOR. The table seems a little strange as it doesn't follow the same naming structure and doesn't exist in databricks as part of the searchable metadata command.

Any help is really appreciated.

FYI
We're using v1.5.9.
I've done a seach for similar but the final conversation seemed to have gone offline (#164)

[boonhapus]

@333-Spur-Tmps You're correct -- I've yet to push this change yet. We can resolve this in a few steps.

1. Upgrade to the 1.5.11 beta (or wait until Monday when I release it)
2. Delete the tables both from ThoughtSpot & Databricks
3. Re-run the metadata , bi-server , and deploy commands in this order

To grab the beta, you can run the following command (it's the install command, but with --beta flag)

powershell -c "IRM https://raw.githubusercontent.com/thoughtspot/cs_tools/master/cs_tools/updater/_bootstrapper.py | python - --install --beta"

---

The reason for this is because the JDBC driver that ThoughtSpot uses for Databricks lags behind by a few versions, whereas the DatabricksSyncer here in CS Tools uses the databricks-sql-connector python package directly which is much more up-to-date. What this means is, CS Tools goes and creates TIMESTAMP_NTZ columns in Databricks** and ThoughtSpot can't understand this data type yet.

CS Tools 1.5.11 will account for this and not create TIMESTAMP_NTZ columns.

**this feature is in Databricks Public Preview and requires Databricks Runtime 13.3 LTS and above.

[333-Spur-Tmps]

Thank you @boonhapus , I took the beta and everything ran successfully.

[333-Spur-Tmps]

Hey @boonhapus ... I've been looking into the two CS Tool liveboards and I'm getting alot of visuals that should have data but are returning with null/0.

eg Liveboard = CS Tools - Thoughtspot Adoption / Archiver tab / Total Answers or Total Liveboards

Seems to boil down to 4 of the ts-* tables have no data, in the above answers in the SQL we're looking at ts_dependent_object.name.

the other tables with no values are..
ts_column_synonym
ts_dependent_object
ts_sharing_access
ts_tagged_object

I wonder if there's an issue in the searchable metadata command with syncing the data to databricks for these tables?

Cheers

[boonhapus]

Interesting... can we get a call to look together @333-Spur-Tmps ? Please shoot me an email at [email protected] and we'll find a time/tz that works for both of us.

[333-Spur-Tmps]

email sent :)