Changes for SSL cert errors, SF proxy params, data contracts, SSO SP-initiated, reference check error (#804)

* Troubleshoot SSL certificate error.

* Reference and Snowflake connection troubleshooting

* Clarified scan def names in programmatic scans

* Data contract clarifications and adjustments

Author: janet-can

_data/nav.yml

@@ -226,6 +226,8 @@
     page: soda/connect-trino.md
   - subtitle: Connect to Vertica
     page: soda/connect-vertica.md
+  - subtitle: Troubleshoot connections
+    page: soda/connect-troubleshoot.md
 
 - title: Soda Cloud API
   page: api-docs/public-cloud-api-v1.md

_includes/custom-sampler.md

@@ -1,3 +1,4 @@
 * You can save Soda Library scan results anywhere in your system; the `scan_result` object contains all the scan result information. To import Soda Library in Python so you can utilize the `Scan()` object, [install a Soda Library package]({% link soda-library/programmatic.md %}), then use `from soda.scan import Scan`.
+* If you provide a name for the scan definition to identify inline checks in a programmatic scan as independent of other inline checks in a different programmatic scan or pipeline, be sure to set a unique scan definition name for each programmatic scan. Using the same scan definition name in multiple programmatic scans results in confused check results in Soda Cloud.
 * If you wish to collect samples of failed rows when a check fails, you can employ a custom sampler; see [Configure a failed row sampler]({% link soda-cl/failed-rows-checks.md %}#configure-a-failed-row-sampler).
 * Be sure to include any variables in your programmatic scan *before* the check YAML files. Soda requires the variable input for any variables defined in the check YAML files.

_includes/snowflake-proxy.md

@@ -0,0 +1,18 @@
+**Problem:** While attempting to connect Soda to a Snowflake data source using proxy parameters, you encounter an error that reads something similar to `Could not connect to data source "name_db": 250001 (08001): Failed to connect to DB: mydb.eu-west-1.snowflakecomputing.com:443. Incoming request with IP/Token xx.xxx.xx.xxx is not allowed to access Snowflake.`
+
+```yaml
+data_source: my_data_source
+  type: snowflake
+  ...
+  session_param:
+    QUERY_TAG: soda-test
+    QUOTED_IDENTIFIERS_IGNORE_CASE: false
+  proxy_http: http://a-proxy-o-dd-dddd-net:8000
+  proxy_https: https://a-proxy-o-dd-dddd-net:8000
+```
+
+**Solution:** When connecting to a Snowflake data source by proxyy, be sure to set the new proxy environment variables from the command-line using export statements, as in the following example.
+```shell
+export HTTP_PROXY=http://a-proxy-o-dd-dddd-net:8000
+export HTTPS_PROXY=https://a-proxy-o-dd-dddd-net:8000
+```

assets/images/experimental.png

@@ -105,7 +105,7 @@ To review the failed rows in Soda Cloud, navigate to the **Checks** dashboard, t
 | ✓ | Use quotes when identifying dataset or column names; see [example](#example-with-quotes). <br />Note that the type of quotes you use must match that which your data source uses. For example, BigQuery uses a backtick ({% raw %}`{% endraw %}) as a quotation mark. | [Use quotes in a check]({% link soda-cl/optional-config.md %}#use-quotes-in-a-check) |
 |   | Use wildcard characters ({% raw %} % {% endraw %} or {% raw %} * {% endraw %}) in values in the check. | - |
 |   | Use for each to apply reference checks to multiple datasets in one scan. | - |
-| ✓ | Apply a dataset filter to partition data during a scan; see [example](#example-with-dataset-filter). | [Scan a portion of your dataset]({% link soda-cl/optional-config.md %}#scan-a-portion-of-your-dataset) |
+| ✓ | Apply a dataset filter to partition data during a scan; see [example](#example-with-dataset-filter). If you encounter difficulties, see [Filter not passed with reference check]({% link soda-cl/troubleshoot.md %}#filter-not-passed-with-reference-check). | [Scan a portion of your dataset]({% link soda-cl/optional-config.md %}#scan-a-portion-of-your-dataset) |
 
 #### Example with check name 
 {% include code-header.html %}
@@ -123,6 +123,9 @@ checks for dim_department_group:
 ```
 
 #### Example with dataset filter
+
+Refer to [Troubleshoot SodaCL]({% link soda-cl/troubleshoot.md %}#filter-not-passed-with-reference-check) to address challenges specific to reference checks with dataset filters.
+
 {% include code-header.html %}
 ```yaml
 filter customers_c8d90f60 [daily]:
@@ -132,7 +135,7 @@ checks for customers_c8d90f60 [daily]:
   - values in (cat) must exist in customers_europe (cat2)
 ```
 
-Refer to [Troubleshoot SodaCL]({% link soda-cl/troubleshoot.md %}#filter-not-passed-with-reference-check) to address challenges specific to reference checks with dataset filters.
+
 
 <br />
 

soda-cl/reference.md

@@ -34,6 +34,8 @@ When an organization's IT Admin revokes a user's access to Soda Cloud through th
 
 Once your organization enables SSO for all Soda Cloud users, Soda Cloud blocks all non-SSO login attempts and password changes via <a href="https://cloud.soda.io/login" target="_blank">cloud.soda.io/login<a/>. If an employee attempts a non-SSO login or attempts to change a password using "Forgot password?" on <a href="https://cloud.soda.io/login" target="_blank">cloud.soda.io/login<a/>, Soda Cloud presents a message that explains that they must log in or change their password using their SSO provider. 
 
+Soda Cloud supports both Identity Provider Initiated (IdP-initiated), and Service Provider Initiated (SP-initiated) single sign-on integrations. Be sure to incidate which type of SSO your organization uses when setting it up with the Soda Support team.
+
 
 ## Add Soda Cloud to Azure AD
 
@@ -63,7 +65,7 @@ Once your organization enables SSO for all Soda Cloud users, Soda Cloud blocks a
 * **Azure AD Identifier** (Section 4 in Azure). This is the IdP entity, ID, or Identity Provider Issuer that Soda needs
 * **Login URL** (Section 4 in Azure). This is the IdP SSO service URL, or Identity Provider Single Sign-On URL that Soda needs.
 * **X.509 Certificate**. Click the **Download** link next to **Certificate (Base64)**.
-12. Email the copied and downloaded values to <a href="mailto:[email protected]">[email protected]</a>. With those values, Soda completes the SSO configuration for your organization in cloud.soda.io and notifies you of completion.
+12. Email the copied and downloaded values to <a href="mailto:[email protected]">[email protected]</a>. With those values, Soda completes the SSO configuration for your organization in cloud.soda.io and notifies you of completion. Soda Cloud supports both Identity Provider Initiated (IdP-initiated), and Service Provider Initiated (SP-initiated) single sign-on integrations; be sure to incidate which type of SSO your organization uses.
 13. Test the integration by assigning the Soda application in Azure AD to a single user, then requesting that they log in.
 14. After a successful single-user test of the sign in, assign access to the Soda Azure AD app to users and/or user groups in your organization.
 
@@ -89,7 +91,7 @@ The values for these fields are unique to your organization and are provided to
 * **Identity Provider Single Sign-On URL**
 * **Identity Provider Issuer**
 * **X.509 Certificate**
-11. Email the copied and downloaded values to <a href="mailto:[email protected]">[email protected]</a>. With those values, Soda completes the SSO configuration for your organization in cloud.soda.io and notifies you of completion.
+11. Email the copied and downloaded values to <a href="mailto:[email protected]">[email protected]</a>. With those values, Soda completes the SSO configuration for your organization in cloud.soda.io and notifies you of completion. Soda Cloud supports both Identity Provider Initiated (IdP-initiated), and Service Provider Initiated (SP-initiated) single sign-on integrations; be sure to incidate which type of SSO your organization uses.
 12. Test the integration by assigning the Soda application in Okta to a single user, then requesting that they log in.
 13. After a successful single-user test of the sign in, assign access to the Soda Okta app to users and/or user groups in your organization.
 
@@ -107,7 +109,7 @@ The values for these fields are unique to your organization and are provided to
 5. On the **SAML Attribute mapping** page, add two Google directory attributes and map as follows:
 * Last Name → User.FamilyName
 * First Name → User.GivenName
-6. Email the copied and downloaded values to <a href="mailto:[email protected]">[email protected]</a>. With those values, Soda completes the SSO configuration for your organization in cloud.soda.io and notifies you of completion.
+6. Email the copied and downloaded values to <a href="mailto:[email protected]">[email protected]</a>. With those values, Soda completes the SSO configuration for your organization in cloud.soda.io and notifies you of completion. Soda Cloud supports both Identity Provider Initiated (IdP-initiated), and Service Provider Initiated (SP-initiated) single sign-on integrations; be sure to incidate which type of SSO your organization uses.
 7. In the Google Workspace admin portal, use Google's instructions to <a href="https://support.google.com/a/answer/6087519?hl=en&ref_topic=7559288" target="_blank">Turn on your SAML app</a> and verify that SSO works with the new custom app for Soda.
 
 

soda-cloud/sso.md

@@ -140,6 +140,8 @@ scan.execute()
 scan.set_verbose(True)
 
 # Set scan definition name, equivalent to CLI -s option
+# The scan definition name MUST be unique to this scan, and
+# not duplicated in any other programmatic scan
 ##################
 scan.set_scan_definition_name("YOUR_SCHEDULE_NAME")
 

soda-library/programmatic.md

@@ -282,9 +282,8 @@ Because Soda Library pushes scan results to Soda Cloud, you may not want to chan
 
 **Problem:** In a Windows environment, you see an error that reads `[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (ssl_c:997)`.
 
-**Short-term solution:** Use `pip install pip-system-certs` to temporarily resolve the issue. This install works to resolve the issue only on Windows machines where the Ops team installs all the certificates needed through Group Policy Objects, or similar. However, the fix is short-term because when you try to run this in a pipeline on another machine, the error will reappear.
+**Solution:** Use `pip install pip-system-certs` to potentially resolve the issue. This install works to resolve the issue only on Windows machines where the Ops team installs all the certificates needed through Group Policy Objects, or similar. 
 
-**Short-term solution:** Contact your Operations or System Admin team to obtain the proxy certificate.
 
   </div>
   <div class="panel" id="three-panel" markdown="1">
@@ -359,6 +358,8 @@ scan.execute()
 scan.set_verbose(True)
 
 # Set scan definition name, equivalent to CLI -s option
+# The scan definition name MUST be unique to this scan, and
+# not duplicated in any other programmatic scan
 ##################
 scan.set_scan_definition_name("YOUR_SCHEDULE_NAME")
 

soda-library/run-a-scan.md

@@ -171,6 +171,10 @@ checks for VOLUME:
             name: Trader row count
 ```
 
+<br />
+
+{% include snowflake-proxy.md %}
+
 <br />
 <br />
 

soda/connect-snowflake.md

@@ -0,0 +1,39 @@
+---
+layout: default
+title: Troubleshoot data source connections
+description: 
+parent: 
+---
+
+# Troubleshoot data source connections
+Last modified on {% last_modified_at %}
+
+
+
+## SSL certificate error
+
+**Problem:** You encounter an SSL certificate error while attempting to connect Soda to a data source.
+
+**Solution:** Use `pip install pip-system-certs` to potentially resolve the issue. This install works to resolve the issue only on Windows machines where the Ops team installs all the certificates needed through Group Policy Objects, or similar. 
+
+## Snowflake proxy connection error
+
+{% include snowflake-proxy.md %}
+
+## Go further
+
+* Access [Troubleshoot SodaCL]({% link soda-cl/troubleshoot.md %}) for help resolving issues running scans with SodaCL.
+* Need help? Join the <a href="https://community.soda.io/slack" target="_blank"> Soda community on Slack</a>.
+
+<br />
+
+---
+
+Was this documentation helpful?
+
+<!-- LikeBtn.com BEGIN -->
+<span class="likebtn-wrapper" data-theme="tick" data-i18n_like="Yes" data-ef_voting="grow" data-show_dislike_label="true" data-counter_zero_show="true" data-i18n_dislike="No"></span>
+<script>(function(d,e,s){if(d.getElementById("likebtn_wjs"))return;a=d.createElement(e);m=d.getElementsByTagName(e)[0];a.async=1;a.id="likebtn_wjs";a.src=s;m.parentNode.insertBefore(a, m)})(document,"script","//w.likebtn.com/js/w/widget.js");</script>
+<!-- LikeBtn.com END -->
+
+{% include docs-footer.md %}