Merge branch 'v10.3.0' into v2_main

.github/CONTRIBUTING.md

@@ -1,42 +1,83 @@
-# Contributing
+# Feedback and Contribution
 
-Contributions to `thoughtspot_tml` are encouraged and very welcome!
+We welcome any input, feedback, bug reports, and contributions via [__ThoughtSpot TML__'s GitHub Repository](http://github.com/thoughtspot/thoughtspot_tml/).
 
-Contributions can come in many forms: documentation enhancements, features, bug fixes, creating issues, and participating in the community.
+All contributions, suggestions, and feedback you submitted are accepted under the [Project's license](./LICENSE). You represent that if you do not own copyright in the code that you have the authority to submit it under the [Project's license](./LICENSE). All feedback, suggestions, or contributions are not confidential.
 
-If you're interested in helping out, you might find some inspiration in [Issues](https://github.com/thoughtspot/thoughtspot_tml/issues). If you have an idea, but don't see it there, don't hesitate to open a new issue.
+### Setting Up Your Environment
 
-Before submitting a pull request, please make sure the enhancement or bugfix you've made has been discussed.
+Fork the __ThoughtSpot TML__ repository on GitHub and then clone the fork to you local machine. For more details on forking see the [GitHub Documentation](https://help.github.com/en/articles/fork-a-repo).
 
-This will ensure no work is duplicated, and that a general approach has been agreed.
+```bash
+git clone https://github.com/YOUR-USERNAME/thoughtspot_tml.git
+```
+
+To keep your fork up to date with changes in this repo, you can [use the fetch upstream button on GitHub](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork).
 
-## Local development setup
+__ThoughtSpot TML__ uses `uv` ([__docs__](https://docs.astral.sh/uv/)) to manage its dependencies. Once you have cloned the repository, run the following command from the root of the repository to setup your development environment:
 
-```shell
-python -m venv .venv
-source .venv\bin\activate
-pip install -e .[dev]
+```bash
+cd thoughtspot_tml/
+uv pip install -e ".[dev]"
+uv run hatch run dev:setup
 ```
 
-## Build
+Now you can install the latest version of __ThoughtSpot TML__ locally using `pip`. The `-e` flag indicates that your local changes will be reflected every time you open a new Python interpreter (instead of having to reinstall the package each time).
 
-> [!IMPORTANT]
-> If you're writing support for a new version of ThoughtSpot, this step requires you to download `scriptability`'s EDoc protocol buffer specification from ThoughtSpot's internal version control.
->
-> ```shell
-> python _generate\__main__.py
-> ```
+`[dev]` indicates that pip should also install the development and documentation requirements which you can find  in `pyproject.toml` (`[project.optional-dependencies]/dev`)
+
+__ThoughtSpot TML__ also uses [__Hatch__](https://hatch.pypa.io/latest/) to group together commands (`hatch run dev:setup`) for simpler maintenance of irregular contributors. You can observe the individual commands that are run within each named script under `[tool.hatch.envs.dev.scripts]`.
 
-If you're not building support for a new version of ThoughtSpot, simply **[Fork this repository](https://github.com/thoughtspot/thoughtspot_tml/fork)** and when ready, **[Open a Pull Request](https://github.com/thoughtspot/thoughtspot_tml/compare)** to contribute your changes.
+### Creating a Branch
 
-## Linting
+Once your local environment is up-to-date, you can create a new git branch which will contain your contribution (always create a new branch instead of making changes to the main branch):
 
-```shell
-ruff check src/ --config pyproject.toml
+```bash
+git switch -c <your-branch-name> dev
 ```
 
-## Running Tests
+With this branch checked-out, make the desired changes to the package.
 
-```shell
-coverage run -m ward; coverage report
+### Creating a Pull Request
+
+When you are happy with your changes, you can commit them to your branch by running
+
+```bash
+git add <modified-file>
+git commit -m "Some descriptive message about your change"
+git push origin <your-branch-name>
 ```
+
+You will then need to submit a pull request (PR) on GitHub asking to merge your example branch into the main __ThoughtSpot TML__ repository. For details on creating a PR see GitHub documentation [__Creating a pull request__](https://help.github.com/en/articles/creating-a-pull-request).
+
+You can add more details about your example in the PR such as motivation for the example or why you thought it would be a good addition. You will get feedback in the PR discussion if anything needs to be changed. To make changes continue to push commits made in your local example branch to origin and they will be automatically shown in the PR.
+
+Hopefully your PR will be answered in a timely manner and your contribution will help others in the future.
+
+> [!IMPORTANT]
+> :exclamation: __If you're writing support for a new version of ThoughtSpot__ :exclamation:
+> 
+> this step requires you to download `scriptability`'s EDoc protocol buffer specification from ThoughtSpot's internal version control.
+>
+> Building from source protos will require you to [__install the Protobuf compiler__](https://github.com/protocolbuffers/protobuf?tab=readme-ov-file#protobuf-compiler-installation) (`protoc`).
+>
+>  - __MacOS__ users can `brew install protobuf`
+>  - __Windows__ users can `choco install protobuf`
+>  - or .. manually [__download__](https://github.com/protocolbuffers/protobuf/releases), unzip, and add to your `PATH` variable.
+>
+> From here, you can run the following command to generate the `_scriptability.py` python module.
+>
+> ```bash
+> uv run hatch run dev:compile
+> ```
+>
+> If the edoc spec has significantly changed since the last built version, you _may_ need to investigate fixes. A good place to start is [`_generate/__main__.py`](../_generate/__main__.py), where the code has been heavily commented.
+>
+
+---
+
+### Readying a Release
+
+The __ThoughtSpot TML's__ release process is managed by [__Github Actions__](./workflows/publish.yml)!
+
+Simply publish a new release here on Github with the appropriate next-version tag, and Github will take care of publishing it to PyPI.

.github/actions/setup/action.yml

@@ -0,0 +1,13 @@
+name: "Setup Environment"
+description: "Installs uv using the community GH Action and installs the development dependencies."
+runs:
+  using: composite
+  steps:
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v5
+      with:
+        version: "0.5.14"
+
+    - name: Install development dependencies
+      run: uv pip install -e ".[dev]"

.github/workflows/code_quality.yml

@@ -0,0 +1,54 @@
+name: Enforce Code Quality
+
+on:
+  workflow_dispatch:
+  pull_request:
+  push:
+    branches:
+      - v2_main
+
+jobs:
+
+  lint:
+    name: Ensuring Types are correct.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: ./.github/actions/setup
+
+      - name: Run the Linter and Formatter.
+        run: uv run hatch run dev:lint
+
+
+  typecheck:
+    name: Ensuring Python type-hints are correct.
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: ./.github/actions/setup
+
+      - name: Run the type checker.
+        run: uv run hatch run dev:typecheck
+
+
+  test:
+    name: Testing on Python ${{ matrix.python-version }} and ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - uses: ./.github/actions/setup
+
+      - name: Run testing
+        run: uv run hatch run dev:test

.github/workflows/publish.yml

@@ -1,16 +1,20 @@
 name: Publish to PyPI.org
+
 on:
   release:
     types: [published]
+
 jobs:
-  pypi:
+
+  publish:
+    name: Build and Publish to PyPI
     runs-on: ubuntu-latest
     steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-        with:
-          fetch-depth: 0
-      - run: python3 -m pip install --upgrade build && python3 -m build
+      - uses: actions/checkout@v4
+      - uses: ./.github/actions/setup
+
+      - name: Build our .whl file
+        run: uvx hatch build
 
       - name: Publish package
         uses: pypa/gh-action-pypi-publish@release/v1

.gitignore

@@ -38,6 +38,7 @@ venv.bak/
 
 # ignore workspace files
 *sublime*
+*vscode*
 
 # ignore virtualization
 .python-version
@@ -46,11 +47,7 @@ venv.bak/
 .coverage
 
 # ignore spec generators
-_generate/*.json
-_generate/*.yaml
-_generate/*.proto
-_generate/*.toml
-_generate/archive/*
+_generate/edoc/*
 
 # ignore test temp directory
 tests/data/_temp/*

.pre-commit-config.yaml

@@ -1,25 +1,30 @@
 repos:
+
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.14.1
+    hooks:
+      - id: mypy
+
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.6.0
+    rev: v5.0.0
     hooks:
-      - id: trailing-whitespace
-      - id: end-of-file-fixer
       - id: check-yaml
       - id: check-added-large-files
 
+  - repo: https://github.com/asottile/pyupgrade
+    rev: v3.19.1
+    hooks:
+      - id: pyupgrade
+        # SEE pyproject.toml FOR py{min-support}-plus.
+        args: [--py39-plus, --keep-runtime-typing]
+
   - repo: https://github.com/charliermarsh/ruff-pre-commit
-    # Ruff version.
-    rev: v0.5.5
+    rev: v0.8.6
     hooks:
-      # Run the linter.
+      # FIXING CODE RULES.
       - id: ruff
-        args: [--fix]
+        args: [--config=pyproject.toml, --fix]
 
-      # Run the formatter.
+      # PURELY STYLE FORMATTING RULES.
       - id: ruff-format
-
-  - repo: https://github.com/dhruvmanila/remove-print-statements
-    rev: v0.5.2
-    hooks:
-      - id: remove-print-statements
-        args: [--verbose]
+        args: [--config=pyproject.toml]

README.md

@@ -16,6 +16,8 @@
     <a href="#thoughtspot_tml-reference">Reference</a>
     <span> | </span>
     <a href="#notes-on-thoughtspot-modeling-language">Notes</a>
+    <span> | </span>
+    <a href="#contributing">Contributing</a>
   </h3>
 
 🚨 __If your examples or scripts are built on__ `thoughtspot_tml==1.3.0`__, see our [Migration to v2.0.0](#migration-to-v200) guide__. 🚨
@@ -584,7 +586,7 @@ tml = disambiguate(tml, guid_mapping=name_guid_map)
 - TML is implemented in the YAML 1.1 spec.
 - When importing a TML file, if the `guid` matches to an existing object in the same Org being uploaded into, then that object will be updated. If the `guid` is missing or does not match an object in that Org, a new object is created with a new GUID. GUIDs are unique to the entire ThoughtSpot instance - if no GUID match is found in an Org, but the GUID has not been used on the instance, ThoughtSpot will use the GUID provided in the TML file when creating the new object.
 
-## Want to contribute?
+## Contributing
 
 We welcome all help! :heart: For guidance on setting up a development environment, see our [Contributing Guide][contrib].