Add Carta Issuer API connector - #59
Draft
fivetran-tommy wants to merge 1 commit into
Draft
Conversation
Syncs issuer-level equity from the Carta Issuer API: option grants, RSUs, RSAs, certificates, flattened vesting events and exercises, stakeholders, share classes, 409A fair market values and their per share class valuations, vesting schedule templates, convertible notes, stakeholder cap table holdings, and issuer and corporation metadata. 16 tables. One connection can cover several issuers through a comma separated issuer_ids value, and every incremental cursor is namespaced by issuer. Notable behavior, all documented in the README: - Incremental sync for the four securities resources that accept Carta's lastModifiedDatetimeAfter cursor. The cursor is written only when a resource completes, because Carta does not return records in modified order. - Equity quantities and prices are declared STRING so Carta's high-precision decimals do not lose precision. - Carta grants OAuth scopes all or nothing per app, so a 403 is a permanent fact about the app rather than a transient error. It is raised as InsufficientScopeError, caught per resource, and logged, so an app with a narrower grant still replicates what it can reach. - A 401 triggers one token refresh and one retry, so a first sync longer than the one hour production token lifetime completes. - Checkpoints every 10,000 upserts inside a resource. One security can expand into hundreds of vesting events, and a single very large commit at the end of a resource can fail and roll back rows already sent.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new Carta Issuer API Connector SDK example connector to the community_connectors repo, intended to replicate issuer-level equity/securities data into 16 destination tables with incremental sync where supported.
Changes:
- Adds the
carta/connector implementation, including OAuth2 token handling, pagination, state management, and schema definitions. - Adds connector-specific documentation (
carta/README.md) and a startercarta/configuration.json. - Registers the new connector in the repository’s top-level
README.mdconnector list.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| README.md | Adds Carta to the catalog/list of available community connectors. |
| carta/README.md | Documents connector purpose, setup, configuration, and replicated tables. |
| carta/connector.py | Implements the Carta Issuer API connector (schema + update, auth, pagination, sync logic). |
| carta/configuration.json | Provides placeholder configuration keys required by the connector. |
Suppressed comments (7)
carta/README.md:28
- The
fivetran initcommand should follow the pattern used across this repo (fivetran init --template <folder>). Usingconnectors/cartaand<project-path>is inconsistent with other examples and may not work as intended.
fivetran init <project-path> --template connectors/carta
carta/README.md:31
- The template README includes a short explanation paragraph about what
fivetran initdoes, and notes should be formatted as> Note:(no space before the colon). Adding the missing paragraph here also makes the Getting started section consistent with other connectors.
> Note : Ensure you have updated the `configuration.json` file with the necessary parameters before running `fivetran debug`. See the [Configuration file](#configuration-file) section for details on the required configuration parameters.
carta/README.md:47
- Use fenced code blocks with language hints for configuration examples (e.g., use
jsonfor the configuration snippet) to match the documentation guidelines for connector READMEs.
**carta/README.md:85**
* The Authentication section should include a numbered list of user actions describing how to obtain credentials and set up authentication (the current numbered list describes technical requirements, not user actions).
Carta uses OAuth2 with the client_credentials grant. Register an application in the Carta Developer Portal to obtain a client id and secret, and have Carta promote the application before using it against production data.
**carta/README.md:140**
* The Tables created section needs to include the list of columns (and primary key) for each table (for example by pasting the per-table schema objects from `schema()`), not just the table names and primary keys.
Tables created
Securities and their children:
option_grants(primary keyissuer_id,id), incremental. Stock option grants.
**carta/connector.py:1410**
* The first log statement in `update()` should follow the required format `log.warning("Example: <CATEGORY> : <EXAMPLE_NAME>")` (note the ` : ` separator).
log.warning("Example: Source Examples - Carta Issuer API Connector")
**carta/connector.py:989**
* Per the repo’s Python connector guidelines, each `op.checkpoint()` call should be preceded by the standard checkpoint comment block (the current comment is abbreviated and doesn’t match the required wording).
# Save the progress by checkpointing the state once the resource and its cursor are complete.
op.checkpoint(state)
</details>
---
💡 <a href="/fivetran/community_connectors/new/main?filename=.github/skills/code-review/SKILL.md" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Add a `code-review` agent skill</a> or configure MCP servers for context-aware, tailored reviews. <a href="https://docs.github.com/en/copilot/how-tos/use-copilot-agents/request-a-code-review/use-code-review#mcp-servers-and-agent-skills" class="Link--inTextBlock" target="_blank" rel="noopener noreferrer">Learn more in the docs.</a>
Comment on lines
+15
to
+19
| - [Supported Python versions](https://github.com/fivetran/connector_sdk/blob/main/README.md#requirements) | ||
| - Operating system: | ||
| - Windows: 10 or later (64-bit only) | ||
| - macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64]) | ||
| - Linux: Distributions such as Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64) |
Comment on lines
+1400
to
+1409
| """ | ||
| Define the update function, which is a required function, and is called by Fivetran during | ||
| each sync. | ||
| See the technical reference documentation for more details on the update function | ||
| https://fivetran.com/docs/connector-sdk/technical-reference/connector-sdk-code/connector-sdk-methods#update | ||
| Args: | ||
| configuration: A dictionary containing connection details | ||
| state: A dictionary containing state information from previous runs | ||
| The state dictionary is empty for the first sync or for any full re-sync | ||
| """ |
Comment on lines
+955
to
+956
| # The 'upsert' operation inserts or updates the record in the destination table. | ||
| op.upsert(table=table, data=row_builder(issuer_id, record)) |
Comment on lines
+568
to
+570
| wait_seconds = int( | ||
| response.headers.get("Retry-After", __DEFAULT_RETRY_AFTER_SECONDS) | ||
| ) |
Contributor
|
Hi @fivetran-tommy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A connector for the Carta Issuer API, which exposes the securities a company has issued. It replicates 16 tables: option grants, restricted stock units, restricted stock awards, certificates, flattened vesting events and option grant exercises, stakeholders, share classes, 409A fair market values and their per share class valuations, vesting schedule templates, convertible notes, stakeholder cap table holdings and their per share class breakdown, and issuer and corporation metadata.
Read only. No write calls to Carta.
The common use is total compensation reporting:
stakeholders.employee_idandstakeholders.emailjoin equity to an HRIS or payroll source, which no single Carta export does flexibly. Dilution analysis and vesting forecasts fall out of the same tables.Why these design choices
Four things about this API are not obvious, and each one shaped the implementation. All four are written up in the connector README.
Scopes are granted all or nothing per OAuth application. If an application is not registered for one of the requested scopes, Carta returns a token with no scope at all, and that token is then rejected by every data endpoint. So the connector raises immediately on an empty granted scope, and it treats a 403 on a resource as a permanent fact about the application rather than a transient error:
InsufficientScopeErroris caught per resource and logged as a warning, so an application with a narrower grant replicates what it can reach instead of failing the sync.scopesis configurable for exactly this reason.Tokens are short lived. The production lifetime is one hour, which a first sync can outlive. A 401 triggers one token refresh and one retry. A second consecutive 401 is raised so a genuinely bad credential fails fast.
The incremental cursor cannot be advanced mid-resource. Only four resources accept
lastModifiedDatetimeAfter, and Carta does not return records in modified order, so a cursor written partway through could skip older records not yet fetched. The cursor is therefore written only when a resource completes. Mid-resource checkpoints persist delivered rows without moving it, and an interrupted sync safely refetches from the last completed cursor.One security expands into many child rows. A single option grant can carry hundreds of vesting events, so the checkpoint interval is measured in upserts (10,000) rather than parent records. That keeps each commit small, which matters because a single very large commit at the end of a resource can fail and roll back everything already sent.
Two smaller notes: equity quantities and prices are declared
STRINGbecause Carta returns high-precision decimals such as99.00000000000000000000and a float would silently lose precision; and the stakeholder field Carta callsgrouplands asstakeholder_group, sincegroupis reserved in most warehouses.One connection can sync several issuers through a comma separated
issuer_idsvalue. Every row carries itsissuer_idand every cursor is namespaced by issuer.Validation
black --line-length 99andflake8against the repository.flake8config both pass oncarta/.The connector was exercised end to end offline, with
requestsandtime.sleepstubbed and the Carta API routed by URL, so no credentials or network were involved. The run covered two issuers and every resource, and deliberately injected one expired token and one permanently scope denied resource:What that run confirms:
111_optionGrants_last_modifiedand222_certificates_last_modified.corporationssynced once for the whole sync rather than once per issuer.grouplanded asstakeholder_group.schema().An earlier revision of this description claimed 17 tables. The harness check against
schema()caught it: the correct count is 16.Known limitations
issuersdetail endpoint is not reachable for every application, so that table is best effort and is skipped with a warning when it 404s.corporationscovers legal entity information in that case.