Skip to content

Commit 73fa07d

Browse files
elynch303Evan Lynchclaude
authored
feat(clickup): add ClickUp data-source plugin (folder-scope boards, sprints, DORA) (#9009)
* feat(clickup): scaffold ClickUp data-source plugin (task→issue MVP) Add a new DevLake data-source plugin for ClickUp, mirroring the linear (structure) and trello (REST client) tracker plugins. - models: ClickUpConnection (token auth, no Bearer prefix), ClickUpList scope (keyed by list id), ClickUpTask, ClickUpUser, ClickUpTaskComment (stub), ClickUpScopeConfig (status + type mapping) - migrationscripts: register + addInitTables creating all _tool_clickup_* tables via migrationhelper.AutoMigrateTables - tasks: task/user collectors (page-based + member REST), extractors, convertors to ticket.Issue/BoardIssue/IssueAssignee, crossdomain.Account and ticket.Board; shared status/type/time helpers - api: connection CRUD + test-connection (GET /team), scope, scope-config, hierarchical remote-scopes (Team→Space→Folder→List), proxy, blueprint v200 - impl: all required plugin interfaces incl. GetTablesInfo listing every model, dependency-ordered SubTaskMetas and PrepareTaskData - register clickup in plugins/table_info_test.go go build ./plugins/clickup/... and go vet pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(clickup): add ClickUp connector with folder-scope boards, sprints, and DORA Add a new ClickUp data-source plugin so ClickUp workspaces feed DevLake's issue-tracking and DORA/velocity metrics, modeled on the Jira/Linear connectors. Backend: - Folder is the data-source scope (ClickUpFolder ~= a Jira board); lists are child entities collected every sync, so rolling/archived sprint lists never need re-scoping. - Sprint lists (name-matched) -> ticket.Sprint + board_sprints; start/end parsed from the list name (M/D vs D/M disambiguated by the >12 rule). - Tasks -> issues + board_issues; sprint-list tasks also -> sprint_issues. - Story points from ClickUp native sprint points (or a configured custom field). - Scope config: sprint-name pattern, story-point field, force-issue-type (flag a folder INCIDENT for DORA CFR/MTTR), status/type overrides. - remote-scopes walks Workspace -> Space -> Folder (folders selectable). - migration 20260722 adds _tool_clickup_folders + sprint/story-point columns. - Unit tests for sprint name/date parsing and status/type/story-point mapping. Config UI: - ClickUp connection registration (personal-token auth) with a Beta pill. - Boards data-scope picker + scope-config transformation panel. - getPluginScopeId(clickup) -> folderId so scope delete and per-scope project selection target one scope; fix a pre-existing Connections-page chunk() split that silently dropped plugins. Docs & dashboards: - Plugin README (connection, folder scope, sprints, story points, incidents). - Branded ClickUp Grafana dashboard (grafana/dashboards/ClickUp.json). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Evan Lynch <e.lynch303@gmail.com> * feat(clickup): classify issues by list name (bug/incident list patterns) ClickUp frequently carries no per-task type, so teams group bugs and incidents in dedicated lists (e.g. "QA Bugs", "Incident Backlog") rather than tagging each task. Add scope-config BugListPattern and IncidentListPattern regexes matched against a list's name; tasks in a matching list are classified BUG / INCIDENT (INCIDENT wins when both match). Precedence in the task convertor: forced DefaultIssueType > list-name pattern > per-task type detection > REQUIREMENT default. - models: BugListPattern / IncidentListPattern on ClickUpScopeConfig - migration 20260723 adds the two columns (additive AutoMigrate) - task convertor: loadListTypeOverrides + pure listTypeFor helper - unit test for listTypeFor (precedence, case-insensitivity, nil regexes) - config-ui transformation panel: two list-pattern inputs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Evan Lynch <e.lynch303@gmail.com> --------- Signed-off-by: Evan Lynch <e.lynch303@gmail.com> Co-authored-by: Evan Lynch <elynch@freedomeledger.net> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 2127e84 commit 73fa07d

47 files changed

Lines changed: 6573 additions & 2 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎backend/plugins/clickup/README.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
-->
17+
18+
# ClickUp
19+
20+
The ClickUp plugin collects issues, boards, and sprints from ClickUp so they
21+
feed DevLake's issue-tracking and DORA/velocity metrics, modeled on the Jira
22+
and Linear connectors.
23+
24+
## Authentication
25+
26+
ClickUp authenticates with a **personal API token** (ClickUp → Settings → Apps
27+
→ API Token, starts with `pk_`). The token is sent verbatim in the
28+
`Authorization` header. OAuth is not yet supported.
29+
30+
Create a connection with:
31+
32+
- **Endpoint** — `https://api.clickup.com/api/v2/`
33+
- **Token** — your `pk_...` personal token
34+
35+
## Data scope: the folder is the board
36+
37+
Unlike a raw list, the **scope you select is a ClickUp folder** (e.g. a team's
38+
`Dev Team` / `Sprint Folder`). This mirrors a Jira board: selecting the folder
39+
collects every list inside it on each sync, so rolling and archived sprint
40+
lists never need to be re-scoped.
41+
42+
Domain mapping:
43+
44+
| ClickUp | DevLake domain |
45+
| --- | --- |
46+
| Folder | `board` |
47+
| Sprint list (name-matched) | `sprint` + `board_sprint` |
48+
| Task | `issue` + `board_issue` |
49+
| Task in a sprint list | `sprint_issue` |
50+
| Folder member | `account` |
51+
52+
## Sprints
53+
54+
A list is treated as a sprint when its name matches the **sprint name pattern**
55+
(default `(?i)sprint\s*\d+`, e.g. `v4.3.0 Sprint 40 (7/6/26 - 7/19/26)`). The
56+
start/end dates are parsed from the parenthesised date span in the name;
57+
`M/D/YY` vs `D/M/YY` ordering is disambiguated automatically (a component > 12
58+
must be the day). Lists that don't match are collected as plain board issues.
59+
Archived sprint lists are collected too, so historical velocity is retained.
60+
61+
## Story points
62+
63+
Story points default to ClickUp's native sprint **`points`** field. To read
64+
them from a custom field instead (e.g. a Fibonacci "LOE" field), set
65+
**Story point field** in the scope config to the custom-field name.
66+
67+
## Incidents (DORA Change Failure Rate / MTTR)
68+
69+
ClickUp tasks have no universal "type" field, so incidents are modeled by
70+
**scope**: add the folder that holds incidents (e.g. a "Security Incidents"
71+
folder) as its own board and set the scope config's **Force issue type** to
72+
`INCIDENT`. Every issue on that board is then classified as an incident.
73+
74+
## Scope configuration (transformation)
75+
76+
Per board, the scope config lets you override:
77+
78+
- **Sprint name pattern** — which lists are sprints.
79+
- **Story point field** — native `points` (blank) or a custom field name.
80+
- **Force issue type** — flag the whole board as `REQUIREMENT` / `BUG` /
81+
`INCIDENT` (leave blank to detect per task).
82+
- **Issue type patterns** — RegExes matched against a task's type; precedence
83+
is `INCIDENT` > `BUG` > `REQUIREMENT`.
84+
- **Status mapping** — ClickUp statuses are auto-mapped by their `type`
85+
(`open`/`unstarted` → `TODO`, `custom` → `IN_PROGRESS`, `done`/`closed` →
86+
`DONE`); list raw status names to override where a team's workflow differs.
87+
88+
## Metrics
89+
90+
A branded **ClickUp** Grafana dashboard (`grafana/dashboards/{mysql,postgresql}/ClickUp.json`)
91+
renders issue throughput and lead-time metrics; DORA and sprint/velocity
92+
metrics are computed from the source-agnostic domain layer.

‎backend/plugins/clickup/api/blueprint_v200.go‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
/*
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
*/
17+
18+
package api
19+
20+
import (
21+
"github.com/apache/incubator-devlake/core/errors"
22+
coreModels "github.com/apache/incubator-devlake/core/models"
23+
"github.com/apache/incubator-devlake/core/models/domainlayer/didgen"
24+
"github.com/apache/incubator-devlake/core/models/domainlayer/ticket"
25+
"github.com/apache/incubator-devlake/core/plugin"
26+
"github.com/apache/incubator-devlake/core/utils"
27+
helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
28+
"github.com/apache/incubator-devlake/helpers/srvhelper"
29+
"github.com/apache/incubator-devlake/plugins/clickup/models"
30+
"github.com/apache/incubator-devlake/plugins/clickup/tasks"
31+
)
32+
33+
func MakePipelinePlanV200(
34+
subtaskMetas []plugin.SubTaskMeta,
35+
connectionId uint64,
36+
bpScopes []*coreModels.BlueprintScope,
37+
) (coreModels.PipelinePlan, []plugin.Scope, errors.Error) {
38+
connection, err := dsHelper.ConnSrv.FindByPk(connectionId)
39+
if err != nil {
40+
return nil, nil, err
41+
}
42+
scopeDetails, err := dsHelper.ScopeSrv.MapScopeDetails(connectionId, bpScopes)
43+
if err != nil {
44+
return nil, nil, err
45+
}
46+
plan, err := makePipelinePlanV200(subtaskMetas, scopeDetails, connection)
47+
if err != nil {
48+
return nil, nil, err
49+
}
50+
scopes, err := makeScopesV200(scopeDetails, connection)
51+
return plan, scopes, err
52+
}
53+
54+
func makePipelinePlanV200(
55+
subtaskMetas []plugin.SubTaskMeta,
56+
scopeDetails []*srvhelper.ScopeDetail[models.ClickUpFolder, models.ClickUpScopeConfig],
57+
connection *models.ClickUpConnection,
58+
) (coreModels.PipelinePlan, errors.Error) {
59+
plan := make(coreModels.PipelinePlan, len(scopeDetails))
60+
for i, scopeDetail := range scopeDetails {
61+
stage := plan[i]
62+
if stage == nil {
63+
stage = coreModels.PipelineStage{}
64+
}
65+
scope, scopeConfig := scopeDetail.Scope, scopeDetail.ScopeConfig
66+
task, err := helper.MakePipelinePlanTask(
67+
"clickup",
68+
subtaskMetas,
69+
scopeConfig.Entities,
70+
tasks.ClickUpOptions{
71+
ConnectionId: connection.ID,
72+
FolderId: scope.FolderId,
73+
ScopeConfigId: scope.ScopeConfigId,
74+
},
75+
)
76+
if err != nil {
77+
return nil, err
78+
}
79+
stage = append(stage, task)
80+
plan[i] = stage
81+
}
82+
return plan, nil
83+
}
84+
85+
func makeScopesV200(
86+
scopeDetails []*srvhelper.ScopeDetail[models.ClickUpFolder, models.ClickUpScopeConfig],
87+
connection *models.ClickUpConnection,
88+
) ([]plugin.Scope, errors.Error) {
89+
scopes := make([]plugin.Scope, 0, len(scopeDetails))
90+
idgen := didgen.NewDomainIdGenerator(&models.ClickUpFolder{})
91+
for _, scopeDetail := range scopeDetails {
92+
scope, scopeConfig := scopeDetail.Scope, scopeDetail.ScopeConfig
93+
id := idgen.Generate(connection.ID, scope.FolderId)
94+
if utils.StringsContains(scopeConfig.Entities, plugin.DOMAIN_TYPE_TICKET) {
95+
scopes = append(scopes, ticket.NewBoard(id, scope.Name))
96+
}
97+
}
98+
return scopes, nil
99+
}

‎backend/plugins/clickup/api/connection_api.go‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
/*
2+
Licensed to the Apache Software Foundation (ASF) under one or more
3+
contributor license agreements. See the NOTICE file distributed with
4+
this work for additional information regarding copyright ownership.
5+
The ASF licenses this file to You under the Apache License, Version 2.0
6+
(the "License"); you may not use this file except in compliance with
7+
the License. You may obtain a copy of the License at
8+
9+
http://www.apache.org/licenses/LICENSE-2.0
10+
11+
Unless required by applicable law or agreed to in writing, software
12+
distributed under the License is distributed on an "AS IS" BASIS,
13+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
See the License for the specific language governing permissions and
15+
limitations under the License.
16+
*/
17+
18+
package api
19+
20+
import (
21+
"context"
22+
"net/http"
23+
24+
"github.com/apache/incubator-devlake/core/errors"
25+
"github.com/apache/incubator-devlake/core/plugin"
26+
helper "github.com/apache/incubator-devlake/helpers/pluginhelper/api"
27+
"github.com/apache/incubator-devlake/plugins/clickup/models"
28+
"github.com/apache/incubator-devlake/plugins/clickup/tasks"
29+
"github.com/apache/incubator-devlake/server/api/shared"
30+
)
31+
32+
type ClickUpTestConnResponse struct {
33+
shared.ApiBody
34+
Connection *models.ClickUpConn
35+
}
36+
37+
func testConnection(ctx context.Context, connection models.ClickUpConn) (*ClickUpTestConnResponse, errors.Error) {
38+
if vld != nil {
39+
if err := vld.Struct(connection); err != nil {
40+
return nil, errors.Default.Wrap(err, "error validating target")
41+
}
42+
}
43+
if connection.Endpoint == "" {
44+
connection.Endpoint = tasks.DefaultEndpoint
45+
}
46+
apiClient, err := helper.NewApiClientFromConnection(ctx, basicRes, &connection)
47+
if err != nil {
48+
return nil, err
49+
}
50+
// GET /team lists the authenticated user's workspaces; it verifies the token.
51+
res, err := apiClient.Get("team", nil, nil)
52+
if err != nil {
53+
return nil, errors.BadInput.Wrap(err, "verify token failed")
54+
}
55+
if res.StatusCode == http.StatusUnauthorized || res.StatusCode == http.StatusForbidden {
56+
return nil, errors.HttpStatus(http.StatusBadRequest).New("authentication failed, please check your API token")
57+
}
58+
if res.StatusCode != http.StatusOK {
59+
return nil, errors.HttpStatus(res.StatusCode).New("unexpected status code while testing connection")
60+
}
61+
connection = connection.Sanitize()
62+
body := ClickUpTestConnResponse{}
63+
body.Success = true
64+
body.Message = "success"
65+
body.Connection = &connection
66+
return &body, nil
67+
}
68+
69+
// TestConnection test clickup connection
70+
// @Summary test clickup connection
71+
// @Description Test clickup Connection
72+
// @Tags plugins/clickup
73+
// @Param body body models.ClickUpConn true "json body"
74+
// @Success 200 {object} ClickUpTestConnResponse "Success"
75+
// @Failure 400 {string} errcode.Error "Bad Request"
76+
// @Failure 500 {string} errcode.Error "Internal Error"
77+
// @Router /plugins/clickup/test [POST]
78+
func TestConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
79+
var connection models.ClickUpConn
80+
if err := helper.Decode(input.Body, &connection, vld); err != nil {
81+
return nil, err
82+
}
83+
result, err := testConnection(context.TODO(), connection)
84+
if err != nil {
85+
return nil, plugin.WrapTestConnectionErrResp(basicRes, err)
86+
}
87+
return &plugin.ApiResourceOutput{Body: result, Status: http.StatusOK}, nil
88+
}
89+
90+
// TestExistingConnection test clickup connection by ID
91+
// @Summary test clickup connection
92+
// @Description Test clickup Connection
93+
// @Tags plugins/clickup
94+
// @Param connectionId path int true "connection ID"
95+
// @Success 200 {object} ClickUpTestConnResponse "Success"
96+
// @Failure 400 {string} errcode.Error "Bad Request"
97+
// @Failure 500 {string} errcode.Error "Internal Error"
98+
// @Router /plugins/clickup/connections/{connectionId}/test [POST]
99+
func TestExistingConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
100+
connection, err := dsHelper.ConnApi.GetMergedConnection(input)
101+
if err != nil {
102+
return nil, errors.BadInput.Wrap(err, "find connection from db")
103+
}
104+
if err := helper.DecodeMapStruct(input.Body, connection, false); err != nil {
105+
return nil, err
106+
}
107+
result, testErr := testConnection(context.TODO(), connection.ClickUpConn)
108+
if testErr != nil {
109+
return nil, plugin.WrapTestConnectionErrResp(basicRes, testErr)
110+
}
111+
return &plugin.ApiResourceOutput{Body: result, Status: http.StatusOK}, nil
112+
}
113+
114+
// PostConnections create clickup connection
115+
// @Summary create clickup connection
116+
// @Description Create clickup connection
117+
// @Tags plugins/clickup
118+
// @Param body body models.ClickUpConnection true "json body"
119+
// @Success 200 {object} models.ClickUpConnection
120+
// @Failure 400 {string} errcode.Error "Bad Request"
121+
// @Failure 500 {string} errcode.Error "Internal Error"
122+
// @Router /plugins/clickup/connections [POST]
123+
func PostConnections(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
124+
return dsHelper.ConnApi.Post(input)
125+
}
126+
127+
// PatchConnection patch clickup connection
128+
// @Summary patch clickup connection
129+
// @Description Patch clickup connection
130+
// @Tags plugins/clickup
131+
// @Param body body models.ClickUpConnection true "json body"
132+
// @Success 200 {object} models.ClickUpConnection
133+
// @Failure 400 {string} errcode.Error "Bad Request"
134+
// @Failure 500 {string} errcode.Error "Internal Error"
135+
// @Router /plugins/clickup/connections/{connectionId} [PATCH]
136+
func PatchConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
137+
return dsHelper.ConnApi.Patch(input)
138+
}
139+
140+
// DeleteConnection delete a clickup connection
141+
// @Summary delete a clickup connection
142+
// @Description Delete a clickup connection
143+
// @Tags plugins/clickup
144+
// @Success 200 {object} models.ClickUpConnection
145+
// @Failure 400 {string} errcode.Error "Bad Request"
146+
// @Failure 409 {object} services.BlueprintProjectPairs "References exist to this connection"
147+
// @Failure 500 {string} errcode.Error "Internal Error"
148+
// @Router /plugins/clickup/connections/{connectionId} [DELETE]
149+
func DeleteConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
150+
return dsHelper.ConnApi.Delete(input)
151+
}
152+
153+
// ListConnections get all clickup connections
154+
// @Summary get all clickup connections
155+
// @Description Get all clickup connections
156+
// @Tags plugins/clickup
157+
// @Success 200 {object} []models.ClickUpConnection
158+
// @Failure 400 {string} errcode.Error "Bad Request"
159+
// @Failure 500 {string} errcode.Error "Internal Error"
160+
// @Router /plugins/clickup/connections [GET]
161+
func ListConnections(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
162+
return dsHelper.ConnApi.GetAll(input)
163+
}
164+
165+
// GetConnection get clickup connection detail
166+
// @Summary get clickup connection detail
167+
// @Description Get clickup connection detail
168+
// @Tags plugins/clickup
169+
// @Success 200 {object} models.ClickUpConnection
170+
// @Failure 400 {string} errcode.Error "Bad Request"
171+
// @Failure 500 {string} errcode.Error "Internal Error"
172+
// @Router /plugins/clickup/connections/{connectionId} [GET]
173+
func GetConnection(input *plugin.ApiResourceInput) (*plugin.ApiResourceOutput, errors.Error) {
174+
return dsHelper.ConnApi.GetDetail(input)
175+
}

0 commit comments

Comments
 (0)