Skip to content

feat: 支持供应商gpustack与xinference #353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+22 −42
Open

feat: 支持供应商gpustack与xinference #353

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 4 additions & 8 deletions backend/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,9 @@ go 1.25.0
require (
entgo.io/ent v0.14.4
github.com/GoYoko/web v1.4.0
github.com/chaitin/ModelKit/v2 v2.0.6
github.com/chaitin/ModelKit/v2 v2.3.1
github.com/doquangtan/socket.io/v4 v4.0.8
github.com/golang-migrate/migrate/v4 v4.18.3
github.com/google/go-github/v74 v74.0.0
github.com/google/uuid v1.6.0
github.com/google/wire v0.6.0
github.com/labstack/echo/v4 v4.13.4
Expand All @@ -18,10 +17,7 @@ require (
github.com/pgvector/pgvector-go v0.3.0
github.com/redis/go-redis/v9 v9.7.3
github.com/rokku-c/go-openai v1.35.7-fix2
github.com/sashabaranov/go-openai v1.41.1
github.com/sirupsen/logrus v1.9.3
github.com/spf13/viper v1.20.1
gitlab.com/gitlab-org/api/client-go v0.142.6
golang.org/x/crypto v0.41.0
golang.org/x/oauth2 v0.30.0
golang.org/x/text v0.28.0
Expand Down Expand Up @@ -80,16 +76,13 @@ require (
github.com/gofiber/websocket/v2 v2.2.1 // indirect
github.com/google/generative-ai-go v0.20.1 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/gax-go/v2 v2.15.0 // indirect
github.com/goph/emperror v0.17.2 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-retryablehttp v0.7.8 // indirect
github.com/hashicorp/hcl/v2 v2.23.0 // indirect
github.com/invopop/yaml v0.1.0 // indirect
github.com/joho/godotenv v1.5.1 // indirect
Expand Down Expand Up @@ -118,11 +111,13 @@ require (
github.com/rs/xid v1.6.0 // indirect
github.com/sagikazarmark/locafero v0.9.0 // indirect
github.com/savsgio/gotils v0.0.0-20230208104028-c358bd845dee // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/spf13/afero v1.14.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/pflag v1.0.6 // indirect
github.com/stretchr/testify v1.11.1 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
Expand All @@ -141,6 +136,7 @@ require (
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/otel/trace v1.37.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/mock v0.6.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/arch v0.19.0 // indirect
golang.org/x/exp v0.0.0-20250718183923-645b1fa84792 // indirect
Expand Down
22 changes: 2 additions & 20 deletions backend/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,8 @@ github.com/bytedance/sonic/loader v0.3.0/go.mod h1:N8A3vUdtUebEY2/VQC0MyhYeKUFos
github.com/certifi/gocertifi v0.0.0-20190105021004-abcd57078448/go.mod h1:GJKEexRPVJrBSOjoqN5VNOIKJ5Q3RViH6eu3puDRwx4=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chaitin/ModelKit/v2 v2.0.6 h1:SkghflCZu9daadWm6VJtBY6ZTWhjuebPCxwKNOm45FE=
github.com/chaitin/ModelKit/v2 v2.0.6/go.mod h1:ArjD6QyI3u9L01mphKoO/6saIxpRJLEgQxXTK5vt57A=
github.com/chaitin/ModelKit/v2 v2.3.1 h1:AAOsmOqm3/GSteS2dJq9jkCdmZPEIy8AT8KqutGklXM=
github.com/chaitin/ModelKit/v2 v2.3.1/go.mod h1:ArjD6QyI3u9L01mphKoO/6saIxpRJLEgQxXTK5vt57A=
github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M=
github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU=
github.com/cloudwego/eino v0.4.7 h1:wwqsFWCuzCQuhw1dYKqHjGWULzjDjFfN9sTn/cezYV4=
Expand Down Expand Up @@ -101,8 +101,6 @@ github.com/evanphx/json-patch v0.5.2 h1:xVCHIVMUu1wtM/VkR9jVZ45N3FhZfYMMYGorLCR8
github.com/evanphx/json-patch v0.5.2/go.mod h1:ZWS5hhDbVDyob71nXKNL0+PWn6ToqBHMikGIFbs31qQ=
github.com/fasthttp/websocket v1.5.3 h1:TPpQuLwJYfd4LJPXvHDYPMFWbLjsT91n3GpWtCQtdek=
github.com/fasthttp/websocket v1.5.3/go.mod h1:46gg/UBmTU1kUaTcwQXpUxtRwG2PvIZYeA8oL6vF3Fs=
github.com/fatih/color v1.16.0 h1:zmkK9Ngbjj+K0yRhTVONQh1p/HknKYSlNT+vZCzyokM=
github.com/fatih/color v1.16.0/go.mod h1:fL2Sau1YI5c0pdGEVCbKQbLXB6edEj1ZgiY4NijnWvE=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
Expand Down Expand Up @@ -161,13 +159,8 @@ github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6
github.com/google/generative-ai-go v0.20.1 h1:6dEIujpgN2V0PgLhr6c/M1ynRdc7ARtiIDPFzj45uNQ=
github.com/google/generative-ai-go v0.20.1/go.mod h1:TjOnZJmZKzarWbjUJgy+r3Ee7HGBRVLhOIgupnwR4Bg=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-github/v74 v74.0.0 h1:yZcddTUn8DPbj11GxnMrNiAnXH14gNs559AsUpNpPgM=
github.com/google/go-github/v74 v74.0.0/go.mod h1:ubn/YdyftV80VPSI26nSJvaEsTOnsjrxG3o9kJhcyak=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
Expand All @@ -190,14 +183,8 @@ github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/ad
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
github.com/hashicorp/go-hclog v1.6.3 h1:Qr2kF+eVWjTiYmU7Y31tYlP1h0q/X3Nl3tPGdaB11/k=
github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M=
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
github.com/hashicorp/go-retryablehttp v0.7.8 h1:ylXZWnqa7Lhqpk0L1P1LzDtGcCR0rPVUrx/c8Unxc48=
github.com/hashicorp/go-retryablehttp v0.7.8/go.mod h1:rjiScheydd+CxvumBsIrFKlx3iS0jrZ7LvzFGFmuKbw=
github.com/hashicorp/hcl/v2 v2.23.0 h1:Fphj1/gCylPxHutVSEOf2fBOh1VE4AuLV7+kbJf3qos=
github.com/hashicorp/hcl/v2 v2.23.0/go.mod h1:62ZYHrXgPoX8xBnzl8QzbWq4dyDsDtfCRgIq1rbJEvA=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
Expand Down Expand Up @@ -322,8 +309,6 @@ github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
github.com/sagikazarmark/locafero v0.9.0 h1:GbgQGNtTrEmddYDSAH9QLRyfAHY12md+8YFTqyMTC9k=
github.com/sagikazarmark/locafero v0.9.0/go.mod h1:UBUyz37V+EdMS3hDF3QWIiVr/2dPrx49OMO0Bn0hJqk=
github.com/sashabaranov/go-openai v1.41.1 h1:zf5tM+GuxpyiyD9XZg8nCqu52eYFQg9OOew0gnIuDy4=
github.com/sashabaranov/go-openai v1.41.1/go.mod h1:lj5b/K+zjTSFxVLijLSTDZuP7adOgerWeFyZLUhAKRg=
github.com/savsgio/gotils v0.0.0-20230208104028-c358bd845dee h1:8Iv5m6xEo1NR1AvpV+7XmhI4r39LGNzwUL4YpMuL5vk=
github.com/savsgio/gotils v0.0.0-20230208104028-c358bd845dee/go.mod h1:qwtSXrKuJh/zsFQ12yEE89xfCrGKK63Rr7ctU/uCo4g=
github.com/sebdah/goldie/v2 v2.5.5 h1:rx1mwF95RxZ3/83sdS4Yp7t2C5TCokvWP4TBRbAyEWY=
Expand Down Expand Up @@ -411,8 +396,6 @@ github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940 h1:4r45xpDWB6
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940/go.mod h1:CmBdvvj3nqzfzJ6nTCIwDTPZ56aVGvDrmztiO5g3qrM=
github.com/zclconf/go-cty-yaml v1.1.0 h1:nP+jp0qPHv2IhUVqmQSzjvqAWcObN0KBkUl2rWBdig0=
github.com/zclconf/go-cty-yaml v1.1.0/go.mod h1:9YLUH4g7lOhVWqUbctnVlZ5KLpg7JAprQNgxSZ1Gyxs=
gitlab.com/gitlab-org/api/client-go v0.142.6 h1:RjqPb7XxJypn9DzkSTuQUOJN7wpRGXZFH8rJCLj4Bg8=
gitlab.com/gitlab-org/api/client-go v0.142.6/go.mod h1:t02B5oJWYEzalBlYIh+PmEJm2H4LPC/VFM1xks5qtG8=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 h1:q4XOmH/0opmeuJtPsbFNivyl7bCt7yRBbeEm2sC/XtQ=
Expand Down Expand Up @@ -515,7 +498,6 @@ golang.org/x/tools v0.17.0/go.mod h1:xsh6VxdV005rRVaS6SSAf9oiAqljS7UZUacMZ8Bnsps
golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.239.0 h1:2hZKUnFZEy81eugPs4e2XzIJ5SOwQg0G82bpXD65Puo=
google.golang.org/api v0.239.0/go.mod h1:cOVEm2TpdAGHL2z+UwyS+kmlGr3bVWQQ6sYEqkKje50=
google.golang.org/genai v1.13.0 h1:LRhwx5PU+bXhfnXyPEHu2kt9yc+MpvuYbajxSorOJjg=
Expand Down
12 changes: 7 additions & 5 deletions backend/internal/model/handler/http/v1/model.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ import (
)

type ModelHandler struct {
usecase domain.ModelUsecase
logger *slog.Logger
usecase domain.ModelUsecase
logger *slog.Logger
modelkit *modelkit.ModelKit
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在 ModelHandler 中添加 modelkit 字段后，建议添加相应的注释说明其用途和生命周期，以提高代码的可读性和可维护性。

Suggested change
modelkit *modelkit.ModelKit
type ModelHandler struct {
usecase domain.ModelUsecase
logger *slog.Logger
modelkit *modelkit.ModelKit // 用于处理与模型相关的操作
}

}

func NewModelHandler(
Expand All @@ -26,7 +27,8 @@ func NewModelHandler(
readonly *middleware.ReadOnlyMiddleware,
logger *slog.Logger,
) *ModelHandler {
m := &ModelHandler{usecase: usecase, logger: logger.With("handler", "model")}
modelkit := modelkit.NewModelKit(logger)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在 NewModelHandler 函数中初始化 modelkit 实例时，建议考虑是否需要将其作为依赖注入的一部分，以提高组件的可测试性和灵活性。

Suggested change
modelkit := modelkit.NewModelKit(logger)
func NewModelHandler(
w *web.Group,
usecase domain.ModelUsecase,
auth *middleware.AuthMiddleware,
active *middleware.ActiveMiddleware,
readonly *middleware.ReadOnlyMiddleware,
logger *slog.Logger,
modelkit *modelkit.ModelKit, // 作为依赖注入传入
) *ModelHandler {
m := &ModelHandler{usecase: usecase, logger: logger.With("handler", "model"), modelkit: modelkit}
g := w.Group("/api/v1/model")
g.Use(auth.Auth(), active.Active("admin"), readonly.Guard())
g.POST("/check", m.Check)
g.GET("/provider/supported", m.GetProviderModelList)
return m
}

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

在初始化ModelKit实例时，应该添加错误处理逻辑以确保其正确初始化。

Suggested change
modelkit := modelkit.NewModelKit(logger)
\tmodelkit, err := modelkit.NewModelKit(logger)
\tif err != nil {
\t\t// handle error appropriately
\t\tlogger.Error(\"Failed to initialize ModelKit\", \"error\", err)
\t\treturn nil
\t}
\tm := &ModelHandler{usecase: usecase, logger: logger.With("handler", "model"), modelkit: modelkit}

m := &ModelHandler{usecase: usecase, logger: logger.With("handler", "model"), modelkit: modelkit}

g := w.Group("/api/v1/model")
g.Use(auth.Auth(), active.Active("admin"), readonly.Guard())
Expand Down Expand Up @@ -55,7 +57,7 @@ func NewModelHandler(
// @Success 200 {object} web.Resp{data=domain.CheckModelResp}
// @Router /api/v1/model/check [post]
func (h *ModelHandler) Check(c *web.Context, req domain.CheckModelReq) error {
modelkitRes, err := modelkit.CheckModel(c.Request().Context(), &modelkitDomain.CheckModelReq{
modelkitRes, err := h.modelkit.CheckModel(c.Request().Context(), &modelkitDomain.CheckModelReq{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

应该检查modelkit.CheckModel方法返回的错误，并进行适当的处理。

Suggested change
modelkitRes, err := h.modelkit.CheckModel(c.Request().Context(), &modelkitDomain.CheckModelReq{
\tmodelkitRes, err := h.modelkit.CheckModel(c.Request().Context(), &modelkitDomain.CheckModelReq{
\t\tProvider: string(req.Provider),
\t\tModel: req.ModelName,
\t\tBaseURL: req.APIBase,
\t\tAPIKey: req.APIKey,
\t\tAPIVersion: req.APIVersion,
\t})
\tif err != nil {
\t\t// handle error appropriately
\t\th.logger.Error("Failed to check model", "error", err)
\t\treturn err
\t}

Provider: string(req.Provider),
Model: req.ModelName,
BaseURL: req.APIBase,
Expand Down Expand Up @@ -199,7 +201,7 @@ func (h *ModelHandler) GetTokenUsage(c *web.Context, req domain.GetTokenUsageReq
// @Success 200 {object} web.Resp{data=domain.GetProviderModelListResp}
// @Router /api/v1/model/provider/supported [get]
func (h *ModelHandler) GetProviderModelList(c *web.Context, req domain.GetProviderModelListReq) error {
modelkitRes, err := modelkit.ModelList(c.Request().Context(), &modelkitDomain.ModelListReq{
modelkitRes, err := h.modelkit.ModelList(c.Request().Context(), &modelkitDomain.ModelListReq{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

应该检查modelkit.ModelList方法返回的错误，并进行适当的处理。

Suggested change
modelkitRes, err := h.modelkit.ModelList(c.Request().Context(), &modelkitDomain.ModelListReq{
\tmodelkitRes, err := h.modelkit.ModelList(c.Request().Context(), &modelkitDomain.ModelListReq{
\t\tProvider: string(req.Provider),
\t\tType: string(req.Type),
\t\tBaseURL: req.BaseURL,
\t\tAPIKey: req.APIKey,
\t})
\tif err != nil {
\t\t// handle error appropriately
\t\th.logger.Error("Failed to get provider model list", "error", err)
\t\treturn err
\t}

Provider: string(req.Provider),
Type: string(req.Type),
BaseURL: req.BaseURL,
Expand Down
2 changes: 1 addition & 1 deletion ui/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"@mui/icons-material": "^6.4.12",
"@mui/lab": "6.0.0-beta.19",
"@mui/material": "^6.4.12",
"@yokowu/modelkit-ui": "2.0.6",
"@yokowu/modelkit-ui": "2.3.1",
"@tailwindcss/vite": "^4.1.12",
"ahooks": "^3.8.4",
"axios": "^1.9.0",
Expand Down
16 changes: 8 additions & 8 deletions ui/pnpm-lock.yaml
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading