feat: OpenAPI integration #984

NexVeridian · 2024-11-13T04:06:49Z

related #855

cargo check -F all_openapi
cargo nextest run --test-threads 1 -F all_openapi

docker run -d \
  -e POSTGRES_DB=postgres_test \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=postgres \
  -p 5432:5432 \
  --health-cmd="pg_isready" \
  --health-interval=10s \
  --health-timeout=5s \
  --health-retries=5 \
  postgres

export DATABASE_URL="postgresql://postgres:postgres@localhost:5432/postgres_test"

cargo nextest run --all-features --workspace --exclude loco-gen --exclude loco --test-threads 1

TODO:

cc @DenuxPlays

DenuxPlays · 2024-11-13T08:24:55Z

Looks like a good start.

I think the yaml also should contain a few other things:

openapi and ui endpoints (I am not sure if this is want you mean in your description)
whether it serves json, yaml or both (Perhaps this can be controlled by using features to exclude unnecessary dependencies)

cfg for tests

DenuxPlays · 2024-11-26T15:17:44Z

Hey @NexVeridian

Can I help you finish this pr?

NexVeridian · 2024-11-27T04:37:54Z

@DenuxPlays yeah of course

DenuxPlays · 2024-11-27T07:09:48Z

How can I help?

Also just one Note to your pr description.
I wouldn't wrap the utoipa path macro.
Its working very well just how it is and wrapping it would make upgrading and maintaining very difficult

NexVeridian · 2024-11-27T08:24:27Z

Also just one Note to your pr description. I wouldn't wrap the utoipa path macro. Its working very well just how it is and wrapping it would make upgrading and maintaining very difficult

thanks!

this would be nice if possible:

impl Routes {
/// .add_openapi(routes!(get_action, post_action))
pub fn add_openapi(mut self, method: UtoipaMethodRouter<AppContext>) -> Self {}
}

any of the unchecked ones in the pr description would be great:
maybe cargo loco generate controller --openapi and docs first

NexVeridian · 2025-01-29T20:00:21Z

src/auth/openapi.rs

+static JWT_LOCATION: OnceLock<Option<JWTLocation>> = OnceLock::new();
+
+#[must_use]
+pub fn get_jwt_location_from_ctx(ctx: &AppContext) -> JWTLocation {
+    ctx.config
+        .auth
+        .as_ref()
+        .and_then(|auth| auth.jwt.as_ref())
+        .and_then(|jwt| jwt.location.as_ref())
+        .unwrap_or(&JWTLocation::Bearer)
+        .clone()
+}
+
+pub fn set_jwt_location_ctx(ctx: &AppContext) {
+    set_jwt_location(get_jwt_location_from_ctx(ctx));
+}
+
+pub fn set_jwt_location(jwt_location: JWTLocation) -> &'static Option<JWTLocation> {
+    JWT_LOCATION.get_or_init(|| Some(jwt_location))
+}
+
+fn get_jwt_location() -> Option<&'static JWTLocation> {
+    JWT_LOCATION.get().unwrap_or(&None).as_ref()
+}


seems like this might be flaky in cargo test because the global state isn't reset between test runs
tests are good in cargo nextest and should be fine for users

NexVeridian · 2025-01-29T20:15:17Z

Just one small thing I noticed. Structs like these: https://github.com/NexVeridian/loco/blob/d72c46032a93a676004ade03266f76e6268b26bc/src/controller/views/pagination.rs would need to implement/derive the ToSchema trait right?

if it's used in the utoipa::path then yeah, I'm not sure if they would be, I haven't used those before

DenuxPlays · 2025-01-29T23:06:42Z

Just one small thing I noticed. Structs like these: https://github.com/NexVeridian/loco/blob/d72c46032a93a676004ade03266f76e6268b26bc/src/controller/views/pagination.rs would need to implement/derive the ToSchema trait right?

if it's used in the utoipa::path then yeah, I'm not sure if they would be, I haven't used those before

They would to provide Basic paginagtion metadata

SorenEdwards · 2025-01-30T17:01:25Z

@DenuxPlays @NexVeridian

I had to do something similar recently and If I understand correctly you would need to generate something like this. I had a problem where the generated openapi schema didn't see that X struct was a query parameter.

#[derive(Debug, Deserialize, Serialize,ToSchema,IntoParams)]
#[into_params(parameter_in=Query)]
pub struct PagerMeta {
    #[serde(rename(serialize = "page"))]
    pub page: u64,
    #[serde(rename(serialize = "page_size"))]
    pub page_size: u64,
    #[serde(rename(serialize = "total_pages"))]
    pub total_pages: u64,
}

…ms)" This reverts commit 9a74966. Revert "some derive(ToSchema) and derive(IntoParams)" This reverts commit fb9e237.

NexVeridian · 2025-01-30T17:46:37Z

5f4c63f still has some errors that I don't know how to fix, one of you two should take a look maybe

DenuxPlays · 2025-01-30T20:07:28Z

I think the Solution would be something like cfg_of and we have to define the struct two times.

SorenEdwards · 2025-01-31T01:14:32Z

I think this would work, I tested it on your branch. The cfg_attr for the trait bounds is pretty ugly so I think @DenuxPlays has a point with using cfg_of and having two structs.

Pragnation.rs


#[cfg_attr(
    any(
        feature = "openapi_swagger",
        feature = "openapi_redoc",
        feature = "openapi_scalar"
    ),
    derive(utoipa::ToSchema)
)]
#[derive(Debug, Deserialize, Serialize)]
pub struct Pager<
#[cfg(any(
    feature = "openapi_swagger",
    feature = "openapi_redoc",
    feature = "openapi_scalar"
))] T: utoipa::ToSchema,
#[cfg(not(any(
    feature = "openapi_swagger",
    feature = "openapi_redoc",
    feature = "openapi_scalar"
),))] T,
> {
    #[serde(rename(serialize = "results"))]
    pub results: T,

    #[serde(rename(serialize = "pagination"))]
    pub info: PagerMeta,
}


#[cfg_attr(
    any(
        feature = "openapi_swagger",
        feature = "openapi_redoc",
        feature = "openapi_scalar"
    ),
    derive(utoipa::ToSchema)
)]
#[derive(Debug, Deserialize, Serialize)]
pub struct PagerMeta {
    #[serde(rename(serialize = "page"))]
    pub page: u64,
    #[serde(rename(serialize = "page_size"))]
    pub page_size: u64,
    #[serde(rename(serialize = "total_pages"))]
    pub total_pages: u64,
    #[serde(rename(serialize = "total_items"))]
    pub total_items: u64,
}

impl<
#[cfg(any(
    feature = "openapi_swagger",
    feature = "openapi_redoc",
    feature = "openapi_scalar"
))] T: utoipa::ToSchema,
#[cfg(not(any(
    feature = "openapi_swagger",
    feature = "openapi_redoc",
    feature = "openapi_scalar"
),))] T,
> Pager<T> {
    #[must_use]
    pub const fn new(results: T, meta: PagerMeta) -> Self {
        Self {
            results,
            info: meta,
        }
    }
}

pragnate/mod.rs

#[cfg_attr(
    any(
        feature = "openapi_swagger",
        feature = "openapi_redoc",
        feature = "openapi_scalar"
    ),
    derive(utoipa::IntoParams)
)]
#[derive(Debug, Deserialize, Serialize)]
pub struct PaginationQuery {
    #[serde(
        default = "default_page_size",
        rename = "page_size",
        deserialize_with = "deserialize_pagination_filter"
    )]
    pub page_size: u64,
    #[serde(
        default = "default_page",
        rename = "page",
        deserialize_with = "deserialize_pagination_filter"
    )]
    pub page: u64,
}

DenuxPlays · 2025-01-31T06:32:36Z

I had something like the following in mind.
Also this isn't tested I do not have time for this atm.

pagination.rs

use cfg_if::cfg_if;

cfg_if! {
    if #[cfg(any(
        feature = "openapi_swagger",
        feature = "openapi_redoc",
        feature = "openapi_scalar"
    ))] {
        #[derive(Debug, serde::Deserialize, serde::Serialize, utoipa::ToSchema)]
        pub struct Pager<T: utoipa::ToSchema> {
            #[serde(rename(serialize = "results"))]
            pub results: T,

            #[serde(rename(serialize = "pagination"))]
            pub info: PagerMeta,
        }
    } else {
        #[derive(Debug, serde::Deserialize, serde::Serialize)]
        pub struct Pager<T> {
            #[serde(rename(serialize = "results"))]
            pub results: T,

            #[serde(rename(serialize = "pagination"))]
            pub info: PagerMeta,
        }
    }
}

cfg_if! {
    if #[cfg(any(
        feature = "openapi_swagger",
        feature = "openapi_redoc",
        feature = "openapi_scalar"
    ))] {
        #[derive(Debug, serde::Deserialize, serde::Serialize, utoipa::ToSchema)]
        pub struct PagerMeta {
            #[serde(rename(serialize = "page"))]
            pub page: u64,
            #[serde(rename(serialize = "page_size"))]
            pub page_size: u64,
            #[serde(rename(serialize = "total_pages"))]
            pub total_pages: u64,
            #[serde(rename(serialize = "total_items"))]
            pub total_items: u64,
        }
    } else {
        #[derive(Debug, serde::Deserialize, serde::Serialize)]
        pub struct PagerMeta {
            #[serde(rename(serialize = "page"))]
            pub page: u64,
            #[serde(rename(serialize = "page_size"))]
            pub page_size: u64,
            #[serde(rename(serialize = "total_pages"))]
            pub total_pages: u64,
            #[serde(rename(serialize = "total_items"))]
            pub total_items: u64,
        }
    }
}

cfg_if! {
    if #[cfg(any(
        feature = "openapi_swagger",
        feature = "openapi_redoc",
        feature = "openapi_scalar"
    ))] {
        impl<T: utoipa::ToSchema> Pager<T> {
            #[must_use]
            pub const fn new(results: T, meta: PagerMeta) -> Self {
                Self {
                    results,
                    info: meta,
                }
            }
        }
    } else {
        impl<T> Pager<T> {
            #[must_use]
            pub const fn new(results: T, meta: PagerMeta) -> Self {
                Self {
                    results,
                    info: meta,
                }
            }
        }
    }
}

I think that @SorenEdwards implementation for the query is good but we should also add ToSchema to it.
It may be useful for some use cases.
So this is what I would do:

#[cfg_attr(
    any(
        feature = "openapi_swagger",
        feature = "openapi_redoc",
        feature = "openapi_scalar"
    ),
    derive(utoipa::IntoParams, utoipa::ToSchema)
)]
#[derive(Debug, serde::Deserialize, serde::Serialize)]
pub struct PaginationQuery {
    #[serde(
        default = "default_page_size",
        rename = "page_size",
        deserialize_with = "deserialize_pagination_filter"
    )]
    pub page_size: u64,
    #[serde(
        default = "default_page",
        rename = "page",
        deserialize_with = "deserialize_pagination_filter"
    )]
    pub page: u64,
}

kaplanelad · 2025-02-03T13:33:23Z

After reviewing this implementation again, I feel we should consider moving it into an initializer maintained in a separate repository. For example:

Could you outline the changes needed in Loco to allow managing OpenAPI as an initializer?

DenuxPlays · 2025-02-03T13:41:27Z

Note:

I don't think thats possible.
we need loco to implement certain traits for this to work.
examples are the above mentioned Pager and PagerMeta types

kaplanelad · 2025-02-03T18:04:49Z

Note:

I don't think thats possible. we need loco to implement certain traits for this to work. examples are the above mentioned Pager and PagerMeta types

Everything is possible. When initializing is public create, you need to load Loco (see the other implementation), which allows us to make the dependent struct public.
I want to count the changes that we need to make in the framework to make it happen

SorenEdwards · 2025-02-04T03:58:12Z

@kaplanelad @DenuxPlays @NexVeridian I am actually using it in the initializer right now on my project/work. If you are interested I can add a post about it.

I think the real upside of adding a feature in loco would be the code generation aspect as I do a lot of this by hand currently and it can get pretty tedious. If you have any suggestions I would love to help out making the initializer approach work.

That being said seaorms --extra-model-derive could fix ToSchema not being added to the generated db entities.

DenuxPlays · 2025-02-04T05:47:04Z

Note:

I don't think thats possible. we need loco to implement certain traits for this to work. examples are the above mentioned Pager and PagerMeta types

Everything is possible. When initializing is public create, you need to load Loco (see the other implementation), which allows us to make the dependent struct public.

I want to count the changes that we need to make in the framework to make it happen

I am not sure what you mean.
AFAIK traits can only be implemented on internal, your own types or if you created the trait.

So unless you copy the structs it isnt possible right?

DenuxPlays · 2025-02-04T05:52:10Z

@kaplanelad @DenuxPlays @NexVeridian I am actually using it in the initializer right now on my project/work. If you are interested I can add a post about it.

I think the real upside of adding a feature in loco would be the code generation aspect as I do a lot of this by hand currently and it can get pretty tedious. If you have any suggestions I would love to help out making the initializer approach work.

That being said seaorms --extra-model-derive could fix ToSchema not being added to the generated db entities.

Yeah I also use it as an initializer.
But I don't think that Code Generation is my Main aspect.
Currently I use utoipauto to collect all the paths, schemas etc. But this would actually use the utoipa-Axum Router which is far cleaner then using utoipauto.
Also you do not have to copy types like Pager.

It is just a Smoother experience.

Just a small note you should Never Serve entities over your api.
Just use DTOs/Views for this.
Ofcourse I do Not know your project and it may be not the best Solution for your I just wanted to mention it.

kaplanelad · 2025-02-04T06:37:59Z

@kaplanelad @DenuxPlays @NexVeridian I am actually using it in the initializer right now on my project/work. If you are interested I can add a post about it.

I think the real upside of adding a feature in loco would be the code generation aspect as I do a lot of this by hand currently and it can get pretty tedious. If you have any suggestions I would love to help out making the initializer approach work.

That being said seaorms --extra-model-derive could fix ToSchema not being added to the generated db entities.

Yes, it can be great. can you also share how you implement it with an example repo?

DenuxPlays · 2025-02-04T06:51:09Z

You can take look here:
https://github.com/financrr/financrr-app/blob/backend/feature/sync-bank-data/backend/src/initializers/openapi.rs

NexVeridian force-pushed the OpenAPI-integration branch 4 times, most recently from 9055d99 to 3e106d0 Compare November 22, 2024 09:21

NexVeridian added 17 commits November 23, 2024 04:38

inti

8ed9348

hook initial_openapi_spec

6deca2b

config.yaml and merge hosted doc with router

def3dd5

update existing tests

8db4443

test: OpenAPI config from file

34d10e6

snapshot tests

c887042

fix snapshot path

4ed2668

match title, upload snapshots

2aa3d66

OpenAPI json snapshot test

0a304c1

Another snapshot

8c0fd96

LocoMethodRouter

5f63b08

fix AppContext

1c67cc6

missing cfg for tests

c1e953e

cfg for tests

clippy

74f78cc

openapi.josn and openapi.yaml endpoints for all types

ac0f1ce

SecurityAddon

2082e12

fix cfg flag for import

ea48919

NexVeridian force-pushed the OpenAPI-integration branch from 3e106d0 to ea48919 Compare November 23, 2024 14:13

NexVeridian added 2 commits November 26, 2024 19:50

Merge branch 'upstream' into OpenAPI-integration

c849c73

fix: snapshots and imports

022909d

Merge branch 'master' into OpenAPI-integration

165d162

allow and defualt to None jwt_location

819b45e

NexVeridian force-pushed the OpenAPI-integration branch from 9acc912 to 819b45e Compare January 21, 2025 20:58

NexVeridian added 4 commits January 28, 2025 08:12

Merge branch 'master' into OpenAPI-integration

6c3d1aa

clippy

05d90af

drop demo test

9107d3e

drop generator command for now

0a7df8c

NexVeridian force-pushed the OpenAPI-integration branch from 14e09c1 to 0a7df8c Compare January 28, 2025 16:40

NexVeridian added 4 commits January 28, 2025 08:59

clippy

9f6bac2

clippy fix from main branch

8a0c996

fmt from main branch

0a09a5c

fix doc tests

902f65a

NexVeridian commented Jan 29, 2025

View reviewed changes

some derive(ToSchema) and derive(IntoParams)

fb9e237

NexVeridian added 2 commits January 30, 2025 09:24

more ToSchema

9a74966

Revert "more ToSchema" and "some derive(ToSchema) and derive(IntoPara…

5f4c63f

…ms)" This reverts commit 9a74966. Revert "some derive(ToSchema) and derive(IntoParams)" This reverts commit fb9e237.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: OpenAPI integration #984

feat: OpenAPI integration #984

NexVeridian commented Nov 13, 2024 •

edited

Loading

DenuxPlays commented Nov 13, 2024

DenuxPlays commented Nov 26, 2024

NexVeridian commented Nov 27, 2024

DenuxPlays commented Nov 27, 2024

NexVeridian commented Nov 27, 2024 •

edited

Loading

NexVeridian Jan 29, 2025

NexVeridian commented Jan 29, 2025

DenuxPlays commented Jan 29, 2025

SorenEdwards commented Jan 30, 2025 •

edited

Loading

NexVeridian commented Jan 30, 2025

DenuxPlays commented Jan 30, 2025 •

edited

Loading

SorenEdwards commented Jan 31, 2025

DenuxPlays commented Jan 31, 2025

kaplanelad commented Feb 3, 2025

DenuxPlays commented Feb 3, 2025

kaplanelad commented Feb 3, 2025

SorenEdwards commented Feb 4, 2025

DenuxPlays commented Feb 4, 2025

DenuxPlays commented Feb 4, 2025

kaplanelad commented Feb 4, 2025

DenuxPlays commented Feb 4, 2025

feat: OpenAPI integration #984

Are you sure you want to change the base?

feat: OpenAPI integration #984

Conversation

NexVeridian commented Nov 13, 2024 • edited Loading

TODO:

DenuxPlays commented Nov 13, 2024

DenuxPlays commented Nov 26, 2024

NexVeridian commented Nov 27, 2024

DenuxPlays commented Nov 27, 2024

NexVeridian commented Nov 27, 2024 • edited Loading

NexVeridian Jan 29, 2025

Choose a reason for hiding this comment

NexVeridian commented Jan 29, 2025

DenuxPlays commented Jan 29, 2025

SorenEdwards commented Jan 30, 2025 • edited Loading

NexVeridian commented Jan 30, 2025

DenuxPlays commented Jan 30, 2025 • edited Loading

SorenEdwards commented Jan 31, 2025

Pragnation.rs

pragnate/mod.rs

DenuxPlays commented Jan 31, 2025

kaplanelad commented Feb 3, 2025

DenuxPlays commented Feb 3, 2025

kaplanelad commented Feb 3, 2025

SorenEdwards commented Feb 4, 2025

DenuxPlays commented Feb 4, 2025

DenuxPlays commented Feb 4, 2025

kaplanelad commented Feb 4, 2025

DenuxPlays commented Feb 4, 2025

NexVeridian commented Nov 13, 2024 •

edited

Loading

NexVeridian commented Nov 27, 2024 •

edited

Loading

SorenEdwards commented Jan 30, 2025 •

edited

Loading

DenuxPlays commented Jan 30, 2025 •

edited

Loading