Add ability to call arbitrary database function with payload

[ivarref]

Hi, and thanks for a fine library.

Currently it does not seem easy to call an arbitrary database function on the given JSON payload.

I would like the following possibility:

"/user"             {:post #vase/transact {:name  :accounts.v1/user-create
                                                  :db-fn :my-great-fn
                                                  :properties [:db/id
                                                               :user/userId
                                                               :user/email]}}}

and that this should expand to:

[[:my-great-fn {:user/userId 42 :user/email "hello@example.com"}]]

It is then up to :my-great-fn to generate / expand to the proper data as needed.

I think this is preferable over using custom interceptors with request db or request conn as this will preserve ACID properties.

Would this be possible and desirable to add to Vase?

Regards

[mtnygard]

Your request makes a lot of sense.

There's one wrinkle to consider, which is that the payload can contain multiple entities. It would be necessary for every entity to go through the same db fn, so the transaction might look more like this:

[[:my-great-fn {:user/userId 42 :user/email "hello@example.com"}]
 [:my-great-fn {:user/userId 86 :user/email "max.smart@control.gov"}]
 [:my-great-fn {:user/userId 99 :user/email "agent99@control.gov"}]]

[mtnygard]

Also, if your need is urgent, it is possible for you to supply a new literal of your own to invoke db fns. The steps would be:

1. Pick a tag
2. Add it to your service's data_readers.clj file, mapped to a reader function you write.
3. Copy and modify com.cognitect.vase.literals/transact plus its related record and action.

Vase is open to extension precisely so you can define your own tags and use them in a descriptor file.

[ohpauleez]

Hi @ivarref - Thanks for using Vase and reaching out with your request!

Originally, :db-op was a multimethod, open for extension but on release, we decided to make it a closed dispatch map, as you can see here: https://github.com/cognitect-labs/vase/blob/master/src/com/cognitect/vase/actions.clj#L84

At the time, the rationale was that if a user wanted more specific control over DB interactions, they were better off creating a new action literal for their use-case, or creating the transaction data within a #vase/intercept call. This ensured that #vase/transact stayed as simple and predictable as possible and directed users to the other vase extension mechanisms.

All of that said, the Vase team will talk about opening :db/op back up.

[ivarref]

Update

Whoops. Seems the error below was a REPL-only thing. Loading works fine using the default Vase server.

Comment
If there is desire to keep #vase/transact as simple as possible, maybe a separate #vase/transact-fn could be added. I've now (locally) added a transact-fn reader which copies a bunch from the original source code. It works with arrays as well.

Thanks for the input.

Original comment

Hi @mtnygard and @ohpauleez

and thank you for your input.

I've given @mtnygard 's suggestion a go with a custom reader, but encountered a new problem.

Given the following edn file:

{:activated-apis [:accounts/v1]
 :datomic-uri    "datomic:mem://bergen"
 :descriptor
                 {:vase/norms {:accounts/item {
                                               :vase.norm/txes [[{:db/id    #db/id [:db.part/user]
                                                                  :db/ident :my-great-fn
                                                                  :db/fn    #db/fn
                                                                                {:lang   :clojure
                                                                                 :params [db m]
                                                                                 :code   (do (println m) m)}}
                                                                 {:db/id          #db/id[:db.part/db]
                                                                  :db/ident       :item/name
                                                                  :db/valueType   :db.type/string
                                                                  :db/unique      :db.unique/identity
                                                                  :db/cardinality :db.cardinality/one
                                                                  :db/doc         "The name of an item"}]]}}
                  :vase/specs {}
                  :vase/apis  {:accounts/v1
                               {:vase.api/routes {}}}}}

gives the following error in the REPL:

CompilerException java.lang.RuntimeException: Can't embed object in code, maybe print-dup not defined: clojure.lang.Delay@48459a27, compiling:(/home/ire/code/learn/your-first-api/resources/your-first-api_service.edn:1:1) 

I'm not sure exactly what's going wrong. This style (the #db/fn part) works fine inside a clj file. Do you have any suggestions or clues about how to fix this?

Regards

[ivarref]

I've added a sample project that adds this functionality as suggested by @mtnygard if others are interested:

https://gitlab.nsd.uib.no/ire/vase-transact-fn

The edn file looks like this:

{:activated-apis
 [:accounts/v1]
 :datomic-uri
 "datomic:mem://bergen"
 :descriptor
 {:vase/norms
  {:accounts/item
   {:vase.norm/txes [[{:db/id    #db/id [:db.part/user]
                       :db/ident :my-great-fn
                       :db/fn    #db/fn
                                     {:lang   :clojure
                                      :params [db m]
                                      :code   (do
                                                (println "hello from DB land:: ==>>" m)
                                                [m])}}
                      {:db/id          #db/id[:db.part/db]
                       :db/ident       :item/name
                       :db/valueType   :db.type/string
                       :db/unique      :db.unique/identity
                       :db/cardinality :db.cardinality/one
                       :db/doc         "The name of an item"}]]}}
  :vase/specs
  {}
  :vase/apis
  {:accounts/v1
   {:vase.api/routes {"/hello" {:post [#nsd/transact-fn {:name       :accounts.v1/item-create-with-function
                                                         :db-fn      :my-great-fn
                                                         :properties [:item/name]}]}}}}}}

[ivarref]

I could probably write a pull request adding #vase/transact-fn with this functionality if that would be of interest? @mtnygard @ohpauleez

[ohpauleez]

It has always been our goal to support and foster a community around the creation/sharing of extensions and action literals for Vase. It's unlikely that we'll add anymore action literals to the core of Vase (there are a few we've been considering, but haven't committed to), but we are going to open up :db-op again as a multimethod

Thanks for opening this issue, starting the conversation back up, and creating an action literal that solves the problem! It is great to see the community jump in!