RESTless version

[d-beltran]

Hi everyone,

I already discussed this with @Bud-Macaulay in Vilinus but I was wondering if we could have a version of this tool which works with in-memory data instead of constant communication with a REST API.
In our case we have aiida data but we don't have an aiida API.
I can reshape these data to match what the aiida-explorer expects, but I can not give it an URL.

I suppose this may be a strong change of how the tool works internally so if this is not doable I totally understand, but I would like to know it so I can start making my own makeshift explorer.

Best regards

[eimrek]

hi! just wondering, do you have an AiiDA instance running that you communicate using the python API?

If yes, you could just pip install aiida-core[rest] and run verdi restapi, which will start the API on localhost (see here). Just curious which restrictions prevent this.

[d-beltran]

Hey @eimrek ,

Sorry for the late reply, sometimes I don't get github notifications.

The contexts where AiiDA data is produced and where it is consumed are very separated in my case.

Anyway I also considered running the AiiDA API where data is to be consumed. For sure I need this data in my database, associated to other non-AiiDA data, since I will run queries over it. This would mean duplicating AiiDA data.

[eimrek]

Hey, thanks for the clarification, but still it's not fully clear to me.

If data is generated on machine 1, and to use it on machine 2 (either using the python API or the REST API), you would need to transfer the data and "duplicate it" either way, no? Or maybe i'm misunderstanding.

Could you give some more details?

[d-beltran]

Data in machine 1 would be deleted after transferred. We only need to keep data in machine 2 to make it available.

[eimrek]

Ok, thanks. And on Machine 2, you are importing it to a new AiiDA instance/server, right? So basically you are using AiiDA Python API on Machine 2 to access the data. But why not also start the REST API on machine 2?

[d-beltran]

No, there is no AiiDA instance/server in machine 2. Not even in machine 1 (yet).

So, the whole pipeline is:

machine 0 - This machine is not mine and here is where the actual data is originally generated. Here is where AiiDA may be running and at the end it will export provenance data to a '.aiida' file (so a zipped sqlite). Then this .aiida data is to be sent to machine 1 next to the actual data.

machine 1 - HPC environment where actual data is processed. Here AiiDA data ideally should be updated (not done yet). The .aiida file is finally read. The links and nodes data is mined and saved to a JSON file, so it can be properly uploaded to a mongo database.

machine 2 - Receives the JSON file and uploads its content to the database. No AiiDA at all.

[eimrek]

Thanks for the details. Unfortunately I don't think we have plans to maintain a "custom" API interface. Currently the idea is to fully reflect an underlying AiiDA database/instance, and that is most conveniently done by just accessing the corresponding AiiDA REST API.