918 ability to freeze datasets or version (#941)

* initial naive approach of frozen dataset

* lock works

* use the view instead of just datset db

* all get endpoint now change to use the dataset view that has both dataset and freeze dataset

* refactoring GET public dataset related endpoints

* refactoring every GET endpoint related to dataset

* write a published wrapper to control showing of edit or not

* use wrapper whenever possible to avoid hardcoding criterias

* add more publish wrapper

* wrap proper auth and published check to various action menu

* add lock to file

* fix metadata edit auth and published

* something wrong with tab and file role

* dataset tab condition set

* fix fileRoleType

* fix file tabs

* freeze button works

* frozen button works

* codegen

* list the latest version of published dataset is working

* update freeze endpoint to take versioning into consideration

* enable the cability of frozen and frozen draft

* messy logic

* change pipeline logic to always have both the latest draft and latest published

* add authorization logic

* sorting is working now

* fix the origin id logic

* there is a bug in wrong origin id when second time publish

* fix backend bug

* change version text

* fix frontend logic

* temp map origin_id to id

* fix query with origin id

* fix the logic that a dataset id on the client point of view will never change

* freeze draft action coded in redux

* fix id

* revert change on file role since it's not in the scope of this PR

* wrong method

* separate out dataset and frozen dataset

* remove freeze version chip

* backend rewrite

* write the logic of frozen

* add snackbar

* refactor the latest endpoint

* add a selection

* selection is working now

* connect version change with backend

* separate out freeze dataset

* fix a metadata get bug

* new endpoint and new widgets

* union is working now

* add an endpoint to list and paginate through all the versions

* add to reducer

* switching between versions working

* file and folder listing using the view

* add frozen chip ugly

* update file page

* recursively freeze file and folder

* download file is working

* normal download nested folders are working

* fix bug recursive freeze folder

* download dataset is fixed now

* metadata is working now

* folder path

* fix the current tab error

* update the chip

* create new version now reflected

* update based on the feedbacks

* fix bug in recursively freeze file/folder

* fix bug in switching to file version

* fix bug after renaming versioning is not updating

* backend public dataset reflect freeze dataset

* implementing public related

* fix public file page

* list public dataset version is working now

* fix all the public related bugs I can think of

* unhide visualization

* add models to handle vis for frozen datasets

* add support for visualization and thumnail

* fix mongoview bug

* fix thumbnail bug

* fix frontend bug

* fix thumbnail bug with public dataset

* change the wording

* duplicate dataset details

* 1039 dataset versions show in links (#1057)

* functionality is working, but not the color

* some detail logic not right

* highlight select is working

* snackbar is working now

* paginatio flipping problem

* fix pagination bug

* completely fix pagination

* public dataset

* remove the now chip for current version

* add folder to action text

* wrap style into function; fix bugs

* turn off scrollbutton to avoid tab position change when switching versions

* change the wording to latest; don't show snackbar when it's latest

* fix viewing -999 bug

* 1040 delete given dataset version (#1063)

* fix bug with get/delete a specific version of dataset; frontend delete button is working

* dataset versioning styling

* public dataset versioning styling

* delete thumbnail and vis endpoint considering versioning

* might be a bug deleting thumbnails

* switch the order

* write utility funcitons to help manage delete

* backend delete is configured

* frontend delete wired in correctly

* delete modal color and prompt updated

* add docstrings

* add last modified

* add specific pytest

* add visualization tests to dataset versioning

* add more pytest after deleting the latest dataset

* fix a potential bug when updating dataset

* Versioning documentation (#1044)

* first draft of the versioning documentation

* update documentation

* wording

Author: longshuicy

.flake8

@@ -1,5 +1,5 @@
 [flake8]
 max-line-length = 180
-extend-ignore = E203, E266, E501, W503, E741, E711
+extend-ignore = E203, E266, E501, W503, E741, E711, E712
 exclude = .svn,CVS,.bzr,.hg,.git,__pycache__,venv/*,src/*,.rst,build
 max-complexity=16

backend/app/db/dataset/download.py

@@ -0,0 +1,21 @@
+from app.models.datasets import DatasetDB, DatasetFreezeDB
+from beanie import PydanticObjectId
+from beanie.odm.operators.update.general import Inc
+
+
+async def _increment_data_downloads(dataset_id: str):
+    # if working draft
+    if (
+        dataset := await DatasetDB.find_one(
+            DatasetDB.id == PydanticObjectId(dataset_id)
+        )
+    ) is not None:
+        await dataset.update(Inc({DatasetDB.downloads: 1}))
+
+    # if published version
+    if (
+        dataset := await DatasetFreezeDB.find_one(
+            DatasetFreezeDB.id == PydanticObjectId(dataset_id)
+        )
+    ) is not None:
+        await dataset.update(Inc({DatasetFreezeDB.downloads: 1}))