mega cleaning

co-demos · Feb 21, 2019 · 7f7abb8 · 7f7abb8
1 parent d0dddfa
commit 7f7abb8
Show file tree

Hide file tree

Showing 38 changed files with 427 additions and 1,686 deletions.
diff --git a/GUIDELINES_DEV.md b/GUIDELINES_DEV.md
@@ -0,0 +1,37 @@
+# GUIDELINES FOR DEVELOPMENT
+
+This project is prone to be developped by several developpers, so we agreed on some basic rules...
+
+## GENERAL 
+
+- check out our "project" board to have an idea about the priorities we collectively decided to work on
+- "see something, say something" : open new issues when you see problems to solve
+- document your own issues with everything you have : images, references, snippets, etc... 
+- we update the `README.md` and the `GUIDELINES_DEV.md` directly on github
+- for the markdown files try to [follow those advices](http://www.cirosantilli.com/markdown-style-guide/#dollar-signs-in-shell-code)
+
+## PYTHON 
+
+- comment your code
+- 
+
+## SWAGGER
+
+- ...
+
+
+## GIT MANAGEMENT
+
+We are following this git principles : [link to blog](https://guillim.github.io/git/2018/04/24/Git-workflow.html). In brief : 
+- there are 2 main branches : `master` and `develop`
+- new features must be code reviewed and then (if accepted) pulled into the branch `develop`
+- once every new feature is accepted and merged into `develop`, `develop` must be merged into `master` to constitute a new release
+
+please follow those rules when developping new features : 
+- locally : create a new branch to work on your own feature(s)
+- push your branch to the remote repo (github repo) : 
+> 
+  git push origin <your_branch>
+- ask for a pull request on Github from `your_branch` to `develop`
+- someone else must be assigned as a code reviewer, whose in charge to accept your code changes and merge
+- once accepted / reviewed delete `your_branch` from repo to keep the branches clean
diff --git a/README.md b/README.md
@@ -1,16 +1,25 @@
-<h1 align=center> AUTH SERVICE </h1>
+<div align=center> 
+	<h1>OPEN AUTH MICROSERVICE </h1>
+	<h3>aka "TOKTOK"</h3>
+</div>
 
 
 -------
 ## PRESENTATION
 
-#### An REST API for an authentication microservice 
+#### An REST API for an open source user manager and authentication microservice
 
 - this auth server was extracted / insulated / adapted from solidata_backend
+- compatible with TADATA! sofware suite ([ApiViz](https://github.com/entrepreneur-interet-general/CIS-front) / [Solidata_frontend](https://github.com/entrepreneur-interet-general/solidata_frontend) / [OpenScraper](https://github.com/entrepreneur-interet-general/OpenScraper) )
 
-compatible with TADATA! sofware suite ([ApiViz](https://github.com/entrepreneur-interet-general/CIS-front) / [Solidata_frontend](https://github.com/entrepreneur-interet-general/solidata_frontend) / [OpenScraper](https://github.com/entrepreneur-interet-general/OpenScraper) )
 
 
+--------
+
+## DEVELOPPERS
+
+- Please check out our **[guidelines](./GUIDELINES_DEV.md)** first
+- Check also the **[`prod_snippets`](./prod_snippets)** if you encounter problems while installing locally or setting your server : [install mongodb](./prod_snippets/prod_mongodb.md), [set up supervisor](./prod_snippets/prod_supervisor.md), [set up git](./prod_snippets/prod_git.md), [set up nginx](./prod_snippets/prod_nginx.md), [set up ubuntu](./prod_snippets/prod_ubuntu.md)...
 
 -------
 
@@ -25,11 +34,12 @@ compatible with TADATA! sofware suite ([ApiViz](https://github.com/entrepreneur-
 ## TECHNICAL POINTS
 
 #### Tech stack
-- _Language_  : **[Python 3.6](https://www.python.org/)**... yes ... I know ... Python again ... hmmm ... gnnn ... don't judge me ?!
+- _Language_  : **[Python 3.6](https://www.python.org/)**... praise be...
 - _Framework_ : **[Flask](http://flask.pocoo.org/)**... minimalistic Python framework
-- _API_       : **[Flask-RestPlus](http://flask-restplus.readthedocs.io/en/stable/)**... Swagger documentation integrated
+- _API_       : **[Flask-RestPlus](http://flask-restplus.readthedocs.io/en/stable/)**... Swagger documentation integrated, praise be noirbizarre...
 - _Security_  : **[Flask-JWT-extended](https://flask-jwt-extended.readthedocs.io/en/latest/)**... wrapper JWT for Flask
 - _Emailing_  : **[Flask-email](https://pythonhosted.org/Flask-Mail/)**... templating, sending, etc...
+- and more...
 
 #### Features :
 
@@ -84,7 +94,7 @@ compatible with TADATA! sofware suite ([ApiViz](https://github.com/entrepreneur-
 - run the app in `default` mode (without possibility to send email) : 
 
 	```bash
-	$ python appserver.py
+	python appserver.py
 	``` 
 
 - test the following urls 
@@ -114,24 +124,24 @@ In practice :
 - you can run the app in dev mode (with possibility to send email) : 
 
 	```bash
-	$ python appserver.py --mode=dev_email
+	python appserver.py --mode=dev_email
 	``` 
 
 - you can choose to deactivate the integrated RSA decryption in the `login` and `register` endpoints
 
 	```bash
-	$ python appserver.py --salt=no
+	python appserver.py --salt=no
 	``` 
 
 - you can choose to deactivate the check for a anonymous JWT in the `login` and `register` endpoints
 
 	```bash
-	$ python appserver.py --anojwt=no
+	python appserver.py --anojwt=no
 	``` 
 
 - you can add up those options in the command line
 	```bash
-	$ python appserver.py --anojwt=no --salt=yes --mode=dev_email
+	python appserver.py --anojwt=no --salt=yes --mode=dev_email
 	``` 
 
 ### _PRODUCTION_
@@ -179,8 +189,8 @@ In practice :
 
 - run the app in production mode (with possibility to send email) : 
 
-	```bash
-	$ python appserver.py --mode=production
+	```bash 
+	python appserver.py --mode=production
 	``` 
 
 ------

diff --git a/auth_api/_choices/__init__.py b/auth_api/_choices/__init__.py
@@ -5,5 +5,5 @@
 from ._choices_user import *
 from ._choices_docs import *
 from ._choices_updates import *
-from ._choices_mapping import *
+# from ._choices_mapping import *
 from ._choices_f_types import * 
diff --git a/auth_api/_choices/_choices_docs.py b/auth_api/_choices/_choices_docs.py
@@ -15,19 +15,30 @@
 
 doc_src_type_list 	= ["api","xls","xlsx","xml","csv"]
 
-doc_type_list		= ["usr","prj","dmt","dmf","dsi","dsr","rec","dso","tag","lic"]
+doc_type_list		= [
+	"usr",
+	# "prj",
+	# "dmt",
+	# "dmf",
+	# "dsi",
+	# "dsr",
+	# "rec",
+	# "dso",
+	# "tag",
+	"lic"
+]
 
 doc_type_dict		= {
 
 	"usr" : "user",
-	"prj" : "project",
-	"dmt" : "datamodel_template",
-	"dmf" : "datamodel_field",
-	"dsi" : "dataset_input",
-	"dsr" : "dataset_raw",
-	"rec" : "recipe",
-	"dso" : "dataset_output",
-	"tag" : "tag",
+	# "prj" : "project",
+	# "dmt" : "datamodel_template",
+	# "dmf" : "datamodel_field",
+	# "dsi" : "dataset_input",
+	# "dsr" : "dataset_raw",
+	# "rec" : "recipe",
+	# "dso" : "dataset_output",
+	# "tag" : "tag",
 	"lic" : "licence",
 
 }
diff --git a/auth_api/_choices/_choices_mapping.py b/auth_api/_choices/_choices_mapping.py
diff --git a/auth_api/_core/queries_db/__init__.py b/auth_api/_core/queries_db/__init__.py
@@ -21,39 +21,16 @@
 
 ### declaring collections as app variables
 
-# mongo_tags 									= mongo.db[ app.config["MONGO_COLL_TAGS"] ]
-mongo_users 								= mongo.db[ app.config["MONGO_COLL_USERS"] ]
-# mongo_projects 							= mongo.db[ app.config["MONGO_COLL_PROJECTS"] ]
-# mongo_datamodels_templates 	= mongo.db[ app.config["MONGO_COLL_DATAMODELS_TEMPLATES"] ]
-# mongo_datamodels_fields 		= mongo.db[ app.config["MONGO_COLL_DATAMODELS_FIELDS"] ]
-# mongo_connectors	 				= mongo.db[ app.config["MONGO_COLL_CONNECTORS"] ]
-# mongo_datasets_inputs 			= mongo.db[ app.config["MONGO_COLL_DATASETS_INPUTS"] ]
-# mongo_datasets_raws 				= mongo.db[ app.config["MONGO_COLL_DATASETS_RAWS"] ]
-# mongo_recipes 							= mongo.db[ app.config["MONGO_COLL_RECIPES"] ]
-# mongo_corr_dicts 					= mongo.db[ app.config["MONGO_COLL_CORR_DICTS"] ]
-# mongo_datasets_outputs 			= mongo.db[ app.config["MONGO_COLL_DATASETS_OUTPUTS"] ]
-# mongo_datasets_outputs_doc 	= mongo.db[ app.config["MONGO_COLL_DATASETS_OUTPUTS_DOC"] ]
-
-# mongo_licences 							= mongo.db[ app.config["MONGO_COLL_LICENCES"] ]
-mongo_jwt_blacklist 				= mongo.db[ app.config["MONGO_COLL_JWT_BLACKLIST"] ]
+mongo_users 				= mongo.db[ app.config["MONGO_COLL_USERS"] ]
+mongo_jwt_blacklist = mongo.db[ app.config["MONGO_COLL_JWT_BLACKLIST"] ]
 
 
 log.debug(">>> _core.queries_db.__init__.py / INDEXING COLLECTIONS ... " )
 
 ### drop previous text indexes (only one text index per collection)
 main_text_fields_to_drop = '$**_text'
 try :
-	# mongo_tags.drop_index(main_text_fields_to_drop)
 	mongo_users.drop_index(main_text_fields_to_drop)
-	# mongo_projects.drop_index(main_text_fields_to_drop)
-	# mongo_datamodels_templates.drop_index(main_text_fields_to_drop)
-	# mongo_datamodels_fields.drop_index(main_text_fields_to_drop)
-	# mongo_datasets_inputs.drop_index(main_text_fields_to_drop)
-	# mongo_datasets_raws.drop_index(main_text_fields_to_drop)
-	# mongo_recipes.drop_index(main_text_fields_to_drop)
-	# mongo_datasets_outputs.drop_index(main_text_fields_to_drop)
-	# mongo_datasets_outputs_doc.drop_index(main_text_fields_to_drop)
-	# mongo_licences.drop_index(main_text_fields_to_drop)
 	mongo_jwt_blacklist.drop_index(main_text_fields_to_drop)
 except:
 	pass
@@ -78,57 +55,19 @@
 	("data_raw.f_code"		,"text") ,
 ]
 
-# log.debug(">>> _core.queries_db.__init__.py / INDEXING COLLECTIONS : all fields... " )
-# mongo_datasets_inputs.create_index(				all_text_fields_to_index, name=all_text_fields_index_name) 	### index all fields
-# mongo_datasets_raws.create_index(					all_text_fields_to_index, name=all_text_fields_index_name)	### index all fields
-# mongo_datasets_outputs.create_index(			all_text_fields_to_index, name=all_text_fields_index_name) 	### index all fields
-# mongo_datasets_outputs_doc.create_index(	all_text_fields_to_index, name=all_text_fields_index_name) 	### index all fields
 
 log.debug(">>> _core.queries_db.__init__.py / INDEXING COLLECTIONS : main fields... " )
-mongo_users.create_index(									main_text_fields_to_index, name=main_text_fields_index_name)
-# mongo_tags.create_index(									main_text_fields_to_index, name=main_text_fields_index_name)
-# mongo_projects.create_index(							main_text_fields_to_index, name=main_text_fields_index_name)
-# mongo_datamodels_templates.create_index(	main_text_fields_to_index, name=main_text_fields_index_name)
-# mongo_datamodels_fields.create_index(			main_text_fields_to_index, name=main_text_fields_index_name)
-# mongo_recipes.create_index(								main_text_fields_to_index, name=main_text_fields_index_name)
-
-# mongo_licences.create_index(							main_text_fields_to_index, name=main_text_fields_index_name)
-mongo_jwt_blacklist.create_index(					main_text_fields_to_index, name=main_text_fields_index_name)
+mongo_users.create_index(					main_text_fields_to_index, name=main_text_fields_index_name)
+mongo_jwt_blacklist.create_index(	main_text_fields_to_index, name=main_text_fields_index_name)
 
 
 db_dict = {
-					# "mongo_tags"									: mongo_tags,
-					"mongo_users"									: mongo_users,
-					# "mongo_projects"							: mongo_projects,
-					# "mongo_datamodels_templates"	: mongo_datamodels_templates,
-					# "mongo_datamodels_fields"			: mongo_datamodels_fields,
-					# "mongo_connectors"					: mongo_connectors,
-					# "mongo_datasets_inputs"				: mongo_datasets_inputs,
-					# "mongo_datasets_raws"					: mongo_datasets_raws,
-					# "mongo_recipes"								: mongo_recipes,
-					# "mongo_corr_dicts"					: mongo_corr_dicts,
-
-					# "mongo_datasets_outputs"			: mongo_datasets_outputs,
-					# "mongo_datasets_outputs_doc"	: mongo_datasets_outputs_doc,
-
-					# "mongo_licences"							: mongo_licences,
-					"mongo_jwt_blacklist"					: mongo_jwt_blacklist,
+					"mongo_users"					: mongo_users,
+					"mongo_jwt_blacklist"	: mongo_jwt_blacklist,
 			}
 db_dict_by_type = {
-					# "tag"						: mongo_tags,
-					"usr"						: mongo_users,
-					# "prj"						: mongo_projects,
-					# "dmt"						: mongo_datamodels_templates,
-					# "dmf"						: mongo_datamodels_fields,
-					# "dsi"						: mongo_datasets_inputs,
-					# "dsr"						: mongo_datasets_raws,
-					# "rec"						: mongo_recipes,
-
-					# "dso"						: mongo_datasets_outputs,
-					# "dso_doc"				: mongo_datasets_outputs_doc,
-
-					# "lic"						: mongo_licences,
-					"jwt_blacklist"	: mongo_jwt_blacklist,
+					"usr"									: mongo_users,
+					"jwt_blacklist"				: mongo_jwt_blacklist,
 			}
 
 def select_collection(coll_name):
@@ -195,8 +134,8 @@ def get_results():
 from .query_list import *
 from .query_delete import *
 from .query_update import *
-# from .query_solidify import *
 from .query_insert_doc import *
+# from .query_solidify import *
 # from .query_build_dso import *
 
 print()

diff --git a/auth_api/_core/queries_db/query_delete.py b/auth_api/_core/queries_db/query_delete.py
@@ -33,19 +33,19 @@ def Query_db_delete (
 	# marshaller = Marshaller(ns, models)
 
 	### default values
-	db_collection				= db_dict_by_type[document_type]
-	document_type_full 	= doc_type_dict[document_type]
-	user_id = user_oid	= None
-	user_role						= "anonymous"
-	doc_oid							= ObjectId(doc_id)
-	document_out				= None
-	response_code				= 401
+	db_collection						= db_dict_by_type[document_type]
+	document_type_full 			= doc_type_dict[document_type]
+	user_id = user_oid			= None
+	user_role								= "anonymous"
+	doc_oid									= ObjectId(doc_id)
+	document_out						= None
+	response_code						= 401
 	user_allowed_to_delete 	= False
-	message 						= "dear user, you don't have the credentials to delete this {} with this oid : {}".format(document_type_full, doc_id) 
+	message 								= "dear user, you don't have the credentials to delete this {} with this oid : {}".format(document_type_full, doc_id) 
 
 	if claims or claims!={}  :
 		user_role 		= claims["auth"]["role"]
-		user_id	 		= claims["_id"] ### get the oid as str
+		user_id	 			= claims["_id"] ### get the oid as str
 		if user_role != "anonymous" : 
 			user_oid 		= ObjectId(user_id)
 			log.debug("user_oid : %s", user_oid )
@@ -56,14 +56,14 @@ def Query_db_delete (
 		# log.debug( "document : \n%s", pformat(document) )
 	else :
 		response_code	= 400
-		document		= None
+		document			= None
 
 	### sum up all query arguments
 	query_resume = {
-		"document_type"		: document_type,	
-		"doc_id" 			: doc_id,
-		"user_id" 			: user_id,
-		"user_role"			: user_role,
+		"document_type"			: document_type,	
+		"doc_id" 						: doc_id,
+		"user_id" 					: user_id,
+		"user_role"					: user_role,
 		"is_member_of_team" : False
 	}
 

diff --git a/auth_api/_core/queries_db/query_doc.py b/auth_api/_core/queries_db/query_doc.py
@@ -8,7 +8,7 @@
 import random
 
 import pandas as pd
-import 	numpy as np
+import numpy as np
 from pandas.io.json import json_normalize
 
 from log_config import log, pformat
@@ -93,11 +93,11 @@ def get_dso_docs(doc_oid, query_args) :
 
 	dso_doc_collection	= db_dict_by_type['dso_doc']
 
-	query = build_first_term_query(doc_oid, query_args)
+	query 	= build_first_term_query(doc_oid, query_args)
 	log.debug('query : \n%s', pformat(query) )  
 
 	# results = dso_doc_collection.find({'oid_dso' : doc_oid })
-	cursor = dso_doc_collection.find(query)
+	cursor 	= dso_doc_collection.find(query)
 
 	results = list(cursor)