Mark 3.1.0 (CTFd#1634)

# 3.1.0 / 2020-09-08 **General** - Loosen team password confirmation in team settings to also accept the team captain's password to make it easier to change the team password - Adds the ability to add custom user and team fields for registration/profile settings. - Improve Notifications pubsub events system to use a subscriber per server instead of a subscriber per browser. This should improve the reliability of CTFd at higher load and make it easier to deploy the Notifications system **Admin Panel** - Add a comments functionality for admins to discuss challenges, users, teams, pages - Adds a legal section in Configs where users can add a terms of service and privacy policy - Add a Custom Fields section in Configs where admins can add/edit custom user/team fields - Move user graphs into a modal for Admin Panel **API** - Add `/api/v1/comments` to manipulate and create comments **Themes** - Make scoreboard caching only cache the score table instead of the entire page. This is done by caching the specific template section. Refer to CTFd#1586, specifically the changes in `scoreboard.html`. - Add rel=noopener to external links to prevent tab napping attacks - Change the registration page to reference links to Terms of Service and Privacy Policy if specified in configuration **Miscellaneous** - Make team settings modal larger in the core theme - Update tests in Github Actions to properly test under MySQL and Postgres - Make gevent default in serve.py and add a `--disable-gevent` switch in serve.py - Add `tenacity` library for retrying logic - Add `pytest-sugar` for slightly prettier pytest output - Add a `listen()` method to `CTFd.utils.events.EventManager` and `CTFd.utils.events.RedisEventManager`. - This method should implement subscription for a CTFd worker to whatever underlying notification system there is. This should be implemented with gevent or a background thread. - The `subscribe()` method (which used to implement the functionality of the new `listen()` function) now only handles passing notifications from CTFd to the browser. This should also be implemented with gevent or a background thread.
DA-ETSIINF · Sep 8, 2020 · 9264e96 · 9264e96
1 parent c1d7910
commit 9264e96
Show file tree

Hide file tree

Showing 145 changed files with 4,716 additions and 366 deletions.
diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml
@@ -11,7 +11,6 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.6']
-        TESTING_DATABASE_URL: ['sqlite://']
 
     name: Linting
     steps:
@@ -30,6 +29,8 @@ jobs:
 
       - name: Lint
         run: make lint
+        env:
+            TESTING_DATABASE_URL: 'sqlite://'
 
       - name: Lint Dockerfile
         uses: brpaz/hadolint-action@master

diff --git a/.github/workflows/mysql.yml b/.github/workflows/mysql.yml
@@ -9,9 +9,12 @@ jobs:
     runs-on: ubuntu-latest
     services:
       mysql:
-        image: mysql
+        image: mysql:5.7
+        env:
+          MYSQL_ROOT_PASSWORD: password
         ports:
-          - 3306:3306
+          - 3306
+        options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
       redis:
         image: redis
         ports:
@@ -20,7 +23,6 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.6']
-        TESTING_DATABASE_URL: ['mysql+pymysql://root@localhost/ctfd']
 
     name: Python ${{ matrix.python-version }}
     steps:
@@ -43,6 +45,7 @@ jobs:
         env:
             AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
             AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+            TESTING_DATABASE_URL: mysql+pymysql://root:password@localhost:${{ job.services.mysql.ports[3306] }}/ctfd
 
       - name: Codecov
         uses: codecov/[email protected]

diff --git a/.github/workflows/postgres.yml b/.github/workflows/postgres.yml
@@ -15,6 +15,7 @@ jobs:
         env:
           POSTGRES_HOST_AUTH_METHOD: trust
           POSTGRES_DB: ctfd
+          POSTGRES_PASSWORD: password
         # Set health checks to wait until postgres has started
         options: >-
           --health-cmd pg_isready
@@ -29,7 +30,6 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.6']
-        TESTING_DATABASE_URL: ['postgres://postgres@localhost/ctfd']
 
     name: Python ${{ matrix.python-version }}
     steps:
@@ -52,6 +52,7 @@ jobs:
         env:
             AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
             AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+            TESTING_DATABASE_URL: postgres://postgres:password@localhost:${{ job.services.postgres.ports[5432] }}/ctfd
 
       - name: Codecov
         uses: codecov/[email protected]

diff --git a/.github/workflows/sqlite.yml b/.github/workflows/sqlite.yml
@@ -11,7 +11,6 @@ jobs:
     strategy:
       matrix:
         python-version: ['3.6']
-        TESTING_DATABASE_URL: ['sqlite://']
 
     name: Python ${{ matrix.python-version }}
     steps:
@@ -35,6 +34,7 @@ jobs:
         env:
             AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
             AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+            TESTING_DATABASE_URL: 'sqlite://'
 
       - name: Codecov
         uses: codecov/[email protected]

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,39 @@
+# 3.1.0 / 2020-09-08
+
+**General**
+
+- Loosen team password confirmation in team settings to also accept the team captain's password to make it easier to change the team password
+- Adds the ability to add custom user and team fields for registration/profile settings.
+- Improve Notifications pubsub events system to use a subscriber per server instead of a subscriber per browser. This should improve the reliability of CTFd at higher load and make it easier to deploy the Notifications system
+
+**Admin Panel**
+
+- Add a comments functionality for admins to discuss challenges, users, teams, pages
+- Adds a legal section in Configs where users can add a terms of service and privacy policy
+- Add a Custom Fields section in Configs where admins can add/edit custom user/team fields
+- Move user graphs into a modal for Admin Panel
+
+**API**
+
+- Add `/api/v1/comments` to manipulate and create comments
+
+**Themes**
+
+- Make scoreboard caching only cache the score table instead of the entire page. This is done by caching the specific template section. Refer to #1586, specifically the changes in `scoreboard.html`.
+- Add rel=noopener to external links to prevent tab napping attacks
+- Change the registration page to reference links to Terms of Service and Privacy Policy if specified in configuration
+
+**Miscellaneous**
+
+- Make team settings modal larger in the core theme
+- Update tests in Github Actions to properly test under MySQL and Postgres
+- Make gevent default in serve.py and add a `--disable-gevent` switch in serve.py
+- Add `tenacity` library for retrying logic
+- Add `pytest-sugar` for slightly prettier pytest output
+- Add a `listen()` method to `CTFd.utils.events.EventManager` and `CTFd.utils.events.RedisEventManager`.
+  - This method should implement subscription for a CTFd worker to whatever underlying notification system there is. This should be implemented with gevent or a background thread.
+  - The `subscribe()` method (which used to implement the functionality of the new `listen()` function) now only handles passing notifications from CTFd to the browser. This should also be implemented with gevent or a background thread.
+
 # 3.0.2 / 2020-08-23
 
 **Admin Panel**

diff --git a/CTFd/__init__.py b/CTFd/__init__.py
@@ -26,7 +26,7 @@
 from CTFd.utils.sessions import CachingSessionInterface
 from CTFd.utils.updates import update_check
 
-__version__ = "3.0.2"
+__version__ = "3.1.0"
 __channel__ = "oss"
 
 

diff --git a/CTFd/api/__init__.py b/CTFd/api/__init__.py
@@ -3,6 +3,7 @@
 
 from CTFd.api.v1.awards import awards_namespace
 from CTFd.api.v1.challenges import challenges_namespace
+from CTFd.api.v1.comments import comments_namespace
 from CTFd.api.v1.config import configs_namespace
 from CTFd.api.v1.files import files_namespace
 from CTFd.api.v1.flags import flags_namespace
@@ -48,3 +49,4 @@
 CTFd_API_v1.add_namespace(pages_namespace, "/pages")
 CTFd_API_v1.add_namespace(unlocks_namespace, "/unlocks")
 CTFd_API_v1.add_namespace(tokens_namespace, "/tokens")
+CTFd_API_v1.add_namespace(comments_namespace, "/comments")
diff --git a/CTFd/api/v1/comments.py b/CTFd/api/v1/comments.py
@@ -0,0 +1,159 @@
+from typing import List
+
+from flask import request, session
+from flask_restx import Namespace, Resource
+
+from CTFd.api.v1.helpers.request import validate_args
+from CTFd.api.v1.helpers.schemas import sqlalchemy_to_pydantic
+from CTFd.api.v1.schemas import APIDetailedSuccessResponse, APIListSuccessResponse
+from CTFd.constants import RawEnum
+from CTFd.models import (
+    ChallengeComments,
+    Comments,
+    PageComments,
+    TeamComments,
+    UserComments,
+    db,
+)
+from CTFd.schemas.comments import CommentSchema
+from CTFd.utils.decorators import admins_only
+from CTFd.utils.helpers.models import build_model_filters
+
+comments_namespace = Namespace("comments", description="Endpoint to retrieve Comments")
+
+
+CommentModel = sqlalchemy_to_pydantic(Comments)
+
+
+class CommentDetailedSuccessResponse(APIDetailedSuccessResponse):
+    data: CommentModel
+
+
+class CommentListSuccessResponse(APIListSuccessResponse):
+    data: List[CommentModel]
+
+
+comments_namespace.schema_model(
+    "CommentDetailedSuccessResponse", CommentDetailedSuccessResponse.apidoc()
+)
+
+comments_namespace.schema_model(
+    "CommentListSuccessResponse", CommentListSuccessResponse.apidoc()
+)
+
+
+def get_comment_model(data):
+    model = Comments
+    if "challenge_id" in data:
+        model = ChallengeComments
+    elif "user_id" in data:
+        model = UserComments
+    elif "team_id" in data:
+        model = TeamComments
+    elif "page_id" in data:
+        model = PageComments
+    else:
+        model = Comments
+    return model
+
+
+@comments_namespace.route("")
+class CommentList(Resource):
+    @admins_only
+    @comments_namespace.doc(
+        description="Endpoint to list Comment objects in bulk",
+        responses={
+            200: ("Success", "CommentListSuccessResponse"),
+            400: (
+                "An error occured processing the provided or stored data",
+                "APISimpleErrorResponse",
+            ),
+        },
+    )
+    @validate_args(
+        {
+            "challenge_id": (int, None),
+            "user_id": (int, None),
+            "team_id": (int, None),
+            "page_id": (int, None),
+            "q": (str, None),
+            "field": (RawEnum("CommentFields", {"content": "content"}), None),
+        },
+        location="query",
+    )
+    def get(self, query_args):
+        q = query_args.pop("q", None)
+        field = str(query_args.pop("field", None))
+        CommentModel = get_comment_model(data=query_args)
+        filters = build_model_filters(model=CommentModel, query=q, field=field)
+
+        comments = (
+            CommentModel.query.filter_by(**query_args)
+            .filter(*filters)
+            .order_by(CommentModel.id.desc())
+            .paginate(max_per_page=100)
+        )
+        schema = CommentSchema(many=True)
+        response = schema.dump(comments.items)
+
+        if response.errors:
+            return {"success": False, "errors": response.errors}, 400
+
+        return {
+            "meta": {
+                "pagination": {
+                    "page": comments.page,
+                    "next": comments.next_num,
+                    "prev": comments.prev_num,
+                    "pages": comments.pages,
+                    "per_page": comments.per_page,
+                    "total": comments.total,
+                }
+            },
+            "success": True,
+            "data": response.data,
+        }
+
+    @admins_only
+    @comments_namespace.doc(
+        description="Endpoint to create a Comment object",
+        responses={
+            200: ("Success", "CommentDetailedSuccessResponse"),
+            400: (
+                "An error occured processing the provided or stored data",
+                "APISimpleErrorResponse",
+            ),
+        },
+    )
+    def post(self):
+        req = request.get_json()
+        # Always force author IDs to be the actual user
+        req["author_id"] = session["id"]
+        CommentModel = get_comment_model(data=req)
+
+        m = CommentModel(**req)
+        db.session.add(m)
+        db.session.commit()
+
+        schema = CommentSchema()
+
+        response = schema.dump(m)
+        db.session.close()
+
+        return {"success": True, "data": response.data}
+
+
+@comments_namespace.route("/<comment_id>")
+class Comment(Resource):
+    @admins_only
+    @comments_namespace.doc(
+        description="Endpoint to delete a specific Comment object",
+        responses={200: ("Success", "APISimpleSuccessResponse")},
+    )
+    def delete(self, comment_id):
+        comment = Comments.query.filter_by(id=comment_id).first_or_404()
+        db.session.delete(comment)
+        db.session.commit()
+        db.session.close()
+
+        return {"success": True}