3.0.0rc1 Release (#769)

* 756 development tooling updates (#758)

* update isort and black versions. run linting

* add [tool.poetry] section to support poetry venv

* 757 remove support for unused frameworks - pt 1 (#760)

* update isort and black versions. run linting

* add [tool.poetry] section to support poetry venv

* remove old django versions from tox.ini

* remove pyramid and cherrypy support.
limit django support to >3.2

* ignore .idea/, replace celery in tox

* update manual testing example
re-link to broken celery issue for clarification

* remove eol python versions (#762)

* remove eol python versions

* updtae CI workflows to only handle 3.8+

* remove pyramid from setup, delete ubuntu 18 job

* restore comment for celery version

---------

Co-authored-by: Mitch Hartweg <[email protected]>

---------

Co-authored-by: Mitch Hartweg <[email protected]>
Co-authored-by: mitchh456 <[email protected]>

* remove nameko and starlett/api files for python <3.6 (#763)

Co-authored-by: Mitch Hartweg <[email protected]>

* 764 update agent to fully support python 311 (#765)

* add py311 to test envs, use partial to create wrapt decorator for error service thread

* remove py3.5+ check for one test, specify <3.11 for realloc test because forcing realloc doesn't work in 3.11

---------

Co-authored-by: Mitch Hartweg <[email protected]>

* Dev tim (#768)

* Resolve CI errors (#752)

* Allow exceptions for pkg_resources usages from pyramid and cherrypy.

* Move python 3.6 to ubuntu 20 GitHub action runner.

Py36 isn't included with ubuntu 22

* Update rq test dependency for dropped 3.6 support.

* Use a specific python 2.7 image for tests.

See actions/setup-python#672

Don't use setup-python for python2.7

* Pin importlib_metadata for < py3.8 in the tests.

* Handle sqlalchemy deprecations and start of v2 support.

* Disable tests for SQLAlchemy v2 to allow CI to pass.

* Update pre-commit repo versions.

* Support flask_sqlalchemy v3

get_engine was deprecated in favor of make_engine.

* Support ElasticSearch 8.7 health_report API.

Update python3.6 tests to use elasticsearch 7

* Update tests to be compatible with rq v1.13+

* Switch python 3.4 GitHub action to run on a custom container.

* Update changelog for CI fixes.

* Fix build for python 3.11 (#748)

* Fix build for python 3.11

* Skip cp311 for manylinux2010 build.

* Update python and django versions. (#753)

* simplify test and build

* cleanup test-ubuntu-20 workflow

* use text method in flask_sqlalchemy tests
skip 3.12+ wheel builds

* fix test sqlalchemy tests

---------

Co-authored-by: Tim Schilling <[email protected]>

* update pymongo collection methods

* replace deprecated manylinux image with 2_28

* add xpack.security.enabled flag to elasticserach

* use elasticsearch <8.0 for <=3.9

* reformat cibw_skip argument

* reformat cibw_skip argument

* remove manylinux1 and manylinux2010 wheel builds

* remove manylinux1 and manylinux2010 wheel builds

* remove i686 from manylinux_2_28 build because it doesn't exist

* consolidate aarch config into main. We want aarch for all, no need for conditional anymore

* fix delete aarch step

* bump version to 3.0.0

* re-add django 3.2 to tests, remove erroneous django version checks

* remove in-code python version checks (#771)

* remove in-code python version checks that no longer apply when supporting py3.8+ only

* remove usage of text_type and string_type

* fix SimpleNamespace imports

* fix mocked side effect error

---------

Co-authored-by: Mitch Hartweg <[email protected]>

* remove __future__ imports (#772)

* remove in-code python version checks that no longer apply when supporting py3.8+ only

* remove usage of text_type and string_type

* fix SimpleNamespace imports

* fix mocked side effect error

* remove __future__ imports

* fix expected line for error in test

---------

Co-authored-by: Mitch Hartweg <[email protected]>

* update changelog and set version to 3.0.0rc1

* update changelog and set version to 3.0.0rc1

* adjust formatting of changelog

---------

Co-authored-by: Quinn Milionis <[email protected]>
Co-authored-by: Mitch Hartweg <[email protected]>
Co-authored-by: Tim Schilling <[email protected]>
Co-authored-by: Quinn Milionis <[email protected]>

Author: 4 people

.github/workflows/main.yml

@@ -19,15 +19,14 @@ jobs:
       fail-fast: false
       matrix:
         python-version:
-        - "3.6"
-        - "3.7"
         - "3.8"
         - "3.9"
         - "3.10"
+        - "3.11"
 
     services:
       elasticsearch:
-        image: elasticsearch:8.2.3
+        image: elasticsearch:8.7.0
         ports:
         - 9200:9200
         env:
@@ -71,55 +70,8 @@ jobs:
       fail-fast: false
       matrix:
         python-version:
-        - "2.7"
-        - "3.5"
-
-    services:
-      elasticsearch:
-        image: elasticsearch:7.10.1
-        ports:
-        - 9200:9200
-        env:
-          discovery.type: single-node
-
-      mongodb:
-        image: mongo:4
-        ports:
-        - 27017:27017
-
-      redis:
-        image: redis:6
-        ports:
-        - 6379:6379
-
-    env:
-      ELASTICSEARCH_URL: http://localhost:9200/
-      MONGODB_URL: mongodb://localhost:27017/
-      REDIS_URL: redis://localhost:6379/0
-
-    steps:
-    - uses: actions/checkout@v2
-    - uses: actions/setup-python@v2
-      with:
-        python-version: ${{ matrix.python-version }}
-    - name: Upgrade packaging tools
-      run: python -m pip install --upgrade pip setuptools virtualenv
-    - name: Install dependencies
-      run: python -m pip install --upgrade tox
-    - name: Run tox targets for ${{ matrix.python-version }}
-      run: |
-        ENV_PREFIX=$(tr -C -d "0-9" <<< "${{ matrix.python-version }}" | cut -c -3)
-        TOXENV=$(tox --listenvs | grep "^py$ENV_PREFIX" | tr '\n' ',') python -m tox
-
-  tests-ubuntu-18:
-    name: Python ${{ matrix.python-version }} Ubuntu 18
-    runs-on: ubuntu-18.04
-
-    strategy:
-      fail-fast: false
-      matrix:
-        python-version:
-        - "3.4"
+        - "3.8"
+        - "3.9"
 
     services:
       elasticsearch:
@@ -128,6 +80,7 @@ jobs:
         - 9200:9200
         env:
           discovery.type: single-node
+          xpack.security.enabled: false
 
       mongodb:
         image: mongo:4

.github/workflows/release.yml

@@ -9,22 +9,16 @@ on:
   pull_request:
 
 jobs:
-  cibuildwheel_py37plus:
-    name: Build python 3.7+ ${{ matrix.manylinux_image }} wheels
+  cibuildwheel_py38plus:
+    name: Build python 3.8+ ${{ matrix.manylinux_image }} wheels
     runs-on: ${{ matrix.os }}
     strategy:
       matrix:
         os: [ ubuntu-20.04 ]
-        python: [ '3.10' ]
-        manylinux_image: [ manylinux2010, manylinux2014, manylinux_2_24 ]
+        python: ['3.10']
+        manylinux_image: [ manylinux2014, manylinux_2_28 ]
         # Disable for platforms where pure Python wheels would be generated
-        cibw_skip: [ "pp37-* pp38-* pp39-* pp310-*" ]
-        include:
-          # Exclude building 3.10 wheel for manylinux1.
-          - os: ubuntu-20.04
-            python: 3.9
-            manylinux_image: manylinux1
-            cibw_skip: "pp37-* pp38-* pp39-* pp310-* cp310-*"
+        cibw_skip: [ "pp38-* pp39-* pp310-* pp311-* pp312-* cp312-*" ]
     steps:
       - uses: actions/checkout@v2
 
@@ -44,53 +38,12 @@ jobs:
         uses: docker/setup-qemu-action@v1
         with:
           platforms: all
-      - name: Set AArch64 env if needed
-        if: matrix.manylinux_image != 'manylinux1' && matrix.manylinux_image != 'manylinux2010'
-        env:
-          CIBW_MANYLINUX_AARCH64_IMAGE: ${{ matrix.manylinux_image }}
-          CIBW_ARCHS_LINUX: "auto aarch64"
-        run: echo "set CIBW_MANYLINUX_AARCH64_IMAGE=${{ env.CIBW_MANYLINUX_AARCH64_IMAGE }}"
       - name: Build binary wheels
         env:
           CIBW_SKIP: ${{ matrix.cibw_skip }}
           CIBW_BUILD_VERBOSITY: 1
           CIBW_MANYLINUX_X86_64_IMAGE: ${{ matrix.manylinux_image }}
-          CIBW_MANYLINUX_I686_IMAGE: ${{ matrix.manylinux_image }}
-        run: python -m cibuildwheel
-
-      - uses: actions/upload-artifact@v2
-        with:
-          path: ./wheelhouse/*.whl
-
-  cibuildwheel_py35py36:
-    name: Build python 3.5, 3.6 platform wheels
-    runs-on: ${{ matrix.os }}
-    strategy:
-      matrix:
-        os: [ ubuntu-18.04 ]
-        python: [ 3.9 ]
-    steps:
-      - uses: actions/checkout@v2
-      - uses: actions/setup-python@v2
-        name: Install Python
-        with:
-          python-version: ${{ matrix.python }}
-
-      - name: Install cibuildwheel
-        run: |
-          python -m pip install --upgrade pip "cibuildwheel==1.12.0"
-
-      - name: Set up QEMU
-        # For cross-architecture builds
-        # https://cibuildwheel.readthedocs.io/en/stable/faq/#emulation
-        if: runner.os == 'Linux'
-        uses: docker/setup-qemu-action@v1
-        with:
-          platforms: all
-      - name: Build binary wheels
-        env:
-          CIBW_BUILD_VERBOSITY: 1
-          CIBW_BUILD: "cp35-* cp36-*"
+          CIBW_MANYLINUX_AARCH64_IMAGE: ${{ matrix.manylinux_image }}
           CIBW_ARCHS_LINUX: "auto aarch64"
         run: python -m cibuildwheel
 
@@ -103,7 +56,7 @@ jobs:
     runs-on: ubuntu-20.04
     strategy:
       matrix:
-        python: [3.9, 2.7]
+        python: [3.9]
     steps:
       - uses: actions/checkout@v2
 
@@ -143,7 +96,7 @@ jobs:
           path: dist/*.tar.gz
 
   upload_pypi:
-    needs: [cibuildwheel_py37plus, cibuildwheel_py35py36, build_pure_wheels, build_sdist]
+    needs: [cibuildwheel_py38plus, build_pure_wheels, build_sdist]
     runs-on: ubuntu-20.04
     steps:
       - name: Download distributions for publishing.

.gitignore

@@ -17,3 +17,5 @@ src/*.egg-info/
 /.pytest_cache
 .tox/
 .coverage
+
+.idea/

.pre-commit-config.yaml

@@ -4,7 +4,7 @@ ci:
 
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.1.0
+    rev: v4.4.0
     hooks:
     - id: check-added-large-files
     - id: check-case-conflict
@@ -17,17 +17,17 @@ repos:
     - id: end-of-file-fixer
     - id: trailing-whitespace
   - repo: https://github.com/psf/black
-    rev: 21.12b0
+    rev: 23.7.0
     hooks:
     - id: black
       language_version: python3
       additional_dependencies: ['click<8.1']
   - repo: https://github.com/pycqa/isort
-    rev: 5.10.1
+    rev: 5.12.0
     hooks:
     - id: isort
   - repo: https://github.com/PyCQA/flake8
-    rev: 4.0.1
+    rev: 6.0.0
     hooks:
     - id: flake8
       additional_dependencies:

CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## [3.0.0rc1] 2023-09-08
+
+Version 3.0.0 drops support for Python 2.7 and < 3.8, as well as Django < 3.2.
+If you need to continue using Python < 3.8 or Django < 3.2, please continue using v2.26.1.
+
+### Added
+- Add formal support for Python 3.11 and Django 4.2.
+
+### Fixed
+- Fix CI for 3.8+.
+- Remove support for Python 2.7, 3.4, 3.5, 3.6.
+- Remove support for Django < 3.2.
+- Remove support for CherryPy, Nameko, and Pyramid.
+- Update ElasticSearch support for 8.7.0 (``health_check``).
+- Support flask_sqlalchemy v3. This requires an integration
+  change where ``instrument_sqlalchemy`` is called before ``db.init_app()``.
+
 ## [2.26.1] 2022-07-28
 
 ### Fixed

DEVELOPMENT.md

@@ -34,7 +34,7 @@ key in `tox.ini`, which you can list with `tox -l`. Pick one for a Python
 version that you have installed and run the tests, for example:
 
 ```
-tox -e py39-django31
+python -m tox -e py39-django31
 ```
 
 If you run `tox` with no arguments, it will test all environments. This will
@@ -70,7 +70,7 @@ by environment variables. These are:
 
 * `ELASTICSEARCH_URL` - point to a running Elasticsearch instance, e.g.
   "http://localhost:9200/" . You can start it with:
-  `docker run --detach --name elasticsearch --publish 9200:9200 -e "discovery.type=single-node" -e "xpack.security.enabled=false" elasticsearch:8.2.3` .
+  `docker run --detach --name elasticsearch --publish 9200:9200 -e "discovery.type=single-node" -e "xpack.security.enabled=false" elasticsearch:8.7.0` .
 * `MONGODB_URL` - point to a running MongoDB instance e.g.
   "mongodb://localhost:27017/" . You can start it with:
   `docker run --detach --name mongo --publish 27017:27017 mongo:4.0` .

README.md

@@ -11,21 +11,19 @@ Monitor the performance of Python Django apps, Flask apps, and Celery workers wi
 
 ## Requirements
 
-Python 2.7 or 3.4+.
+Python 3.8+.
+For legacy Python versions, including 2.7 and 3.4+, pin scout-apm to <=2.26.1.
 
 Scout APM has integrations for the following frameworks:
 
 * Bottle 0.12+
-* CherryPy 18.0.0+
 * Celery 3.1+
-* Django 1.8+
+* Django 3.2+
 * Dramatiq 1.0+
 * Falcon 2.0+
 * Flask 0.10+
 * Huey 2.0+
 * Hug 2.5.1+
-* Nameko 2.0+
-* Pyramid 1.8+
 * RQ 1.0+
 * Starlette 0.12+
 

pyproject.toml

@@ -1,15 +1,16 @@
+[tool.poetry]
+name="scout_apm_python"
+description = "Scout APM Python Agent"
+readme = "README.md"
+version = "0.1.0"
+authors = ["Developers <[email protected]>"]
+
 [build-system]
 requires = ["setuptools >= 40.6.0", "wheel"]
 build-backend = "setuptools.build_meta"
 
 [tool.black]
-target-version = ["py27"]
+target-version = ["py311"]
 
 [tool.isort]
 profile = "black"
-add_imports = [
-    "from __future__ import absolute_import",
-    "from __future__ import division",
-    "from __future__ import print_function",
-    "from __future__ import unicode_literals",
-]

pytest.ini

@@ -1,37 +1,6 @@
 [pytest]
 addopts = --cov=scout_apm
           --cov-report=term-missing:skip-covered
-          # Disable Nameko's built-in Pytest plugin because it does
-          # eventlet.monkey_patch() early on, but we want to test things in a
-          # non-eventlet world!
-          # It lead to this failure:
-          # cls = <class 'eventlet.green.ssl.GreenSSLSocket'>, sock = <eventlet.greenio.base.GreenSocket object at 0x10667e588>, keyfile = None, certfile = None, server_side = False, cert_reqs = <VerifyMode.CERT_NONE: 0>
-          # ssl_version = <_SSLMethod.PROTOCOL_TLS: 2>, ca_certs = None, do_handshake_on_connect = True, args = ()
-          # kw = {'_context': <eventlet.green.ssl.GreenSSLContext object at 0x104d6f480>, 'server_hostname': 's3-us-west-1.amazonaws.com'}
-          #
-          #     def __new__(cls, sock=None, keyfile=None, certfile=None,
-          #                 server_side=False, cert_reqs=CERT_NONE,
-          #                 ssl_version=PROTOCOL_SSLv23, ca_certs=None,
-          #                 do_handshake_on_connect=True, *args, **kw):
-          #         if _is_under_py_3_7:
-          #             return super(GreenSSLSocket, cls).__new__(cls)
-          #         else:
-          #             if not isinstance(sock, GreenSocket):
-          #                 sock = GreenSocket(sock)
-          #             with _original_ssl_context():
-          #                 ret = _original_wrap_socket(
-          #                     sock=sock.fd,
-          #                     keyfile=keyfile,
-          #                     certfile=certfile,
-          #                     server_side=server_side,
-          #                     cert_reqs=cert_reqs,
-          #                     ssl_version=ssl_version,
-          #                     ca_certs=ca_certs,
-          #                     do_handshake_on_connect=False,
-          # >                   *args, **kw
-          #                 )
-          # E               TypeError: wrap_socket() got an unexpected keyword argument '_context'
-          -p no:pytest_nameko
 norecursedirs = src
 filterwarnings =
     error
@@ -77,5 +46,13 @@ filterwarnings =
     ; Triggered by cherrypy -> jarco/text -> importlib_resources
     ignore:read_text is deprecated\. Use files\(\) instead\.:DeprecationWarning
     ignore:open_text is deprecated\. Use files\(\) instead\.:DeprecationWarning
+    ; Triggered by cherrypy -> cherrypy/__init__.py, pyramid/asset.py
+    ignore:pkg_resources is deprecated as an API:DeprecationWarning
+    ignore:.*pkg_resources.declare_namespace:DeprecationWarning
     ; ElasticSearch v8 will drop support for python < 3.6
     ignore:Support for Python 3\.5 and earlier is deprecated and will be removed in v8\.0\.0:DeprecationWarning
+    ; Triggered by sqlalchemy -> sqlalchemy/engine/base.py
+    ignore:.*These feature\(s\) are not compatible with SQLAlchemy 2\.0:DeprecationWarning
+    ; Triggered by rq -> rq/job.py
+    ignore:The `push_connection` function is deprecated\. Pass the `connection` explicitly instead\.:DeprecationWarning
+    ignore:'cgi' is deprecated and slated for removal in Python 3.13