Merge from master (July 26th, 2019)

CHANGES.rst

@@ -32,6 +32,11 @@ dev (master)
 * Fix edge case where Retry-After header was still respected even when
   explicitly opted out of. (Pull #1607)
 
+* Remove dependency on ``rfc3986`` for URL parsing.
+
+* Fix issue where URLs containing invalid characters within ``Url.auth`` would
+  raise an exception instead of percent-encoding those characters.
+
 
 1.25.3 (2019-05-23)
 -------------------

CONTRIBUTORS.txt

@@ -278,5 +278,9 @@ In chronological order:
 * James Meickle <https://permadeath.com/>
   * Improve handling of Retry-After header
 
+* Chris Jerdonek <[email protected]>
+  * Remove a spurious TypeError from the exception chain inside
+    HTTPConnectionPool._make_request(), also for BaseExceptions.
+
 * [Your name or handle] <[email or website]>
   * [Brief summary of your changes]

README.rst

@@ -96,8 +96,8 @@ Tidelift will coordinate the fix and disclosure with maintainers.
 Maintainers
 -----------
 
-- `@theacodes <https://github.com/theacodes>`_ (Thea Flowers)
 - `@sethmlarson <https://github.com/sethmlarson>`_ (Seth M. Larson)
+- `@theacodes <https://github.com/theacodes>`_ (Thea Flowers)
 - `@haikuginger <https://github.com/haikuginger>`_ (Jess Shapiro)
 - `@lukasa <https://github.com/lukasa>`_ (Cory Benfield)
 - `@sigmavirus24 <https://github.com/sigmavirus24>`_ (Ian Cordasco)
@@ -130,7 +130,7 @@ development <https://urllib3.readthedocs.io/en/latest/contributing.html#sponsors
 
 Sponsors include:
 
-- Google Cloud Platform (2018-present), sponsors `@theacodes <https://github.com/theacodes>`_'s work on an ongoing basis
-- Abbott (2018-present), sponsors `@sethmlarson <https://github.com/sethmlarson>`_'s work on an ongoing basis
+- Abbott (2018-2019), sponsored `@sethmlarson <https://github.com/sethmlarson>`_'s work on urllib3.
+- Google Cloud Platform (2018-2019), sponsored `@theacodes <https://github.com/theacodes>`_'s work on urllib3.
 - Akamai (2017-2018), sponsored `@haikuginger <https://github.com/haikuginger>`_'s work on urllib3
-- Hewlett Packard Enterprise (2016-2017), sponsored `@Lukasa’s <https://github.com/Lukasa>`_ work on urllib3
+- Hewlett Packard Enterprise (2016-2017), sponsored `@Lukasa’s <https://github.com/Lukasa>`_ work on urllib3.

_travis/upload_coverage.sh

@@ -4,5 +4,5 @@ set -exo pipefail
 
 if [[ -e .coverage ]]; then
     python -m pip install codecov
-    codecov --env TRAVIS_OS_NAME,NOX_SESSION
+    python -m codecov --env TRAVIS_OS_NAME,NOX_SESSION
 fi

dev-requirements.txt

@@ -6,12 +6,11 @@ tornado==5.1.1
 PySocks==1.6.8
 pkginfo==1.4.2
 pytest-random-order==1.0.4;python_version>="3.5"
-pytest-timeout==1.3.1
-pytest==4.0.1
-pytest-cov==2.6.0
+pytest-timeout==1.3.3
+pytest==4.6.4
+pytest-cov==2.7.1
 h11==0.8.0
 cryptography==2.6.1
-pluggy==0.11.0
 
 # https://github.com/ionelmc/python-lazy-object-proxy/issues/30
 lazy-object-proxy==1.4.0

docs/contributing.rst

@@ -13,54 +13,62 @@ If you wish to add a new feature or fix a bug:
    to start making your changes.
 #. Write a test which shows that the bug was fixed or that the feature works
    as expected.
+#. Format your changes with black using command `$ nox -s blacken` and lint your
+   changes using command `nox -s lint`.
 #. Send a pull request and bug the maintainer until it gets merged and published.
    :) Make sure to add yourself to ``CONTRIBUTORS.txt``.
 
 
 Setting up your development environment
 ---------------------------------------
 
-It is recommended, and even enforced by the make file, that you use a 
-`virtualenv
-<http://docs.python-guide.org/en/latest/dev/virtualenvs/>`_::
+In order to setup the development environment all that you need is 
+`nox <https://nox.thea.codes/en/stable/index.html>`_ installed in your machine::
 
-  $ python3 -m venv venv3
-  $ source venv3/bin/activate
-  $ pip install -r dev-requirements.txt
+  $ pip install --user --upgrade nox
 
 
 Running the tests
 -----------------
 
 We use some external dependencies, multiple interpreters and code coverage
-analysis while running test suite. Our ``Makefile`` handles much of this for
-you as long as you're running it `inside of a virtualenv
-<http://docs.python-guide.org/en/latest/dev/virtualenvs/>`_::
-
-  $ make test-quick
-  [... magically installs dependencies and runs tests on your virtualenv]
-  Ran 182 tests in 1.633s
-
-  OK (SKIP=6)
-
-There is also a make target for running all of our tests and multiple python
+analysis while running test suite. Our ``noxfile.py`` handles much of this for
+you::
+
+  $ nox --sessions test-2.7 test-3.7
+  [ Nox will create virtualenv, install the specified dependencies, and run the commands in order.]
+  nox > Running session test-2.7
+  .......
+  .......
+  nox > Session test-2.7 was successful.
+  .......
+  .......
+  nox > Running session test-3.7
+  .......
+  .......
+  nox > Session test-3.7 was successful.
+
+There is also a nox command for running all of our tests and multiple python
 versions.
 
-  $ make test-all
+  $ nox --sessions test
 
 Note that code coverage less than 100% is regarded as a failing run. Some
 platform-specific tests are skipped unless run in that platform.  To make sure
 the code works in all of urllib3's supported platforms, you can run our ``tox``
 suite::
 
-  $ make test-all
-  [... tox creates a virtualenv for every platform and runs tests inside of each]
-  py27: commands succeeded
-  py34: commands succeeded
-  py35: commands succeeded
-  py36: commands succeeded
-  py37: commands succeeded
-  pypy: commands succeeded
+  $ nox --sessions test
+  [ Nox will create virtualenv, install the specified dependencies, and run the commands in order.]
+  .......
+  .......
+  nox > Session test-2.7 was successful.
+  nox > Session test-3.4 was successful.
+  nox > Session test-3.5 was successful.
+  nox > Session test-3.6 was successful.
+  nox > Session test-3.7 was successful.
+  nox > Session test-3.8 was successful.
+  nox > Session test-pypy was successful.
 
 Our test suite `runs continuously on Travis CI
 <https://travis-ci.org/urllib3/urllib3>`_ with every pull request.

dummyserver/testcase.py

@@ -1,5 +1,4 @@
 import threading
-import unittest
 
 import pytest
 from tornado import ioloop, web
@@ -20,7 +19,7 @@ def consume_socket(sock, chunks=65536):
         pass
 
 
-class SocketDummyServerTestCase(unittest.TestCase):
+class SocketDummyServerTestCase(object):
     """
     A simple socket-based server is created for this class that is good for
     exactly one request.
@@ -67,7 +66,7 @@ def start_basic_handler(cls, **kw):
         )
 
     @classmethod
-    def tearDownClass(cls):
+    def teardown_class(cls):
         if hasattr(cls, "server_thread"):
             cls.server_thread.join(0.1)
 
@@ -102,10 +101,10 @@ def _start_server(cls, socket_handler):
         cls.port = cls.server_thread.port
 
 
-class HTTPDummyServerTestCase(unittest.TestCase):
+class HTTPDummyServerTestCase(object):
     """ A simple HTTP server that runs when your test class runs
 
-    Have your unittest class inherit from this one, and then a simple server
+    Have your test class inherit from this one, and then a simple server
     will start when your tests run, and automatically shut down when they
     complete. For examples of what test requests you can send to the server,
     see the TestingApp in dummyserver/handlers.py.
@@ -132,11 +131,11 @@ def _stop_server(cls):
         cls.server_thread.join()
 
     @classmethod
-    def setUpClass(cls):
+    def setup_class(cls):
         cls._start_server()
 
     @classmethod
-    def tearDownClass(cls):
+    def teardown_class(cls):
         cls._stop_server()
 
 
@@ -151,7 +150,7 @@ class IPV6HTTPSDummyServerTestCase(HTTPSDummyServerTestCase):
     host = "::1"
 
 
-class HTTPDummyProxyTestCase(unittest.TestCase):
+class HTTPDummyProxyTestCase(object):
 
     http_host = "localhost"
     http_host_alt = "127.0.0.1"
@@ -164,7 +163,7 @@ class HTTPDummyProxyTestCase(unittest.TestCase):
     proxy_host_alt = "127.0.0.1"
 
     @classmethod
-    def setUpClass(cls):
+    def setup_class(cls):
         cls.io_loop = ioloop.IOLoop.current()
 
         app = web.Application([(r".*", TestingApp)])
@@ -185,7 +184,7 @@ def setUpClass(cls):
         cls.server_thread = run_loop_in_thread(cls.io_loop)
 
     @classmethod
-    def tearDownClass(cls):
+    def teardown_class(cls):
         cls.io_loop.add_callback(cls.http_server.stop)
         cls.io_loop.add_callback(cls.https_server.stop)
         cls.io_loop.add_callback(cls.proxy_server.stop)

noxfile.py

@@ -69,6 +69,8 @@ def blacken(session):
     session.install("black")
     session.run("black", "src", "dummyserver", "test", "noxfile.py", "setup.py")
 
+    lint(session)
+
 
 @nox.session
 def lint(session):

setup.cfg

@@ -23,4 +23,5 @@ requires-dist =
     brotlipy>=0.6.0; extra == 'brotli'
 
 [tool:pytest]
-xfail_strict=true
+xfail_strict = true
+python_classes = Test *TestCase

setup.py

@@ -56,7 +56,6 @@
         "urllib3.packages",
         "urllib3.packages.ssl_match_hostname",
         "urllib3.packages.backports",
-        "urllib3.packages.rfc3986",
         "urllib3.contrib",
         "urllib3.contrib._securetransport",
         "urllib3.util",

src/urllib3/_async/connectionpool.py

@@ -27,7 +27,6 @@
 from urllib3.packages.ssl_match_hostname import CertificateError
 from urllib3.packages import six
 from urllib3.packages.six.moves import queue
-from urllib3.packages.rfc3986.normalizers import normalize_host
 from .request import RequestMethods
 from .response import HTTPResponse
 from .connection import HTTP1Connection
@@ -43,7 +42,7 @@
     BaseSSLError,
 )
 from ..util.timeout import Timeout
-from ..util.url import get_host, Url, NORMALIZABLE_SCHEMES
+from ..util.url import get_host, Url, _normalize_host as normalize_host
 from ..util.queue import LifoQueue
 
 try:
@@ -908,14 +907,14 @@ def _normalize_host(host, scheme):
     Normalize hosts for comparisons and use with sockets.
     """
 
+    host = normalize_host(host, scheme)
+
     # httplib doesn't like it when we include brackets in IPv6 addresses
     # Specifically, if we include brackets but also pass the port then
     # httplib crazily doubles up the square brackets on the Host header.
     # Instead, we need to make sure we never pass ``None`` as the port.
     # However, for backward compatibility reasons we can't actually
     # *assert* that.  See http://bugs.python.org/issue28539
     if host.startswith("[") and host.endswith("]"):
-        host = host.strip("[]")
-    if scheme in NORMALIZABLE_SCHEMES:
-        host = normalize_host(host)
+        host = host[1:-1]
     return host