windowsservercore: urllib SSL errors

[lazka]

I can't get SSL to work in the container, any ideas welcome:

docker run -it python:3.7.1-windowsservercore
>>> import urllib.request; urllib.request.urlopen("https://letsencrypt.org/")
Traceback (most recent call last):
  File "C:\Python\lib\urllib\request.py", line 1317, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "C:\Python\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Python\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Python\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Python\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "C:\Python\lib\http\client.py", line 956, in send
    self.connect()
  File "C:\Python\lib\http\client.py", line 1392, in connect
    server_hostname=server_hostname)
  File "C:\Python\lib\ssl.py", line 412, in wrap_socket
    session=session
  File "C:\Python\lib\ssl.py", line 853, in _create
    self.do_handshake()
  File "C:\Python\lib\ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python\lib\urllib\request.py", line 525, in open
    response = self._open(req, data)
  File "C:\Python\lib\urllib\request.py", line 543, in _open
    '_open', req)
  File "C:\Python\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Python\lib\urllib\request.py", line 1360, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "C:\Python\lib\urllib\request.py", line 1319, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1051)>
>>>

[tianon]

I've verified that this is indeed an issue, but I've no idea where to turn to figure out what's going on. 😞

[lazka]

Thanks for checking. Yeah, information about this seems sparse.

One workaround (for users) would be to pip install certifi and set the SSL_CERT_FILE env var to the contained cacert.pem

[tianon]

As another data point, whether the error drops seems to depend on the site you're trying to hit; for example, doing urllib.request.urlopen("https://bootstrap.pypa.io/get-pip.py"), I do not get an error. 🤷‍♂️

[mika-fischer]

I have the same issue, maybe this whould be forwarded to the Python people?

[tianon]

Has anyone been able to reproduce outside the minimal Docker environment Microsoft provides? The most likely culprit is something the image is missing that a normal install isn't. (I'm not aware of anyone having filed anything upstream yet, but it's worth a search before doing so.)

[mika-fischer]

Well I have the issue also with mcr.microsoft.com/windows which is not so minimal anymore. Not sure what else to test.

I'll have a look and file a bug on the Python tracker.

[mika-fischer]

Created https://bugs.python.org/issue36137

[mika-fischer]

As a (limited) workaround, maybe this can be added to the images:

certutil -generateSSTFromWU roots.sst && certutil -addstore -f root roots.sst && del roots.sst

But this really needs to be fixed in Python...

[tianon]

Looks like the latest on this is in https://bugs.python.org/issue36011 python/cpython#80192 (and that it's still a problem 😞).

[tianon]

I just tested this fresh, just to be certain, and it's definitely still an issue on the latest builds. 🙈

[LaurentGoderre]

ServerCore seems to come with significantly fewer root authority (Only 2 compared to the dozen on my Windows host.

[LaurentGoderre]

It seems the certificates are managed either by Windows Update or by Active Directory policy.

To install them in the container you can run the following Powershell script

certutil.exe -generateSSTFromWU .\rootcerts.sst
$sstFile = (Get-ChildItem -Path .\rootcerts.sst)
$sstFile | Import-Certificate -CertStoreLocation Cert:\LocalMachine\Root