Commiting latest changes.

Author: grivescorbett

ChangeLog

@@ -1,3 +1,8 @@
+5.4.0 -- 4/25/17:
+* Resolved issues:
+- Issue 200: https://github.com/googleads/googleads-python-lib/issues/200
+- Issue 196: https://github.com/googleads/googleads-python-lib/issues/196
+
 5.3.0 -- 3/20/17:
 * Removed support for AdWords v201605.
 * Removed examples for AdWords v201605.

README.md

@@ -1,12 +1,12 @@
-#The googleads Python client library
+# The googleads Python client library
 
 
 This client library simplifies accessing Google's SOAP Ads APIs - AdWords,
 and DoubleClick for Publishers. The library provides easy ways to store your
 authentication and create SOAP web service clients. It also contains example
 code to help you get started integrating with our APIs.
 
-##Getting started
+## Getting started
 1. Download and install the library
 
    *[setuptools](https://pypi.python.org/pypi/setuptools) is a pre-requisite
@@ -46,7 +46,7 @@ code to help you get started integrating with our APIs.
 
    * [Developing a web application (AdWords or DFP)](https://github.com/googleads/googleads-python-lib/wiki/API-access-on-behalf-of-your-clients-(web-flow))
 
-####Where can I find samples?
+#### Where can I find samples?
 
 You can find code examples for the latest versions of AdWords or DFP on the
 [releases](https://github.com/googleads/googleads-python-lib/releases) page.
@@ -55,11 +55,11 @@ Alternatively, you can find [AdWords](https://github.com/googleads/googleads-pyt
 or [DFP](https://github.com/googleads/googleads-python-lib/tree/master/examples/dfp)
 samples in the examples directory of this repository.
 
-####Where can I find the pydocs?
+#### Where can I find the pydocs?
 
 Our pydocs can be found [here](http://googleads.github.io/googleads-python-lib).
 
-####Caching authentication information
+#### Caching authentication information
 
 It is possible to cache your API authentication information. The library
 includes a sample file showing how to do this named `googleads.yaml`. Fill
@@ -76,7 +76,7 @@ adwords_client = adwords.AdWordsClient.LoadFromStorage()
 dfp_client = dfp.DfpClient.LoadFromStorage('C:\My\Directory\googleads.yaml')
 ```
 
-####How do I change the Client Customer Id at runtime?
+#### How do I change the Client Customer Id at runtime?
 You can change the Client Customer Id with the following:
 
 ```
@@ -85,7 +85,7 @@ adwords_client.SetClientCustomerId('my_client_customer_id')
 ```
 
 
-##Where do I submit bug reports and/or feature requests?
+## Where do I submit bug reports and/or feature requests?
 
 If you have issues directly related to the client library, use the [issue
 tracker](https://github.com/googleads/googleads-python-lib/issues).
@@ -100,7 +100,7 @@ Make sure to subscribe to our [Google Plus page](https://plus.google.com/+Google
 for API change announcements and other news.
 
 
-##How do I log SOAP interactions?
+## How do I log SOAP interactions?
 The library uses Python's built in logging framework. If you wish to log your
 SOAP interactions to stdout, you can do the following:
 ```python
@@ -111,7 +111,7 @@ If you wish to log to a file, you'll need to attach a log handler to this source
 which is configured to write the output to a file.
 
 
-##How do I disable log filters?
+## How do I disable log filters?
 By default, this library will apply log filters to the `googleads.common`,
 `suds.client`, and `suds.transport` loggers in order to omit sensitive data. If
 you need to see this data in your logs, you can disable the filters with the
@@ -126,7 +126,7 @@ logging.getLogger('suds.transport').removeFilter(
 ```
 
 
-##I'm familiar with suds. Can I use suds features with this library?
+## I'm familiar with suds. Can I use suds features with this library?
 Yes, you can. The services returned by the `client.GetService()` functions all
 have a reference to the underlying suds client stored in the `suds_client`
 attribute. You can retrieve the client and use it in familiar ways:
@@ -162,7 +162,7 @@ suds_client.set_options(
 suds_client.service.mutate([operation])
 ```
 
-##How can I configure or disable caching for the suds client?
+## How can I configure or disable caching for the suds client?
 
 By default, the suds clients are cached because reading and digesting the WSDL
 can be expensive. However, the default caching method requires permission to
@@ -187,26 +187,15 @@ adwords_client = adwords.AdWordsClient(
   client_customer_id=client_customer_id, cache=suds.cache.NoCache())
 ```
 
-##Timeout Tips
-The requests sent by this library are sent via urllib, which is consequently
-where the timeout is set. If you set a system timeout elsewhere, the googleads
-library will respect it.
 
-You can do the following if you wish to override the timeout:
+## Requirements
 
-```python
-import socket
-socket.setdefaulttimeout(15 * 60)
-```
-
-##Requirements
-
-###Python Versions
+### Python Versions
 
 This library supports both Python 2 and 3. To use this library, you will need to
 have Python 2.7.9 (or higher) or Python 3.4 (or higher) installed.
 
-###External Dependencies:
+### External Dependencies:
 
     - httplib2             -- https://pypi.python.org/pypi/httplib2/
     - oauth2client         -- https://pypi.python.org/pypi/oauth2client/
@@ -221,5 +210,5 @@ have Python 2.7.9 (or higher) or Python 3.4 (or higher) installed.
                               (only needed to run unit tests)
 
 
-##Authors:
+## Authors:
     Mark Saniscalchi

googleads.yaml

@@ -49,21 +49,22 @@ dfp:
   #############################################################################
   # The network_code is required for all services except NetworkService:
   # network_code: INSERT_NETWORK_CODE_HERE
+  # delegated_account: INSERT_DOMAIN_WIDE_DELEGATION_ACCOUNT
   #############################################################################
   # OAuth2 Configuration                                                      #
   # Below you may provide credentials for either the installed application or #
-  # service account flows. Remove or comment the lines for the flow you're    #
-  # not using.                                                                #
+  # service account (recommended) flows. Remove or comment the lines for the  #
+  # flow you're not using.                                                    #
   #############################################################################
-  # The following values configure the client for the installed application
-  # flow.
-  client_id: INSERT_OAUTH_2_CLIENT_ID_HERE
-  client_secret: INSERT_CLIENT_SECRET_HERE
-  refresh_token: INSERT_REFRESH_TOKEN_HERE
   # The following values configure the client for the service account flow.
+  path_to_private_key_file: INSERT_PATH_TO_FILE_HERE
+  # Only needed when using P12
   # service_account_email: INSERT_SERVICE_ACCOUNT_EMAIL_HERE
-  # path_to_private_key_file: INSERT_PATH_TO_FILE_HERE
-  # delegated_account: INSERT_DOMAIN_WIDE_DELEGATION_ACCOUNT
+  # The following values configure the client for the installed application
+  # flow.
+  # client_id: INSERT_OAUTH_2_CLIENT_ID_HERE
+  # client_secret: INSERT_CLIENT_SECRET_HERE
+  # refresh_token: INSERT_REFRESH_TOKEN_HERE
 
 
 # Common configurations:

googleads/common.py

@@ -60,7 +60,7 @@
     'compatibility with this library, upgrade to Python 2.7.9 or higher.')
 
 
-VERSION = '5.3.0'
+VERSION = '5.4.0'
 _COMMON_LIB_SIG = 'googleads/%s' % VERSION
 _HTTP_PROXY_YAML_KEY = 'http_proxy'
 _HTTPS_PROXY_YAML_KEY = 'https_proxy'
@@ -649,6 +649,7 @@ def __init__(self, handlers, **kwargs):
         handlers: an iterable of urllib2.BaseHandler subclasses.
         **kwargs: Keyword arguments.
       """
+      kwargs['timeout'] = 3600
       suds.transport.http.HttpTransport.__init__(self, **kwargs)
       self.handlers = handlers
 

googleads/util.py

@@ -85,6 +85,13 @@ def _ApplySudsJurkoSendPatch(self):
     enabled. For more details on SOAP Compression, see:
     https://developers.google.com/adwords/api/docs/guides/bestpractices?hl=en#use_compression
     """
+    def GetInflateStream(msg):
+      stream = io.BytesIO()
+      stream.write(msg)
+      stream.flush()
+      stream.seek(0)
+      return gzip.GzipFile(fileobj=stream, mode='rb')
+
     def PatchedHttpTransportSend(self, request):
       """Patch for HttpTransport.send to enable gzip compression."""
       msg = request.message
@@ -102,6 +109,12 @@ def PatchedHttpTransportSend(self, request):
         if e.code in (202, 204):
           return None
         else:
+          if e.headers.get('content-encoding') == 'gzip':
+            # If gzip encoding is used, decompress here.
+            # Need to read and recreate a stream because urllib result objects
+            # don't fully implement the file-like API
+            e.fp = GetInflateStream(e.fp.read())
+
           raise suds.transport.TransportError(e.msg, e.code, e.fp)
 
       self.getcookies(fp, u2request)
@@ -110,11 +123,7 @@ def PatchedHttpTransportSend(self, request):
 
       if result.headers.get('content-encoding') == 'gzip':
         # If gzip encoding is used, decompress here.
-        stream = io.BytesIO()
-        stream.write(result.message)
-        stream.flush()
-        stream.seek(0)
-        result.message = gzip.GzipFile(fileobj=stream, mode='rb').read()
+        result.message = GetInflateStream(result.message).read()
 
       suds.transport.http.log.debug('received:\n%s', result)
       return result

tests/common_test.py

@@ -15,7 +15,9 @@
 """Unit tests to cover the common module."""
 
 
+import io
 import unittest
+from urllib import addinfourl
 import urllib2
 import warnings
 
@@ -339,6 +341,18 @@ def testLoadFromString_passesWithHTTPAndHTTPSProxy(self):
                       'proxy_config': proxy_config.return_value,
                       googleads.common.ENABLE_COMPRESSION_KEY: False}, rval)
 
+  def testHTTPSProxyExtendsSSLSockTimeout(self):
+    with mock.patch('googleads.oauth2.GoogleRefreshTokenClient'):
+      with mock.patch('googleads.common.open', self.fake_open, create=True):
+        with mock.patch('suds.transport.http.'
+                        'HttpTransport.__init__') as mock_transport:
+          proxy_config = googleads.common.ProxyConfig()
+          proxy_config.GetSudsProxyTransport()
+          mock_transport.assert_called_once()
+          # 90 is the default, don't expect an exact value so that we can change
+          # the timeout in the code without returning here.
+          self.assertGreater(mock_transport.call_args_list[0][1]['timeout'], 90)
+
   def testLoadFromString_passesWithHTTPProxy(self):
     yaml_doc = self._CreateYamlDoc(
         {'adwords': {},
@@ -985,5 +999,61 @@ def testProxyConfigWithHTTPAndHTTPSProxy(self):
         self.assertEqual(t.return_value, transport)
 
 
+class TestSudsRequestPatcher(unittest.TestCase):
+
+  class MockHeaders(object):
+
+    def __init__(self, d):
+      self.dict = d
+
+  def testInflateSuccessfulRequestIfGzipped(self):
+    with mock.patch('suds.transport.http.HttpTransport.getcookies'):
+      with mock.patch('urllib2.OpenerDirector.open') as mock_open:
+        with mock.patch('gzip.GzipFile.read') as mock_read:
+          resp_fp = io.BytesIO()
+          resp_fp.write('abc')
+          resp_fp.flush()
+          resp_fp.seek(0)
+          resp = addinfourl(resp_fp, self.MockHeaders(
+              {'content-encoding': 'gzip'}), 'https://example.com', code=200)
+          mock_open.return_value = resp
+
+          req = suds.transport.Request('https://example.com',
+                                       message='hello world')
+          suds.transport.http.HttpTransport().send(req)
+          mock_read.assert_called_once()
+
+  def testInflateFailedRequestIfGzipped(self):
+
+    with mock.patch('suds.transport.http.HttpTransport.getcookies'):
+      with mock.patch('urllib2.OpenerDirector.open') as mock_open:
+        with mock.patch('gzip.GzipFile') as mock_gzip:
+          gzip_instance_mock = mock.MagicMock
+          mock_gzip.return_value = gzip_instance_mock
+          resp_fp = io.BytesIO()
+          resp_fp.write('abc')
+          resp_fp.flush()
+          resp_fp.seek(0)
+          resp = addinfourl(resp_fp,
+                            self.MockHeaders({'content-encoding': 'gzip'}),
+                            'https://example.com', code=200)
+
+          def fail_to_open(*args, **kwargs):
+            raise urllib2.HTTPError(
+                'https://example.com',
+                500,
+                'oops!',
+                {'content-encoding': 'gzip'},
+                resp
+            )
+          mock_open.side_effect = fail_to_open
+
+          req = suds.transport.Request('https://example.com',
+                                       message='hello world')
+          with self.assertRaises(suds.transport.TransportError) as exc:
+            suds.transport.http.HttpTransport().send(req)
+          self.assertEqual(exc.exception.fp, gzip_instance_mock)
+
+
 if __name__ == '__main__':
   unittest.main()