AdWords v201705/v201708 sunset, Zeep/google-auth migration

Author: msaniscalchi

ChangeLog

@@ -1,3 +1,44 @@
+11.0.0 -- 03/21/18
+* Removed support and examples for AdWords v201705, and v201708.
+* The default SOAP backend is now zeep. If you encounter any issues,
+  please report them and use your googleads.yaml file to revert back
+  to suds.
+- Zeep logs to the googleads.soap channel.
+- When using zeep (the default), you no longer need to encode your
+  image assets to base64, this will be done automatically.
+* Sunset oauth2client, replaced with google-auth.
+  https://google-auth.readthedocs.io/en/latest/
+- Updated googleads.yaml to remove service_account_email configurations that
+  are no longer necessary.
+- Removed httplib2 dependency.
+- Added google-auth, google-auth-oauthlib, and requests as dependencies.
+- AdWords and DFP auth examples have been updated to use new OAuth 2.0
+  dependencies.
+* Fixed a bug in IncrementalUploadHelper where the specified server would be
+  lost in serialization/deserialization.
+* Breaking changes:
+* Connection faults are now raised as GoogleAdsSoapTransportError
+  instances instead of suds errors.
+- Soap faults are now raised as GoogleAdsServerFault instances instead of suds
+  errors.
+- GoogleServiceAccountClient no longer supports P12 key files.
+- GoogleServiceAccountClient's initializer arguments have been refactored;
+  `key_file` is now a required positional argument, `client_email` and
+  `private_key_password` have been removed.
+- Removed Proxy from common.py. When creating a ProxyConfig, you must now
+  provide the full URI for http_proxy and https_proxy rather than a Proxy
+  instance. For an example of what these would look like, see:
+  http://docs.python-requests.org/en/master/user/advanced/#proxies
+- The proxy_config used in googleads.yaml has been updated to better resemble
+  proxy configuration for the underlying HTTP library. The http_proxy and
+  https_proxy configurations have been removed, and replaced with http and
+  https. These take the full proxy URI as their value.
+- Removed proxy_info attribute from ProxyConfig.
+- Refactored proxy_option attribute in ProxyConfig, renamed to proxies.
+* Resolved issues:
+- Issue 251: https://github.com/googleads/googleads-python-lib/issues/251
+- Issue 254: https://github.com/googleads/googleads-python-lib/issues/254
+
 10.1.1 -- 03/16/18
 * Accepted PR #222, to improve AdWords App Engine sample.
 

README.md

@@ -106,7 +106,14 @@ configuration via the configuration file; see [googleads.yaml](https://github.co
 for an example.
 
 Alternatively, you can manually specify your logging configuration. For example,
-if you want to log your SOAP interactions to stdout, you can do the following:
+if you want to log your SOAP interactions to stdout, and you are using the Zeep soap library, you
+can do the following:
+```python
+logging.basicConfig(level=logging.INFO, format=googleads.util.LOGGER_FORMAT)
+logging.getLogger('googleads.soap').setLevel(logging.DEBUG)
+```
+
+If you are using suds, you can achieve the same like this:
 ```python
 logging.basicConfig(level=logging.INFO, format=googleads.util.LOGGER_FORMAT)
 logging.getLogger('suds.transport').setLevel(logging.DEBUG)
@@ -116,7 +123,22 @@ which is configured to write the output to a file.
 
 
 ## How do I disable log filters?
-By default, this library will apply log filters to the `googleads.common`,
+The zeep plugin used for logging strips sensitive data from its output. If you would like this data
+included in logs, you'll need to implement your own simple logging plugin. For example:
+```python
+class DangerousZeepLogger(zeep.Plugin):
+  def ingress(self, envelope, http_headers, operation):
+    logging.debug('Incoming response: \n%s', etree.tostring(envelope, pretty_print=True))
+    return envelope, http_headers
+
+  def egress(self, envelope, http_headers, operation, binding_options):
+    logging.debug('Incoming response: \n%s', etree.tostring(envelope, pretty_print=True))
+    return envelope, http_headers
+
+adwords_client.zeep_client.plugins.append(DangerousZeepLogger())
+```
+
+When using suds, this library will apply log filters to the `googleads.common`,
 `suds.client`, and `suds.transport` loggers in order to omit sensitive data. If
 you need to see this data in your logs, you can disable the filters with the
 following:
@@ -134,10 +156,10 @@ logging.getLogger('suds.transport').removeFilter(
 ```
 
 
-## I'm familiar with suds. Can I use suds features with this library?
+## I'm familiar with suds/zeep. Can I use those features in the library?
 Yes, you can. The services returned by the `client.GetService()` functions all
-have a reference to the underlying suds client stored in the `suds_client`
-attribute. You can retrieve the client and use it in familiar ways:
+have a reference to the underlying client stored in the `zeep_client` or `suds_client`
+attributes. You can retrieve the client and use it in familiar ways:
 ```python
 client = AdWordsClient.LoadFromStorage()
 campaign_service = client.GetService('CampaignService')
@@ -170,25 +192,40 @@ suds_client.set_options(
 suds_client.service.mutate([operation])
 ```
 
-## How can I configure or disable caching for the suds client?
+## How can I configure or disable caching?
 
-By default, the suds clients are cached because reading and digesting the WSDL
+By default, clients are cached because reading and digesting the WSDL
 can be expensive. However, the default caching method requires permission to
 access a local file system that may not be available in certain hosting
 environments such as App Engine.
 
-You can pass an implementation of `suds.cache.Cache` to the `AdWordsClient` or
+You can pass an implementation of `suds.cache.Cache` or `zeep.cache.Base` to the `AdWordsClient` or
 `DfpClient` initializer to modify the default caching behavior.
 
-For example, configuring a different location and duration of the cache file:
+For example, configuring a different location and duration of the cache file with zeep
+```python
+doc_cache = zeep.cache.SqliteCache(path=cache_path)
+adwords_client = adwords.AdWordsClient(
+  developer_token, oauth2_client, user_agent,
+  client_customer_id=client_customer_id, cache=doc_cache)
+```
+
+And with suds:
 ```python
 doc_cache = suds.cache.DocumentCache(location=cache_path, days=2)
 adwords_client = adwords.AdWordsClient(
   developer_token, oauth2_client, user_agent,
   client_customer_id=client_customer_id, cache=doc_cache)
 ```
 
-You can also disable caching in similar fashion:
+You can also disable caching in similar fashion with zeep
+```python
+adwords_client = adwords.AdWordsClient(
+  developer_token, oauth2_client, user_agent,
+  client_customer_id=client_customer_id, cache=None)
+```
+
+And with suds:
 ```python
 adwords_client = adwords.AdWordsClient(
   developer_token, oauth2_client, user_agent,
@@ -212,6 +249,7 @@ have Python 2.7.9 (or higher) or Python 3.4 (or higher) installed.
     - pytz                 -- https://pypi.python.org/pypi/pytz
     - pyYAML               -- https://pypi.python.org/pypi/pyYAML/
     - xmltodict            -- https://pypi.python.org/pypi/xmltodict/
+    - zeep                 -- https://pypi.python.org/pypi/zeep
     - mock                 -- https://pypi.python.org/pypi/mock
                               (only needed to run unit tests)
     - pyfakefs             -- https://pypi.python.org/pypi/pyfakefs

examples/adwords/adwords_appengine_demo/appengine_config.py

@@ -1,3 +1,5 @@
+#!/usr/bin/env python
+#
 # Copyright 2018 Google Inc. All Rights Reserved.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
@@ -18,6 +20,6 @@
 import os
 from google.appengine.ext import vendor
 
-# Add any libraries install in the "lib" folder.
+# Add any libraries installed in the "lib" folder to the path.
 vendor.add(os.path.join(os.path.dirname(os.path.realpath(__file__)), 'lib'))
 

examples/adwords/authentication/create_adwords_client_with_service_account.py

@@ -22,7 +22,6 @@
 
 # OAuth2 credential information. In a real application, you'd probably be
 # pulling these values from a credential storage.
-SERVICE_ACCOUNT_EMAIL = 'INSERT_SERVICE_ACCOUNT_EMAIL_HERE'
 KEY_FILE = 'INSERT_PATH_TO_KEY_FILE'
 SERVICE_ACCOUNT_USER = 'INSERT_IMPERSONATED_EMAIL_HERE'
 
@@ -32,11 +31,10 @@
 CLIENT_CUSTOMER_ID = 'INSERT_CLIENT_CUSTOMER_ID_HERE'
 
 
-def main(service_account_email, key_file, service_account_user,
-         developer_token, user_agent, client_customer_id):
+def main(key_file, service_account_user, developer_token, user_agent,
+         client_customer_id):
   oauth2_client = oauth2.GoogleServiceAccountClient(
-      oauth2.GetAPIScope('adwords'), service_account_email, key_file,
-      sub=service_account_user)
+      key_file, oauth2.GetAPIScope('adwords'), sub=service_account_user)
 
   adwords_client = adwords.AdWordsClient(
       developer_token, oauth2_client, user_agent,
@@ -53,5 +51,5 @@ def main(service_account_email, key_file, service_account_user,
 
 
 if __name__ == '__main__':
-  main(SERVICE_ACCOUNT_EMAIL, KEY_FILE, SERVICE_ACCOUNT_USER,
-       DEVELOPER_TOKEN, USER_AGENT, CLIENT_CUSTOMER_ID)
+  main(KEY_FILE, SERVICE_ACCOUNT_USER, DEVELOPER_TOKEN, USER_AGENT,
+       CLIENT_CUSTOMER_ID)

examples/adwords/authentication/generate_refresh_token.py

@@ -20,7 +20,8 @@
 import argparse
 import sys
 
-from oauth2client import client
+from google_auth_oauthlib.flow import InstalledAppFlow
+from oauthlib.oauth2.rfc6749.errors import InvalidGrantError
 
 # Your OAuth2 Client ID and Secret. If you do not have an ID and Secret yet,
 # please go to https://console.developers.google.com and create a set.
@@ -29,6 +30,9 @@
 
 # The AdWords API OAuth2 scope.
 SCOPE = u'https://www.googleapis.com/auth/adwords'
+# The redirect URI set for the given Client ID. The redirect URI for Client ID
+# generated for an installed application will always have this value.
+_REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oob'
 
 parser = argparse.ArgumentParser(description='Generates a refresh token with '
                                  'the provided credentials.')
@@ -42,31 +46,68 @@
                     'refresh token. Each scope should be separated by a comma.')
 
 
+class ClientConfigBuilder(object):
+  """Helper class used to build a client config dict used in the OAuth 2.0 flow.
+  """
+  _DEFAULT_AUTH_URI = 'https://accounts.google.com/o/oauth2/auth'
+  _DEFAULT_TOKEN_URI = 'https://accounts.google.com/o/oauth2/token'
+  CLIENT_TYPE_WEB = 'web'
+  CLIENT_TYPE_INSTALLED_APP = 'installed'
+
+  def __init__(self, client_type=None, client_id=None, client_secret=None,
+               auth_uri=_DEFAULT_AUTH_URI, token_uri=_DEFAULT_TOKEN_URI):
+    self.client_type = client_type
+    self.client_id = client_id
+    self.client_secret = client_secret
+    self.auth_uri = auth_uri
+    self.token_uri = token_uri
+
+  def Build(self):
+    """Builds a client config dictionary used in the OAuth 2.0 flow."""
+    if all((self.client_type, self.client_id, self.client_secret,
+            self.auth_uri, self.token_uri)):
+      client_config = {
+          self.client_type: {
+              'client_id': self.client_id,
+              'client_secret': self.client_secret,
+              'auth_uri': self.auth_uri,
+              'token_uri': self.token_uri
+          }
+      }
+    else:
+      raise ValueError('Required field is missing.')
+
+    return client_config
+
+
 def main(client_id, client_secret, scopes):
   """Retrieve and display the access and refresh token."""
-  flow = client.OAuth2WebServerFlow(
-      client_id=client_id,
-      client_secret=client_secret,
-      scope=scopes,
-      user_agent='Ads Python Client Library',
-      redirect_uri='urn:ietf:wg:oauth:2.0:oob')
+  client_config = ClientConfigBuilder(
+      client_type=ClientConfigBuilder.CLIENT_TYPE_WEB, client_id=client_id,
+      client_secret=client_secret)
+
+  flow = InstalledAppFlow.from_client_config(
+      client_config.Build(), scopes=scopes)
+  # Note that from_client_config will not produce a flow with the
+  # redirect_uris (if any) set in the client_config. This must be set
+  # separately.
+  flow.redirect_uri = _REDIRECT_URI
 
-  authorize_url = flow.step1_get_authorize_url()
+  auth_url, _ = flow.authorization_url(prompt='consent')
 
-  print ('Log into the Google Account you use to access your AdWords account'
-         'and go to the following URL: \n%s\n' % (authorize_url))
+  print ('Log into the Google Account you use to access your AdWords account '
+         'and go to the following URL: \n%s\n' % auth_url)
   print 'After approving the token enter the verification code (if specified).'
   code = raw_input('Code: ').strip()
 
   try:
-    credential = flow.step2_exchange(code)
-  except client.FlowExchangeError, e:
-    print 'Authentication has failed: %s' % e
+    flow.fetch_token(code=code)
+  except InvalidGrantError as ex:
+    print 'Authentication has failed: %s' % ex
     sys.exit(1)
-  else:
-    print ('OAuth2 authorization successful!\n\n'
-           'Your access token is:\n %s\n\nYour refresh token is:\n %s'
-           % (credential.access_token, credential.refresh_token))
+
+  print 'Access token: %s' % flow.credentials.token
+  print 'Refresh token: %s' % flow.credentials.refresh_token
 
 
 if __name__ == '__main__':