Initial commit of the scripts

Author: Stanley Shi

CODE_OF_CONDUCT.md

@@ -0,0 +1,76 @@
+# Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to make participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+  advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+  address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies within all project spaces, and it also applies when
+an individual is representing the project or its community in public spaces.
+Examples of representing a project or community include using an official
+project e-mail address, posting via an official social media account, or acting
+as an appointed representative at an online or offline event. Representation of
+a project may be further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the project team at <[email protected]>. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

CONTRIBUTING.md

@@ -0,0 +1,37 @@
+# Contributing to Ads-Library-API-Script-Repository
+We want to make contributing to this project as easy and transparent as
+possible.
+
+## Pull Requests
+We actively welcome your pull requests.
+
+1. Fork the repo and create your branch from `master`.
+2. If you've added code that should be tested, add tests.
+3. If you've changed APIs, update the documentation.
+4. Ensure the test suite passes.
+5. Make sure your code lints.
+6. If you haven't already, complete the Contributor License Agreement ("CLA").
+
+## Contributor License Agreement ("CLA")
+In order to accept your pull request, we need you to submit a CLA. You only need
+to do this once to work on any of Facebook's open source projects.
+
+Complete your CLA here: <https://code.facebook.com/cla>
+
+## Issues
+We use GitHub issues to track public bugs. Please ensure your description is
+clear and has sufficient instructions to be able to reproduce the issue.
+
+Facebook has a [bounty program](https://www.facebook.com/whitehat/) for the safe
+disclosure of security bugs. In those cases, please go through the process
+outlined on that page and do not file a public issue.
+
+## Coding Style  
+* 4 spaces for indentation rather than tabs
+* 80 character line length
+* PEP8 formatting following [Black](https://black.readthedocs.io/en/stable/)
+
+## License
+By contributing to Ads-Library-API-Script-Repository, you agree that your
+contributions will be licensed under the LICENSE file in the root directory
+of this source tree.

LICENSE

@@ -0,0 +1,17 @@
+Copyright (c) Facebook, Inc. and its affiliates. All rights reserved.
+
+You are hereby granted a non-exclusive, worldwide, royalty-free license to use,
+copy, modify, and distribute this software in source code or binary form for use
+in connection with the web services and APIs provided by Facebook.
+
+As with any software that integrates with the Facebook platform, your use of
+this software is subject to the Facebook Platform Policy
+[http://developers.facebook.com/policy/]. This copyright notice shall be
+included in all copies or substantial portions of the software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
+FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
+COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
+IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -0,0 +1,33 @@
+# Ads-Library-API-Script-Repository
+Ads-Library-API-Script-Repository is a set of code examples to help user/researchers understand how the Facebook Ads Library API works. It also provides a simple command-line interface(CLI) for users to easily use the Facebook Ads Library API.
+
+## Examples
+Here's an example on how to use the CLI:
+
+    $ python fb_ads_library_api_cli.py -t {access_token} -f 'page_id,ad_snapshot_url,funding_entity,ad_delivery_start_time' -c 'CA' -s '.' -v count
+
+It count the number of active polictical ads that are running in CA(Canada);
+
+Note: please replace the '{access_token}' with your [Facebook Developer access token](https://developers.facebook.com/tools/accesstoken/).
+
+## Requirements
+Ads-Library-API-Script-Repository requires or works with
+* Mac OS X or Linux or Window
+* Python 3.0+
+
+
+## How Ads-Library-API-Script-Repository works
+The script will query the [Facebook Ads library API](https://www.facebook.com/ads/library/api) to get all the Ads Library information on the Facebook platform;
+
+## Full documentation
+You can find the full documentation here: (--to-be-added--)
+
+## More about Facebook Ads Library
+* Website: https://www.facebook.com/ads/library
+* Report: https://www.facebook.com/ads/library/report
+* API: https://www.facebook.com/ads/library/api
+
+See the [CONTRIBUTING](CONTRIBUTING.md) file for how to help out.
+
+## License
+Ads-Library-API-Script-Repository is licensed under the Facebook Platform License, as found in the LICENSE file.

python/fb_ads_library_api.py

@@ -0,0 +1,126 @@
+#!/usr/bin/env python3
+
+# Copyright (c) Facebook, Inc. and its affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the license found in the
+# LICENSE file in the root directory of this source tree.
+
+from datetime import datetime
+import json
+import re
+import requests
+
+
+def get_ad_archive_id(data):
+    """
+    Extract ad_archive_id from ad_snapshot_url
+    """
+    return re.search(r"/\?id=([0-9]+)", data["ad_snapshot_url"]).group(1)
+
+
+class FbAdsLibraryTraversal:
+    default_url_pattern = (
+        "https://graph.facebook.com/{}/ads_archive?access_token={}&"
+        + "fields={}&search_terms={}&ad_reached_countries={}&search_page_ids={}&"
+        + "ad_active_status={}&limit={}"
+    )
+    default_api_version = "v4.0"
+
+    def __init__(
+        self,
+        access_token,
+        fields,
+        search_term,
+        country,
+        search_page_ids="",
+        ad_active_status="active",
+        after_date="1970-01-01",
+        page_limit=500,
+        api_version=None,
+        retry_limit=3,
+    ):
+        self.page_count = 0
+        self.access_token = access_token
+        self.fields = fields
+        self.search_term = search_term
+        self.country = country
+        self.after_date = after_date
+        self.search_page_ids = search_page_ids
+        self.ad_active_status = ad_active_status
+        self.page_limit = page_limit
+        self.retry_limit = retry_limit
+        if api_version is None:
+            self.api_version = self.default_api_version
+        else:
+            self.api_version = api_version
+
+    def generate_ad_archives(self):
+        next_page_url = self.default_url_pattern.format(
+            self.api_version,
+            self.access_token,
+            self.fields,
+            self.search_term,
+            self.country,
+            self.search_page_ids,
+            self.ad_active_status,
+            self.page_limit,
+        )
+        return self.__class__._get_ad_archives_from_url(
+            next_page_url, after_date=self.after_date, retry_limit=self.retry_limit
+        )
+
+    @staticmethod
+    def _get_ad_archives_from_url(
+        next_page_url, after_date="1970-01-01", retry_limit=3
+    ):
+        last_error_url = None
+        last_retry_count = 0
+        start_time_cutoff_after = datetime.strptime(
+            after_date, "%Y-%m-%d"
+        ).timestamp()
+
+        while next_page_url is not None:
+            response = requests.get(next_page_url)
+            response_data = json.loads(response.text)
+            if "error" in response_data:
+                if next_page_url == last_error_url:
+                    # failed again
+                    if last_retry_count >= retry_limit:
+                        raise Exception(
+                            "Error message: [{}], failed on URL: [{}]".format(
+                                json.dumps(response_data["error"]), next_page_url
+                            )
+                        )
+                else:
+                    last_error_url = next_page_url
+                    last_retry_count = 0
+                last_retry_count += 1
+                continue
+
+            filtered = list(
+                filter(
+                    lambda ad_archive: datetime.strptime(
+                        ad_archive["ad_delivery_start_time"], "%Y-%m-%dT%H:%M:%S%z"
+                    ).timestamp()
+                    >= start_time_cutoff_after,
+                    response_data["data"],
+                )
+            )
+            if len(filtered) == 0:
+                # if no data after the after_date, break
+                next_page_url = None
+                break
+            yield filtered
+
+            if "paging" in response_data:
+                next_page_url = response_data["paging"]["next"]
+            else:
+                next_page_url = None
+
+    @classmethod
+    def generate_ad_archives_from_url(cls, failure_url, after_date="1970-01-01"):
+        """
+        if we failed from error, later we can just continue from the last failure url
+        """
+        return cls._get_ad_archives_from_url(failure_url, after_date=after_date)