Skip to content

Commit

Permalink
Add a rate_per_hour option to each_page
Browse files Browse the repository at this point in the history
I was having issues even when sending only 200
requests per hour, so I've added this for
additional flexibility.
  • Loading branch information
naiyt committed Jul 9, 2017
1 parent 0f65a32 commit 405812e
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 5 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ end
- `limit`: the number of results to return per page. Defaults to 100 (the max allowed through the Giant Bomb API)
- `offset`: defaults to 0
- `should_rate_limit`: pass this in as true if you want to ensure that you are rate limiting your requests to under the required 200 per resource per hour (which is important if you're trying to iterate through every page of a resource)
- `rate_per_hour`: defaults to 200 (the standard Giant Bomb hourly request limit). Override this value if you want to send more requests per hour (not recommended) or less.

Example using the optional arguments:

Expand Down
2 changes: 1 addition & 1 deletion giant_bomb_api.gemspec
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Gem::Specification.new do |s|
s.name = "giant_bomb_api"
s.version = "0.5.2"
s.version = "0.5.3"
s.default_executable = "giantbomb"

s.authors = ["Tim Adler"]
Expand Down
8 changes: 4 additions & 4 deletions lib/giant_bomb_api/collection_resource.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ def collection_resource_name(collection_resource_name = nil)
self.instance_variable_get("@collection_resource_name") || self.resource_name.pluralize
end

def each_page(sort: {}, limit: 100, offset: 0, should_rate_limit: false)
rate_limit(should_rate_limit) do
def each_page(sort: {}, limit: 100, offset: 0, should_rate_limit: false, rate_per_hour: 200)
rate_limit(should_rate_limit, rate_per_hour) do
response = where(sort: sort, limit: limit, offset: offset, tries: 5)
yield response
offset += limit
Expand Down Expand Up @@ -51,7 +51,7 @@ def where(params = {})

private

def rate_limit(should_rate_limit, &block)
def rate_limit(should_rate_limit, rate_per_hour, &block)
started_at = Time.now
num_of_requests = 0

Expand All @@ -69,7 +69,7 @@ def rate_limit(should_rate_limit, &block)

request_time = t2 - t1
time_to_one_hour = (started_at + 1.hour) - t2
remaining_requests = 200 - num_of_requests
remaining_requests = rate_per_hour - num_of_requests
min_time_per_request = time_to_one_hour / remaining_requests

sleep(min_time_per_request - request_time) if request_time < min_time_per_request
Expand Down

0 comments on commit 405812e

Please sign in to comment.