DEV Community

Cover image for My $646 mistake with PHP
Jonathon Ringeisen
Jonathon Ringeisen

Posted on

My $646 mistake with PHP

Yesterday I learned a very valuable lesson when iterating a while loop using the Google Places API - Web Service that cost me $646. That's right, I pinged the Places API 17,000 times in one hour 😳 🤦🏻‍♂️. So I decided to write an article today documenting my mistake in the hopes that this prevents someone else from running into this same issue. Let's dive in.

What Happened?
Well...I'm currently building a Web and Mobile Application that uses the Google Places API and I needed to return results using the nearby search. The Google API only returns 20 results with a max of 60 which you can get by using their version of pagination. So let's go through the code and explain what went wrong.

// This is the initial request that we send to get the initial 20 results.
        $response = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=' . $request->lat .'%2C'. $request->lng.'&radius='.$request->radius.'&type=restaurant&key=' . env('GOOGLE_MAP_KEY'))->json();

        // We push those results to our google_places collection
        $this->google_places->push(...$response['results']);

        // Here we check the status of the response, if it's ok we continue.
        if ($response['status'] == 'OK') {
            // If the response has a next page token we loop through until it doesn't, getting the next 20 results.
            while ($response['next_page_token']) {
                // The google places api has a delay from when the next_page_token is created and those results are actually available.
                // So we need to create a delay before we try and grab the next results otherwise we will get an INVALID_REQUEST response.
                sleep(2);

                // Now we get the next 20 results.
                $new_results = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?pagetoken=' . $response['next_page_token'] . '&key=' . env('GOOGLE_MAP_KEY'))->json();

                // Then we push those results to our collection.
                $this->google_places->push(...$new_results['results']);
            }
        }
Enter fullscreen mode Exit fullscreen mode

Were you able to figure out where I went wrong? A while loop in PHP will run until it gets a false back, the value that I put in the while loop always returns true and therefore sends the while loop into an infinite loop. I was using the $response['next_page_token'] from the initial request which will always return true.

How did I fix this issue?

$response = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?location=' . $request->lat .'%2C'. $request->lng.'&radius='.$request->radius.'&type=restaurant&key=' . env('GOOGLE_MAP_KEY'))->json();

        $this->google_places->push(...$response['results']);

        // We check to see if the above response includes the key next_page_token, if it does we set the next_page_token property to true
        // This is super important, this is how we're going to ensure our loop doesn't run infinitely. 
        if (array_key_exists('next_page_token', $response)) {
            $this->next_page_token = true;
        }

        if ($response['status'] == 'OK') {
            // For the while loop we check to see if the next_page_token is true, it will end when it returns false.
            while ($this->next_page_token) {
                sleep(2);

                $new_results = Http::get('https://maps.googleapis.com/maps/api/place/nearbysearch/json?pagetoken=' . $response['next_page_token'] . '&key=' . env('GOOGLE_MAP_KEY'))->json();

                $this->google_places->push(...$new_results['results']);

                // This is also super important, here we check to see if next_page_token does not exist or if we're getting an invalid request response.
                // If we do we want to set the next_page_token to false so that the loop ends.
                if (!array_key_exists('next_page_token', $new_results) || $new_results['status'] == 'INVALID_REQUEST') {
                    $this->next_page_token = false;
                }
            }
        }

        // Being that the Google API only returns 60 results that means the loop should only have to run a max of 2 times.
        // So you could create a $count variable and add that like so while($this->next_page_token || $count <= 2) and increment the count each time.
        // This will act as a safety in case you accidentally throw a forever truthy statement in there like I did.
Enter fullscreen mode Exit fullscreen mode

Google Places API Chart

Google Places API Billing

Conclusion
Be cautious when working with APIs that charge and running them inside a loop. As a precaution you can always create a $count variable and stop the loop at a set count in case you make a mistake. This is what I will do for now on just to ensure I don't do this again. Hopefully this has helped you or someone else from making the same mistake. ✌🏻

Discussion (21)

Collapse
slyfirefox profile image
Peter Fox

I think the mistake here is more, always write unit tests when you can. A test running this function could easily find that a while loop is never completing.

Equally never hurts to put a limit on a loop for an API so it never iterates more than say 50 times etc.

Collapse
khalyomede profile image
Khalyomede

Good idea, and since I see some Laravel code up there, just mock the API return so that you never really consume credits (and you still find out infinite loops, but for free)

Collapse
jaywhy13 profile image
Jean-Mark Wright

Thanks for sharing this, and capturing the lessons you learned along the way! I believe this example underscores the importance of targeted testing to confirm expected behavior.
Assuming that you have a mock setup for the test, some examples of tests that come to mind are:

  • Verify that there's only one call to the API when there's no next page token
  • Verify that sleeps happen when there's a next page token
  • Verify the number of calls to the API when there are multiple pages
Collapse
euperia profile image
Andrew McCombe • Edited on

I did a similar thing but with the lookup taking place in a background job using SQS. A bug in the code meant the job always failed and got retried.

£20,000 later...

Luckily Google were cool about it.

Lesson: set decent retry counts on your Laravel worker jobs.

Collapse
dgrigg profile image
Derrick Grigg

Could you not have changed this one line

$new_results Http::get('maps.googleapis.com ...

to

$response = Http::get('maps.googleapis.com ...

so the loop would work correctly by referencing the latest result set?

Collapse
michaeltharrington profile image
Michael Tharrington (he/him)

Ooof. 😅 Really happy that you can see the humor in this, and thanks for sharing!

Maybe you can pass on this story and they'll give ya some sorta credits for the pain ya went through here. 🤞

It would be a good publicity move, Google. We're watching... 👀

Collapse
ricardoboss profile image
Ricardo Boss

Looks like you also need to use the correct next_key_token. You are using the initial next_key_token from the first request in your URL.

Collapse
jringeisen profile image
Jonathon Ringeisen Author

Good catch! I'm actually not using the Google Web Service anymore so I won't be using this code but, still good to know.

Collapse
stack111 profile image
Daniel Stack • Edited on

Full disclosure, I am a software engineer on Microsoft Azure Maps who built a feature to help address this exact problem because API calls == $.

My other suggestions if you consider another product like Azure maps is to leverage entry level Skus which have upper limits resulting in throttling on requests. Second option is look into the authentication support; SAS token authentication allows you to configure an upper bound how many requests per second to limit charges.

I have no affiliation with Google maps or other competitors.

Collapse
khalyomede profile image
Khalyomede

Thanks a lot for sharing your valuable eperience about this! I see this resonated on some other folks experiences as well, including mine.

I definitely advice, even if you have a great code, to put an hard coded limit on the maximum calls, wether it matches the plan limit or something lower.

We ran into the same issue at my job, where one of the plans we subscribed for a given service was mistakenly showing us limit reached, but in small characters we could actually go further by... Paying an extra per additional calls (knowing we already paid for the initial bucket of credit). Now that we have the hard limit, we get an error and we can decide to up the limit or just give up on this service until next month.

Collapse
shanks25 profile image
Kunal Rajput

I did similar thing, causing 500$ for Google timezone api.

I just sent them a email explaining them it was a mistake, they gave me waiver off on everything lol

Collapse
roestvrijstaal profile image
RoestVrijStaal

You might want to cache the results from Google Places API for at least a day.

And use the same cached result for the same and 5-10 meters around the given latitude and radius.

Collapse
jringeisen profile image
Jonathon Ringeisen Author

As much as I would love to, Google's TOS don't allow for this.

Collapse
roestvrijstaal profile image
RoestVrijStaal

But how would Google verify that?

In both cases, the server(s) of Google Places API see only the IP-address of your application server.

But less frequently when you cache the results.

Collapse
c0ldf0x profile image
ColdFox

It happened in my company as well, after a 2k€ bill we set the max limit equal to the free tier usage. This way we only pay if there's a need to increase the limit in any given month.

Collapse
yoursunny profile image
Junxiao Shi

Another error:
.status == "OK" should be checked first before saving the results.

Collapse
dimkiriakos profile image
dimkiriakos

The pay as you go is a big trap for people. be careful about these services. you are making the rich richer and yourselves more poor.

Collapse
jeremymoorecom profile image
Jeremy Moore

Thanks for sharing. Lots of good advice going on.

Collapse
tez123z profile image
tez123z

$7000 was my mistake. 😅

Not a bug in my case just overlooked the number of API calls that were attached to the external library I was using and Google had just started charging at the time.

Collapse
tkutru profile image
Comment marked as low quality/non-constructive by the community. View Code of Conduct
tkutru

I am sorry, but this code looks crappy anyway. Check out dependency injection, class constants and code style.

Collapse
jringeisen profile image
Jonathon Ringeisen Author • Edited on

Word of advise, if you're going to call someones code crappy, you should provide examples on how you would of made it better.