James Shah

Posted on Mar 26, 2020

How I Automated The Google Form Filling For My College Attendance Using Python

#python #script #showdev #productivity

To fight Corona Virus, our Prime Minister has declared lockdown for 21 days in all over the country. And so, our college decided to take online lectures and also, sent us a google form which is to be filled every day for each lecture to fill attendance. Filling form 4 times a day with the subject name and corresponding time and professor details is a tedious task. But also, Attendance is Engineer's Only Motivation to attend lectures.

And So, I decided to use Python and to write a script to automate this tedious task.

As usual, I started by searching on google to see how can we submit the google form using python script. And I came to know that Every Google Form Input element has a "name" attribute attached to it and we can use it to send the data for the corresponding field.

So, I opened the google form in chrome and Started inspecting Input element in which I will be filling the data, and I found a bunch of ss-form-entry objects with name attribute like " name = entry.<id> "

I copied every name attribute value of the field.

Now, In Python I created a dictionary with the value of name attributes that we copied from the browser as keys and the data that I want to fill in the corresponding field as it's value.

Then I created a dictionary with Day Name(Monday, Tuesday, etc.) as keys and the lectures' details as list as values.

And put both the dictionaries in a function that returns the list of different data dictionaries of the current day.

And, finally the function for sending the form response using POST method of requests library. It takes two arguments - URL and data(here, data_dictionary), respectively.

Note: Change the URL of google form by replacing '/viewform' at the end of the URL with 'formResponse'. Now, your URL should look like this https://docs.google.com/forms/d/<form_id>/formResponse

And that's it, now I just have to run this script once a day and it will fill my attendance for all the lectures of that day.
I can also deploy it on a server like PythonAnyWhere or DigitalOcean and it can run by itself every day, but as I'm home quarantine for the 21 Days, I have more than enough time to run it manually every day.🤓🤓

You can find the full code here : AutoFill Google Form

Top comments (30)

Yechiel Kalmenson • Sep 9 '20

Thanks! This is very useful! I came here because I was searching to see if there's some kind of API to submit Google forms (my kids' school also has us submitting forms every day :)

I didn't think it would be as simple as sending a POST request to the /formResponse endpoint 😂

James Shah • Sep 10 '20

I'm glad that this helped you. 👍😄

Simon Richard • May 18 '20 • Edited

Great post, James! I needed to do a similar task with google forms, and this was very helpful. However, it didn't work right off the bat for me. After using requests.post(url, data=d), I got the status code 401 (which stands for an Unauthorized Error).

Side Note: no error is actually thrown. To view the status code, print the object or use r.status_code.

However, I was able to find a solution, and for the sake of the other readers I wanted to share it here. Here's what I did:

First, I filled out the form using chrome. After submitting the form, I used the developer tools in Chrome (on the landing page of the form) to look at the headers. That can be found on the Network tab after selecting the formResponse entry. For anyone following along, there should be a list of several header categories including General, Response Headers, Request Headers, and Form Data.

Another Side Note: all of the field names with the data you entered in the Google Form can be found under Form Data; I found that using this was a lot easier than hunting through html tag attributes.

Anyways, Request Headers is the one you want. Now, I'm not sure if you need every single field under the Request Headers, but I decided to use all of them. To do this, I just copied the plain text under Request Headers and assigned it to a variable like this:

header_text = '''
field_one: value
field_two: value
etc.
'''

Then, I parsed the string using this function:

def parse_headers(text):
    # Init dictionary
    headers = {}

    # Loop through lines
    for line in text.split("\n"):
        # Split field name and value up so they can be assigned
        header = line.split(": ")

        if len(header) < 2:
            continue # no ": ", so it's probably just an empty line
        elif len(header) > 2:
            print("Help!") # This shouldn't happen
        else:
            headers[header[0]] = header[1]

    return headers

This returns a dictionary with all of the fields and values under Request Headers. Now, let's parse the header and make the request (to submit the Google Form).

headers = parse_headers(header_text)
r = requests.post(url, data=d, headers=headers)

I hope this helps!

William Skellenger • Mar 12 '21 • Edited

I found this helpful, Simon. I did some experiments to figure out what parts of the headers were needed and I was able to get it down to only one: The cookie: line.

My concern is that this cookie will eventually expire, so hardcoding it may not be a long term solution.

James Shah • May 19 '20 • Edited

Hey Simon, Thank you for your feedback. I'm glad, it helped you. About Error 401, I didn't ran into this, and I think it's because of the url. Have you changed your google form url as mentioned in the post? You've to change the /viewform to /formResponse at the end of the url. I'm not sure if this will prevent the 401 error but I guess, this should work.

And about Form Data, thank you so much about that. Finding attributes from the developer tools is the most tiresome work in this script.

Again, thanks for your comment! And it feels good when someone uses your script and tells you how can you improve it!😄

Simon Richard • May 19 '20

I did use /formResponse. However, I didn't make the form, so there might be some setting differences... I'm not sure. Thanks again, though.

rizdaagisa • Feb 20 '21

how i can get the header text please?

Simon Richard • Feb 21 '21

Here's a screenshot:

It's under Network, click the fromResponse request, then make sure you're on the header tab and look for "Response Headers."

rizdaagisa • Feb 21 '21

Thanks a lot

dipakp2726 • Jun 24 '20

its not working in form which has set response receipt always,

James Shah • Jun 24 '20

No that's not the case. As I've tried it on such form which sends a response receipt.

Mike Kolcun • Feb 22 '21

I'm having the same issue with a test form. If I set "response receipts" to "Always" I only ever get error 400. if I set "response receipts" to "If respondent requests it" I am able to get the form to submit.

Any thoughts?

dipakp2726 • Jun 24 '20

try again bro, i have tried today, its working when i disable this option and getting error 400 when i activate it

Kang Min Zhe • Jul 7 '20

The reason why you are getting a response error of 400 is because when using the requests module, the posted data is automatically url encoded. Hence posting options that contains characters such as "+", "@" ..etc will cause a bad form error.

Hence, one way you can work around this is to have the request post http URL instead by passing a string.

string = "entry.xxx=Option+1&entry.xxx=option2"
r = requests.post(url, params = string)

Hope this helps.

DJDragonfruit • Nov 3 '20 • Edited

Hi, this is a really great guide! However I'm trying to fill out a google doc that is locked to access by members of a certain organization. Can I make this code automatically sign me into my school account before filling out the form? (I've looked through a gazillion solutions for this on GitHub and none of them seem to work. Chances are I'm using them wrong, but try as I might I can't figure out how.)

dipakp2726 • Jun 23 '20

Error occured HTTPSConnectionPool(host='docs.google.com', port=443): Max retries exceeded with url: /forms/d/1JxbxYl7ZnWTEKtYqVMQJOi_6_cZvTYrDbGsPeNNnGSY/formResponse?edit_requested=true (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available."))

i am having these error ,

sahilcodes-max • Mar 4 '21

I need help. I am trying to use this but for a multiple choice problem. For example, the question is 1 + 1.
The first option is 1, second is 2. How would we do this?
Image of my code: (I basicilly removed everything for a basic one question. )
Url: docs.google.com/forms/d/e/1FAIpQLS...

Immoralskun • Jun 5 '20

Good work! I needed to do something like this too with google forms. But, it didn't work because the form had different pages. Do you have a way of solving this? I would like to point out that the response is received for the first page but not received in the subsequent pages.
Your help will be very much appreciated! Thanks!!

James Shah • Jun 6 '20

Hey, Thank you for the appreciation but I'm afraid I don't know about submission of multipage google forms, but I guess you can search it on google and find a solution. In my opinion, if there isn't a solution for multi page, you can use selenium to automate the process. Hope this helps!

ananya-boop • Sep 30 '20

Hey, great work! I am trying to follow the same step but as I am a beginner, not really sure how to go about it. I have a form to fill every day and was thinking about how to automate it. It consists of several fields as well. Let me know if you could help in any way :)

Comment deleted

James Shah • Jun 23 '20

You can find it from the elements tab in dev tools. It's a pretty cumbersome task to find all the entry is, especially when there are lots of fields. In my opinion, you should take a look at the method suggested by Simon in the comments. It's very easy comparing to my entry id method.