Happy Thursday! I hope things are going well. I have enjoyed working through some (ha, many) issues while learning Flask to integrate Python with Twilio's API. This week I will focus on the issues and learnings I had with using both Flask and Twilio for the first time.
At the moment, I spend time logging migraines for the doctors office but don't remember the format she wanted, or the details she found important. This application is being created as a way to request a survey for a migraine through SMS, quickly fill out the details, and log the data for later use at the doctors appointment.
To clone the code, please visit the GitHub project here. This project is a current work in progress and is not fully functional at this time, the app will walk through a survey but data is not currently saved anywhere.
The first issue I ran into was connecting to a port when starting my application using
python3 -m app run. I have run into this issue in the past when I use to work on servers and would get similar messages to the one below.
socket.error: [Errno 48] Address already in use
This just means a process is bound to the port I am trying to use. This is caused by the same Python app being called before and that process still being bound to the port. To fix this, I looked at what Python processes were still running using
ps -fA | grep python.
Using this, you can spot the running Python process that is still active. You may want to test if
http://localhost:<port>/ still shows a directory listing for local files. The second number is the process number that you will need to stop. This can be done using the command
kill as in the example below:
ps -fA | grep python >> 501 39548 16817 0 9:50PM ttys001 0:00.20 Python -m app run kill 39548 >> + Terminated: 15 python3 -m app run
What did I learn here? This is caused by the same Python app being called before and that process still being bound to the port. This has only happened twice so far, and it is a simple and quick fix to just kill the process.
The next issue was one I didn't realize was happening until I ran into it. When running the application, I would answer the first question sent by the survey and received the response below:
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) 127.0.0.1 - - [08/Apr/2020 18:09:20] "POST /sms HTTP/1.1" 200 - [2020-04-08 18:09:20,296] ERROR in app: Exception on /question/1 [GET] TypeError: cannot unpack non-iterable NoneType object 127.0.0.1 - - [08/Apr/2020 18:09:20] "GET /question/1?
After an initial search into this issue, it was clear that results were all over the place. With that, I looked into the first question and coded up what I expected to be happening. I set the
question_id to 1, read in the JSON for the survey, and passed the question ID into the survey to grab the question information for it. After returning the first question and its type, I printed the information.
from parse import parseJson question_id = 1 surv = parseJson('../survey.json') question, type = surv.question_metadata(question_id) print(question) print(type)
When running this code, it produced the expected results, therefore something was wrong with the way the
question_id was being passed. To recreated the problem outside of the Flask application, I changed the
question_id to a string instead of int. Rerunning that code, reproduced the expected error:
TypeError: cannot unpack non-iterable NoneType object
What did I learn here? Therefore, the issue had been resolved. It was a type issue in which I was expecting an integer and receiving a string instead. Don't assume the type of the value being passed in, make it explicit.
As I had mentioned, I have not used Flask before, so I expected to end up with issues learning how to use it. The main issue I had when learning Flask was how to use session secrets. After resolving the above issues, the next that came up was the about not having a session secret setup.
"The session is unavailable because no secret " RuntimeError: The session is unavailable because no secret key was set. Set the secret_key on the application to something unique and secret.
Sessions allow you to store user data from one request to the next. This was needed in the survey in order to store and track the question that the user was on and iterate to the next question.
What did I learn here? When researching sessions, I learned that the secret key is used in a session to implement on top of the cookies and is used so that the data cannot be modified without the key used for signing. Here is more information on Sessions.
The last issue I had came about when I was interacting with the code I developed previously to grab the JSON survey and read it. When implementing this code and trying to get to the next question in the survey, I ran into two main problems I had when working with the survey questions:
- How to find out if there was another question in the JSON or not based on the currently stored
- What does the app do if there is another question? What does the app do if there is not another question?
With that, let's take a look at question 1, how do you use
question_id to determine if another question exists. To start, I looked back at the function I created in
parse.py to see how I was gathering the data and was using a filter object which grabbed out the question whose ID matched and would return the body and type of the data.
try: survey_dict = self._json_to_dict()['questions'] data = list(filter(lambda item: item['id'] == question_id, survey_dict)) if data: return data['body'], data['type'] except: print("The index povided for the question does not appear to exist.")
After some further reading into this section of the code, I changed the line that filters to a way that will return a list instead of filter object. The line
filter(function, iterable) is equivalent to
[item for item in iterable if function(item)] which means I can change the code to substitute:
data = [item for item in survey_dict if (item['id'] == question_id)]
This is a more Pythonic way to find the next question in the JSON that we read in.
With that, we can get onto question 2, what does the app do now that it has the data for the next question? The data returned for the next question is either
None or the next question's information. Using the
answer() function reads in the question ID given, (example: question 1 returns 1) and then iterates to the next questions ID by 1. Therefore, when question 1 is provided, the next ID would be 2. We use that ID, 2, and feed it into the
surv.question_metadata() function which looks to see if the data returned is None or another question.
Before proceeding with that data collected, the
answer() function extracts out the content from the previous question and stores it, which will be discussed more next week. After storing the data, the function looks to see if the data for the next question is None, which will end the survey, OR the if the data contains the next question, it will then pass the ID of the question and pass the ID to
redirect_twiml() which will handle the response to the user.
@app.route("/answer/<question_id>", methods=["POST"]) def answer(question_id): id = int(question_id) + 1 # Cast to an int, comes in as a string data = surv.question_metadata(id) # Return first question data_collected.append(str(extract_content())) # Append the data collected if data: return redirect_twiml(id) else: return goodbye_twiml()
What did I learn here? After working through these two problems, I gained a better understanding in how Flask operates to redirect from one function to the next and how I needed to setup the survey to redirect correctly as the user answered each question. During this time, I began to understand more about
Sessionsas well, which I talked about earlier. The smallest tidbit of info I took away from this issue was that the line
filter(function, iterable)is equivalent to
[item for item in iterable if function(item)]which is a more Pythonic way to work with the data.
So far, I have worked on the code needed to interact with Google Sheets API and Twilio API in a Flask application...
Stay tuned for next weeks post where I discuss how I integrate the two API's together in the Python Flask app that generates a survey for migraines. I look forward to sharing with you the issues and learnings I have encountered while learning to use Flask with these API's!