How to download data in multiple file formats (CSV, XLS, TXT) with Django REST Framework

A lot of the time, our typical server response is in the form of JSON or XML. This serves our use cases a good number of times, however, there are times when the need to provide data in the form of a file arises.

In this article, we will be exploring how to convert our model data into files and send them as responses in Django REST Framework (DRF). We will do this by building out a simple project.

Prerequisites

You should have some basic Django and Django REST Framework knowledge. You should also have python already installed on your system.

The project, Setting up and Installation

We will be building out a simple Student Management App. The purpose of the app will be to:

1. Allow a user to input student data

2. Download all the available data as either a CSV, Excel or TXT file.

Creating the project and Installing Dependencies

We will install Django and DRF to start. To do this, first, create your virtual environment and activate it.

python3 -m venv studentappenv
source studentappenv/bin/activate

Now install Django and DRF. We will also be installing openpyxl which is a python package that helps when interacting with data and excel files.

pip install django djangorestframework openpyxl

Next, create the Django project and create an app.

django_admin startproject student_management .
python manage.py startapp student_data

Update your settings.py and include the new app inside the INSTALLED_APPS list.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    'student_data', #new line
]

Let's create our model. Head over to student_data/models.py

from django.db import models

class StudentData(models.Model):

    STUDENT_GENDER = ((1, 'Male'), (2, 'Female'), (3, 'Other'))
    STUDENT_LEVEL = ((1, 'Junior'), (2, 'Senior'))

    first_name = models.CharField(max_length=100)
    last_name = models.CharField(max_length=100)
    age = models.IntegerField()
    gender = models.IntegerField(choices=STUDENT_GENDER)
    level = models.IntegerField(choices=STUDENT_LEVEL)

A pretty simple model that stores basic student information. This will be good enough for our needs.

Next up, we create our serializer. Create a new file serializers.py inside the student_data folder and add the following inside.

from rest_framework import serializers
from .models import StudentData

class StudentDataSerializer(serializers.ModelSerializer):

    class Meta:
        model = StudentData
        fields = ' __all__'

    def to_representation(self, instance):
        data = super().to_representation(instance)

        data['gender'] = instance.get_gender_display()
        data['level'] = instance.get_level_display()

        return data

This is also a basic serializer for our StudentData model. We override our to_representation to change how the server would return values for the gender and level fields. Instead of returning something like 'gender': '1', we will instead get 'gender': 'Male'.

We have our model and serializer, now, let's create our view. Inside student_data/views.py, add the following lines of code:

from django.utils import timezone
from rest_framework.response import Response

from rest_framework.viewsets import ModelViewSet
from rest_framework.decorators import action
from .models import StudentData
from .serializers import StudentDataSerializer

class StudentDataViewset(ModelViewSet):
    serializer_class = StudentDataSerializer
    queryset = StudentData.objects.all()

    @action(detail=False, methods=["get"])
    def download(self, request):
        queryset = self.get_queryset()
        serializer = StudentDataSerializer(queryset, many=True)

        return Response(serializer.data)

This is our view which will handle POST requests to create our student data. We have also defined a new viewset action download which we will use for downloading our file responses. Right now it will simply return the data as a JSON response, we will tweak it soon to meet our needs.

Let's connect our URL paths. Create a urls.py file in the student_data folder.

# student_data/urls.py

from .views import StudentDataViewset
from django.urls import path, include

from rest_framework import routers

router = routers.DefaultRouter()
router.register("", StudentDataViewset, basename="student-data")

urlpatterns = [
    path("", include(router.urls)),
]

Connect this to the main urls file at student_management folder.

# student_management/urls.py
from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('student-data/', include('student_data.urls')),
]

Run makemigrations and migrate

python manage.py makemigrations
python manage.py migrate

Start the server with python manage.py runserver.

Make a few POST requests at http://127.0.0.1:8000/student-data/ to populate our database with some data.

Now test the http://127.0.0.1:8000/student-data/download/ endpoint with a GET request to see the response type. You will get JSON data as a response similar to what we saw from our POST request. This is because DRF sets a global default renderer rest_framework.renderers.JSONRenderer which will return server responses in application/json media type.

We will create our own renderers which will convert our data into the response formats we want.

Create a renderers.py file inside the student_data folder.

# student_data/renderers.py
import io
import csv
from rest_framework import renderers

STUDENT_DATA_FILE_HEADERS = ["id", "first_name", "last_name", "age", "gender", "level"]

class CSVStudentDataRenderer(renderers.BaseRenderer):

    media_type = "text/csv"
    format = "csv"

    def render(self, data, accepted_media_type=None, renderer_context=None):

        csv_buffer = io.StringIO()
        csv_writer = csv.DictWriter(csv_buffer, fieldnames=STUDENT_DATA_FILE_HEADERS, extrasaction="ignore")
        csv_writer.writeheader()

        for student_data in data:
            csv_writer.writerow(student_data)

        return csv_buffer.getvalue()

This is going to be the renderer that will convert our data into CSV format. Let's break this code down.

1. We set a STUDENT_DATA_FILE_HEADERS list which contains all our model fields.

2. Our CSVStudentDataRendererclass subclasses BaseRenderer and defines the media_type and format attributes. We have many different media types which you can find here at IANA defined media types (mime types).

3. We define a render method where a bulk of the work is done. Inside it:

• We create a string buffer named csv_buffer.

• We create an instance of csv.DictWriter and pass in our buffer as an argument.

• We call csv_writer.writeheader() which will write the values in STUDENT_DATA_FILE_HEADERS as the first line of our CSV file.

• We use the csv helper module to write our data as comma separated values in the buffer location.

• We retrieve the content written into our buffer with csv_buffer.getvalue()

Let's add renderers for Excel and Text files next. Update renderers.py file.

import io
import csv
import openpyxl # new
from rest_framework import renderers

STUDENT_DATA_FILE_HEADERS = ["id", "first_name", "last_name", "age", "gender", "level"]

...

# ------ NEW LINES --------
class TextStudentDataRenderer(renderers.BaseRenderer):

    media_type = "text/plain"
    format = "txt"

    def render(self, data, accepted_media_type=None, renderer_context=None):

        text_buffer = io.StringIO()
        text_buffer.write(' '.join(header for header in STUDENT_DATA_FILE_HEADERS) + '\n')

        for student_data in data:
            text_buffer.write(' '.join(str(sd) for sd in list(student_data.values())) + '\n')

        return text_buffer.getvalue()

class ExcelStudentDataRenderer(renderers.BaseRenderer):

    media_type = "application/vnd.ms-excel"
    format = "xls"

    def render(self, data, accepted_media_type=None, renderer_context=None):    

        workbook = openpyxl.Workbook()
        buffer = io.BytesIO()
        worksheet = workbook.active
        worksheet.append(STUDENT_DATA_FILE_HEADERS)

        for student_data in data:
            worksheet.append(list(student_data.values()))

        workbook.save(buffer)

        return buffer.getvalue()

The idea is the same as our CSVStudentDataRenderer. We define the right media_type and format of our expected file. We then create a buffer which we write our headers and model data into. After that, we return the content of the buffer. In the case of our ExcelRenderer, we use the openpyxl library to simplify the buffer writing process.

Now that we have our renderers, let's update our view to use it. Update your student_data/views.py file.

from django.utils import timezone
from rest_framework.response import Response

from rest_framework.viewsets import ModelViewSet
from rest_framework.decorators import action
from .models import StudentData
from .renderers import CSVStudentDataRenderer, ExcelStudentDataRenderer, TextStudentDataRenderer
from .serializers import StudentDataSerializer

class StudentDataViewset(ModelViewSet):
    serializer_class = StudentDataSerializer
    queryset = StudentData.objects.all()

    @action(detail=False, methods=["get"], renderer_classes=[CSVStudentDataRenderer, ExcelStudentDataRenderer, TextStudentDataRenderer])
    def download(self, request):
        queryset = self.get_queryset()

        now = timezone.now()        
        file_name = f"student_data_archive_{now:%Y-%m-%d_%H-%M-%S}.{request.accepted_renderer.format}"
        serializer = StudentDataSerializer(queryset, many=True)
        return Response(serializer.data, headers={"Content-Disposition": f'attachment; filename="{file_name}"'})

The three main takeaways from the new changes are:

1. We define a renderers_classes parameter in our @action decorator. This will be used by the download endpoint when choosing a renderer. How will it decide? we will talk about that in a minute.

2. We create a distinct file name based on the current time and append the format of the renderer which would be used to serve the particular response, found at request.accepted_renderer.format.

3. We set a content-dispostion header which lets the client know that the response should be treated as an attachment that should be downloaded under the value of filename.

Let's head back to postman to test our download endpoint.

I get a CSV file downloaded onto my system.

Great. This means Django decided to use our CSVRenderer, but how did it decide? Django has a content negotiation mechanism which it uses to determine how to render a response back to the client. It looks into the Accept header in the request and tries to map it to an available renderer. If it is unable to map it to any specific one, by default, Django chooses the first renderer in the renderers_classes list. This is why we get a CSV file in return.

Let's edit our Accept request header and specify the Excel media type(application/vnd.ms-excel) then retry the request.

I now get an Excel file downloaded.

And if we update the Accept header to text/plain we get a txt file downloaded as Django chooses the text renderer class.

That's it.

Conclusion

In this article, we learned how to create our own custom renderers for specific media types and formats like excel, CSV and text. We also saw how to help our application pick the right renderer using the Accept request header.

If you found this article useful or learned something new, consider dropping a heart up and following me to keep up-to-date with any recent postings!

You can also find me on Twitter at akinsola232 and on LinkedIn at Ademola.

Till next time, happy coding!

Levi