Using Django Rest Framework we can create rest endpoints, which can be used to create, read, update and delete (CRUD) objects. However, those endpoints are limited to modifying one object at a time. This becomes a huge problem when we want to create and update multiple objects at a time.
In this article, we are going to call endpoints only once instead of thousands of calls to endpoints for thousands of objects. By the end of this article, you should be able to perform bulk operations in the most efficient way.
So without further ado let’s get started!!
1. We will implement Create API using standard ListCreateView.
2. We will use ListSerializer
class by customizing its behavior and bulk_create
and bulk_update
methods to handle multiple data input objects.
Create a new Django project named app
, then start a new app called inventory
.
# Create the project directory mkdir Inventory cd Inventory
# Create a virtual environment to isolate our package dependencies locally
python3 -m venv venv
source venv/bin/activate # On Windows use `venv\Scripts\activate`
# Install Django and Django REST framework into the virtual environment
pip install django
pip install djangorestframework
# Set up a new project with a single application
django-admin startproject app . # Note the trailing '.' character
python manage.py startapp inventory
Add `'rest_framework' and 'inventory'` to your `INSTALLED_APPS` setting.
INSTALLED_APPS = [
...
'rest_framework',
'inventory',
]
We will implement the first approach as per what we mentioned in the objectives.
Data Model: models.py
class Product(models.Model):
title = models.CharField(max_length=255)
description = models.TextField(max_length=500)
price = models.IntegerField()
type = models.CharField(max_length=255)
visible = models.BooleanField(default=False)
discount = models.IntegerField()
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
def __str__(self):
return self.title
Serializer: serializers.py
1. Serializers allow complex data such as querysets and model instances to be converted to native Python datatypes that can then be easily rendered into JSON
, XML
, or other content types. Serializers also provide deserialization, allowing parsed data to be converted back into complex types, after first validating the incoming data.
2. For this example, we will create ProductSerializer
which inherited from ModelSerializer
.
from rest_framework import serializers
from inventory import models
class ProductSerializer(serializers.ModelSerializer):
class Meta:
model = models.Product
fields = ["id", "title", "description", "price", "type", "visible", "discount", "created_at", "updated_at"]
Views: views.py
1. The generic views provided by the REST framework allow you to quickly build API views that map closely to your database models.
2. If the generic views don't suit the needs of your API, you can drop down to using the regular APIView
class, or reuse the mixins and base classes used by the generic views to compose your own set of reusable generic views.
3. For this example, we will create ProductView
which inherited from ListCreateAPIView
.
from rest_framework import generics, status
from rest_framework.response import Response
from inventory import models, serializers
class ProductView(generics.ListCreateAPIView):
queryset = models.Product.objects.all()
serializer_class = serializers.ProductSerializer
def create(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data, many=True)
serializer.is_valid(raise_exception=True)
try:
self.perform_create(serializer)
return Response(serializer.data, status=status.HTTP_201_CREATED)
except:
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
URLs: urls.py
1. Create API URLs
from django.urls import path, include
from inventory import views
urlpatterns = [
path('product-bulk-create-update/', views.ProductView.as_view(), name='product-bulk-create-update')
]
Result
1. In this approach, we overrided the create
method and explicitly mentioned many=True
in the serializer in order to handle multiple data input objects at the same time.
2. Even though we set many=True
in the serializer, It took half a minute (531 milliseconds) to insert data consisting of 100 objects. In this approach, create
method gets called 100 times to create 100 objects.
3. We have used Insomnia, an API client, to test our API endpoints.
4. We will implement the second approach using the ListSerializer
class to speed up the performance.
5. The code inside models.py and urls.py remains the same, we don't need to modify it.
6. ListSerializer
the class provides the behavior for serializing and validating multiple objects at once. The list serializer will allow you to submit a request for multiple creates in a single call.
7. To do this, let's override the get_serializer
method of the ListCreateAPIView
class to check whether the input data is list or not. If it is a list then set the property of kwargs["many"]=True
. By setting this property it will tell the serializer that it should use the list_serilizer_class
before calling individual creates for each object.
8. Now let's modify our ProductView inside of views.py.
class ProductView(generics.ListCreateAPIView):
serializer_class = serializers.ProductSerializer
def get_serializer(self, *args, **kwargs):
if isinstance(kwargs.get("data", {}), list):
kwargs["many"] = True
return super(ProductView, self).get_serializer(*args, **kwargs)
9. Customizing ListSerializer
behavior
10. There are a few use cases when you might want to customize the ListSerializer
behavior. For example:
11. You want to provide particular validation of the lists, such as checking that one element does not conflict with another element in a list.
12. You want to customize the create or update behavior of multiple objects.
13. For these cases, you can modify the class that is used when many=True
is passed, by using the list_serializer_class
option on the serializer Meta
class.
14. The default implementation for multiple object creation is to simply call .create()
for each item in the list. If you want to customize this behavior, you'll need to customize the .create()
method on the ListSerializer
class that is used when many=True
is passed.
15. Let's modify the ProductSerializer inside of serializers.py
class ProductBulkCreateUpdateSerializer(serializers.ListSerializer):
def create(self, validated_data):
product_data = [models.Product(**item) for item in validated_data]
return models.Product.objects.bulk_create(product_data)
class ProductSerializer(serializers.ModelSerializer):
class Meta:
model = models.Product
fields = ["id", "title", "description", "price", "type", "visible", "discount", "created_at", "updated_at"]
read_only_fields = ['id',]
list_serializer_class = ProductBulkCreateUpdateSerializer
16. That's it. We have implemented efficient bulk creation by customizing ListSerializer
class behavior.
1. The main advantage of ListSerializer
is that it uses bulk_create
method instead of save
on objects.
2. By using ListSerializer
and bulk_create
, It took less than 80 milliseconds to insert data consisting of 100 objects within a single POST request.
We have seen efficient bulk create by customizing the ListSerializer
class behavior and how it can be used to increase our app performance. A full working project can be found on GitHub in the Django bulk create a repository.
That's all for today. Stay tuned for the next part where learn how to implement bulk updates using Django Rest Framework.