As a web developer, often times you will be responsible for creating an API, or Application Programming Interface. Quite simply, your API takes in requests and returns structured responses. API design is one of the most useful skills you can master though it involves a lot of different moving parts: authentication, authorization, rate limiting for requests, serializing the data from the database into a structured format, etc. We’ll walk through the most important parts of creating an API along with a framework to help us implement it in Django.

As an example, go to https://aws.random.cat/meow. This very simple API returns a random image url of a cat. You’ll see an output like the following in JSON format:

{"file": "https:\/\/purr.objects-us-west-1.dream.io\/i\/UwL2m.jpg"}

Project Goals

For this project, we will convert our Reddit application into an API and create an API explorer so we can easily test out some actions such as creating and editing a post, creating comments, creating subreddits, etc.

We will be using Django Rest Framework (DRF) to build our API. It has a ton of great features including a browsable API, authentication policies, serialization of our data, and much more.

Why use API’s?

Companies use APIs either for internal use or to expose their APIs to their clients. For example, Google offers a Maps API to let developers leverage their maps data.

With the proliferation of front-end web frameworks like Angular.js, Redux & React, and Vue.js, it’s become increasingly popular to have the backend be a simple API and the frontend be handled by one of these frontend frameworks.

Moreover, Single Page Apps (SPA’s) have become increasingly more popular. With SPA’s, there is no need to constantly request new pages from the server. All the rendering logic is fetched on the first page load by the user. After that as a user navigates the site, data is simply fetched via AJAX calls so the user has a smoother experience. AJAX calls are asynchronous so that the client requests the data in the background — there is no reloading of the page. This data fetching is generally done via…a backend API!

RESTful API’s

You’ll often hear the term REST associated with API’s. REST stands for REpresentational State Transfer. It has a 6 principles which need to be satisfied for an interface to be deemed RESTful. We won’t cover all 6 principles here, but you can read this for a quick overview.

Most RESTful API’s support the following important operations:

• Creating new data
• Retrieving data
• Updating data
• Deleting data

You may have heard of these operations referred to as CRUD.

Each of these operations is associated with its own HTTP method.

• Creating new data → POST
• Retrieving data → GET
• Updating data → PUT or PATCH
• Deleting data → DELETE

Perfect, now we’ll dive right into some features of DRF.

Serializers

Serializers are super helpful when we want to convert complex data such as querysets or instances of a model into something the consumer of our API can use. This will likely be JSON, XML, or another common content type.

Serializers also help provide deserialization, i.e. going from parsed data to model instances, querysets, etc. They also provide validation on this parsed data so we can ensure that the data is valid and in the format that we expect.

Viewsets

A ViewSet provides the logic for a set of actions such as create, retrieve, list, update, and destroy a model. DRF further provides default implementations of all these methods in their ModelViewSet, i.e. .list(), .retrieve(), .create(), .update(), .partial_update(), and .destroy().

DRF also provides mixins such as the RetrieveModelMixin. A mixin is just a simple class that provides a specific functionality — in this case, it’s the ability to retrieve a model. Sometimes we don’t want a model to be able to be updated or destroyed. In this case, we’ll specifically inherit from the mixins that we need.

Standard Attributes

• queryset — The queryset associated with a particular ViewSet. For example the UserViewSet may have queryset = User.objects.all().
• serializer_class — The serializer class (mentioned earlier) associated with a viewset. This will be used for creation, updates, retrieves, etc.
• permission_classes — the API policy for the viewset. I.e. can anyone access the ViewSet? Is it only limited to authenticated users (those who’ve logged in)? Is it only limited to users who have created a particular item?

Step 0: Installation

Download the proj4-starter code from here. (For Django 1.11 users, use the link here). It is the same as the final version of our Project 2 (Reddit) with the migrations, virtual environment, etc. removed.

Go to your requirements.txt located at proj4-starter/requirements.txt) file and add

djangorestframework==3.8.2

The final file should now look like:

django==1.11.0
djangorestframework==3.8.2

Next, go to your settings file located at proj4-starter/mysite/settings.py. Add rest_framework it to the INSTALLED_APPS like below.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'reddit',
    'rest_framework'
]

Let’s get the project all set up! Run the following commands in your terminal:

python3 -m venv myvenv
source myvenv/bin/activate
pip install -r requirements.txt
python manage.py makemigrations
python manage.py migrate
python manage.py createsuperuser

Step 1: Users API

Let’s go ahead and expose users through our API.

Create the Serializer

First, we’ll need to create the UserSerializer. Go to proj4-starter/reddit and create a file called serializers.py.

Copy in the following:

from .models import *
from django.contrib.auth.models import User
from rest_framework import serializers
class UserSerializer(serializers.ModelSerializer):
    class Meta:
        model = User
        fields = ('id', 'first_name', 'last_name')

For every DRF serializer that we define, we’ll have to create a Meta class that contains information about which model this serializer is associated with and the fields of the model to be used for the serializer, any read_only fields, any related fields, etc.

Our UserSerializer class is very simple so we only have to define the model and fields.

Note that users don’t have eid’s since we’re using Django’s built in User object. We’ll leave it as an exercise to you to use your own custom user model. (Hint: Assignment #1 covered this).

Create the ViewSet

Let’s go ahead and create our UserViewSet. Go to proj4-starter/reddit and create a file called viewsets.py.

Copy in the following:

from .models import *
from .serializers import *
from django.db.models import Q
from django.contrib.auth.models import User
from rest_framework import viewsets, mixins
from rest_framework.decorators import detail_route, list_route
from rest_framework.response import Response
from rest_framework import permissions
class UserViewSet(viewsets.GenericViewSet, mixins.RetrieveModelMixin):
    """
    API endpoint that allows users to be viewed.
    """
    queryset = User.objects.all().order_by('-date_joined')
    serializer_class = UserSerializer
    permission_classes = (permissions.IsAuthenticatedOrReadOnly,)

The first thing to notice is that our UserViewSet inherits from viewsets.GenericViewSet and mixins.RetrieveModelMixin instead of viewsets.ModelViewSet that we had mentioned earlier. The only thing we want to enable through the user API is to get the public details of a user, i.e. their first_name, last_name, and id.

We don’t want to allow user creation, editing, etc, hence we’ll just use a GenericViewSet and the RetrieveModelMixin.

Next, we define the queryset associated with this ViewSet. This will be used across all the various actions such as retrieve().

Next, we set the appropriate serializer_class, i.e. UserSerializer in our case.

Finally, we set the permission_classes to the (permissions.IsAuthenticatedOrReadOnly,) tuple. We don’t want non-logged in users to be able to make any changes, so this permission will ensure that. In this case, it doesn’t matter as much, since we only expose the retrieve action anyways. However, it will become a lot more important for our other viewsets that allow creating, editing, deleting, etc via the API. We’ll even create our own custom permissions later on!

Why don’t we want to allow our API to list all users?

We don’t want to reveal the list of all users who use our application, our API won’t expose that information.

Introducing the Browsable API & Testing it

Let’s test whether our API work!

First, we need to hook up the urls so that we can test our API. Go to reddit/urls.py and copy in the following.

Django 2.0 and above

from django.urls import path, include
from . import views
from . import viewsets
from rest_framework import routers
router = routers.DefaultRouter()
router.register(r'users', viewsets.UserViewSet)
router.register(r'subreddits', viewsets.SubRedditViewSet)
router.register(r'posts', viewsets.PostViewSet)
router.register(r'comments', viewsets.CommentViewSet)
urlpatterns = [
    path('api/v1/', include(router.urls)),
    path('api-auth/', include('rest_framework.urls', namespace='rest_framework')),
    ### From the last assignment
    path('', views.post_list, name='post_list'),
    path('post/<uuid:pk>/', views.post_detail, name='post_detail'),
    path('sub/<uuid:pk>/', views.sub_detail, name='sub_detail'),
    path('post/new/', views.post_new, name='post_new'),
    path('post/<uuid:pk>/edit/', views.post_edit, name='post_edit'),
    path('post/<uuid:pk>/comment/', views.add_comment, name='add_comment_to_post'),
    path('post/<uuid:pk>/comment/<uuid:parent_pk>/', views.add_comment, name='add_reply_to_comment'),
    path('content/<uuid:pk>/upvote/', views.vote, {'is_upvote': True}, name='upvote'),
    path('content/<uuid:pk>/downvote/', views.vote, {'is_upvote': False}, name='downvote')
]

Django 1.11

from django.conf.urls import url, include
from . import views
from . import viewsets
from rest_framework import routers
router = routers.DefaultRouter()
router.register(r'users', viewsets.UserViewSet)
urlpatterns = [
    url(r'^api/v1/', include(router.urls)),
    url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework')),
    ### From the last assignment
    url(r'^$', views.post_list, name='post_list'),
    url(r'^post/(?P<pk>[0-9a-f-]+)/$', views.post_detail, name='post_detail'),
    url(r'^sub/(?P<pk>[0-9a-f-]+)/$', views.sub_detail, name='sub_detail'),
    url(r'^post/new/$', views.post_new, name='post_new'),
    url(r'^post/(?P<pk>[0-9a-f-]+)/edit/$', views.post_edit, name='post_edit'),
    url(r'^post/(?P<pk>[0-9a-f-]+)/comment/$', views.add_comment, name='add_comment_to_post'),
    url(r'^post/(?P<pk>[0-9a-f-]+)/comment/(?P<parent_pk>[0-9a-f-]+)/$', views.add_comment, name='add_reply_to_comment'),
    url(r'^content/(?P<pk>[0-9a-f-]+)/upvote/$', views.vote, {'is_upvote': True}, name='upvote'),
    url(r'^content/(?P<pk>[0-9a-f-]+)/downvote/$', views.vote, {'is_upvote': False}, name='downvote')
]

Alright, back to the common instructions, regardless of Django version!

A few important things to note about our urls.py file

• On lines 6, we instantiate the DefaultRouter provided by DRF, and on line 7, we register our UserViewSet with the prefix users. We’ll cover more on this after the next point.
• On line 10, we include all of the routers url’s behind a prefix of api/v1/. We’re using a default of v1 here so that if you have multiple versions of the API running, you can easily manage it here. Imagine if you’re a public API and you keep providing new functionality or changing existing endpoints (or urls). You’d want a way to not break the existing applications that use your API, so you version your changes. For example, you may release the latest changes under api/v1.1/.
• On line 11, we add the endpoint for the authentication endpoints for using DRF’s browsable api. You can access these endpoints at http://localhost:8000/api-auth/login/.

Note that we’re not actually removing any of the older urls, so you can still continue to use your Reddit application like you did earlier. In a real production application, you wouldn’t generally have both an API and use the default views and template setup, but we keep both around so you can compare the different approaches.

So now if we want to access the list of all the users, we’d simply navigate to http://localhost:8000/api/v1/users/. Try going to this endpoint now! (Make sure your server is running with python manage.py runserver).

Alas — you should have gotten a Page not found (404) error. This is entirely expected. Remember we blocked all actions from our UserViewSet besides the retrieve action.

So, let’s try actually retrieving the details of our superuser. Go to http://localhost:8000/api/v1/users/1/. You should see something like the following:

Note that your user probably won’t have a first or last name (since you created the superuser via your terminal). You can always change this via the admin page (i.e. by going to http://localhost:8000/admin/auth/ user/).

You can click on the arrow next to the blue GET button on the right to see the different content types your API can serialize your response to. Try clicking on the JSON setting. You should see something like:

{"id":1,"first_name":"","last_name":""}

Most of the time you’ll be using JSON to interface with whatever frontend framework you’re using.

Step 2: SubReddit API

Create the initial Serializer

Great, now we’ll move on to creating our SubRedditSerializer. Let’s create a few helper classes for ourselves first.

We know that all of our Post, Comment, and SubReddit objects have a UUID as their primary key. We want our serializer to out the string version of this UUID string for all of these models. So, lets create a BaseSerializer that everyone will inherit from. Add this to your serializers.py that we created earlier.

class BaseSerializer(serializers.ModelSerializer):
    eid = serializers.UUIDField(read_only=True)

We haven’t declared any class Meta since that will be defined in all the children classes. Here, we simply let DRF know that the eid field will be a UUIDField that is read_only, i.e. it can’t be modified by the API.

Let’s create the SubRedditSerializer now.

class SubRedditSerializer(BaseSerializer):
    posts = serializers.PrimaryKeyRelatedField(many=True, required=False, read_only=True)
    moderators = serializers.PrimaryKeyRelatedField(many=True, queryset=User.objects.all(), required=True, read_only=False)
    class Meta:
        model = SubReddit
        fields = ('eid', 'name', 'cover_image_url', 'posts', 'moderators')

This can be a bit confusing, so let’s walk through this step by step.

• We declare posts as a PrimaryKeyRelatedField but since a SubReddit can have multiple posts, we pass in many=True. It is not required to be provided by the API and is thus read_only. The PostSerializer that we will create later will handle adding the post to the various subreddit’s.
• moderators is very similar to posts, except that we DO want the API to accept a list of users. Whenever a field is not just read_only, we must provide a queryset argument so that DRF can validate whether the passed in values are actually User objects and automatically lookup the objects associated with these eid’s. In our case, queryset is just defined as User.objects.all().

We only have to explicitly declare fields when we want to customize them in some way. Hence you can see that we provided a lot of fields to the Meta class such as name, cover_image_url, etc, that we don’t need to explicitly declare.

Great, the basic serializer is now done, but we still have a few things to add to it.

Let’s think about creation first. We want to ensure that the list of moderators that is passed in exists and has at least one user’s eid. We can make use of DRF’s validation capabilities to do this.

Simply define a validate_moderators method, that just checks that the value exists and it has more than one element. If not, it throws a ValidationError.

class SubRedditSerializer(BaseSerializer):
    ...
    class Meta:
        model = SubReddit
       ...
    ###### ADD THIS METHOD ###########
    def validate_moderators(self, value):
        if not value or len(value) == 0:
            raise serializers.ValidationError('Need to include at least one moderator!')
        return value
    ##################################

DRF will automatically associate the validate_moderators method with the moderators field and will pass that method the proper value. It just looks for a method matching validate_x where x is the field name. A little DRF magic at play!

Great, now the final step before we can create a post through the API is to make sure we add the moderators to all the newly created subreddit. Let’s create the create method!

class SubRedditSerializer(BaseSerializer):
    ...
    def validate_moderators(self, value):
        ...
    ###### ADD THIS METHOD ###########
    def create(self, validated_data):
        user = self.context['request'].user
        moderators = validated_data.pop('moderators')
        if user not in moderators: moderators.append(user)
        subreddit = SubReddit.objects.create(**validated_data)
        for mod in moderators: subreddit.moderators.add(mod)
        return subreddit
    ##################################

The create method will be passed the validated_data. This validated_data has gone through all the validate methods that have been defined as well as any default validation provided by the various fields (for example the queryset we defined on the moderators field will automatically ensure that the passed in eid values are actually User objects).

On line 8, we figure out who the currently logged in user is. Whenever we instantiate a serializer in one of our ViewSet’s, or if it is automatically done via one of the mixins, the serializer is passed a context, which is a dictionary. This dictionary includes the request object. So, we can just access the logged in user via self.context['request'].user.

Next, on line 9, we remove the moderators data from the validated_data dictionary. We will be passing this dictionary to SubReddit.objects.create on line 7, and it will throw an error since it won’t know what to do with the moderators list.

Now this moderators variable doesn’t just contain a list of user eid‘s. After it has gone through validation, DRF has automatically queried the eid’s and found the associated objects. So now, moderators refers to a list of actual User model instances.

Next, we ensure that the passed in list of moderators contains the user who is currently logged in. If it doesn’t, we’ll automatically add the user. This makes sense since we want the user who created a subreddit to automatically be it’s moderator.

Finally we simply create both the SubReddit object and add all of the moderators to the newly created subreddit object.

Create the initial ViewSet

Add the SubRedditViewSet class in reddit/viewsets.py.

class BaseSerializer(serializers.ModelSerializer):
    ...
class UserSerializer(serializers.ModelSerializer):
    ...
class SubRedditViewSet(viewsets.GenericViewSet, mixins.RetrieveModelMixin, mixins.CreateModelMixin, mixins.UpdateModelMixin):
    """
    API endpoint that allows SubReddits to be viewed or edited.
    """
    queryset = SubReddit.objects.all()
    serializer_class = SubRedditSerializer
    permission_classes = (permissions.IsAuthenticatedOrReadOnly,)

Note that we want to be able to create, update, and retrieve the SubReddit model so we’ll inherit from GenericViewSet, RetrieveModelMixin, CreateModelMixin, and UpdateModelMixin.

Next, go to reddit/urls.py and register the subreddit urls.

router = routers.DefaultRouter()
router.register(r'users', viewsets.UserViewSet)
## ADD THIS
router.register(r'subreddits', viewsets.SubRedditViewSet)

Testing Creating SubReddits

Finally, we’re now ready to test creating our first subreddit.

Navigate to http://localhost:8000/api/v1/subreddits/. You’ll have to log in by clicking the ‘Log in’ button in the black topbar on the right. Simply enter your superuser’s username and password.

You’ll notice that when you navigate to http://localhost:8000/api/v1/subreddits/, you see a HTTP 405 Method Not Allowed. This is great because we explicitly blocked the list action by not inheriting from ListModelMixin in our SubRedditViewSet.

Click on the Raw Data tab, leave the Media type as application/json. Let’s enter in some data for our first subreddit!

Enter in the following into the Content section.

{
    "name": "Our first SubReddit",
    "cover_image_url": "https://www.aspcapetinsurance.com/media/1064/mountain-dog.jpg",
    "moderators": []
}

Click on the “Post” button. Remember our discussion earlier of HTTP Verbs, where POST corresponds to the creation of new objects.

You should have received a HTTP 400 Bad Request error. These are HTTP response codes and let you know the status of your request. 400 corresponds to a Bad Request.

HTTP 400 Bad Request
Allow: POST, OPTIONS
Content-Type: application/json
Vary: Accept
{
    "moderators": [
        "Need to include at least one moderator!"
    ]
}

This is expected because you didn’t pass in any moderators!

Let’s modify this to actually pass in a moderator. Enter in the following into the Content section:

{
    "name": "Our first SubReddit",
    "cover_image_url": "https://www.aspcapetinsurance.com/media/1064/mountain-dog.jpg",
    "moderators": [1]
}

After you click ‘Post’, you should see a successful output with a HTTP 201 Created response.

HTTP 201 Created
Allow: POST, OPTIONS
Content-Type: application/json
Vary: Accept
{
    "eid": "97f56643-36f6-4330-810b-a974c589ef07",
    "name": "Our first SubReddit",
    "cover_image_url": "https://www.aspcapetinsurance.com/media/1064/mountain-dog.jpg",
    "posts": [],
    "moderators": [
        1
    ]
}

Notice how our output has an automatically generated eid, and returns an empty list for posts since we haven’t created any posts yet. Keep this eid handy as we’ll use it to test out our SubReddit modifications in a bit.

Adding Functionality for Modifying SubReddits

We want to allow our API to support modifying subreddits, such as it’s name and cover_image_url. Let’s assume we also want to block it from modifying the list of moderators.

How do we instruct the API to allow moderators to be set when creating a SubReddit object, but not allow it to be modified when it’s being updated? Since we’ll likely run into this issue multiple times with other serializers as well, let’s try to bake this functionality into the BaseSerializer.

Add the following update method to your BaseSerializer class.

class BaseSerializer(serializers.ModelSerializer):
    eid = serializers.UUIDField(read_only=True)
    def update(self, instance, validated_data):
        if hasattr(self, 'protected_update_fields'):
            for protected_field in self.protected_update_fields:
                if protected_field in validated_data:
                    raise serializers.ValidationError({
                        protected_field: 'You cannot change this field.',
                    })
        return super().update(instance, validated_data)

The basic logic is the following — whenever a serializer inherits from BaseSerializer , it can choose to define a protected_update_fields variable which will be a list of field names that shouldn’t be allowed in any modification call via the API.

We first check if the serializer object has the field defined on line 5. Next, we just loop through the list of fields, and if it is included in validated_data (meaning it was included in the API call), we raise a ValidationError. Otherwise, we just call super and have it do it’s default update behavior.

All we’ve done here is overriden the default update method to do our checks.

Great, let’s add protected_update_fields to SubRedditSerializer. Here is the final version:

class SubRedditSerializer(BaseSerializer):
    posts = serializers.PrimaryKeyRelatedField(many=True, required=False, read_only=True)
    moderators = serializers.PrimaryKeyRelatedField(many=True, queryset=User.objects.all(), required=True, read_only=False)
    ### ADD THIS ##################
    protected_update_fields = ['moderators']
    ###############################
    class Meta:
        ...

Now when we’re updating a SubReddit, we only want to allow the moderator to make changes, otherwise we’ll want to raise a permissions error. We can use the built-in DRF Permissioning framework.

Remember, how we used permissions.IsAuthenticatedOrReadOnly earlier? We’ll now create our custom implementation of a permission similar to that.

To implement a custom permission, we will override BasePermission and implement either, or both, of the following methods:

• .has_permission(self, request, view) — this is run against all requests from viewsets that have this permission attached to them.
• .has_object_permission(self, request, view, obj) — this is only run against particular object instances from viewsets that have this permission attached to them.

For our case, we’ll only need to override has_object_permission, since we only want to block access for modifying existing subreddits.

Create a file called permissions.py inside the reddit folder, and paste in the following:

from rest_framework import permissions
class IsModeratorOrReadOnly(permissions.BasePermission):
    """
    Object-level permission to only allow owners of an object to edit it.
    Assumes the model instance has an `owner` attribute.
    """
    def has_object_permission(self, request, view, obj):
        # Read permissions are allowed to any request,
        # so we'll always allow GET, HEAD or OPTIONS requests.
        if request.method in permissions.SAFE_METHODS:
            return True
        # Instance must have an attribute named `owner`.
        return obj.moderators.filter(id=request.user.id).exists()

The implementation of the method is fairly straightforward. On line 12, we check if the request method is safe (i.e. a GET request) that doesn’t cause anything to be changed in our database, and if so, we allow everyone to access it by returning True. Otherwise, we check if the currently logged in user is part of the moderators for this subreddit. The exists command for the queryset will return True if request.user is part of the moderators set or False otherwise.

Let’s make sure this permission is now being used! Go back to viewsets.py and add IsModeratorOrReadOnly to the permission_classes for the SubRedditViewSet. Make sure to import it first!

from .permissions import IsModeratorOrReadOnly
class SubRedditViewSet(viewsets.GenericViewSet, mixins.RetrieveModelMixin, mixins.CreateModelMixin, mixins.UpdateModelMixin):
    """
    API endpoint that allows SubReddits to be viewed or edited.
    """
    queryset = SubReddit.objects.all()
    serializer_class = SubRedditSerializer
    permission_classes = (permissions.IsAuthenticatedOrReadOnly, IsModeratorOrReadOnly)

Testing Modifying SubReddits

Let’s test! We’ll use the eid of our SubReddit that we had saved earlier.

Navigate to http://localhost:8000/api/v1/subreddits/97f56643-36f6-4330-810b-a974c589ef07/. Replace this eid with whatever your’s was.

Enter in the following into the Content section of the Raw Data tab.

{
    "name": "Our first SubReddit - Modified"
}

Click on the “Patch” button. Remember our earlier discussion of HTTP Verbs, where PATCH corresponds to the modification of new objects.

You should have received a HTTP 200 OK response.

HTTP 200 OK
Allow: GET, PUT, PATCH, HEAD, OPTIONS
Content-Type: application/json
Vary: Accept
{
    "eid": "97f56643-36f6-4330-810b-a974c589ef07",
    "name": "Our first SubReddit - Modified",
    "cover_image_url": "https://www.aspcapetinsurance.com/media/1064/mountain-dog.jpg",
    "posts": [],
    "moderators": [
        1
    ]
}

Perfect, this all seems to work!

You can ensure that another user can’t modify this subreddit by creating another user account (you could just create another superuser for now), logging in as that user, and then trying to go to the url above. You won’t even see an option to enter in data, as DRF already knows your new user doesn’t have permissions to edit!

Part 1 Completed!

Congratulations!! In the next Part, you’ll get a lot of hands on practice creating your own API for Posts, Comments, as well as the voting logic. Let’s move on to Part 2.