Sunday, 24 May 2015

Serializing Self Referential Model Recursively in Django REST Framework

Consider a Comment model with replies to it. Each comment can have multiple replies. Or in other words each reply may have a parent comment to which it is posted as a reply. Then how to serialize recursively? 

I can't use the following code - 

class CommentSerializer(serializers.ModelSerializer):
    replies = CommentSerializer(many=True)    
    
    class Meta:
        model = Comment
        fields = ('content', 'stamp_date', 'replies')

Because as the 1st line of the class is reached, till then CommentSerializer is not completely defined and this makes it as compile time error.

Whenever you find yourself in this soup, the solution is simple. Just have the following special recursive serializer defined in your serializers.py

class RecursiveField(serializers.Serializer):
    def to_representation(self, value):
        serializer = self.parent.parent.__class__(value, context=self.context)
        return serializer.data

And in your CommentSerializer simply use it - 

class CommentSerializer(serializers.ModelSerializer):
    replies = RecursiveField(many=True)

    class Meta:
        model = Comment
        fields = ('content', 'stamp_date', 'replies')

Monday, 4 May 2015

CORS in Django REST Framework

Django REST Framework doesn't allow cross browser requests via AJAX. When we build a REST API it is usually exposed on a separate server and the applications using it have their own servers on which they run. Hence when we hit any of the REST URL via AJAX we get No 'Access-Control-Allow-Origin' header is present on the requested resource error. For instance,
Ext.Ajax.request({    
 url: 'http://localhost:8080/portal/categories',    
 method: 'GET',
 success: function(response){console.log(response)},
 failure: function(){console.log('failure');}
});

the above request, will result into following error.

XMLHttpRequest cannot load http://localhost:8080/portal/categories?_dc=1430698970456. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost' is therefore not allowed access.

This is because of no provision in the REST API to handle Cross-Origin-Resource-Request (CORS). To enable it in the REST API, we need django-cors-headers as recommended in Django REST Framework documentation. The minimal settings required are as follows.

Step 0 - install django-cors-headers,
>>> pip install django-cors-headers

Step 1 - In settings.py, add the following entry in INSTALLED_APPS tuple.
INSTALLED_APPS = (
 ...
 ...

 'corsheaders'
)

Step 2 - In settings.py, add the following entries at the top of MIDDLEWARE_CLASSES tuple.
MIDDLEWARE_CLASSES = (
 'corsheaders.middleware.CorsMiddleware',
 'django.middleware.common.CommonMiddleware',
 
 ...
 ...
) 

Step 3 - Add the following flag variable in settings.py
CORS_ORIGIN_ALLOW_ALL = True 

We are done !!

Now you can again make the AJAX request as given in the beginning of this post & you will get the response in success handler as follows.
{
 requestId: 7
 responseText: "[{"name":"Database","href":"/portal/categories/1/scripts"},{"name":"RFB","href":"/portal/categories/2/scripts"},{"name":"Info Miner","href":"/portal/categories/3/scripts"}]"
 responseXML: null
 status: 200
 statusText: "OK"
 
 ...
 ...
}

For more configuration options visit django-cors-headers.

Sunday, 3 May 2015

Django - Model Operations: Part 1

This post will demonstrate performing DML (data manipulation language) operations on the model structure created in previous post, Django - Creating Models. Its rudimentary focus is record insertion. 

Start shell by executing following command.
>>> python manage.py shell

Following commands are needed to perform initial setup. If not executed then you will not be able to refer to an entity via foreign key while creating a record. E.g. referring to production house while creating movie.
>>> import django
>>> django.setup()
>>> from bollywood.models import *

Let's insert actor records,
>>> actor = Actor()
>>> actor.first_name = 'Shahrukh'
>>> actor.last_name = 'Khan'
>>> actor.gender = 'M'
>>> import datetime
>>> actor.dob = datetime.date(1965,11,2)
>>> actor.save()

actor.save() is important because only after that the record is inserted into the database table.

Another way to insert a record is -
>>> Country.objects.create(name = 'USA')

This will create the record & also insert it into database table.

To do a select *
>>> Actor.objects.all()
[<Actor: Aamir Khan>, <Actor: Salman Khan>, <Actor: Akshay Khanna>, <Actor: Shahrukh Khan>]

To get a specific record,
>>> Actor.objects.get(first_name='Salman')
<Actor: Salman Khan>

Inserting records for Aishwarya Rai & Zarine Khan, both were introduced by Salman Khan,
>>> ar = Actor()
>>> ar.first_name = 'Aishwarya'
>>> ar.last_name = 'Rai'
>>> ar.dob = datetime.date(1973,11,1)
>>> ar.gender = 'F'
>>> ar.introduced_by = Actor.objects.get(first_name='Salman')
>>> ar.save()

>>> zk = Actor()
>>> zk.first_name = 'Zarine'
>>> zk.last_name = 'Khan'
>>> zk.gender = 'F'
>>> zk.dob = datetime.date(1987,5,14)
>>> zk.introduced_by = Actor.objects.get(first_name='Salman')
>>> zk.save()

Note that the Assignment of introduced_by would failed if you don't do django.setup() initially.

Now to find all the actors whom Salman has introduced,
>>> Actor.objects.get(first_name='Salman').introduced.all()
[<Actor: Aishwarya Rai>, <Actor: Zarine Khan>]

We can use introduced because while creating the self referential foreign key, we have specified related_name='introduced'. (Click here to find the mode definition)

Creating an entry in production house,
>>> ProductionHouse.objects.create(banner='Aamir Khan Productions', start_date=datetime.date(2001,1,1))
<ProductionHouse: Aamir Khan Productions>

Inserting a movie into the table,
>>> m = Movie()
>>> m.title = 'Taare Zameen Par'
>>> m.release_date = datetime.date(2007,12,21)
>>> m.production_house = ProductionHouse.objects.get(banner__startswith='Aamir')
>>> m.save()

Note that none of the many-to-many field values can be set till the object is saved i.e. inserted into database. The reason is for many-to-many fields, a separate table is created and to make entry into that table, primary keys are needed & PKs are not generated till the object is saved.

To set the country (a movie can release in multiple countries & country can have multiple movie releases in it)
>>> m.release_country.add(Country.objects.get(name='India'))

In this way you can add all the countries in which the movie is released.

To find which all movies are released in 'India',

>>> Country.objects.get(name='India').movie_set.all()
[<Movie: Kal Ho Na Ho>, <Movie: Taare Zameen Par>]

As related_name field is not specified in Movie model while referring to Country, hence from country to get all movies, it is model name i.e. movie appended with _set (movie_set)

A movie can have multiple actors and an actor can act in multiple movies. Each actor charges some amount for each movie. Hence we ahave defined a many-to-many relation between movie and actor via a through model MovieActor. Aamir Khan is the actor in the movie 'Taare Zameen Par' & he charged 52,00,000. Therefore
>>> tzp = Movie.objects.get(title__startswith='Taare')
>>> ak = Actor.objects.get(first_name='Aamir')
>>> MovieActor.objects.create(movie=tzp, actor=ak, charges=520000)
<MovieActor: Aamir Khan - Taare Zameen Par>

Another way of inserting such a record can be,
>>> srk = Actor.objects.get(first_name='Shahrukh')
>>> khnh = Movie.objects.get(title__startswith='Kal')
>>> assignment = MovieActor()
>>> assignment.actor = srk
>>> assignment.movie = khnh
>>> assignment.charges = 600025.5
>>> assignment.save()

And then you can see all the actors in the movie,
>>> khnh.actors.all()
[<Actor: Akshay Khanna>, <Actor: Shahrukh Khan>]

With this tutorial we have corely covered insertion of records. In the upcoming post we will focus on select querys via Django's ORM.

Django - Creating Models

To build familiarity with Models and ORM (Object relational mapping) offered by Django, we are going to use a bollywood relational structure depicted in the following ERD.


We will express the above entities, their attributes and relationships as Django models. 

Module - django.db.models

Actor

class Actor(models.Model):
    gender_choices = (
        ('M', 'Male'),
        ('F', 'Female')
    )

    first_name = models.CharField(max_length=30)
    last_name = models.CharField(max_length=30, null=True, blank=True)
    gender = models.CharField(max_length=1, choices=gender_choices),
    dob = models.DateField()
    is_active = models.BooleanField(default=True)
    rec_created_date = models.DateTimeField(auto_now_add=True)
    last_updated = models.DateTimeField(auto_now = True)
    base_salary = models.DecimalField(max_digits=8, decimal_places=2, default=1000)
    introduced_by = models.ForeignKey('self', related_name='introduced', null=True, on_delete=models.SET_NULL)

Key points to focus -
  • You must always specify max_length with models.CharField
  • blank determines whether the field should be validated as required or not in forms. False means the form will generate an error if not provide, while True means empty values are allowed.
  • null determines whether the field should be set as NULL or NOT NULL at the DB level. This has nothing to do with form validation.
  • auto_now_add sets the value when record is created.
  • auto_now updates the value when any part of the record is updated.
  • Self referential foreign key can be created by using keyword 'self'

ProductionHouse

class ProductionHouse(models.Model):
    banner = models.CharField(max_length=50, unique=True)
    start_date = models.DateField()

Movie

class Movie(models.Model):
    title = models.CharField(max_length=50, unique_for_date='release_date')
    release_date = models.DateField()
    actors = models.ManyToManyField('Actor', through='MovieActor')
    production_house = models.ForeignKey('ProductionHouse')
    release_country = models.ManyToManyField('Country')

    class Meta:
        get_latest_by = 'release_date'
        ordering = ('title',)

    def __str__(self):
        return self.title

Key points to focus -
  • unique_for_date refers to a model.DateField and adds a constraint that the title should be unique for a particular release_date
  • If the model to be referenced is not already defined then its name must be given in quotes. E.g. 'Country'
  • When a many-to-many relation has other attributes then the relation goes through another model specified by through='MovieActor'
  • The __str__ function returns a friendly name to be printed when the object is accessed via console.

MovieActor

class MovieActor(models.Model):
    actor = models.ForeignKey('Actor')
    movie = models.ForeignKey('Movie')
    charges = models.DecimalField(max_digits=9, decimal_places=2)

    class Meta:
        unique_together = ('actor', 'movie')

    def __str__(self):
        return "{} - {}".format(self.actor, self.movie)

Key points to focus -
  • This is a through model. It must have foreign keys. While creating database columns for this table, foreign key fields are appended with _id. E.g. actor becomes actor_id
  • This model also illustrates unique_together constraint.

Country

class Country(models.Model):
    name = models.CharField(max_length=50)
    
    def __str__(self):
        return self.name

Finally to move these models to database,

>>> python manage.py makemigrations
>>> python manage.py migrate

We have learned about creating models. In the upcoming posts we will discuss CRUD operations via Django ORM. Click Here to go to next article.

Saturday, 2 May 2015

Building RESTful APIs - Cheat Sheet

Let's discuss the key points to focus on while designing a RESTful API.

REST (REpresentational State Transfer) has no standards but there are some data points to keep in mind while designing the URLs.

Why it is called REST? It is called as Representation State Transfer as the response data format (XML or JSON) can be interchanged based on client-server agreement which happens via HTTP Request and Response Header field Content Type.

URI Designing Guidelines

The URIs should be resource based & not action based

Action based - /getmessages.do?user=1


Resource based - /messages/1


Identify the entities (resources) in the system and create unique URIs for them. The URIs should have plural form of entities (nouns) & not verbs i.e.

No singular noun - /message/1

No verb/getmessage/1

Plural noun/messages/1


Designing Collection URIs

To get a collection of resources in response, the URI will be

To get all messages - /messages


Designing URIs around resource relations

To get all comments of message with id 1 - /messages/1/comments

To get comment with id 2 for message with id 1 - /messages/1/comments/2


Filtering criteria should be passed as query parameter (query string)

To get all the messages for the year 2014, with pagination (start at 30 and 10 records at a time) -
/messages?year=2014&offset=30&limit=10

Using HTTP Methods

As we have identified designing instance and collection URIs, next let's look at the operations that can be performed and data that's exchanged.

The following table brings in a bright comparison between non-RESTful and RESTful URLs.

Non-REST
RESTful
URL
URL
HTTP Method
/getMessages.do?id=1
/messages/1
GET
/postMessges/do?id=1
/messages/1
POST
/updateMessages.do?id=1
/messages/1
PUT
/deleteMessages.do?id=1
/messages/1
DELETE

Delete comment with id = 2 for message with id = 1
HTTP Method: DELETE
URL: /messages/1/comments/2 

Delete all comments for message with id = 1 (used seldom)
HTTP Method: DELETE
URL: /messages/1/comments

Update all comments for message with id = 2 with a list of new comments (used seldom)
HTTP Method: PUT
URL: /messages/1/comments
Body: List of new comments

Add a new comment to message with id = 1
HTTP Method: POST
URL: /messages/1/comments
Body: Comment

Note - You can't create comments in bulk. For each comment in the body, you need to hit the URL.

Method Idempotence

There are two ways in which we can classify popular HTTP methods.

GET - Read Only
POST, PUT, DELETE - Write

or

GET, PUT, DELETE - Repeatable
POST - Non-repeatable

E.g. count = 6
If you execute the above statement 100 times, at the end the value of count will remain 6. Hence this is repeatable and safe.

Methods (GET, PUT, DELETE) which are safely repeatable are called Idempotent methods. Others (POST) are called non-Idempotent methods.

The advantage is that the client can safeguard itself from non-idempotent methods correctly. E.g. when you do refresh, browser simply hits the same last hit URL. When the method is GET, it goes ahead because it is safely repeatable. But when the method of last hit URL is POST, it asks for confirmation before proceeding with the request.

HTTP Status Codes

The status code and a description in response informs the client about what has happened to the request it has made to the server. The codes are from 100 to 599.

Not all the values in the range are used as code, which means there are not 500 different status codes.

The first digit of code represents a class. Hence there are 5 classes from 1 to 5.

Informational Codes - 1


This is used for acknowledgment. The codes in this class are not used in REST services.

Success Codes - 2


200 - OK
201 - Created
203 - Created and acknowledged (this is better than 201)
204 - No content to send. This is used in response to a DELETE request.

Redirection Codes - 3


These are used by server to do further action to complete it's request.

302 - Found
307 - Temporary Redirect

Any of the above codes are used by server to tell the client don't ask me & ask to another server.

304 - Tells client nothing has changed since last request

Client Error - 4


Status code from his class is sent when the client makes an error; the request syntax might be incorrect or the client is requesting for something which it is not intended to see.

400 - Bad Request
401 - Unauthorized (The client needs to sign in)
403 - Forbidden (though client is signed in but s/he is not having rights to view the resource requested)
404 - Not Found
415 - Unsupported Media Type - (client is speaking in some language which the server can't understand)

Server Error - 5


500 - Internal Server Error (Error details must be sent in the body of the response)

Following chart gives a comprehensive list of HTTP Request Methods and Response codes for respective scenarios.


Friday, 1 May 2015

Django's Component Architecture

In this post we will have a rudimentary discussion about how a URL hits a server, how the request is served and a response is returned to the client. The following diagram depicts the entire life cycle of a request and response in context of Django Components.

A high-level overview of Django's Component Architecture
Credits: This diagram is taken from book Python Web Development with Django by Jeff Forcier, Paul Bissex and Wesley Chun.

When a user (browser) hits a URL, e.g. http://192.168.10.1:8080/library/books/, then over the Internet it finds the machine with IP address 192.168.10.1 and hits the service running on port 8080. This service is the Server (which we run using manage.py for non-production environments). The server picks up the request and forwards it to the Django framework.

Inside Django, the request is accepted by Request Middleware. This middleware enriches the Response object and forwards the request along with URL /library/books/ to the file specified by ROOT_URLCONF variable in settings.py (which is a urls.py file). 

In urls.py file there is a list of url patterns and the corresponding views.The entry in the list matching /library/books is identified, it's associated view is picked and this information along with request object is forwarded to View Middleware.

The View Middleware enriches response object and delivers the request (in **args parameter) along with **kwargs to the appropriate function or class-based view (generic, sem-generic or custom) in views.py. The **args contain request object as parameter (and the unnamed parameter groups extracted from URL) and **kwargs contain the parameters as key value pairs extracted from URL e.g. library/books/(?P<book_id>\d+)$, then value of book_id will be passed via **kwargs as argument.

The view can communicate with database via model -> ORM -> Python DB-API || Database Adapter -> RDBMS. It can also pick a template from template stack, feed data into it and pass it back to the client by adding it into the response object.

Finally the Response object via Response Middleware gets converted into HTTP Response and gets sent to the client browser over Internet by the Web Server.

This is the simplest description and panoramic view of the Django Architecture and interaction of its components at different stages while serving a client request.