Chatting in real-time with WebSockets and Django Channels

Lately we see a constant switch towards real-time applications. With the wide support for WebSockets in recent browsers, more and more frameworks are giving us the ability to use them.

Django, which was primarily a request/response based framework, is doing the switch with Django Channels. Django channels is a lot more than just support for WebSockets, it's a complete architectural change for Django and - in my honest opinion - a great move towards the new era of frameworks.

I'll leave this for another post though and I'll stick to our main theme: leveraging the power of WebSockets to create a real-time application using Django Channels.

Setting the stage

Django Channels Chat

Django Channels are supported in the most recent version of Django (1.10). All you have to do is install some libraries and they will make all the hard work for you. In order to get started, use this Blueprint. This will give the following goodies out of the box:

  1. a new Django 1.10 project with Python 3.5
  2. the following libraries, which will be automatically installed for you

    # requirements.txt
    
    asgi-redis==1.0.0
    channels==0.17.3
    daphne==0.15.0
    Django==1.10.4
    
  3. a redis add-on directly connected to your project - more on this later on

Creating an echo WebSocket endpoint

As a first example, let's create a simple WebSocket endpoint that will reply the same message we send to it. We will need a new Django app - which we'll name "chat" so that we can use it later on. In order to start your app open the command palette using Ctrl + Shift + P (or Cmd on Mac) and type "start app".

Command Palette Start App

This will create a Django app - like if you'd run ./manage.py startapp chat in your terminal.

When using channels, views consumers and urls are routes. Let's start by creating our first consumer, which will be used to echo back messages sent through WebSockets. Create a new file named consumers.py inside the "chat" directory with the following contents:

# consumers.py

def ws_echo(message):
    message.reply_channel.send({
        'text': message.content['text'],
    })

Since you have created your consumer now, you will need to:

  1. add channels and chat to installed apps and append the needed settings in your settings.py file

    # settings.py
    
    INSTALLED_APPS = (
        ...
        'channels',
        'chat',
    )
    
    ...
    
    CHANNEL_LAYERS = {
        'default': {
            'BACKEND': 'asgiref.inmemory.ChannelLayer',
            'ROUTING': 'django_channels.routing.channel_routing',
        },
    }
    
  2. create the needed routes - create a file named routing.py inside the "django_channels" directory, with the following contents

    # routing.py
    from channels.routing import route
    
    channel_routing = [
        route('websocket.receive', 'chat.consumers.ws_echo'),
    ]
    

In order to test this, open your project's Public URL using the eye-like icon in the sidebar. You should be presented with the default Django screen, but you'll have a nice surprise if you try to use WebSockets. Open your browser's console and type in the following JavaScript code:

// Create a new WebSocket
ws = new WebSocket((window.location.protocol == 'http') ? 'ws://' : 'wss://' +  window.location.host + '/')
// Make it show an alert when a message is received
ws.onmessage = function(message) {
  alert(message.data);
}
// Send a new message when the WebSocket opens
ws.onopen = function() {
  ws.send('Hello, world');
}

Hooray! You just created a simple WebSocket echo server. That's a nice first step, but we're talking real-time chats here so let's move on to more advanced usage of sockets.

If you have any issues, you can always fast-forward to a working copy by typing git checkout -f step/0-echo-WebSocket in your SourceLair terminal

Say hello to group chatting

Now that we have our simple echo server ready, the next big step is to create a group chat. We will extend our simple echo server, so that it echoes every message it receives to every connected WebSocket.

First, let's create a new consumer function in our consumers.py file which will add connected WebSockets to a group and change the ws_echo consumer to send the echo to the whole group:

# consumers.py

from channels import Group


def ws_add(message):
    Group('chat').add(message.reply_channel)


def ws_echo(message):
    Group('chat').send({
        'text': message.content['text'],
    })

To do that, we will used the Group concept of channels. Now, we'll use a new route called ws_connect, which is used each time a new WebSocket connects to the server in order to add it to a group. Add the following line to your routing.py file:

#routing.py

channel_routing = [
    ...
    route('websocket.connect', 'chat.consumers.ws_add'),
]

Now that everything is in place, let's test that everything is working nicely. Open your Public URL again and paste the following code in the console:

for (var i = 0; i < 3; ++i) {
  // Create a new WebSocket
  var ws = new WebSocket((window.location.protocol == 'http') ? 'ws://' : 'wss://' +  window.location.host + '/')
  // Assign it an id
  ws.id = i;
  // Make it show a console message when a message is received
  ws.onmessage = function(message) {
    console.log('W' + this.id + ': ' + message.data);
  }
  // Send a new message when the WebSocket opens
  ws.onopen = function() {
    this.send('Hello, world');
  }
}

You'll notice that first you'll get the echo from WebSocket 0, then from 0 and 1 and finally for all 3 of them. It's this awesome? 😍 Ok, every decent chat application at least sports chat rooms right? Let's get on top of this!

If you have any issues, you can always fast-forward to a working copy by typing git checkout -f step/1-simple-group-messaging in your SourceLair terminal

Get a room!

Routing for channels allows you to route WebSockets based on URIs, which leads to the great decision of identifying each chat room from its URI. We'll make sure now that if you connect to /chat/awesome you chat only with awesome people, while if you're a rockstar 🎸 you can always connect to /chat/rockstar.

Let's tweek our routing.py file now to include the URI definition - don't worry, this is very familiar to what you would do in a normal Django application

# routing.py

channel_routing = [
    ...
    route('websocket.connect', 'chat.consumers.ws_add',
          path=r'^/chat/(?P<room>\w+)$'),
]

Now that our routes are in place, we need to make our consumers understand and remember different channels. For that we'll Django's sessions and thus first we'll need to migrate our database. To do so, open the Command Palette and the type "migrate". This is exactly the same as running ./manage.py migrate in your terminal.

After we have migrated our database, we can use Django sessions in our consumers for storing data. Do the follwoing update in your consumers.py file:

# consumers.py

from channels import Group
from channels.sessions import channel_session


@channel_session
def ws_add(message, room):
    Group('chat-%s' % room).add(message.reply_channel)
    message.channel_session['room'] = room


@channel_session
def ws_echo(message):
    room = message.channel_session['room']
    Group('chat-%s' % room).send({
        'text': message.content['text'],
    })

Next, let's test this in a quick and hacky way. Open your Public URL and type in the following code in your console:

for (var i = 0; i < 3; ++i) {
  for (var j = 0; j < 2; ++j) {
    var ws = new WebSocket((window.location.protocol == 'http') ? 'ws://' : 'wss://' +  window.location.host + '/chat/' + i)
    ws.id = i * 10 + j;
    ws.channel = i;
    // Make it show an alert when a message is received
    ws.onmessage = function(message) {
      console.log('W' + this.id + ': ' + message.data);
    }
    // Send a new message when the WebSocket opens
    ws.onopen = function() {
      this.send('Hello, channel ' + this.channel + ', from id: ' + this.id);
    }
  }
}

As you can see, now each message to the channel is printed only by the WebSockets of that channel!

If you have any issues, you can always fast-forward to a working copy by typing git checkout -f step/2-chat-rooms in your SourceLair terminal

Sorry, did I forget to introduce myself?

Up until now everything is nice and fun. Though chats need to have at least a name behind each message. For this reason, we'll add a basic - and when I say basic, I really mean it - authentication. We'll let WebSockets to give their username in a GET parameter as they connect, so that they can be easily identified.

Open your consumers.py and add the following lines of code. As you'll notice, we are also changing the structure of messages to JSON in order to have all the information in the same message.

# consumers.py

import json
from urllib import parse

from channels import Group
from channels.sessions import channel_session


@channel_session
def ws_add(message, room):
    query = parse.parse_qs(message['query_string'])
    if 'username' not in query:
        return
    Group('chat-%s' % room).add(message.reply_channel)
    message.channel_session['room'] = room
    message.channel_session['username'] = query['username'][0]


@channel_session
def ws_echo(message):
    if 'username' not in message.channel_session:
        return
    room = message.channel_session['room']
    Group('chat-%s' % room).send({
        'text': json.dumps({
            'message': message.content['text'],
            'username': message.channel_session['username']
        }),
    })

Let's test this, once more using your project's Public URL and your browser's console:

for (var i = 0; i < 3; ++i) {
  for (var j = 0; j < 2; ++j) {
    var id = i * 10 + j,
    ws = new WebSocket((window.location.protocol == 'http') ? 'ws://' : 'wss://' +  window.location.host + '/chat/' + i + '?username=' + id);
    ws.id = id;
    ws.channel = i;
    // Make it show an alert when a message is received
    ws.onmessage = function(message) {
      console.log('W' + this.id + ': ' + message.data);
    }
    // Send a new message when the WebSocket opens
    ws.onopen = function() {
      this.send('Hello, channel ' + this.channel + ', from id: ' + this.id);
    }
  }
}

If you have any issues, you can always fast-forward to a working copy by typing git checkout -f step/3-usernames in your SourceLair terminal

Feeling the real power of Django Channels

Until now, the only thing we've tasted about Django Channels was the fact that they enable WebSockets. There are a lot of other goodies though in Django Channels and one of them is the new, distributed model of execution. For our example, we were using the in-memory channel layer - remember, we defined this in settings.py when we started - but Django Channels support more options like Redis.

For this reason, we have also included a Redis server to this Blueprint, as Redis is the recommended channel layer for production environments. Let's set this up!

In your settings.py file, change the channel layer declaration to the following:

# settings.py

CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'asgi_redis.RedisChannelLayer',
        'ROUTING': 'django_channels.routing.channel_routing',
        'CONFIG': {
            'hosts': [('redis', 6379), ],
        },
    },
}

This tells Django to use a Redis server with host redis and port 6379 and RedisChannelLayer as the backend. Since you've used the provided Blueprint, we have already provisioned a Redis server for you with the exact same settings and it will simply work now using Redis!

That's not that sexy though, so let's have a quick and nice example in order to see the distributed nature of Django Channels in action. The basic concepts of channels are:

  • Interface servers, which communicate between Django and the outside world. This includes a WSGI adapter as well as a separate WebSocket server - which we used for our WebSockets
  • The channel backend, which is a combination of pluggable Python code and a datastore (a database, or Redis) responsible for transporting messages - this was in-memory up until now and now we're using Redis
  • The workers, that listen on all relevant channels and run consumer code when a message is ready - this was up to now the same process with the server

Now, we'll run a second worker, right inside your SourceLair terminal. To start your worker, type ./manage.py runworker. This will start a second worker. In order to trace which worker handles each request, we'll add some logging.

Open you consumers.py file and add the following logging code:

# consumers.py

...
import logging
...


@channel_session
def ws_add(message, room):
    query = parse.parse_qs(message['query_string'])
    if 'username' not in query:
        return
    logging.info('Adding WebSocket with username %s in room %s',
                 query['username'], room)
     ...


@channel_session
def ws_echo(message):
    if 'username' not in message.channel_session:
        return
    room = message.channel_session['room']
    logging.info('Echoing message %s from username %s in room %s',
                 message.content['text'], message.channel_session['username'],
                 room)
     ...

Open your project's Public URL once more and paste the following code inside your Terminal to test this:

for (var i = 0; i < 3; ++i) {
  for (var j = 0; j < 2; ++j) {
    var id = i * 10 + j,
    ws = new WebSocket((window.location.protocol == 'http') ? 'ws://' : 'wss://' +  window.location.host + '/chat/' + i + '?username=' + id);
    ws.id = id;
    ws.channel = i;
    // Make it show an alert when a message is received
    ws.onmessage = function(message) {
      console.log('W' + this.id + ': ' + message.data);
    }
    // Send a new message when the WebSocket opens
    ws.onopen = function() {
      this.send('Hello, channel ' + this.channel + ', from id: ' + this.id);
    }
  }
}

As you can see now, some of the logs appear on the server tab, while some other in the terminal tab. Your workers (both the server and the worker you started in the Terminal) just grab events from Redis and consume them!

If you have any issues, you can always fast-forward to a working copy by typing git checkout -f step/4-redis in your SourceLair terminal

Wrapping up

We created a pretty sexy chat now, the only thing missing is the UI. That's why we have also created a simple but nice UI so that you can share your chat with your friends. In order to switch to the branch with the UI, type git checkout -f step/final in your terminal and then open your Public URL to see your site. Share it with friends and start chatting!

Django Channels Demo

Django Channels is a big architectural change for Django, allowing for more complex application and at the same time, better handling workloads - since the interfaces servers and workers can be scaled independently. I hope you're as excited as I am about the upcoming stable release of Django Channels. Until then, play as much as you like with them using our Blueprint below.