WSGI Protocol and Django Implementation

When people start Django development, many of them encounter the word “WSGI” and most of them still don’t know what WSGI is and stands for what. I’ll touch on these points in this article.

“WSGI” stands for “Web Server Gateway Interface”. It’s a Python standard that tells how a web server and web applications should be built together. It’s just a simple and universal specification of communication between a web server and web applications, how a web server interacts with a web application and how a web application handles requests.

WSGI has two parts:

• Server/Gateway: HTTP Server (Nginx or Apache), which is responsible for receiving requests from the client and forwarding to the application and returning the application response to the client.
• Application/Framework: A Python web application or web framework that receives requests forwarded by the WSGI Server, process the requests, executes the logic and prepares a response and sends it to the server.

We can write a very simple web application like this:

def application(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain')])
    return b'Hello World!'

environ is a dictionary containing CGI environment and start_response is a callable which includes business logic and takes two required parameters HTTP status and response_headers. Status and headers are returned to the server through the start_response method, and a response body is also returned as an iterable of byte strings.

Why Do We Need WSGI?

The way of handling requests in Django is sequential. While first request is processing, second request waits in queue, it means second request won’t be handled until the first request is finished. To deal with this concurrency problem, we benefit from WSGI implementations such as uWSGI. We can use Nginx+uWSGI to provide high concurrency for Django.

How Django Implements WSGI?

1- Program Entry: runserver command

Since runserver is an implementation of django BaseCommand, you can execute this command method via python manage.py runserver. The first method to be executed will be self.handle method.

output = self.handle(*args, **options) # inside BaseComman.execute()

The runserver command overrides this method and calls the run method, and ongoing method/function calls will be made in this order:

1. Call self.run method inside self.handle
2. Call self.inner_run method inside self.run
3. Call wsgi entry function basehttp.run inside self.inner_run

In this method, both server and application implementation of the WSGI protocol is handled.

Inside basehttp.run, a WSGIServer class is created and instantiated. The server part of the WSGI protocol is done.

We passed a wsgi handler value inside the inner_run method to basehttp.run function call. This parameter is generated via django.core.wsgi.get_wsgi_application function.

Inside this function, django.setup() is called, where the web application part is set. After this call, get_wsgi_application function returns a WSGIHandler instance. In the end, httpd.set_app(wsgi_handler) is executed and the application part of the WSGI protocol is done.

Finally, the server starts to listen port via httpd.serve_forever() call.

2- Processing Requests

We saw earlier that the inside get_wsgi_application function returned a WSGIHandler instance. This class inherits from base.BaseHandler class.

BaseHandler class contains almost whole the request processing flow. We can see load_middleware() and get_response(request) methods to process the incoming requests.

Here we focus on the _get_respons() method which processes the request and prepares a response.

def _get_response(self, request): # inside BaseHandler
	"""
	Resolve and call the view, then apply view, exception, and
	template_response middleware. This method is everything that happens
	inside the request/response middleware.
	"""
	response = None
	callback, callback_args, callback_kwargs = self.resolve_request(request)

	# Apply view middleware
	for middleware_method in self._view_middleware:
		response = middleware_method(request, callback, callback_args, callback_kwargs)
		if response:
			break

	if response is None:
		wrapped_callback = self.make_view_atomic(callback)
		# If it is an asynchronous view, run it in a subthread.
		if asyncio.iscoroutinefunction(wrapped_callback):
			wrapped_callback = async_to_sync(wrapped_callback)
		try:
			response = wrapped_callback(request, *callback_args, **callback_kwargs)
		except Exception as e:
			response = self.process_exception_by_middleware(e, request)
			if response is None:
				raise

	# Complain if the view returned None     (a common error).
	self.check_response(response, callback)

	# If the response supports deferred rendering, apply template
	# response middleware and then render the response
	if hasattr(response, 'render') and callable(response.render):
		for middleware_method in self._template_response_middleware:
			response = middleware_method(request, response)
			# Complain if the template response middleware returned None (a common error).
			self.check_response(
				response,
				middleware_method,
				name='%s.process_template_response' % (
					middleware_method.__self__.__class__.__name__,
				)
			)
		try:
			response = response.render()
		except Exception as e:
			response = self.process_exception_by_middleware(e, request)
			if response is None:
				raise

	return response

Firstly, the request url is resolved via resolve_request(request) method. Then all the middleware methods are applied to request, and finally if there is a match between the request url and any of the endpoints in the url, then the corresponding view will be dispatched for that request.

from django.conf.urls import url

urlpatterns = [
    url(r'^example/', view.ExampleView.as_view(), name='example'),
]

The view will prepare the output for the response. The response will be rendered if it should, then it’ll be returned to WSGIServer.

Sources:
An Introduction to Python WSGI Servers: Part 1
WSGI Servers