Feb 4, 2015

CherryPy as a WSGI server

If you are new to python it might surprise you that there are a lot of web frameworks at your disposal, each one of them with some particular advantages. CherryPy is one of those, but it also have a build-in HTTP server that tend to have a great reputation for production ready deployments, specially if your application is going to be deployed on windows, stand alone or for a low to middle traffic website, for example any typical intranet application.

Yesterday I answered this question on StackOverflow, on which I explained three different ways on how to integrate cherrypy on your web app.

I'll further expand the explanation on the following lines.

This example is using a Bottle application just because that was the original context, but by no means is limited to Bottle, it can be used for any WSGI compatible application.

import logging
import logging.handlers

import cherrypy as cp
from cherrypy.wsgiserver import CherryPyWSGIServer
from cherrypy.process.servers import ServerAdapter
import requestlogger
from bottle import Bottle

app = Bottle()


@app.get('/stuff')
def do_stuff():
    '''
    Method that does stuff.
    '''
    stuff = {'data': 'some data'}
    cp.log.error('some error logging', 'TEST')
    return stuff


def run_decoupled(app, host='0.0.0.0', port=8080, config=None, **kwargs):
    if config is not None:
        cp.config.update(config)
    server = CherryPyWSGIServer((host, port), app, **kwargs)
    try:
        server.start()
    except KeyboardInterrupt:
        server.stop()


def run_in_cp_tree(app, host='0.0.0.0', port=8080, **config):
    cp.tree.graft(app, '')
    cp.config.update(config)
    cp.config.update({
        'server.socket_port': port,
        'server.socket_host': host
    })
    cp.engine.signals.subscribe() # optional
    cp.engine.start()
    cp.engine.block()


def run_with_adapter(app, host='0.0.0.0', port=8080, config=None, **kwargs):
    cp.server.unsubscribe()
    bind_addr = (host, port)
    cp.server = ServerAdapter(cp.engine,
                              CherryPyWSGIServer(bind_addr, app, **kwargs),
                              bind_addr).subscribe()
    if config:
        cp.config.update(config)
    cp.engine.signals.subscribe() # optional
    cp.engine.start()
    cp.engine.block()


def _with_access_log(app, filepath=None, when='d', interval=7, **kwargs):
    if filepath is not None:
        handlers = [logging.handlers.TimedRotatingFileHandler(
        filepath, when, interval, **kwargs)]
    else:
        handlers = [logging.StreamHandler()]
    return requestlogger.WSGILogger(
        app, handlers, requestlogger.ApacheFormatter())

Given this functions and the implementation of the Bottle application, it is possible to use the cherrypy web server on the following ways considering the access log and error log.

Decoupled mode

def run_decoupled(app, host='0.0.0.0', port=8080, config=None, **kwargs):
    if config is not None:
        cp.config.update(config)
    server = CherryPyWSGIServer((host, port), app, **kwargs)
    try:
        server.start()
    except KeyboardInterrupt:
        server.stop()

In this way your are using only the CherryPyWSGIServer class, but you will have to handle the stopping and signal handling on your own, maybe you already have an strategy for such scenarios and this could be the right way.

You can also use the WSGIPathInfoDispatcher to mount different WSGI compatible applications on use it on the same server.

run_decoupled(app)

To enable an access log you can make use of wsgi-request-logger that is been used on the utility function _with_access_log

run_decoupled(_with_access_log(app))

To write the access log on a file use:

run_decoupled(_with_access_log(app, 'access.log'))

The error log case can be covered without the need of anything special, you can gain advantage of the already build-in error log of CherryPy.

run_decoupled(_with_access_log(app, 'access.log'),
              config={'log.screen': False,
                      'log.error_file': 'error.log'})

To log your messages use cherrypy.log('My message').

Alternatively you can use the stdlib module logging.

CherryPy tree mode

def run_in_cp_tree(app, host='0.0.0.0', port=8080, **config):
    cp.tree.graft(app, '')
    cp.config.update(config)
    cp.config.update({
        'server.socket_port': port,
        'server.socket_host': host
    })
    cp.engine.signals.subscribe() # optional
    cp.engine.start()
    cp.engine.block()

In this mode the default CherryPy server can mount native applications and external WSGI applications.

For example:

import cherrypy as cp
from magic_place import wsgi_app

class Root:

    @cp.expose
    def default(self):
        return "Hi!"

cp.tree.mount(Root(), '')
cp.tree.graft(wsgi_app, '/wsgi')
cp.engine.start()
cp.engine.block()

And both applications can live happily ever after.

Notice the cp.engine stuff, you are now using the cherrypy engine, and by the grace of cherrypy you can now gain advantage of the cherrypy plugins (auto-reload, signals, pidfiles, daemon, background tasks, etc) and have a better control on the state of the server and the possibility to create one of your own.

Following the examples on the Decoupled mode this are the equivalents using this method, I'm just basically swapping the function name:

run_in_cp_tree(app)
run_in_cp_tree(_with_access_log(app))
run_in_cp_tree(_with_access_log(app, 'access.log'))
run_in_cp_tree(_with_access_log(app, 'access.log'),
               config={'log.screen': False,
                       'log.error_file': 'error.log'})

And use cherrypy.log('My message') to log your messages.

ServerAdapter mode

def run_with_adapter(app, host='0.0.0.0', port=8080, config=None, **kwargs):
    cp.server.unsubscribe()
    bind_addr = (host, port)
    cp.server = ServerAdapter(cp.engine,
                              CherryPyWSGIServer(bind_addr, app, **kwargs),
                              bind_addr).subscribe()
    if config:
        cp.config.update(config)
    cp.engine.signals.subscribe() # optional
    cp.engine.start()
    cp.engine.block()

This mode is a mixture of both of the previous ones, it uses the cherrypy engine and also is using his own instantiated CherryPyWSGIServer but in this mode you are not allowed to mount native cherrypy applications.

Notice that in this method you can also use the WSGIPathInfoDispatcher to mount different WSGI compatible applications on use it on the same server.

The code uses some of the building block of CherryPy the ServerAdapter class, you can even create a bunch of different servers on different ports and be coordinated by the cherrypy engine!

As on the previous example this are the function calls of the common cases:

run_with_adapter(app)
run_with_adapter(_with_access_log(app))
run_with_adapter(_with_access_log(app, 'access.log'))
run_with_adapter(_with_access_log(app, 'access.log'),
                 config={'log.screen': False,
                         'log.error_file': 'error.log'})

And use cherrypy.log('My message') to log your messages.

Closing thoughts

The use of cherrypy as a WSGI server has become a very common one and as a response of this the cherrypy server is already available on the Cheroot project, which is meant to be the future core of CherryPy4, along with the MagicBus project (the magic behind cherrypy.engine). Currently those two are embedded in cherrypy itself, hopefully in the near future I can make a Cheroot/MagicBus tutorial on how to do most of the stuff that I've just covered.

Tags: cherrypy python