Andrew Wei

Engineer / Designer / Illustrator

Google Compute Engine Setup for Django App with uWSGI and Nginx

Jun 01 15

This log demonstrates how to set up a Django app on Google Compute Engine and serve it using uWSGI and Nginx from scratch. Assume the following:

  1. The app uses a VM instance on Debian OS. This log uses the Debian GNU/Linux 7.8 (wheezy) image on GCE.
  2. The name of the app is example.com and its base path on the VM instance is /srv.
  3. The app has a Git repo.
  4. The database which the app uses will be set up in the same VM instance as the app.
  5. The app is generated by generator-vars-django, which uses Gulp as its asset pipeline, hence depends on Node.js.

Let's begin. SSH into your instance and ensure you have sudo access.

Installing Dependencies

  1. Update APT packages, no explanations needed.

    $ sudo apt-get update
  2. Install Git to pull down the app's source code.

    $ sudo apt-get install git-core
  3. Install Upstart if the system is still using SysVinit. This will make writing startup scripts much easier later on.

    $ sudo apt-get install upstart

    When prompted, enter 'Yes, do as I say!'.

  4. Install nvm so you have the freedom to juggle between different npm versions. Install it globally to save yourself from the trouble of managing global node packages later on. This allows you to run nvm with sudo. Follow this GitHub repo or simply run:

    $ sudo wget -qO- https://raw.githubusercontent.com/xtuple/nvm/master/install.sh | sudo bash
  5. Install node via nvm. Pick the version of your choice.

    $ sudo nvm ls-remote
    $ sudo nvm install x.xx.x
  6. Install gulp globally to build the app (assuming you have Gulp setup, which comes with generator-vars-django:

    sudo npm install -g gulp
  7. Install python and pip as needed by any Django app.

    $ sudo apt-get install python-pip python-virtualenv python-dev build-essential
  8. Install database of your choice. Here are a couple examples:

    MySQL

    $ sudo apt-get install mysql-server
    $ sudo apt-get install python-mysqldb

    PostgreSQL

    $ sudo apt-get install libpq-dev python-dev python-psycopg2
    $ sudo apt-get install postgresql postgresql-contrib
  9. Install uWSGI. Do it through pip instead of apt-get to get the most updated version.

    $ sudo pip install uwsgi
  10. Install Nginx:

    $ sudo apt-get install nginx

Database Configuration

You will need to set up your database before you can run a Django app (if you are not using SQLite). Basically you need to create a new database with a new owner and password. Here's a quick guide for MySQL and PostgreSQL:

MySQL

$ mysql_secure_installation
$ mysql --user=root --password={password}
$ mysql> create database {db_name};
$ mysql> quit;

PostgreSQL

$ sudo su postgres
$ createuser -P
$ createdb --owner username dbname
$ exit

Python Virtual Environment Setup

Python packages required by the app will be installed into a virtual environment. Create one in /srv:

$ sudo virtualenv /srv

Before sourcing it, add any environment variables your app needs to /srv/bin/activate, such as the Django secret key. Assuming the app is generated by generator-vars-django, all the database credentials need to go in there as well.

$ sudo nano /srv/bin/activate

Add the following to the end of the file (replace {xxxx} with your own values):

export DJANGO_SECRET_KEY="{django_secret_key}"
export DJANGO_DB_NAME="{db_name}"
export DJANGO_DB_USER="{db_user}"
export DJANGO_DB_PASS="{db_password}"
export DJANGO_DB_HOST="localhost"
export DJANGO_DB_PORT="{db_port}"

Finally, source it:

$ source /srv/bin/activate

App Setup

  1. Clone the source code to /srv. It is best to have the source code inside a folder like example.com. This should already be taken care of if the app is generated by generator-vars-django.

  2. Install Python dependencies. Remember to activate the virtual environment first.

    $ sudo pip install -r requirements.txt

    Double check that the packages were installed to the virtual environment:

    $ which django-admin

    If it's not, you should redo this step. It might be because the command was executed using sudo. In which case you will either need to give your account write permission for /srv so you can run pip install without sudo, or you can simply login as root by doing:

    $ sudo su -
  3. Install Node modules:

    $ sudo npm install
  4. Do the initial Django migration:

    $ gulp migrate
  5. Build it:

    $ gulp

    You should now have the app built and deployed to the build directory.

Serving the App

Nginx Setup

generator-vars-django comes with a default example.com_nginx.conf file. Symlink that so Nginx can see it.

$ sudo ln -s /srv/example.com_nginx.conf /etc/nginx/sites-enabled/

Edit whatever you need in there. It should look like this:

# The upstream component nginx needs to connect to.
upstream django {
    server unix:///srv/variante.io/variante.io.sock;
}

# Configuration of the server.
server {
    # The port in which the site will be served on.
    listen 80;

    # The domain name it will serve for (or the external IP).
    server_name example.com;

    # Character encoding.
    charset utf-8;

    # Max upload size.
    client_max_body_size 75M;

    # Django media path.
    location /media {
        alias /srv/example.com/build/media;
    }

    # Django static path.
    location /static {
        alias /srv/example.com/build/static;
        expires 90d;
    }

    # Send all non-media requests to the Django server.
    location / {
        uwsgi_pass django;
        include /srv/example.com/uwsgi_params;
    }
}

Make sure that you also have the uwsgi_params file in the specified place. It also comes with generator-vars-django. If not, it looks like this:

uwsgi_param  QUERY_STRING       $query_string;
uwsgi_param  REQUEST_METHOD     $request_method;
uwsgi_param  CONTENT_TYPE       $content_type;
uwsgi_param  CONTENT_LENGTH     $content_length;

uwsgi_param  REQUEST_URI        $request_uri;
uwsgi_param  PATH_INFO          $document_uri;
uwsgi_param  DOCUMENT_ROOT      $document_root;
uwsgi_param  SERVER_PROTOCOL    $server_protocol;
uwsgi_param  HTTPS              $https if_not_empty;

uwsgi_param  REMOTE_ADDR        $remote_addr;
uwsgi_param  REMOTE_PORT        $remote_port;
uwsgi_param  SERVER_PORT        $server_port;
uwsgi_param  SERVER_NAME        $server_name;

Restart Nginx:

$ sudo service nginx restart

At this point you would probably see a 500 error if you try to access the site.

uWSGI Setup

We will be setting uWSGI up in emperor mode which will be spawning instances for each vassal it finds.

First create the directories for storing these vassals.

$ sudo mkdir /etc/uwsgi
$ sudo mkdir /etc/uwsgi/vassals

Symlink example.com_uwsgi.ini so the emperor can see it:

$ sudo ln -s /srv/example.com_uwsgi.ini /etc/uwsgi/vassals/

You should tweak this file accordingly. The file looks like this:

[uwsgi]

# Variables.
base-path = /srv
app-name = example.com
app-path = %(base-path)/%(app-name)
env-path = %(base-path)

# Configurations.
chdir = %(app-path)/build
module = project.wsgi
home = %(env-path)
master = true
processes = 10
socket = %(app-path)/%(app-name).sock
chmod-socket = 666
vacuum = true

This is the most basic configuration you need to get things up and running. Note that chmod-socket is set to 666, which is very permissive. Tweak that accordingly. Also make sure that env-path matches the path to your virtualenv.

Now try running uWSGI in emperor mode:

$ sudo uwsgi --emperor /etc/uwsgi/vassals

You may encounter the following issues:

  1. You get an error about permission issues with writing to the UDP socket. You might need to change the owner of /srv/example.com to the Nginx owner/group, which is probably www-data:www-data:

    $ sudo chown -R www-data /srv/example.com
    $ sudo chmod -R 775 /srv/example.com
  2. You get a warning about running uWSGI in root. You can set the gid and uid to the Nginx user (probably www-data) either in the ini file or in the command, like so:

    $ uwsgi --emperor /etc/uwsgi/vassals --uid www-data --gid www-data

The last step is to make sure that the emperor starts whenever the system reboots. upstart finally comes into play. Create a new file:

$ sudo nano /etc/init/uwsgi.conf

Add the following:

description "uWSGI Emperor"

start on runlevel [2345]
stop on runlevel [!2345]

script
    . /srv/bin/activate
    uwsgi --emperor /etc/uwsgi/vassals
end script

This script will execute everytime the system boots up and it basically first activates the virtual environment so that all the environment variables are loaded (i.e. the Django secret key), and then kickstarts the emperor.

Verifying the Live Site

Open up your browser and navigate to the site. It should be now be served properly with Nginx and uWSGI. You might encounter the following issues:

  1. You are still getting a 500 error. The best thing to do is to check the Nginx logs. It's in /var/log/nginx/error.log.

  2. You are instead getting a 400 error. You can debug this using your browser console to see what is wrong. If you get a GET request fail on the domain itself, chances are you forgot to add your domain to the ALLOWED_HOSTS of your Django project settings.

  1. A Tutorial for Deploying a Django Application that Uses Numpy and Scipy to Google Compute Engine Using Apache2 and modwsgi

  2. Installing Nginx, WWSGI and Python on Google Compute Engine

  3. Setting up Django and your web server with uWSGI and Nginx

  4. Setting up Django with Nginx, Gunicorn, virtualenv, supervisor and PostgreSQL

  5. Using uwsgi the right way

  6. How To Set Up uWSGI and Nginx to Serve Python Apps on Ubuntu 14.04