Beginner DevOps with Django and Fabric

I've been aware of Fabric for quite some time now and knew that it's commonly used with Django to automate DevOps related tasks such as website deployment. I figured that even though starting a new project may not be the normal Fabric-type task that I'd still be able to accomplish my goal. Fabric essentially is a python package that gives you easy to use helper functions for scripting local and remote shell commands. A simple example that Fabric is often used for is connecting to a remote server, updating your codebase from source control, and then restarting your webserver.

Laundry List

As I got ready to create the Fabric script I made sure I had my very basic workflow written down beforehand:

1. Create new virtual environment using virtualenvwrapper
2. Install django, mezzanine, south, and mysql python modules
3. Create a new mezzanine-django project
4. Set up appropriate settings and local_settings files
5. Create a requirements.txt file
6. Create the first app within the django project
7. Set up the database and first South migration
8. Make the first git commit

Fabric Basics

with prefix("workon myvirtualenv"):

The "with prefix" function prepends its contents ahead of every shell command ran underneath it. In this example it will switch your shell to be working on the "myvirtualenv" virtual environment before any command that is ran after it.

with lcd("myprojectdir"):

The "with lcd" function will change the directory of the current shell before running any command.

local("./manage.py syncdb")

The local() function will run the supplied command on a local shell terminal.

Now Let's Look at the Script

Setup

def new_project(virtual_env_name, project_name, app_name):    with prefix("source ~/.bash_profile"):        bash_local("mkvirtualenv %s" % virtual_env_name)        bash_local("mkdir %s" % project_name)        with lcd("%s" % project_name):            with prefix("workon %s" % virtual_env_name):                bash_local("sudo pip install mezzanine")                bash_local("mezzanine-project %s" % project_name)                bash_local("sudo pip install south")

Here's the top portion of the function I created. You'll notice that it takes in 3 inputs from the user, all of which are pretty self explanatory. The first prefix(), which sources our bash profile is due to the virtualenvwrapper commands that we have mapped in there. It then proceeds to created the virtualenv for this project and create the project folder. After cd'ing into the created folder and activating our newly created virtualenv we then install mezzanine (which installs django), create a new mezzanine project, and install south.

SQL Settings

with prefix('export PATH="$PATH:/usr/local/mysql/bin/"'):    bash_local("sudo pip install mysql-python")with lcd("%s" % project_name):    bash_local("sed 's/backends.sqlite3/backends.mysql/g' local_settings.py > local_settings.py.tmp")    bash_local("mv local_settings.py.tmp local_settings.py")

On Macbooks when installing the mysql-python library there is a bug where it cannot find your mysql config. By exporting your mysql path before running the install everything goes smoothly. After the main python libraries are installed we change directory into our newly created project and update the local_settings file to point to mysql instead of sqlite.

Ease of Use

bash_local("chmod +x manage.py")bash_local("pip freeze > requirements/requirements.txt")bash_local("./manage.py startapp %s" % app_name)

Next we make our manage.py executable so we can call "./manage.py" instead of "python manage.py". We take our list of installed modules and write it to a file called requirements.txt. This allows someone else collaborating on this project to just install all modules with one command (pip install -r requirements.txt). Lastly here we create the first app within this django project.

Database Setup

with prefix('export PATH="$PATH:/usr/local/mysql/bin/"'):    bash_local("mysqladmin -u root create %s" % virtual_env_name)bash_local("./manage.py syncdb")bash_local("./manage.py migrate")

This section handles setting up mysql and south. First we create the mysql database using the virtualenv name. Note: This assumes our mysql admin user is root without any password. Once the database is set up we run syncdb and then our first South migration. Mezzanine uses South so running this first migrate call will help get us up and running. I recommend using South with mezzanine so that if you upgrade to a new version of Mezzanine in the future you will have much less of a headache. Also you should be using South to manage database migrations for your own apps as well.

Git Init

bash_local("git init")bash_local("git add .")bash_local("git commit -m'init'")

Lastly we tie up our project by setting up a git repository and making our initial commit.

Now How do I Use It?

1. Open up your shell and navigate to the folder where you want to create your new project
2. fab -f /path/to/fabfile.py new:<virtual_env_name>,<project_name>,<app_name>

The -f argument lets you specify where your fabfile script exists incase fabric is not able to pick up your script on its path.

Assumptions and Gotcha's

• This script makes a lot of assumptions about the development environment you are working on. Specifically it is built around doing django development on a Macbook using MySQL.
• The full script, which you can find linked at the beginning and end of this article has A LOT of TODOs. Since I'm new to Fabric development there is a lot I can improve upon. In general there are a few helper functions I need to abstract out and a lot of testing/prompting I can do when the script fails.
• Something I skipped over in the above examples is the use of bash_local(). This is a copy and pasted version of Fabric's built in local() function. The only difference is that it uses /bin/bash as it's executable instead of /bin/sh. This gave me the use of extra commands that I didn't have access to (specifically virtualenvwrapper). There may be a better fix, but for the time being it got me through the first version of this script.

Fabric to the Next Level

Going forward with Fabric I'd like to start building out remote server functionality that mimics the local behavior. For example let's say you've set up a project locally with this script and have some working functionality checked into your source control. How great would it be to run another Fabric one-liner that spins up a staging server for your project? This script would clone your repository, create a virtual environment with necessary python modules, create/sync your database, and create an apache config. I know for Yeti this would be another huge time saver and eliminate more unnecessary time we spend doing DevOps work.

Checkout github for the full script and any new updates!