Configuring large Django projects can be difficult. Different environments - local, staging, production, testing, etc - each have different requirements, and different settings that need to be configured.

For example, in development, you will want to set DEBUG=True in your settings.py file. But this is a big security problem in your production environment, where end-users should not be exposed to the internals of the backend system. Your local environment should also not connect to a production database, and therefore a reliable way of managing these differences in Django settings is recommended.

Another vital consideration is security - you do not want settings such as database credentials, secret keys, API keys, and other sensitive information finding its way onto your source-control repository (Github, Gitlab, etc).

This article will outline what I consider to be the best practice in the area of configuration management. We will learn about a package called django-environ, which is excellent for handling configuration from the environment - one of the 12 factors of the 12 Factor App, which we will cover in more detail in the tutorial.

Objectives

In this tutorial, we will learn:

• The basics of the 12-factor approach to configuration management
• How to do 12-factor configuration management in Django, with the django-environ package.

Project Setup

To test this out, we can start with a fresh Django project.

django-admin startproject djconfig

This will create a new Django project called djconfig.

Within the project folder of the generated project - alongside thesettings.py file - you should add a .env file. This will be used to securely store the key-value pairs you want to store in your environment, and reference in your Django settings. This should be added to a .gitignore file if you are committing to source control, as it is vital not to leak any secure information.

Let's add a .env file with key-value pairs for some typical Django settings. A basic setup would look something like this:

DEBUG=true
SECRET_KEY=your-secret-key
DATABASE_URL=psql://pguser:password@127.0.0.1:5432/database
SQLITE_URL=sqlite:///sqlite3.db
JWT_SIGNING_KEY=signing-key

We are going to consider how to use django-environ to load these values into our OS environment, and then reference them in our settings.py file.

But first, let's have a brief overview of the concept of configuration management in the 12-Factor philosophy.

12-Factor - Configuration

The 12-Factor principles were created to help developers in the creation of modern, distributed web applications and SAAS applications. They provide a set of 12 guidelines for development, and these have become common-practice in the years since the methodology was first proposed.

For the purposes of this tutorial, we are interested in the Configuration principle. The 12-Factor page on configuration states:

The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard.

The recommendation is to store config in the operating system's environment. We do not want our configuration in our code - separation between code and configuration is vital for the security, maintainability and scalability of your application.

On Linux systems, you can set transient environment variables with the export command - for example, to set the database password:

export DATABASE_PASSWORD=secretpw

This key-value pair will remain in the shell's environment until it is closed. To set persistent environment variables, that last beyond the lifetime of the shell, you can set export commands in your bash_profile file (in your home directory).

For more information about the 12-Factor philosophy on configuration, see here.

django-environ

The package we will use to conform to the above philosophy is django-environ.

The key feature that this package provides is that it enables values from an environment file to be read into the os.environ dictionary in Python (representing OS environment variables), where they can then be easily cast to Python data-types. django-environ also makes it easy to set default values for settings that can help protect your application and prevent misconfiguration in some environments.

Instead of having numerous uncommitted settings_<ENV>.py files containing sensitive information, we can now keep all our configuration in environment files. These can be referenced and the values read in by django-environ, making your life easier and allowing you to scale up your development team (and application) more easily.

Firstly, we need to install it. This can be done with pip:

pip install django-environ

Once installed, we can edit our settings.py file to utilize this package. We import django-environ and then read in our environment file, and set some types and default values, with the following code:

import environ

# create an Env object
env = environ.Env(
    # set casting, default value
    DEBUG=(bool, False)
)

# read in values from the .env file
environ.Env.read_env()

With the env object, we can define a schema for our environment variables - for example, above we specify that the DEBUG variable should be of type bool, and have a default value of False.

This casting can be important, because the values in the .env file are just strings - this gives the code mechanism for converting the string to a boolean value. Values can be cast to strings, integers, floats, lists, tuples, dicts, json, and some custom types. The default values give the developer a way to set sensible, security-conscious defaults - for example, we should default the DEBUG setting to False for security reasons. This way, if it is excluded from our environment file, the application is protected because the default is set to False. Safety-first!

We can now set our DEBUG variable to the value from our .env file.

DEBUG = env('DEBUG')

If the DEBUG variable exists in the environment file, it will be read in here and set as a bool type in Python. Since our dummy environment file had DEBUG=true, this will be read in, and the true value is sensibly cast to a bool type with value True. Had there been no DEBUG value in our environment file, the default value of False would have been applied.

We can also cast values from our .env file using methods available on the env object. For example, to cast DEBUG to a bool, we could write:

DEBUG = env.bool('DEBUG')

This would take care of the casting (although in this case, we have already done that when setting up our env object - this is just another method).

If we don't provide a default value for a variable, and the variable is not in our .env file, then we will get an ImproperlyConfigured exception. For example, if we blindly try and reference an AWS_SECRET_KEY (which is not in our .env file!) then we will get the exception. However, we can provide a default using a second parameter.

### When AWS_SECRET_KEY is *not* in our .env file...

# django.core.exceptions.ImproperlyConfigured: Set the AWS_SECRET_KEY environment variable
env.str('AWS_SECRET_KEY')

# we can provide a default! No error here.
env.str('AWS_SECRET_KEY', '53sdigjisgjit')

Providing a default value safeguards us from the exception.

We can now set our SECRET_KEY, DATABASE_URL and SQLITE_URL settings. We'll use the special env.db() method to set the database URL - this method looks for a DATABASE_URL variable in the environment file, which we have set above.

# set the secret key - default to some random characters
SECRET_KEY = env.str('SECRET_KEY', default='snfjdsghsiogjsdjgsd')

# set the two databases - our default (postgresql) and a sqlite3 dummy database
DATABASES = {
    'default': env.db(),
    'dummy': env.db('SQLITE_URL', default='sqlite:////data/sqlite3.db')
}

The second env.db() call sets up a second database, with a default link to the database file provided.

We are demonstrating that, with django-environ, it is very simple to set operating system environment variables from a .env file. It is easy to cast the string values in these files to a variety of Python data-types, and we can also easily provide security-conscious default values to protect our application.

As well as handy database helpers, django-environ also provides methods for dealing with cache URLs (Redis, memchached, etc), email URLs, and search URLs (ElasticSearch, Solr, and others).

Conclusion

In this tutorial, the key takeaway is the importance of not mixing configuration and code in your settings file. Configuration should be stored separately in the environment, as per the 12-Factor app principles.

Additionally, it is very important to avoid committing sensitive values to source control - your environment files containing sensitive information should be added to a .gitignore file. It is also important to set security-conscious default values for settings when referencing environment values with django-environ.

If you enjoyed this post, please subscribe to our YouTube channel and follow us on Twitter to keep up with our new content!

Please also consider buying us a coffee, to encourage us to create more posts and videos!

Additional Notes

• django-environ also has a Path object, which helps with constructing idiomatic paths in your settings. More info here.