Python packaging with setuptools

We are going create a python egg distribution for a simple helloworld module.

Install tools

Let install two tools we need (consider switch to debian sid repository in order to get latest version of the tools):

deby:~# apt-get -y install python-setuptools python-virtualenv

We are going to work in isolated environment:

user1@deby:~$ virtualenv devenv
New python executable in devenv/bin/python
Installing setuptools............done.
user1@deby:~$ cd devenv/
user1@deby:~/devenv$

Directory structure

Suppose our directory structure looks this way:

~/devenv/
`-- trunk/
    |-- src/
    |   `-- greatings/
    |       |-- __init__.py
    |       |-- helloworld.py
    |       `-- tests/
    |           |-- __init__.py
    |           `-- test_helloworld.py
    `-- README.txt

We are going to place all our python code in src directory.

mkdir -p trunk/src/greatings/tests

Code

The file __init__.py is left empty and makes greating a python package. Here is content of helloworld.py (note that we are using docunits in order to demonstrate dependencies later, main function will be an entry point of our script):

import sys

def say():
    """
    >>> say()
    'hello world'
    """
    return 'hello world'

def main():
    print(say())
    return 0

if __name__ == '__main__':
    sys.exit(main())

The tests will be combined into test suites so they can be easier added for testing our setup later. Here is test_helloworld.py:

from greatings import helloworld
import unittest

class HelloworldTestCase(unittest.TestCase):

    def test_say(self):
        assert 'hello world' == helloworld.say()


def suite():
    loader = unittest.TestLoader()
    suite = unittest.TestSuite()
    suite.addTest(loader.loadTestsFromTestCase(HelloworldTestCase))
    return suite

if __name__ == '__main__':
    unittest.TextTestRunner(verbosity=2).run(suite())

Here is tests package __init__.py file:

from greatings import helloworld
import test_helloworld

def suite():
    import unittest
    import doctest
    suite = unittest.TestSuite()
    suite.addTests(doctest.DocTestSuite(helloworld))
    suite.addTests(test_helloworld.suite())
    return suite

if __name__ == '__main__':
    unittest.TextTestRunner(verbosity=2).run(suite())

Setup files

Here is out ~/devenv/trunk/setup.py file:

import os
from setuptools import setup, find_packages

setup(
    name = 'greatings',
    version = '0.1',

    # Package structure
    #
    # find_packages searches through a set of directories 
    # looking for packages
    packages = find_packages('src', exclude = ['ez_setup',
        '*.tests', '*.tests.*', 'tests.*', 'tests']),
    # package_dir directive maps package names to directories.
    # package_name:package_directory
    package_dir = {'': 'src'},

    # Not all packages are capable of running in compressed form, 
    # because they may expect to be able to access either source 
    # code or data files as normal operating system files.
    zip_safe = True,

    # Entry points
    #
    # install the executable
    entry_points = {
        'console_scripts': ['helloworld = greatings.helloworld:main']
    },

    # Dependencies
    #
    # Dependency expressions have a package name on the left-hand 
    # side, a version on the right-hand side, and a comparison 
    # operator between them, e.g. == exact version, >= this version
    # or higher
    install_requires = [
        '',
    ],

    # Tests
    #
    # Tests must be wrapped in a unittest test suite by either a
    # function, a TestCase class or method, or a module or package
    # containing TestCase classes. If the named suite is a package,
    # any submodules and subpackages are recursively added to the
    # overall test suite.
    test_suite = 'greatings.tests.suite',
    # Download dependencies in the current directory
    tests_require = 'docutils >= 0.6',

    # Meta information
    #
    author = 'Me',
    author_email = 'my@e-mail.com',
    description = 'A sample hello world application',
    url = 'http://mindref.blogspot.com'
)

And configuration (file ~/devenv/setup.cfg):

[global]
# Just silently do your job
quiet = 1

[easy_install]
# Where we are going to look for thrirdparty dependencies
find_links = thirdparty

[build_py]
# No optimization for now
optimize = 0
# Force build everything?
force = True

[egg_info]
# We are doing development build
tag_build = dev
# Do we want to have date in file name?
tag_date = 0
# Add svn revision to the file name
tag_svn_revision = 1

[bdist_egg]
# We do not want to distribute binary with source code
exclude-source-files = True

[rotate]
# Keep only last 10 eggs, clean up older
match = .egg
keep = 10

Third party dependencies

The next thing, we would like keep thirdparty dependencies (e.g. docutils) in a separate folder so each time we build the project it doesn't download dependencies from internet instead look at our folder, so we always have a proper version there. So let create directory thirdparty at the same level as src and download there docutils.

user1@deby:~/devenv/trunk$ mkdir thirdparty
user1@deby:~/devenv/trunk$ wget -P thirdparty/ http://pypi.python.org\
/packages/source/d/docutils/docutils-0.6.tar.gz

The directory structure should look like this:

~/devenv/
`-- trunk/
    |-- src/
    |   `-- greatings/
    |        ...
    `-- thirdparty/
        `-- docutils-0.6.tar.gz

Install docutils that we downloaded into our environment:

../bin/easy_install thirdparty/*

Test, EGG, Source

Let ensure tests are passed:

master@deby:~/devenv/trunk$ ../bin/python setup.py test
..
---------------------------------------------
Ran 2 tests in 0.015s

OK

Here is how to create a binary distribution in egg format (look outcome at ~/devenv/trunk/dist directory):

../bin/python setup.py bdist_egg

... and source code:

../bin/python setup.py sdist

Both source and binary distributions are in ~/devenv/trunk/dist directory.

user1@deby:~/devenv/trunk$ ls dist/
greatings-0.1dev-py2.6.egg  greatings-0.1dev.tar.gz

Version control

Before adding the project to version control (e.g. svn), ensure the following directories are ignored:

1. build
2. dist
3. src/greatings.egg-info

That's it.