Python is a very good language. It’s easy to learn, easy to use, with a rich standard library and even richer library of 3rd party authors. Python is available everywhere and can be used almost for any task.
Unfortunately, the problem is not only in development but during deployment as well. I love how the system of compiled languages work, how you can bundle one binary and just run it, without dependency hell.
Nowadays, Docker can be the rescue. Problem is, Docker has many of its own problems as well. For example, I personally wouldn’t like to use Docker without any container orchestration, for example Kubernetes. But with that, you need to manage many services and solve a lot of new problems you wouldn’t have to. I could write own blog post only for that.
I can see the benefits of containers, in some cases, it doesn’t make sense to use any other solution. But the container is not a hammer and not everything is a nail.
So, how can be Python deployed reasonably in a case when container doesn’t make sense?
I like Debian and Debian-like systems are the most popular ones. Well, at least we can agree on the desktop. Servers are more complicated. Anyway, the rest would be almost similar.
So… my answer after many years of tweaking is debian package, with
dh_virtualenv, systemd services, included configuration files
Makefile taking care of virtualenv and for all actions
needed during development (and deployment).
Let’s elaborate it a little bit more. First is debian package. I saw many times usage of Git or Fabric or scp or whatever, but the problem is, those tools are trying to do jobs which can be handled by the system’s packaging tool. There is no need to make a script to handle configuration files or removing an app and so on, for example. Also, Debian has already defined places where to put different files (logs, pids, binaries and so on).
Problem with debian package is dependencies. Any dependency you have
in your Python’s
setup.py, you need to pass it to
It would be easy if Debian would always have up-to-date versions and,
mostly, all Python libraries available as a debian package. This is not
the case, though. In Seznam.cz, we solved that
problem with custom debian repository and custom builds of Python
libraries. Not very optimal solution, but works well if you can afford it.
I started to use
dh_virtualenv which is a helper for building
debian packages to create a virtualenv during the build and keep all
Python dependencies as simple Python modules in one bundle. I used
this technique also at CZ.NIC and I never saw
any problem with that. Thanks to that solution you don’t mess with
any Python libraries installed in your system. You can install any
Python library as debian package or simply by
pip and your app will
not be affected.
The next thing is to have something to (not just) start up your app.
init.d scripts for many years. It had its problems, but after
years I was able to solve many problems so it was working just fine.
But today we need to move to systemd. It doesn’t matter if you or I like
or not, it’s here. I think it’s good, it’s better than init scripts,
but it lacks good documentation with good examples.
Which is also a purpose why I’m writing this blog post. After years of copy & paste and improving for every new app, I created a working example. I want to keep that example up-to-date to have it as a reference guide. Documentations are not very good, the examples are very hard to find and I haven’t found the best practices for this kind of deployment.
Here it is!
The example contains more ideas, like Makefile
or included configurations. You can have a different opinion and that’s
fine, you don’t have to use it that way. It’s the way I do web
applications. The important part is a debian package with
systemd for uwsgi. Use any framework or configuration system