GeoDjango library problem on Heroku

2018-03-27 by Senko Rašić

If you're using GIS-related capabilities of Django (also known as GeoDjango) and have recently started using Heroku or upgraded to heroku-16 stack from older cedar-14 stack, you might have hit a problem with GDAL and GEOS libraries.

The problem manifests itself as a failure to load gdal or geos library, crashing your web server, deploy procedure, or both:

File "/app/.heroku/python/lib/python3.6/site-packages/django/contrib/gis/geos/libgeos.py", line 57, in load_geos
    '", "'.join(lib_names)
ImportError: Could not find the GEOS library (tried "geos_c", "GEOS"). Try setting GEOS_LIBRARY_PATH in your settings.

Here's what happens:

Missing GCC

Django uses GDAL and GEOS libraries through ctypes, a package in standard library that allows Python programs to load C libraries and use them directly, without going through trouble of writing a Python module in C that access the library (which is the approach taken with most Python wrappers around libraries: for example the Postgres client package psycopg2 does this).

In order to be able to load a library through ctypes, Django has to find it, as it may be located in several locations. When linking C programs directly, the build system (and the runtime system through LD_LIBRARY_PATH) take care of finding and loading a correct library directly, but with ctypes approach, Django has to do it manually.

The way Django does this by default is by using ctypes.util.find_library utility function.

This function (on Unix systems, which is what we're interested here since Heroku runs on Linux) attempts to mimic C library detection approach commonly used at compile-time: try to link a small "Hello World" program with the library from the specific location. If it succeeds, the library's there and usable. In order to do that, there must be a working C compiler on the system.

So here's the problem: the new heroku-16 stack (the operating system or platform that buildpacks and your code ultimately run on) doesn't include the C compiler (whereas the old stack, cedar-14, did). This breaks the find_library code in a way that it assumes the library is not there.

Luckily, GeoDjango gives us a way around it - we can manually set GDAL_LIBRARY_PATH and GEOS_LIBRARY_PATH to point to the exact path of the libraries. Thankfully, the Python buildpack exports the library paths for these to environment libraries. So, instead of hardcoding, we can get the locations based on the env variables:

GDAL_LIBRARY_PATH = os.getenv('GDAL_LIBRARY_PATH')
GEOS_LIBRARY_PATH = os.getenv('GEOS_LIBRARY_PATH')

This is actually documented in PostGis Heroku guide, however it's so discreet that it's easy to miss and you could easily spend hours trawling the web before find it.

Missing libjasper

Even if you do that, there's a possibility that the libraries still won't be loaded (you'll get the same error). This may be due to an error in the Heroku Python buildpack. If you're using an additional buildpack such as cyberdelia/heroku-geo-buildpack, you'll get the same error.

This is due to a missing library that GDAL depends on, libjasper. The TrailStash/heroku-geo-buildpack fixes this problem and is (as of this writing) the recommended buildpack to use for geodjango.

Recap

To recap, if you're having GDAL or GEOS library problems on Heroku

  • check that you're using TrailStash/heroku-geo-buildpack in addition to heorku/python buildpack
  • make sure you've set GDAL_LIBRARY_PATH and GEOS_LIBRARY_PATH in your Django project settings

These things change quickly, so this article may be out of date in a year. If you're reading this then, the problem may already be fixed by Heroku directly or another workaround should be used. In that case, let me know and I'll update the article!

Author
Senko Rašić
We’re small, experienced and passionate team of web developers, doing custom app development and web consulting.