Fixing Pillow segmentation fault

I am currently working a lot with large GeoTIFFs in Python and use Pillow to read them in, then convert them to NumPy arrays for processing. Every now and then, Pillow throws the following error, that I’ve seen on several computers running OS X now:

python TIFFReadDirectory: Warning, Unknown field with tag 42113 (0xa481) encountered. Segmentation fault: 11

Since it always takes me a while to figure out how to fix this, here’s a short note to self, maybe also useful to someone else out there:

  1. Uninstall Pillow:
  2. $ pip uninstall pillow
  3. Install dependencies for building Pillow from source:
  4. $ brew install libtiff libjpeg webp little-cms2
  5. Download Pillow source from PyPI
  6. Unpack and change into the folder with the source code, then build an install via:
    python setup.py install

This has always fixed the problem for me so far. I don’t know whether building from source rather than simply running

pip install pillow

will also fix this problem on other operating systems, but it’s worth a shot if you hit that error.

Calling Bullshit →

Awesome idea for a course by Carl Bergstrom and Jevin West at the University of Washington in Seattle:

The world is awash in bullshit. Politicians are unconstrained by facts. Science is conducted by press release. So-called higher education often rewards bullshit over analytic thought. Startup culture has elevated bullshit to high art. Advertisers wink conspiratorially and invite us to join them in seeing through all the bullshit, then take advantage of our lowered guard to bombard us with second-order bullshit. The majority of administrative activity, whether in private business or the public sphere, often seems to be little more than a sophisticated exercise in the combinatorial reassembly of bullshit.

We’re sick of it. It’s time to do something, and as educators, one constructive thing we know how to do is to teach people. So, the aim of this course is to help students navigate the bullshit-rich modern environment by identifying bullshit, seeing through it, and combatting it with effective analysis and argument.

I would definitely have taken a course that has Harry G. Frankfurt’s On Bullshit as its first reading. Let’s hope they get the university administration to approve it.

Workshop on Reproducible Computational Geosciences @ AGILE 2017

Interesting new workshop to take place at AGILE 2017. I like how they break from the usual submission workflow: In order to submit, you should fork their GitHub repo, add your submission file to the fork, and then send a pull request. If that’s too much hassle, you can also email your submission to Daniel Nüst, who is chairing the workshop.

Papers should be < 1000 words, the deadline is on March 19.

Unconventional academic writing

If you have a few minutes to spare (or need some cheering up after one of your papers has been rejected), here’s a nice read:

Guillaume Cabanac (2015) Unconventional academic writing.

Cabanac wrote this as an addendum to Hartley’s Academic writing and publishing: A practical handbook (2008), and as a present for Hartley’s 75th birthday. It contains lots of unusual – and very funny – titles, papers, and figures, all of which have been published in academic journals. My favorite may be this one-page paper on writer’s block:

Hat tip to Viola Voß for the pointer.

PostgreSQL DISTINCT ON

Getting the highest (or lowest) value from a database column is a bit tricky if you cannot use GROUP BY, because it requires you to aggregate across all columns that you want in the result. Say you have a table with employees, having the columns name, salary, and department, and you want to know the highest-paid employee per department. Then GROUP BY is not an option because you would need to also aggregate by name to have the name in the output, which doesn’t make sense.

I used to tackle this kind of query with unwieldy subqueries, but I just came across PostgreSQL’s DISTINCT ON clause, which makes them a lot more compact and readable. Using that, you can simply do

SELECT DISTINCT ON (department)
       name, salary, department
FROM   salaries_table
ORDER BY department, salary DESC;

So we’ll only get one entry per department, and ORDER BY salary DESC makes sure it is the one with the highest salary. The only bummer is that it is a PostgreSQL-specific function, so it won’t work on other DBMSs.

New toys, no time to play

I came across some really neat new tools this week. Since I don’t have any time  to test them out right now (or in the foreseeable future…), I’ll at least post them here so I don’t forget to check them out later:

  • Chris Whong has made a Docker container running Carto. This should make running your own Carto instance a hell of a lot easier.
  • Riccardo Scalco came up with Textures.js, a  JavaScript library for SVG patterns that builds on D3.
  • And my favorite comes from Geoff Boeing, who wrote OSMnx, a python module for extracting street networks from OpenStreetMap and then do all kinds of smart stuff with them.

COSIT goes Open Access

The first call for papers for COSIT 2017 was circulated last week. The conference will be held in L’Aquila, Italy, from September 4-8 2017. I was happy to see that the COSIT steering committee has decided to move the outlet for the full paper proceedings from Lecture Notes in Computer Science to Schloss Dagstuhl’s Leibniz International Proceedings in Informatics, which is Open Access under CC-BY.

The submission deadlines are:

  • March 1, 2017, for full papers
  • May 15, 2017, for posters