(This is a quick post for search-engine fodder, since I didn’t manage to find anything relevant myself.)
If you’re using
pip install --isolated to install Python packages and find
that it fails with an error like the following:
Complete output from command python setup.py egg_info: usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: -c --help [cmd1 cmd2 ...] or: -c --help-commands or: -c cmd --help error: option --no-user-cfg not recognized
… then you might have run into an incompatibility between pip and Python versions 3.0–3.3.
Running pip in isolated mode also passes the
flag to Python’s
distutils to disable reading
~/.pydistutils.cfg. But that flag isn’t available in Python
versions 3.0–3.3, causing the error above.
I ran into this because I recently migrated the Python code that generates this site to run under Python 3.x. I’m using a virtualenv setup, so once I had everything working under both Python versions, I was reasonably confident that I could switch ‘production’ (i.e. the Compute Engine instance that serves this site) to Python 3 and discard the 2.x-compatibility code.
Good thing I tested it out first, since it didn’t even install.
It turns out that:
- virtualenv ships an embedded copy of pip and setuptools, but setuptools will use the system version of distutils1, and
--no-user-cfgwas added in Python 2.7, but wasn’t ported to 3.x until 3.42, and
- the distribution I’m using on my real server (Debian 7) ships with Python 3.2.3, rather than the 3.4.x I’m using elsewhere.
I worked around this by just omitting the
--isolated flag for Python
verions [3.0, 3.4) — though since I don’t actually have any system config
files in practice, I probably could have set
instead (which has the effect of ignoring all config files).
I’m not the first person to have noticed that virtualenv isn’t actually hermetic. Though some of that rant is out of date now (Python wheel files provide prebuilt binaries), and some isn’t relevant to the way I’m using virtualenv/pip, it’s definitely true that the dependency on the system Python libraries is the main reasons I’d look to something more like Docker or Vagrant for deployment were I doing this professionally.
So did I finally manage to switch to Python 3.x after that? Not even close:
Python 3.x didn’t gain the ability to (redundantly) use the
for Unicode strings until 3.3, and some of my dependencies use that syntax.
So I’m waiting until I can switch to Debian 8 on Compute
Engine3, at which point I can cleanly assume Python 3.4 or
This is a rant for another day, but it looks like virtualenv monkeypatches pip, which monkeypatches setuptools, which either monkeypatches or builds upon distutils. Debugging through this edifice of patched abstractions is… not easy. ↩
It’s a bit more complex than that: Python 3.0 and 3.1 were released first, then the feature was implemented in both 2.7 and 3.2, but then distutils as a whole was rolled back to its 3.1 state before 3.2 was released. That rollback was reverted for Python 3.4. ↩
apt-get dist-upgradefrom the Debian 7 image just fine, but it’s a bit slow and hacky, so I’d rather wait for official images. (I also need to fix some custom mail-related configuration that appears to have broken under Debian 8.) ↩