Zed Shaw, of Learn Python the Hard Way fame, has now written The Case Against Python 3.

I’m not involved with core Python development. The only skin I have in this game is that I like Python 3. It’s a good language. And one of the big factors I’ve seen slowing its adoption is that respected people in the Python community keep grouching about it. I’ve had multiple newcomers tell me they have the impression that Python 3 is some kind of unusable disaster, though they don’t know exactly why; it’s just something they hear from people who sound like they know what they’re talking about. Then they actually use the language, and it’s fine.

I’m sad to see the Python community needlessly sabotage itself, but Zed’s contribution is beyond the pale. It’s not just making a big deal about changed details that won’t affect most beginners; it’s complete and utter nonsense, on a platform aimed at people who can’t yet recognize it as nonsense. I am so mad.

Part of my Python FAQ, which is doomed to never be finished.

Maybe you have a Python 2 codebase. Maybe you’d like to make it work with Python 3. Maybe you really wish someone would write a comically long article on how to make that happen.

I have good news! You’re already reading one.

(And if you’re not sure why you’d want to use Python 3 in the first place, perhaps you’d be interested in the companion article which delves into exactly that question?)

Part of my Python FAQ, which is doomed to never be finished.

The short answer is: because it’s the actively-developed version of the language, and you should use it for the same reason you’d use 2.7 instead of 2.6.

If you’re here, I’m guessing that’s not enough. You need something to sweeten the deal. Well, friend, I have got a whole mess of sugar cubes just for you.

And once you’re convinced, you may enjoy the companion article, how to port to Python 3! It also has some more details on the diffences between Python 2 and 3, whereas this article doesn’t focus too much on the features removed in Python 3.

This post is about the thing in the title.

I used to work for Yelp. For historical reasons — probably “the initial developers preferred it” — their mostly-Python codebase had always been indented with tabs. That’s in stark contrast to the vast majority of the Python ecosystem, which generally uses the standard library’s style guide recommendation of four spaces. The presence of tabs caused occasional minor headaches and grumbles among the Python developers, who now numbered in the dozens and were generally used to spaces.

At the end of 2013, I bestowed Yelp with a Christmas gift: I converted their entire primary codebase from tabs to four spaces. On the off chance anyone else ever wants to do the same, here’s how I did it. Probably. I mean, it’s been two and a half years, but I wrote most of this at the time, so it should be correct.

Please note: I do not care what you think about tabs versus spaces. That’s for a different post! I no longer work for Yelp, anyway — so as compelling as your argument may be, I can no longer undo what I have done.

Nova Dasterin asks, with money:

How about usage of Lua for game development? Love2d etc. Also http://lexaloffle.com/pico-8.php which I recently heard about.

clarification: thoughts on Lua as a ‘good choice’, also Lua vs Python for that subject (gamedev)

There are a couple ways I can interpret this, so I’ll go with: all of them.

(edit: you may be interested in a subsequent post about the game I actually made for the PICO-8!)

Sketch is still buying days of my time, which is super cool of him. Continuing from last month, he asked that I make it possible to disable normal editing and only accept proposals on the wiki.

After some internal debate about how to add a real configuration system, I realized this could just be expressed with permissions, so I wrote some little permissions UI. And actually added them to the proposal code. Which is good.

I wanted to have a nice way to iterate all possible permissions from whatever plugins are currently active, but the way permissions work right now is kind of fucked up anyway, so in the end I just hardcoded a list of existing permissions. Oh, well. I’ll get around to it.

Also I added CSRF protection everywhere. Whoops. Like I said, spline is still lacking in a lot of niceties, such as “being ready for production use”. But it’s getting there, one architecture astronauting session at a time.

While I was in there I finally added UI so Glip can attach videos and cutscenes to Floraverse pages without my intervention. It was pretty easy and I don’t know why I subjected myself to messing with the db manually for so long.

This isn’t very long or exciting; it was my project and I knew what I was doing, and there was a lot of pondering involved, and I don’t have anything to complain about.

---

Which is why I’m using it to start off a dev log, containing shorter posts about things I have done that don’t merit some deep dive into obscure technology. I also started keeping a notebook (a real, physical notebook) for jotting down stuff I do every day, and maybe I’ll summarize it once a week or so. I’ll also post about little “releases” like Mario Maker levels. In fact I might go make backdated posts for all the levels I’ve made so far.

Remember, if you’re following via the Atom feed and only want to see the blog, there’s a feed with only blog posts.

I’m not sure what this means for the projects page, which has always been kind of a mess. It’s also annoying that you can’t easily filter by project, because they’re just tags, and it’s not obvious which tags are projects. I’ll figure this out as I go, I suppose.

Sketch bought a day of my time this month (and, er, last month) to work on adding “pull requests” to a git-backed wiki. I picked spline, because, uh, I wrote it.

Don’t use pickle. Don’t use pickle. Don’t use pickle.

The problems with Python’s pickle module are extensively documented (and repeated). It’s unsafe by default: untrusted pickles can execute arbitrary Python code. Its automatic, magical behavior shackles you to the internals of your classes in non-obvious ways. You can’t even easily tell which classes are baked forever into your pickles. Once a pickle breaks, figuring out why and where and how to fix it is an utter nightmare.

Don’t use pickle.

So we keep saying. But people keep using pickle. Because we don’t offer any real alternatives. Oops.

You can fix pickle, of course, by writing a bunch of __setstate__ and __reduce_ex__ methods, and maybe using the copyreg module that you didn’t know existed, and oops that didn’t work, and it’s trial and error figuring out which types you actually need to write this code for, and all you have to do is overlook one type and all your rigor was for nothing.

What about PyYAML? Oops, same problems: it’s dangerous by default, it shackles you to your class internals, it’s possible to be rigorous but hard to enforce it.

Okay, how about that thing Alex Gaynor told me to do at PyCon, where I write custom load and dump methods on my classes that just spit out JSON? Sure, you can do that. But if you want to serialize a nested object, then you have to manually call dump on it, and it has to not do the JSON dumping itself. There’s also the slight disadvantage that all the knowledge about what the data means is locked in your application, in code — if all you have to look at is the JSON itself, there’s no metadata besides “version”. You can’t even tell if your codebase can still load a document without, well, just trying to load it. We’re really talking about rolling ad-hoc data formats here, so I think that’s a shame.

But I have good news: I have solved all of your problems.

(This article has been translated into Russian by Everycloudtech—thanks!)

I have a hilariously unfinished Python module I work on from time to time named sanpera. It’s an imaging library for Python, with the vain hope that it might replace PIL someday. But this isn’t about sanpera.

sanpera happens to be powered by ImageMagick. I distinguish this from being an “ImageMagick wrapper”, as it explicitly has nothing resembling the ImageMagick API, because said API is insane. But this isn’t about ImageMagick, either.

Using ImageMagick requires binding Python to C, and that’s what this is about. There are several ways to use C libraries from Python:

• Writing an extension module means the Python API is defined in C, so the library is used exactly as it was intended: with C code. Unfortunately this requires writing a lot of C, as well as a lot of careful Python refcounting. My C is passable, but I’ve done far more reading it than writing it, so this is not an appealing option.

• ctypes is a standard-library module that can load shared libraries and call functions from them without the use of a C compiler or any new C code. Convenient, especially if you hate dependencies (in which case why are you binding to C?), but all the ctypes-powered code I’ve read has been tedious and fiddly and ugly.

• Cython, a spiritual port/evolution/fork/something of the older Pyrex, is a language similar to Python that translates to C and then compiles into an extension module. Cython code can define Python classes and functions, but also call C functions and perform other C operations directly. Code with C semantics is translated fairly directly to C; code with Python semantics is translated to appropriate use of the CPython API; and Cython fills in all the bits to translate between the two.

I went with Cython because it looked interesting, it seemed to reduce the number of translation layers I’d have to care about, and it would even let me write hot loops (this is an imaging library) in C without actually writing any C. Plus, since it’s not actually Python code, it can compile to both Python 2 and Python 3 extension modules with very little effort on my part.

Here’s what Cython looks like:

This is a story of how nothing in this story is my fault.

I’ve got a coworker who’s super into Clojure, a Lisp-like that runs on the JVM. In particular he’s super into how it’s got notions of mutability (and, thus, immutability) all throughout.

More than once he’s lamented that Python lacks a frozendict—a dictionary type that can’t be changed. Dictionaries tend to crop up a lot in Python, and in a very large codebase, it’s very easy to end up with this scenario:

1. Some function somewhere generates a dict that’s only used by one caller. The interface is obvious since the keys and values are created right there in one place. No problem.
2. Other code comes along, notices this handy function, and starts using its return value. Some of this code may pass the ad-hoc dict up to callers, too.
3. Some of that other code needs more things added to the dict, but computing the extra data is expensive, so arguments are added to the function that optionally turn on certain keys.
4. Some code needs even more things added to the dict that are outside the purview of the original function, so they add helper functions that take the ad-hoc dict and add more things to it.
5. Since this has all now happened multiple times throughout your codebase, someone addresses the problem by writing adapter code that infers the original dict from some other object describing its structure, thus saving everyone from writing all these functions that return dicts.

A lovely spaghetti dinner. It’s now nigh impossible to trace what the dict contains or where half of it came from.

Returning an object in the first place would have avoided much of this, but when you’re sitting at step 1, that seems like a lot of effort just to return half a dozen things from a function you wrote to another function you wrote. Swapping out dict() for frozendict() is easy.

My opinion on frozendict had never grown stronger than “I guess that would be cool”, so I never sat down and wrote the class, and there must be enough subtleties that nobody else at Yelp has either.

Then today, PEP 416 came to my attention. This PEP proposed adding a frozendict type, but was rejected last year as being largely unnecessary. What’s interesting about it is that the rejection ends with almost a footnote suggesting that perhaps dictproxy ought to be exposed to Python-land, instead. And indeed this was done, and it exists in Python 3.3.

Almost a year ago now, Jack Diederich gave a talk entitled “Stop Writing Classes“, in which he implores Python programmers to stop creating classes just for the hell of it, and specifically calls out the common pattern of a class with only a constructor/initializer and a single method—which should, of course, just be a function.

A few weeks ago, Armin Ronacher wrote a rebuttal entitled “Start Writing More Classes“, which argues that classes are essential for both writing extensible code and smoothing over crappy interfaces. (Hm. Now that I look at it again, if you read the post backwards, it almost sounds like he’s suggesting writing a class to smooth out the crappy interface you get from using too many classes…)

I’m having some trouble here, because I agree with both points of view. There must be a way to resolve this contradiction, a message that resonates with everyone.

I think I’ve found it.

Stop writing stupid classes.

It’s hard to get a feel for a new tool. Is it any good? Does it do anything I can’t already do? What’s the community like? Tough questions to answer without diving in and using it for a significant amount of time—and then you risk not liking the answers you get.

But fear not! I have discovered a new and brilliant way to discern the novel features of a tool, the vibrance of its community, and its range of users all at once. In mere minutes.

Look at its ten highest-voted questions on StackOverflow.

I’m totally serious. Watch.

From TechCrunch:

Anyway, my sympathy for PHP’s deviltry is because I appreciate its ethos. Its just-get-it-done attitude. Or, as Melvin Tercan put it in his recent blog post, “here’s to the PHP Misfits. The pragmatic ones who would pick up anything – even double-clawed hammers – to build their own future. Often ridiculed and belittled by the hip guys in class who write cool code in Ruby or Python, but always the ones who just get shit done.”

He’s on to something there. The best is the enemy of the good, and shipping some working PHP code is approximately a million times better than designing something mindblowing in Haskell that never actually ships. I fully support Jeff Atwood’s call to replace PHP once and for all–but I hope that everyone realizes that eliminating its many, many, multitudinous flaws won’t be enough; they’ll have to somehow duplicate its just-make-it-work ethos, too.

This is a recurring sentiment: developers telling me, well, yeah, Python may be all cool in your ivory tower, man, but like, I just want to write some programs.

To which I say: what the fuck are you people smoking? Whence comes this belief that anything claimed to be a better tool must be some hellacious academic-only monstrosity which actively resists real-world use?

But, hey, I’m sick of talking about PHP. So let’s talk about Python. In honor of the 90s, let’s make a guestbook.

Part of my Python FAQ.

How do I pass by reference? Does Python pass by reference or pass by value?

Part of my Python FAQ.

How does @property work? Why does it call my __getattr__? What’s a “descriptor”?

Part of my Python FAQ.

I only know PHP. How do I write a Web application in Python?

Part of my Python FAQ.

What does is do? Should I use is or ==?