addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupsimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1outlookpersonJoin Group on CardStartprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

Re: [python-189] Python performance anti-patterns

From: glyph
Sent on: Monday, January 7, 2013 10:28 AM

On Jan 7, 2013, at 8:49 AM, Simeon Franklin <[address removed]> wrote:

I'm wondering if there are any language specific anti-patterns in
Python I should be aware of. I'm not thinking things like choosing the
right algorithm or general observations (don't use an ORM) but
wondering if there are any language level features that may not
perform well.

I can think of legacy issues that are mostly fixed: string
concatenation is now optimized in CPython AFAIK (although I still tend
to join arrays of strings) and there is no need for DSU when
sorting... But for current versions of Python the only things that
jump to mind are observations that hold true for most dynamic
languages: minimize dot lookups, don't overuse objects, etc. Are there
any typical Python gotcha's that I'm missing?

Your feedback is appreciated!

I think the most prevalent performance antipattern in modern Python is continuing to care about CPython's performance, rather than optimizing for PyPy :).  PyPy is significantly faster, and also tends to perform better with clearly-expressed, straightforward Python code, whereas CPython's performance gotchas are all things that require weird distortions of your original intent.

Once you've optimized your code at an algorithmic level, switching to PyPy will likely dwarf any benefit you might get from mangling all your code to suit your preferred version of CPython's idiosyncrasies.

For example, one piece of performance advice that crops up every so often is "avoid function calls, they're slow".  Of course, you can't simply avoid one thing, you have to replace it with something else to accomplish the same purpose.  So what to replace it with?  Usually a big hairy ball of manually-inlined code.  PyPy optimizes by finding small loops and then JITing them.  If you manually inline lots of code, that actually makes it *harder* for your code to get automatically JITed.

Another is "use tuples instead of classes, dot lookups are slow".  PyPy can automatically turn a class into something like a C struct.  I don't know if this is faster than using a tuple, but it's definitely fast enough that it's not worth killing readability by changing 'count = self.width * self.height' 'count = info[5] * info[6]'

Another popular bit of advice is "just write it in C if you think it might be slow".  In PyPy, it's actually faster in many cases to just write things in Python than to pay the penalty of the FFI transition into C and back again.

As always, the main performance lesson is profile before you do anything.  The main antipattern across all languages and all eras is to assume you know how a system will perform before you've tried measuring it.


Our Sponsors

  • Yelp

    Platinum Sponsor, also a host for majority of our events.

  • Twilio

    Chipping in for food and drinks for Project Night at Google

  • Homesuite

    Chipping in for food at Project Night at Google

  • Ampush

    Chipping in for food and drinks for Project Night at Google

  • ModuleQ

    Chipping in for food for Project Night at Google

  • Google

    Providing venue, tutorial leaders and food for Project Night at Google

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy