Re: [python-189] Python performance anti-patterns

From: Valentino
Sent on: Thursday, January 10, 2013 11:12 PM
That's true but they also allow to avoid unnecessary loops so while doing:

for i in range(15):
    doSomething()

is equivalent to:

i = 0
items = []
while i < 15:
    items.append(i)
    i += 1
for i in items:
    doSomething()

the version with the generator only loops once and for big values of i will end up in a faster script overall.


On Thu, Jan 10, 2013 at 8:42 PM, David Cramer <[address removed]> wrote:
Anyone reading this should probably ignore most things in this thread.

Generators consume less memory, because they yield results. That is, they don't copy the instance.

What you just did is the same as doing range() except you explicitally added a loop to it.

Generators == less memory, not more speed

On Thursday, January 10, 2013 at 6:45 PM, Bryce Verdier wrote:

I found a little tid bit when playing with code one time. I was told that generators were preferred to list comps... but I found a case where that isn't true.

In [2]: %prun sum(x for x in xrange(1000000))
         1000004 function calls in 0.344 seconds

n [3]: %prun sum([x for x in xrange(1000000)])
         3 function calls in 0.195 seconds

not sure if this is a anti-pattern per se. But I thought I would contribute it to the conversation anyway.

Bryce



On 01/10/2013 06:05 PM, Stephen wrote:
I'm quibbling your comment "overuse of dicts as data structures".

They are different things but dicts and namedtuples can be used to implement lightweight objects.

It is not necessarily an anti-pattern to use a dict of dicts [...] to back an object, unless that actually causes
a real performance issue. Much of the time it doesn't and so this can be  good coding style when writing new code.
Once you know where your bottleneck is you just rewrite.

Not meaning to take a detour from the discussion of performance.


From: [address removed]

To: [address removed]
Subject: Re: [python-189] Python performance anti-patterns
Date: Thu, 10 Jan 2013 20:23:26 -0500

Please, don't confuse objects and dicts. They're entirely different things. One is an approximately fixed set of attributes combined with behaviors, the other is an arbitrary pairing of keys to values, with no behaviors. These are not the same things.

Alex


On Thu, Jan 10, 2013 at 5:06 PM, Stephen <[address removed]> wrote:
Yeah I've also been wondering if namedtuple is a good standin as a lightweight for writing a class whose
members vary a lot or names or are not statically known at creation time or method-writing time.
But that's not a performance question.

You can make navdata.drone.camera.translation.x resolve to navdata['drone']['camera']['translation']['x']
by rehooking __getattribute__() ; see http://stackoverflow.com/questions/371753/python-using-getattribute-method
Then that hides the implementation details of whether you have a 'proper' object or not and how to access its members.

Stephen


From: [address removed]
To: [address removed]
Subject: Re: [python-189] Python performance anti-patterns
Date: Thu, 10 Jan 2013 19:34:18 -0500


How about overuse of dicts as data structures?  Dict ease of use (a really nice thing about Python) can lead to abuse.

There are times when accessing a dict's dict's dict's dict, e.g. navdata['drone']['camera']['translation']['x'] might make sense, but there are times where it feels like a shortcut and I'd like real objects (or at least namedtuples) with the benefits that offers: works with static analysis, better documented/more readable.


On Thu, Jan 10, 2013 at 4:05 PM, Simeon Franklin <[address removed]> wrote:
On Thu, Jan 10, 2013 at 12:34 PM, Alex Gaynor <[address removed]> wrote:
I really cannot imagine anything less import than dict() vs. {}:

I agree with you (and Brendan and Glyph) - I really try to use the language features that make the most sense from a readability perspective. I'm not looking for micro-optimizations here and it sounds like the answer to my question (so far) is a resounding "no".

The example I mentioned (that no longer applies) was worth avoiding. In older versions of Python doing the naively straightforward thing (+= to accumulate a string) could be orders of magnitude slower than appending to an array and then .joining the array on an empty string separator. See eg http://www.skymind.com/~ocrow/python_string/ from circa 2004. All the advice so far is good general advice - profile, use the right data structures, use PyPy - but none of it answers my question directly. In this case a negative result is a good one - I'm happy not to have to mangle my code to avoid "gotchas" in the language*.

-regards
Simeon Franklin

* Actually the one other "gotcha" I've considered is that due to the GIL threads are not as useful as they are in some other languages to deal with performance problems... But that approaches more the "choose the right algorithms and data structures" level of knowledge.




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Simeon Franklin ([address removed]) from San Francisco Python Meetup Group.
To learn more about Simeon Franklin, visit his/her member profile





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by John Wiseman ([address removed]) from San Francisco Python Meetup Group.
To learn more about John Wiseman, visit his/her member profile
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Stephen ([address removed]) from San Francisco Python Meetup Group.
To learn more about Stephen, visit his/her member profile



--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Alex Gaynor ([address removed]) from San Francisco Python Meetup Group.
To learn more about Alex Gaynor, visit his/her member profile
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]




--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Stephen ([address removed]) from San Francisco Python Meetup Group.
To learn more about Stephen, visit his/her member profile
Set my mailing list to email me As they are sent | In one daily email | Don't send me mailing list messages

Meetup, POB 4668 #37895 NY NY USA 10163 | [address removed]





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by Bryce Verdier ([address removed]) from San Francisco Python Meetup Group.
To learn more about Bryce Verdier, visit his/her member profile





--
Please Note: If you hit "REPLY", your message will be sent to everyone on this mailing list ([address removed])
This message was sent by David Cramer ([address removed]) from San Francisco Python Meetup Group.
To learn more about David Cramer, visit his/her member profile



--
Valentino Volonghi

Our Sponsors

  • Yelp

    Providing food, beverages, venue, and a good time!

People in this
Meetup are also in:

Log in

Not registered with us yet?

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy