Home » Coding » Python » Problem of Readability of Python
| Problem of Readability of Python [message #15800] |
Sun, 07 October 2007 13:41  |
 Licheng Fang
Messages: 1
Registered: October 2007 |
Junior Member |
|
|
Python is supposed to be readable, but after programming in Python for
a while I find my Python programs can be more obfuscated than their C/C
++ counterparts sometimes. Part of the reason is that with
heterogeneous lists/tuples at hand, I tend to stuff many things into
the list and *assume* a structure of the list or tuple, instead of
declaring them explicitly as one will do with C structs. So, what used
to be
struct nameval {
char * name;
int val;
} a;
a.name = ...
a.val = ...
becomes cryptic
a[0] = ...
a[1] = ...
Python Tutorial says an empty class can be used to do this. But if
namespaces are implemented as dicts, wouldn't it incur much overhead
if one defines empty classes as such for some very frequently used
data structures of the program?
Any elegant solutions?
|
|
| |
| Re: Problem of Readability of Python [message #17421 is a reply to message #15800 ] |
Sun, 07 October 2007 14:26  |
Bruno Desthuilliers
Messages: 277
Registered: July 2007 |
Senior Member |
|
|
Licheng Fang a écrit :
> Python is supposed to be readable, but after programming in Python for
> a while I find my Python programs can be more obfuscated than their C/C
> ++ counterparts sometimes. Part of the reason is that with
> heterogeneous lists/tuples at hand, I tend to stuff many things into
> the list and *assume* a structure of the list or tuple, instead of
> declaring them explicitly as one will do with C structs. So, what used
> to be
>
> struct nameval {
> char * name;
> int val;
> } a;
>
> a.name = ...
> a.val = ...
>
> becomes cryptic
>
> a[0] = ...
> a[1] = ...
Use dicts, not lists or tuples:
a = dict(name='yadda', val=42)
print a['name']
print a['val']
> Python Tutorial says an empty class can be used to do this. But if
> namespaces are implemented as dicts, wouldn't it incur much overhead
> if one defines empty classes as such for some very frequently used
> data structures of the program?
If you do worry about overhead, then C is your friend !-)
More seriously: what do you use this 'nameval' struct for ? If you
really have an overhead problem, you may want to use a real class using
__slots__ to minimize this problem, but chances are you don't need it.
|
|
| |
| Re: Problem of Readability of Python [message #17423 is a reply to message #15802 ] |
Sun, 07 October 2007 14:44 |
George Sakkis
Messages: 26
Registered: September 2007 |
Junior Member |
|
|
On Oct 7, 2:14 pm, Steven Bethard <steven.beth...@gmail.com> wrote:
> Licheng Fang wrote:
> > Python is supposed to be readable, but after programming in Python for
> > a while I find my Python programs can be more obfuscated than their C/C
> > ++ counterparts sometimes. Part of the reason is that with
> > heterogeneous lists/tuples at hand, I tend to stuff many things into
> > the list and *assume* a structure of the list or tuple, instead of
> > declaring them explicitly as one will do with C structs. So, what used
> > to be
>
> > struct nameval {
> > char * name;
> > int val;
> > } a;
>
> > a.name = ...
> > a.val = ...
>
> > becomes cryptic
>
> > a[0] = ...
> > a[1] = ...
>
> > Python Tutorial says an empty class can be used to do this. But if
> > namespaces are implemented as dicts, wouldn't it incur much overhead
> > if one defines empty classes as such for some very frequently used
> > data structures of the program?
>
> > Any elegant solutions?
>
> You can use __slots__ to make objects consume less memory and have
> slightly better attribute-access performance. Classes for objects that
> need such performance tweaks should start like::
>
> class A(object):
> __slots__ = 'name', 'val'
>
> The recipe below fills in the obvious __init__ method for such classes
> so that the above is pretty much all you need to write:
>
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/5022 37
>
> STeVe
For immutable records, you may also want to check out the named tuples
recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/5002 61
George
|
|
|
| Re: Problem of Readability of Python [message #17424 is a reply to message #17423 ] |
Sun, 07 October 2007 14:48 |
Steven Bethard
Messages: 42
Registered: August 2007 |
Member |
|
|
George Sakkis wrote:
> On Oct 7, 2:14 pm, Steven Bethard <steven.beth...@gmail.com> wrote:
>> Licheng Fang wrote:
>>> Python is supposed to be readable, but after programming in Python for
>>> a while I find my Python programs can be more obfuscated than their C/C
>>> ++ counterparts sometimes. Part of the reason is that with
>>> heterogeneous lists/tuples at hand, I tend to stuff many things into
>>> the list and *assume* a structure of the list or tuple, instead of
>>> declaring them explicitly as one will do with C structs. So, what used
>>> to be
>>> struct nameval {
>>> char * name;
>>> int val;
>>> } a;
>>> a.name = ...
>>> a.val = ...
>>> becomes cryptic
>>> a[0] = ...
>>> a[1] = ...
>>> Python Tutorial says an empty class can be used to do this. But if
>>> namespaces are implemented as dicts, wouldn't it incur much overhead
>>> if one defines empty classes as such for some very frequently used
>>> data structures of the program?
>>> Any elegant solutions?
>> You can use __slots__ to make objects consume less memory and have
>> slightly better attribute-access performance. Classes for objects that
>> need such performance tweaks should start like::
>>
>> class A(object):
>> __slots__ = 'name', 'val'
>>
>> The recipe below fills in the obvious __init__ method for such classes
>> so that the above is pretty much all you need to write:
>>
>> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/5022 37
>
> For immutable records, you may also want to check out the named tuples
> recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/5002 61
Yep, it's linked in the description of the first recipe.
STeVe
|
|
| |
| Re: Problem of Readability of Python [message #17433 is a reply to message #15800 ] |
Sun, 07 October 2007 16:24 |
aleax
Messages: 95
Registered: July 2007 |
Member |
|
|
Licheng Fang <fanglicheng@gmail.com> wrote:
...
> Python Tutorial says an empty class can be used to do this. But if
> namespaces are implemented as dicts, wouldn't it incur much overhead
> if one defines empty classes as such for some very frequently used
> data structures of the program?
Just measure:
$ python -mtimeit -s'class A(object):pass' -s'a=A()' 'a.zop=23'
1000000 loops, best of 3: 0.241 usec per loop
$ python -mtimeit -s'a=[None]' 'a[0]=23'
10000000 loops, best of 3: 0.156 usec per loop
So, the difference, on my 18-months-old laptop, is about 85 nanoseconds
per write-access; if you have a million such accesses in a typical run
of your program, it will slow the program down by about 85 milliseconds.
Is that "much overhead"? If your program does nothing else except those
accesses, maybe, but then why are your writing that program AT ALL?-)
And yes, you CAN save about 1/3 of those 85 nanoseconds by having
'__slots__=["zop"]' in your class A(object)... but that's the kind of
thing one normally does only to tiny parts of one's program that have
been identified by profiling as dramatic bottlenecks, to shave off the
last few nanoseconds in the very last stages of micro-optimization of a
program that's ALMOST, but not QUITE, fast enough... knowing about such
"extreme last-ditch optimization tricks" is of very doubtful value (and
I think I'm qualified to say that, since I _do_ know many of them...:-).
There ARE important performance things to know about Python, but those
worth a few nanoseconds don't matter much.
Alex
|
|
|
| Re: Problem of Readability of Python [message #17436 is a reply to message #15800 ] |
Sun, 07 October 2007 17:24 |
Paul McGuire
Messages: 59
Registered: August 2007 |
Member |
|
|
On Oct 7, 1:07 pm, Licheng Fang <fanglich...@gmail.com> wrote:
> Python is supposed to be readable, but after programming in Python for
> a while I find my Python programs can be more obfuscated than their C/C
> ++ counterparts sometimes. Part of the reason is that with
> heterogeneous lists/tuples at hand, I tend to stuff many things into
> the list and *assume* a structure of the list or tuple, instead of
> declaring them explicitly as one will do with C structs. So, what used
> to be
>
> struct nameval {
> char * name;
> int val;
>
> } a;
>
> a.name = ...
> a.val = ...
>
> becomes cryptic
>
> a[0] = ...
> a[1] = ...
>
> Python Tutorial says an empty class can be used to do this. But if
> namespaces are implemented as dicts, wouldn't it incur much overhead
> if one defines empty classes as such for some very frequently used
> data structures of the program?
>
> Any elegant solutions?
"""Just use a single empty class (such as the AttributeContainer
below)
and then use different instances of the class for different sets
of name/value pairs. (This type of class also goes by the name
Bag,
but that name is too, um, nondescript for me.) You can see from
the
example that there is no requirement for names to be shared,
unshared,
common, or unique.
-- Paul"""
class AttributeContainer(object):
pass
a = AttributeContainer()
a.name = "Lancelot"
a.favorite_color = "blue"
b = AttributeContainer()
b.name = "European swallow"
b.laden = true
b.airspeed = 20
|
|
| |
| Re: Problem of Readability of Python [message #17444 is a reply to message #15802 ] |
Sun, 07 October 2007 18:24 |
John Machin
Messages: 60
Registered: July 2007 |
Member |
|
|
On 8/10/2007 4:14 AM, Steven Bethard wrote:
> Licheng Fang wrote:
>> Python is supposed to be readable, but after programming in Python for
>> a while I find my Python programs can be more obfuscated than their C/C
>> ++ counterparts sometimes. Part of the reason is that with
>> heterogeneous lists/tuples at hand, I tend to stuff many things into
>> the list and *assume* a structure of the list or tuple, instead of
>> declaring them explicitly as one will do with C structs. So, what used
>> to be
>>
>> struct nameval {
>> char * name;
>> int val;
>> } a;
>>
>> a.name = ...
>> a.val = ...
>>
>> becomes cryptic
>>
>> a[0] = ...
>> a[1] = ...
>>
>> Python Tutorial says an empty class can be used to do this. But if
>> namespaces are implemented as dicts, wouldn't it incur much overhead
>> if one defines empty classes as such for some very frequently used
>> data structures of the program?
>>
>> Any elegant solutions?
>
> You can use __slots__ to make objects consume less memory and have
> slightly better attribute-access performance. Classes for objects that
> need such performance tweaks should start like::
>
> class A(object):
> __slots__ = 'name', 'val'
>
> The recipe below fills in the obvious __init__ method for such classes
> so that the above is pretty much all you need to write:
>
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/5022 37
>
>
If not needing/wanting __slots__, something simpler (no metaclasses!)
like the following helps the legibility/utility:
<file record.py>
class BaseRecord(object):
def __init__(self, **kwargs):
for k, v in kwargs.iteritems():
setattr(self, k, v)
def dump(self, text=''):
print '== dumping %s instance: %s' % (self.__class__.__name__,
text)
for k, v in sorted(self.__dict__.iteritems()):
print ' %s: %r' % (k, v)
</file record.py>
Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from record import BaseRecord
>>> class A(BaseRecord):
.... pass
....
>>> class B(BaseRecord):
.... pass
....
>>> a1 = A(foo=1, bar='rab', zot=(1, 2))
>>> a2 = A(foo=2, bar='xxx', zot=(42, 666))
>>> a1.dump()
== dumping A instance:
bar: 'rab'
foo: 1
zot: (1, 2)
>>> a2.dump('more of the same')
== dumping A instance: more of the same
bar: 'xxx'
foo: 2
zot: (42, 666)
>>> a1.ugh = 'poked in'
>>> a1.dump('after poking')
== dumping A instance: after poking
bar: 'rab'
foo: 1
ugh: 'poked in'
zot: (1, 2)
>>> b1 = B()
>>> b1.dump()
== dumping B instance:
>>> b1.esrever = 'esrever'[::-1]
>>> b1.dump()
== dumping B instance:
esrever: 'reverse'
>>>
HTH,
John
|
|
| |
| Re: Problem of Readability of Python [message #17448 is a reply to message #17445 ] |
Sun, 07 October 2007 19:58 |
aleax
Messages: 95
Registered: July 2007 |
Member |
|
|
Steven D'Aprano <steve@REMOVE-THIS-cybersource.com.au> wrote:
> On Sun, 07 Oct 2007 13:24:14 -0700, Alex Martelli wrote:
>
> > And yes, you CAN save about 1/3 of those 85 nanoseconds by having
> > '__slots__=["zop"]' in your class A(object)... but that's the kind of
> > thing one normally does only to tiny parts of one's program that have
> > been identified by profiling as dramatic bottlenecks
>
> Seems to me that:
>
> class Record(object):
> __slots__ = ["x", "y", "z"]
>
>
> has a couple of major advantages over:
>
> class Record(object):
> pass
>
>
> aside from the micro-optimization that classes using __slots__ are faster
> and smaller than classes with __dict__.
>
> (1) The field names are explicit and self-documenting;
> (2) You can't accidentally assign to a mistyped field name without Python
> letting you know immediately.
>
>
> Maybe it's the old Pascal programmer in me coming out, but I think
> they're big advantages.
I'm also an old Pascal programmer (ask anybody who was at IBM in the
'80s who was the most active poster on the TURBO FORUM about Turbo
Pascal, and PASCALVS FORUM about Pascal/Vs...), and yet I consider these
"advantages" to be trivial in most cases compared to the loss in
flexibility, such as the inability to pickle (without bothering to code
an explicit __getstate__) and the inability to "monkey-patch" instances
on the fly -- not to mention the bother of defining a separate 'Record'
class for each and every combination of attributes you might want to put
together.
If you REALLY pine for Pascal's records, you might choose to inherit
from ctypes.Structure, which has the additional "advantages" of
specifying a C type for each field and (a real advantage;-) creating an
appropriate __init__ method.
>>> import ctypes
>>> class Record(ctypes.Structure):
.... _fields_ =
(('x',ctypes.c_float),('y',ctypes.c_float),('z',ctypes.c_flo at)
)
....
>>> r=Record()
>>> r.x
0.0
>>> r=Record(1,2,3)
>>> r.x
1.0
>>> r=Record('zip','zop','zap')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: float expected instead of str instance
See? You get type-checking too -- Pascal looms closer and closer!-)
And if you need an array of 1000 such Records, just use as the type
Record*1000 -- think of the savings in memory (no indirectness, no
overallocations as lists may have...).
If I had any real need for such things, I'd probably use a metaclass (or
class decorator) to also add a nice __repr__ function, etc...
Alex
|
|
| | | |
| Re: Problem of Readability of Python [message #17460 is a reply to message #17448 ] |
Mon, 08 October 2007 00:47 |
Michele Simionato
Messages: 60
Registered: August 2007 |
Member |
|
|
On Oct 7, 7:58 pm, al...@mac.com (Alex Martelli) wrote:
> If you REALLY pine for Pascal's records, you might choose to inherit
> from ctypes.Structure, which has the additional "advantages" of
> specifying a C type for each field and (a real advantage;-) creating an
> appropriate __init__ method.
>
> >>> import ctypes
> >>> class Record(ctypes.Structure):
>
> ... _fields_ =
> (('x',ctypes.c_float),('y',ctypes.c_float),('z',ctypes.c_flo at)
> )
> ...>>> r=Record()
> >>> r.x
> 0.0
> >>> r=Record(1,2,3)
> >>> r.x
> 1.0
> >>> r=Record('zip','zop','zap')
>
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> TypeError: float expected instead of str instance
>
> See? You get type-checking too -- Pascal looms closer and closer!-)
>
> And if you need an array of 1000 such Records, just use as the type
> Record*1000 -- think of the savings in memory (no indirectness, no
> overallocations as lists may have...).
That's very cool Alex! I have just a question: suppose I want
to measure the memory allocation of a million of records made
with ctypes vs the memory allocation of equivalent
records made with __slots__, how do I measure it? Say on Linux,
Mac and Windows?
If ctypes records are more efficient than __slots__ records,
I will ask for deprecation of __slots__ (which is something
I wanted from the beginning!).
Michele Simionato
|
|
|
Goto Forum:
Current Time: Fri May 16 02:41:22 EDT 2008
Total time taken to generate the page: 0.37534 seconds |
Members
Search
Help
Register
Login
Home
