Python Idioms



  • I wrote my very first Python program last night.  I want to start learning it so that I can use it at work for some of my smaller projects.  However, I also just read the book Dreaming in Code, where I learned that programmers coming to Python from the C-like languages end up writing code that is just like their previous language.  So here is my code - let me know if there is a more "Pythony" way of writing it.  Also, if you know of any standard Python idioms go ahead and post those, along with the C-like code they replace.

    The program is a deranger, i.e. it prints all permutations of a list in which every atom is in a different location from the original list:

    def beginDerange(l):
        return derange(l, l, [], [])

    def derange(initial, remaining, assigned, results):
        if len(remaining) == 0:
            results.append(assigned)
        else:
            for i in range(len(remaining)):
                a, r = assigned[:], remaining[:]
                a.append(r.pop(i))
                last = len(a) - 1
                if a[last] != initial[last]:
                    derange(initial, r, a, results)
        return results

    for l in beginDerange(["a", "b", "c", "d"]):
        print l




  • I'd probably go for the simplest conceptual way to do it: generate all the permutations of your list, then remove the original list.

    Here's a recursive permutations generator

    Note that it's not tail-recursive, there is no point in creating a tail-recursive generator as python doesn't implement tail-recursion optimization.

    def perm(lst):
    	if len(lst) <= 1:
    		yield lst
    	else:
    		for p in perm(lst[1:]):
    			for i in range(len(lst)):
    				yield p[:i] + [lst[0]] + p[i:]

    Then the only thing left is to remove the original list, this could be done by removing the first generated list (which is the original list), but we'll go for the conceptually clearer way of filtering out the original list.

    def derange(lst):
    	return filter(lambda l: lst != l, perm(lst))

    Here you are, just call derange() with a list argument and you're done.

    Plus you now have a completely generic list permutation function which you can use wherever you need it



  • Masklinn has given a good method, although it sound like you want to filter out any lists that match the original list at any point, and not just filter out the original list.

    As for idioms, there are a few cases where you can make things more pythonic. The common idiom for looping over something is "for x in remaining", rather than "for i in range(len(remaining))". If you need the index, and the element, then enumerate is generally better:

        for n,a in enumerate([2,3,5,8]): print n,a

    should give:

    0,2

    1,3

    2,5

    3,8

    If you only need the index (the n in the above example), then use whichever method makes most sense to you. However, the method involving len doesn't work for iterables (objects you can loop over), that don't have a pre-determined length. Also, xrange can be better than range, when you only want to loop over a range, as range generates a list of the elements in the range before looping over them, which can take some time for very large ranges.

    In python, you can index lists from the end. a[-1] is the last element in a, a[-2], is the second-to-last, and so on. This could simplify the "a[last] != intial[last]" bit.

    I hope that helps.



  • @Silverfish said:

    Masklinn has given a good method, although it sound like you want to filter out any lists that match the original list at any point, and not just filter out the original list.

    As for idioms, there are a few cases where you can make things more pythonic. The common idiom for looping over something is "for x in remaining", rather than "for i in range(len(remaining))". If you need the index, and the element, then enumerate is generally better:

        for n,a in enumerate([2,3,5,8]): print n,a

    should give:

    0,2

    1,3

    2,5

    3,8

    If you only need the index (the n in the above example), then use whichever method makes most sense to you. However, the method involving len doesn't work for iterables (objects you can loop over), that don't have a pre-determined length. Also, xrange can be better than range, when you only want to loop over a range, as range generates a list of the elements in the range before looping over them, which can take some time for very large ranges.

    In python, you can index lists from the end. a[-1] is the last element in a, a[-2], is the second-to-last, and so on. This could simplify the "a[last] != intial[last]" bit.

    I hope that helps.

    you don't use a colon in those lists? a[:-1] for the end?

    or is that optional?

     

    I am also learning pyhton.



  • @Silverfish said:

    Masklinn has given a good method, although it sound like you want to filter out any lists that match the original list at any point, and not just filter out the original list.

    Aah very good point, I missed it indeed, that part.

    The correction is not that hard though (even if it makes the code noticeably less readable, Python's not ideal for higher order programming)

    Changing derange to

    def derange(lst):
    	return filter(
    		lambda l: reduce(
    			lambda acc, el: acc and (el[0] != el[1]),
    			zip(lst,l),
    			True),
    		perm(lst))

    should do it.

    @GeneWitch said:
    @Silverfish said:

    Masklinn has given a good method, although it sound like you want to filter out any lists that match the original list at any point, and not just filter out the original list.

    As for idioms, there are a few cases where you can make things more pythonic. The common idiom for looping over something is "for x in remaining", rather than "for i in range(len(remaining))". If you need the index, and the element, then enumerate is generally better:

        for n,a in enumerate([2,3,5,8]): print n,a

    should give:

    0,2

    1,3

    2,5

    3,8

    If you only need the index (the n in the above example), then use whichever method makes most sense to you. However, the method involving len doesn't work for iterables (objects you can loop over), that don't have a pre-determined length. Also, xrange can be better than range, when you only want to loop over a range, as range generates a list of the elements in the range before looping over them, which can take some time for very large ranges.

    In python, you can index lists from the end. a[-1] is the last element in a, a[-2], is the second-to-last, and so on. This could simplify the "a[last] != intial[last]" bit.

    I hope that helps.

    you don't use a colon in those lists? a[:-1] for the end?

    or is that optional?

    They're completely different. [X] is an index while [X:Y:Z] is a slice, composed of 0 to 3 indexes (all of them are optional)

    lst[-1] means "take the last element of the list" while lst[:-1] means "take the list up the last element excluded" and lst[-1:] means "take the list from the last element onwards".

    There's also lst[::-1] which means "take the list with steps of -1".

    They yield the following:

    >>> lst = ['a','b','c','d']
    >>> lst[-1]
    'd' # note that this is an item, not a list
    >>> lst[-1:]
    ['d'] # note that this is a list, not an item
    >>> lst[:-1]
    ['a', 'b', 'c']
    >>> lst[::-1]
    ['d', 'c', 'b', 'a'] # this was idiomatic way to reverse a list before `reversed` was included
    >>> lst[:]
    ['a', 'b', 'c', 'd'] # and this is how you clone a list
    


  • Addendum to my previous post:

    by using a feature of Python akin to tuple pattern matching (which I think is going to disappear in Python 3.0, not sure for 2.6, still works n/p in 2.5) we can make derange ever so slightly cleaner:

    def derange(lst):
    	return filter(
    		lambda l: reduce(
    			lambda acc, (e1, e2): acc and (e1 != e2), # nice trick
    			zip(lst,l),
    			True),
    		perm(lst))


  • @GeneWitch said:

    @Silverfish said:

    [snip]

    In python, you can index lists from the end. a[-1] is the last element in a, a[-2], is the second-to-last, and so on. This could simplify the "a[last] != intial[last]" bit.

    I hope that helps.

    you don't use a colon in those lists? a[:-1] for the end?

    or is that optional?

     

    I am also learning pyhton.

    The colon is to distinguish an element of a list, and a sub-list. So a[-1] is the last element of the list. a[:-1] is the list ending at the 2nd to last element. The Python tutorial gives more details on subscripts in the section on strings (http://docs.python.org/tut/node5.html#SECTION005120000000000000000). As you'll see, the full form with a colon is a[x:y], giving the elements from a[x] to a[y-1] (thus not including a[y]). Omitting x or y means you default to listing from the beginning, or to the end.   



  • @Silverfish said:

    As you'll see, the full form with a colon is a[x:y]

    The full form of a slice is actually a[x:y:z] giving every zth element between the elements of index x (included) and y excluded.

    x, y and z are all optional and default to None (which in fact correspond to x=0, y=length(sliceable) and z=1), the only requirement of the slice syntax is the first colon (which means that a[:], a[::], a[b:], a[:b], a[🅱], a[b::], a[::b], a[b::c], a[🅱c], ... are all valid slices, even though some are equivalent with one another)

    </language-lawyer>



  • Ok, I wrote my second ever Python program in response to the coder challenge over here: http://forums.worsethanfailure.com/forums/ShowThread.aspx?PostID=115804#115804

    I tried to make use of some of your suggestions, but I still don't feel very Pythony.  Anything jump out at you?

    Btw masklinn, you have the best icon ever.



  • A few things could be more "Pythony" (actually, pythonic is the usual term).

    In

    <font face="Lucida Console" size="2">import System
    from System import *</font>

    you do the same (or a similar) thing in two different ways. import foo will import the foo module as a seperate namespace, so to run the bar function in foo, you enter foo.bar(). from foo import * will insert all* the functions, and other objects in the foo module into the current scope. So simply bar() will work. You only need one or the other, not both. The former is safer, as it avoids confusion when you have two functions with the same name. You can also import one or more objects from foo, using from foo import bar1,bar2 to import bar1, and bar2 into the current scope. Then bar1(), and bar2() will run the 2 functions.

    Another idiom that can be useful is a,b = b,a. That will simply swap a and b. The swaprow and swapCol could probably benefit with using that. On that subject, you seem to be using those functions to alter the table, whilst also returning the table. In Python it seems to be more common for a function to do either one or the other. That is, a function will generally either alter one of the arguments passed to it, or return a value, not both.

    I'm not sure what r.<font face="Lucida Console" size="2">NextDouble </font>does. It NextDouble is a function, then make sure to use r.NextDouble() to call it. Python won't complain (unless it leads to some other error) if you try to call a functions without the brackets, but it will do something you didn't expect, as you'll get the function object, rather than the result of calling it.

    I'll leave it at that for now. Don't worry too much about getting it all straight away. Over time you'll become more familiar with Python's little quirks. I'm happy to help if you have any more questions.



  • Hey thanks, I've incorporated your suggestions into my next version.  And good catch on the r.NextDouble - that was probably a bug.

    I've spent the past hour or so messing around with map, reduce, filter and the lambda keyword.  Pretty cool stuff.



  • You might want to investigate list comprehensions, by the way.


    [f(x) for x in a] is an equivalent to map(f,a). It's arguably more clear, and you avoid the use of lambda:
    [x+! for x in a] is the same as map(lambda x:x+1,a).

    Similarly [x for x in a if f(x)] is equivalent to filter(f,a).

    Again, you can avoid lambda: [x for x in a if x  > 0] is the equivalent to filter(lambda x:x>0,a).

    The issue with lambda is you generate a new function object each time you run it, which can slow things down. The tutorial has a section on list comprehensions (http://docs.python.org/tut/node7.html#SECTION007140000000000000000), that'll give you an indication of how flexible they can be.


Log in to reply