Functional programming with Python

Christian Harms's picture

To work with lists many c/c++ programmer use a simple for-loop. I will try to explain how this can be made a little bit easier. For that I use the self-explaining script language python.

Here is a simple example to filter all odd integers from a list in simple python 2.x syntax.

def filter_odd(myList):  
   result = []               # initialize the result list  
   for num in myList:               # iterate over the list  
     if num % 2 == 1:        # modulo 2 return 1 for odd integers
        result.append(num)   # add to the result
   return result                

The function call will return a list containing only odd integers (you can use the result of the modulo operation directly without comparing with 1). Try it directly on a interactive python console (as an applet on secapps.com).

>> print filter_odd([1,2,3,4,5])
[1, 3, 5]

try it functional

The function filter_odd could be a more generic method, if the condition could be a variable or used as a parameter:

def condition_odd(num):        # the condition as separate function
  return num % 2 == 1
#condition_odd

def generic_filter(condition, myList):
  result = []
  for num in myList:
    if condition(num):         # the same condition like the first example, only as parameter
      result.append(num)
  return result
#generic_filter

We can define a condition as a function and use it as a parameter in the generic_filter implementation to filter the values of list myList. Ok, you don't have to define a generic filter function, python offers the filter function in the standard language.

>> print filter_generic(condition_odd, [1,2,3,4,5])
[1, 3, 5]
>>print filter(condition_odd, [1,2,3,4,5])
[1, 3, 5]

The new implementation does not say what to do, it say how to use the condition on a list and has much less code compared to first example.

Passing a function as a parameter is a key aspect of functional programming. But the condition_odd function will only be used for the line with the filter call. The next step will show you, how to use this function parameter without defining it separately.

>> print filter(lambda x: x % 2 == 1, [1,2,3,4,5])
[1, 3, 5]

The first parameter is the lambda construct: Create a temporary, anonymous function with x as parameter and return the result of the term "x % 2 == 1". I don't have to define a function and I can use the filter in one line. Save namespace with one-time functions and code lines.

list comprehension

One-line solutions are fine to write short code, but it also has to be readable. If the reader doesn't know what filter and lambda means, it can be confusing.

An other alternative for the one-line filter solution, and more confusing in python, is list comprehension. You can write the above example:

>> print [x for x in [1,2,3,4,5] if x % 2 == 1]
[1, 3, 5]

The result contains items from the constant list, filtered with the term in the if-condition. [Updated] You have to use the exact syntax, not the simple variant like filter.

more functional helper: map and reduce

There are some other list functions. First the map function with calls for every item in the list a function. In a example we will double every integer from the list. This could also be written as a list comprehension.

>>print map(lambda x: x*2, [1,2,3,4,5])
[2, 4, 6, 8, 10]
>>print [x**2 for x in [1,2,3,4,5]]
[2, 4, 6, 8, 10]

Second function is reduce. It get a function and a list as parameter and will call the function with the first two items from the list. Then the result and the next item will be computed and so on. A simple exapmle will calculate the sum of a number list:

>>print reduce(lambda a, b: a + b, [1,2,3,4,5])
15

This simple example can also resolved with the build-in sum function.

two examples

Now we will try some real world examples - project eulers first problem: Find the sum of all the multiples of 3 or 5 below 1000. Simply to resolve with a for-loop:

  1. the condition term is x%3==0 or x%5==0
  2. filter the values beetween 0 and 1000
  3. calculate the sum

def eulerOne(myList):
  mySum = 0
  for num in range(1000):
    if num%3==0 or num%5==0:
      mySum += num
  return mySum

But we want to do it with an one-liner, with functional programming or a list comprehension:

>>print reduce(lambda a,b:a + b, [x for x in range(1000) if x % 3 == 0 or x % 5 == 0])
233168
>>print sum([x for x in range(1000) if not (x%3 and x%5)])
233168

The second line show a shorter version with sum and a little bit boolean algebra but it will not so clear readable.

As a second example filter all persons who older than 18:

>>persons = [{'age': 5, 'name': 'peter'}, {'age': 20, 'name': 'paul'}]
>>print  [x['name'] for x in persons if x['age'] > 18]
['paul']
>>print map(lambda x:x['name'], filter(lambda x: x['age']>19, persons))
['paul']

summary

Here the summary with the python doc-strings to the new learned functions:

filter(function or None, sequence) -> list, tuple, or string
Return those items of sequence for which function(item) is true. If
function is None, return the items that are true. If sequence is a tuple
or string, return the same type, else return a list.
map(function, sequence[, sequence, ...]) -> list
Return a list of the results of applying the function to the items of
the argument sequence(s). If more than one sequence is given, the
function is called with an argument list consisting of the corresponding
item of each sequence, substituting None for missing values when not all
sequences have the same length. If the function is None, return a list of
the items of the sequence (or a list of tuples if more than one sequence).
reduce(function, sequence[, initial]) -> value
Apply a function of two arguments cumulatively to the items of a sequence,
from left to right, so as to reduce the sequence to a single value.
For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates
((((1+2)+3)+4)+5). If initial is present, it is placed before the items
of the sequence in the calculation, and serves as a default when the
sequence is empty.
lambda - syntax
normal function definition:
def name(arguments):
  return expression

anonymous lambda function definition:
lambda arguments: expression

[Updated]: filter and map are still in python 3000 in the iterator variant, the reduce-function is moved in the functools module.

Comments

timb's picture

"The result contains items from the constant list, filtered with the term in the if-condition. But you can't call it with the pre-defined function"

I may be misunderstanding you, but you can filter with a function:

>>> [x for x in [1,2,3,4,5,6] if condition_odd(x)]
[1, 3, 5]

On another topic, requiring users to register in order to leave a comment is discouraging.

Compare your current flow:

1. Click register
2. Fill out form
3. Check email for verification key, click email link
4. Change one-time password
5. Navigate back to original post
6. Fill out comment form

It's too many hoops to jump through versus something like:

1. Fill out comment form

Nico Heid's picture

on leaving comments:

I agree with you on the workflow. The comments are now available for everyone without registration. It was caused by wrong configuration and not intention. Sorry for that.

on the coding part:
Christian will look into it for sure, but may take a little time.

thanks again

Christian Harms's picture

Yes, your are right. The difference between filter and a list comprehension with if-Condition is only the syntax. With the functional syntax you need only a function and the list, with the list comprehension you need the "for x in list" and the "if ".

The more benefit for the filter-variant I will post the follow-up article next week.

@Nico: Yes free comments or a integrationg like disqus would be fine ;-)

Anonymous's picture

The function filter, map and lambda are still in python 3000 in the functools module, only reduce is skipped.

This is completely and utterly wrong.

lambda is a statement, not a function, and it doesn't move (statements aren't namespaced after all). filter and map (as well as zip) stay in __builtins__, but they now return an iterator instead of a list. As a result, itertools.imap, itertools.ifilter and itertools.izip are redundant and have been removed.

reduce on the other hand, was initially proposed for removal and ends up being moved to functools.

Proof:

$ python3.0
Python 3.0.1 (r301:69556, Apr 1 2009, 11:42:54)
[GCC 4.0.1 (Apple Inc. build 5490)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> map
<class 'map'>
>>> filter
<class 'filter'>
>>> zip
<class 'zip'>
>>> import functools
>>> functools.reduce
<built-in function reduce>
>>> import itertools
>>> itertools.imap
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'imap'
>>> itertools.ifilter
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'ifilter'
>>> itertools.izip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'izip'
>>>

Christian Harms's picture

Thanx for correcting me - in my first article I had to check carefully all parts of the article. I will update the article.

Anonymous's picture

Hi Christian,

I was really supprised to see your article today on "reddit-python" :) Hope things are still fine in KA...

In the intro to map/reduce you say "..we will square every integer from the list" but you then just multiplied each element by 2.

Maybe you missed one asterix:

print map(lambda x: x**2, [1,2,3,4,5])

Greets from Munich,
Lothar K.

Christian Harms's picture

Yes, very attentive - I have edit the word square to double ;-)

Greets to munich - nice to see you here with my first article. We will try with this blog to bring some interessting aspect of our daily study to the world ... do subscribe our litte blog - NOW!

Anonymous's picture

Actually, it is correct to say square. The ** in python is referred to as the exponent operator. In this case, you're raising each list element to the power of 2 (aka: squaring it).

Anonymous's picture

This article has been shared on favSHARE.net.

IPv6 request crashed my google app engine application | unit's picture

[...] you want to calculate a number from a IPv6 address try this short functional python code: def [...]

The rotate google contest in 15 lines | united-coders.com's picture

[...] And to do this with a complete list of lines use a list comprehension (read more about functional programming in python). [...]

Google code jam solution for alien language | united-coders.'s picture

[...] the complete solution uses the filter function again (read more about the functional part of python) to shrink the [...]

Javascript Functional programming with Underscore or ECMAScr's picture

[...] time to code more functional with javascript (last year's entry was about functional programming with python). First I will show the normal loop-variant. After the old variant I will show the functional [...]

10 one-line solutions for project euler | united-coders.com's picture

[...] solving some project euler problems with python I got some one-line solutions with nice usage of list comprehensions and functional programming. These code examples are not clean code but a challenge to find more one-line solutions. This [...]