There are three functions in python that provide vast practicality and usefulness when programming. These three functions, which provide a functional programming style within the object-oriented python language, are the map(), filter(), and reduce() functions. Not only can these functions be used individually, but they can also be combined to provide even more utility.

In this tutorial, we will cover these three functions and some examples of when they can be useful. But before we start, we need to review what lambda (or anonymous) functions are and how to write them.

Lambda Expressions

Lambda expressions are used to create anonymous functions, or functions without a name. They are useful when we need to create a function that will only need to be used once (a throw-away function) and can be written in one line. As such, they can be very useful to use in functions that take in another function as an argument.

Lambda functions can have any number of parameters but can only have one expression. They generally have this format which yields a function object:

lambda parameters: expression

Lambda Expression with One Parameter
Let’s create a lambda expression that returns the square of a number:

lambda num: num**2

And that’s it! We first start with the lambda keyword, then the parameter num, a colon, and what you want that function to return, which is num**2.

Note that this function is anonymous, or does not have a name. So we cannot invoke the function at a later point. In addition, we did not write the return keyword. Everything after the colon is part of the expression that will be returned.

If we want to assign a lambda function to a variable so that it can be invoked later, we can do so by using an assignment operator:

square = lambda num: num**2

We can then invoke or call this function the same way we would with a function that was defined with the def keyword. For example:

square(3) # will return 9 as the output

Lambda Expression with Multiple Parameters
Let’s write a lambda function that has two parameters instead of just one. For example, we can write a lambda expression that returns the sum of two numbers:

lambda num1, num2: num1+num2

As we can see, if we want our function to have multiple parameters in a lambda expression, we would just separate those parameters by a comma.

Conditional Statements in Lambda Expressions
We can also include if else statements in lambda expressions. We would just need to make sure that it is all on one line. For example, let’s create a function that takes in two numbers and returns the greater of those numbers:

lambda num1, num2: num1 if num1>num2 else num2

Our lambda expression takes in two numbers, num1 and num2, and returns num1 if num1 is greater than num2, else, it returns num2. Obviously this function doesn’t take into account if the numbers are equal, as it will just return num2 in that case, however, we are just illustrating how we can use conditional statements within a lambda expression.

For more on lambda expressions:Lambda Expressions in PythonHow to write anonymous functions in pythontowardsdatascience.com

Now that we understand what lambda functions are, let’s talk about the first function on today’s agenda: the map function.

Map Function

Simply put, the map function provides us a way to apply a function to each element in an iterable object, such as lists, strings, tuples, etc… Thus, the map function takes in two arguments: the function we want to apply, and the iterable object we want to apply it to.

map(function, iterable)

For example, we have a list of numbers (the iterable object), and we want to create a new list that contains the squares of those numbers.

num_list = [1,2,3,4,5]squared_list = list(map(lambda x:x**2, num_list))print(squared_list)
# [1, 4, 9, 16, 25]

The map function will return a map object, which is an iterator. If we want to create a list from this map object, we would need to pass in our map object to the built-in list function.

What happened in this line of code?

squared_list = list(map(lambda x:x**2, num_list))

The map function took the first element from num_list, which is a 1, and passed it in as an argument to the lambda function (since we passed that function in as the first argument to the map function). The lambda function took that argument, 1, and returned 1**2, or 1 squared, and that was added to our map object. The map function then took the second element from num_list, which is 2, and passed it in as an argument to the lambda function. The lambda function returned the square of 2, which is 4, which was then added to our map object. After it finished going through the num_list elements and the rest of our squared numbers were added to the map object, the list function casts this map object onto a list, and that list was assigned to the variable squared_list.

Cryptography: Caesar Cipher

Let’s try a slightly more interesting example involving cryptography, specifically the caesar cipher. The caesar cipher encrypts a message by taking each letter in the message, and replacing it with a shifted letter, or a letter that is a specified number of spaces away in the alphabet. So if we choose the number of spaces to be 1, then each letter will be replaced with the letter 1 space away in the alphabet. So the letter a will be replaced with the letter b, the letter b will be replaced with the letter c, and so on. If we choose the number of spaces to be 2, then a will be replaced with c, and b will be replaced with d.

If while we are counting spaces we get to the end of the alphabet, then we go back to the beginning of the alphabet. In other words, the letter z will be replaced with the letter a (if we shift 1 space), or with the letter b (if we shift 2 spaces).

For example, if the message we want to encrypt is ‘abc’ and we choose the number of spaces to be 1, then the encrypted message will be ‘bcd’. If the message is ‘xyz’, then the encrypted message will be ‘yza’.

How can we use the map() function to accomplish this?
Well, we are applying something to each element of an iterable object. In this case, the iterable object is a string, and we would like to replace each letter/element in our string by a different one. And the map function can do exactly that!

Let’s assume that all messages will be lowercase letters only. And the number of spaces will be a number between 0–26. Remember, we only want to replace letters with other letters. Thus, any non-letter elements, such as a space or symbol, will be unchanged.

We first need access to the lowercase alphabet. We can either write out a string with all the lowercase letters, or we can use the string module as follows:

abc = 'abcdefghijklmnopqrstuvwxyz'orimport stringabc = string.ascii_lowercaseprint(abc)
# 'abcdefghijklmnopqrstuvwxyz'

Then, we can write our encrypt function as follows:

def encrypt(msg, n):
    return ''.join(map(lambda x:abc[(abc.index(x)+n)%26] if x in abc else x, msg))encrypt('how are you?',2)# 'jqy ctg aqw?'

We create the function encrypt with two parameters: the message we want to encrypt, msg, and the number of spaces n we want to shift the letters by. The iterable we pass in to the map function is the message, msg. The function we pass in to the map function will be a lambda function, which takes each element from the msg string, and if the element is a letter in the alphabet, it replaces it with the shifted letter depending on the n value we pass in. It does so by taking the current index of that letter in the alphabet, or abc.index(x), adds the n value to it, and then takes the modulus of that sum. The modulus operator is used to start back at the beginning of the alphabet if we get to the end (if abc.index(x)+n is a number greater than 25). In other words, if the original letter was z (which will have the index 25 in the abc string we created above), and the n value is 2, then (abc.index(x)+n)%26 will end up being 27%26, and that will yield a remainder of 1. Thus replacing the letter z with the letter of the index 1 in the alphabet, which is b.

map(lambda x:abc[(abc.index(x)+n)%26] if x in abc else x,msg)

Remember that the map function will return a map object. Thus, we can use the string method join, which takes in an iterable (which the map object is, since it is an iterator, and all iterators are iterable), and then joins it into a string. The function then returns this string.

To decrypt a message, we can use the following decrypt function (notice how we are subtract n from abc.index(x) instead of adding it):

def decrypt(coded, n):
    return ''.join(map(lambda x:abc[(abc.index(x)-n)%26] if x in abc else x, coded))decrypt('jqy ctg aqw?',2)
# 'how are you?'

Filter Function

Simply put, the filter function will “filter out” an iterable object based on a specified condition. This condition will be decided by the function that we pass in.

filter(function, iterable)

The filter function takes in two arguments: the function that checks for a specific condition and the iterable we want to apply it to (such as a list). The filter function takes each element from our list (or other iterable) and passes it in to the function we give it. If the function with that specific element as an argument returns True, the filter function will add that value to the filter object (that we can then create a list from just like we did with the map object above). If the function returns False, then that element will not be added to our filter object. In other words, we can think of the filter function as filtering our list or sequence based on some condition.

For example, we have a list of numbers, and we want to create a new list that contains only the even numbers from our list.

num_list = [1,2,3,4,5,6]list_of_even_nums = list(filter(lambda x:x%2==0, num_list))print(num_list)
# [2,4,6]

This filter function will return a filter object, which is an iterator. If we want to create a list from this filter object, we would need to pass in our filter object to the built-in list function (just like we did with the map object).

So let’s break down what happened in this line of code:

list_of_even_nums = list(filter(lambda x:x%2==0, num_list))

The filter function took the first element from num_list, which is a 1, and passed it in as an argument to the lambda function (since we passed that function in as the first argument to the filter function). The lambda function then returns False, since 1 is not even, and 1 is not added to our filter object. The filter function then took the second element from num_list, which is 2, and passed it in as an argument to the lambda function. The lambda function returns True, since 2 is even, and thus 2 is added to our filter object. After it goes through the rest of the elements in num_list and the rest of the even numbers are added to our filter object, the list function casts this filter object onto a list, and that list was assigned to the variable list_of_even_nums.

List Comprehensions vs. Map and Filter

If we recall, list comprehensions are used to create lists out of other sequences, either by applying some operation to the elements, by filtering through the elements, or some combination of both. In other words, list comprehensions can have the same functionality as the built-in map and filter functions. The operation applied to each element is similar to the map function, and if we add a condition to which elements are added to the list in the list comprehension, that’s similar to the filter function. Also, the expression that is added in the beginning of a list comprehension is similar to the lambda expression that can be used inside the map and filter functions.

Example:

This is a list comprehension that adds the square of the elements from 0 to 9, only if the element is even:

[x**2 for x in range(10) if x%2==0]# [0,4,16,36,64]

We can use the map and filter functions, along with lambda functions, to accomplish the same thing:

list(map(lambda x:x**2, filter(lambda x:x%2==0, range(10))))# [0,4,16,36,64]

The function passed into the map function is a lambda expression that takes input x and returns its square. The list passed into the map function is a filtered list that contains the even elements from 0 to 9.

For more on list comprehensions:List Comprehensions in PythonA more elegant and concise way to create lists in pythontowardsdatascience.com

Reduce Function

Simply put, the reduce function is taking an iterable, such as a list, and reduces it down to a single cumulative value. The reduce function can take in three arguments, two of which are required. The two required arguments are: a function (that itself takes in two arguments), and an iterable (such as a list). The third argument, which is an initializer, is optional.

reduce(function, iterable[, initializer])

To use the reduce function, we need to import it as follows:

import functoolsorfrom functools import reduce

The first argument that reduce takes in, the function, must itself take in two arguments. Reduce will then apply this function cumulatively to the elements of the iterable (from left to right), and reduces it to a single value.

Examples:

Let’s say the iterable we use is a list of numbers, such as num_list:

num_list = [1,2,3,4,5]

We would like to take the product of all the numbers in num_list. We can do so as follows:

from functools import reduceproduct = reduce(lambda x,y:x*y, num_list)print(product)
# 120

Our iterable object is num_list, which is the list: [1,2,3,4,5]. Our lambda function takes in two arguments, x and y. Reduce will start by taking the first two elements of num_list, 1 and 2, and passes them in to the lambda function as the x and y arguments. The lambdafunction returns their product, or 1 * 2, which equals 2. Reduce will then use this accumulated value of 2 as the new or updated x value, and uses the next element in num_list, which is 3, as our new or updated y value. It then sends these two values (2 and 3) as x and y to our lambda function, which then returns their product, 2 * 3, or 6. This 6 will then be used as our new or updated x value, and the next element in num_list will be used as our new or updated y value, which is 4. It then sends 6 and 4 as our x and y values to the lambda function, which returns 24. Thus, our new x value is 24, and our new y value is the next element from num_list, or 5. These two values are passed to the lambda function as our x and y values, and it returns their product, 24 * 5, which equals 120. Thus, reduce took an iterable object, in this case num_list, and reduced it down to a single value, 120. This value is then assigned to the variable product. In other words, the x argument gets updated with the accumulated value, and the y argument gets updated from the iterable.

Mathematically, you can think of reduce as doing the following:

f(f(f(f(f(x,y),y),y),y),y)

So the first time the lambda function runs, it takes x and y (the first two elements in the list), and returns an output. That output for f(x,y) will then be used as the new x for the next iteration, and y will be the next element in the list, thus f(f(x,y),y). And so on.

Reduce Third Argument: Initializer

Remember how we said that reduce can take in an optional third argument, the initializer? The default value for it is None. If we pass in an initializer, it will be used as the first x value by reduce (instead of x being the first element of the iterable). So if we pass in the number 2 in the above example as the initializer:

product = reduce(lambda x,y:x*y, num_list, 2)print(product) 
#240

Then the first two values or arguments for x and y will be 2 and 1, respectively. All subsequent steps will be the same. In other words, the initializer is placed before the elements of our iterable in the calculation.

Other Examples of Using Reduce

There are so many other scenarios of when we can use reduce.

For example, we can find the sum of the numbers in a list:

num_list = [1,2,3,4,5]sum = reduce(lambda x,y: x + y, num_list)print(sum) 
#15

Or we can find the maximum number in a list:

num_list = [1,2,3,4,5]max_num = reduce(lambda x,y: x if x > y else y, num_list)print(max_num) 
#5

Or the minimum number in a list:

num_list = [1,2,3,4,5]min_num = reduce(lambda x,y: x if x < y else y, num_list)print(min_num) 
#1

And so many other applications!

Note: Python does have built-in functions such as max(), min(), and sum() that would have been easier to use for these three examples. However, the goal was to show how reduce() can be used to accomplish many different tasks.

We can also combine the reduce function along with the filter or map function. For example, if we have a list of numbers, but we only want to return the product of the odd numbers only, we can do so using reduce and filter as follows:

num_list = [1,2,3,4,5]product_of_odd_nums = reduce(lambda x,y: x*y, filter(lambda x:x%2!=0, num_list))print(product_of_odd_nums) 
# 15 (since 1*3*5=15)

In other words, the iterable we pass in to the reduce function is a filter object, which only includes the odd numbers. We can think of it as a list of only the odd numbers. Then, the reduce function will take that filtered list and returns the product of those numbers.

Conclusion

In this tutorial, we first looked at how we can create lambda (or anonymous) functions. We then learned about the map function, which gives a way to apply some function to each element in an iterable object. We then looked at the filter function, and how it can be used to filter an iterable object based on some condition. Furthermore, we compared the map and filter functions to list comprehensions. Next, we looked at how the reduce function can be used to reduce an entire iterable object down into a single value. We also saw examples of all of these functions being used, including an example involving cryptography. Lastly, we learned how these functions can be combined in order to provide even more functionality.