4  Useful functions

• map(): works as an iterator to return a result after applying a function to every item of an iterable. It is used when you want to apply a single transformation function to all the iterable elements.

Code

numbers = (1,2,3,4)
squared_numbers = map(lambda n: n**2, numbers)

# list(squared_numbers) = [1, 4, 9, 16]

• filter(): is a function that extracts elements from an iterable for which a function returns True. It takes a function and some iterable (set, list, tuple,..) and returns a filter object.

Code

def check_even(number):
    if number % 2 == 0:
        return True  

    return False

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
even_numbers = filter(check_even, numbers)

# list(even_numbers) = [2, 4, 6, 8, 10]

• reduce(): does not return a new list based on the function and iterable we’ve passed. Instead, it returns a single value. It can be found in functools module.

Code

from functools import reduce

a = [1, 2, 3, 4]
result = reduce(lambda x,y: x*y, a)

# result = 24

Tip

For map(), filter() and reduce() functions it is more convenient, if possible, to use lambda functions as argument.

• zip(): creates an iterator that will aggregate elements from two or more iterables

Code

list1 = ["a", "b", "c"]
list2 = [1, 2, 3]
dict_result = {key: val for key, val in zip(list1, list2)}

# dict_result = {'a': 1, 'b': 2, 'c': 3}

Tip

Instead of bunch of print statements use logging module.

• partial(): this function allows us to fix a certain number of arguments of a function and generate a new function.

Code

from functools import partial

def f(a, b, c, x):
    return 1000*a + 100*b + 10*c + x

g = partial(f, 3, 1, 4)

# g(1) = 3141

• argsort(): returns the indices of array elements that would sort an array
• del method can be used for erasing variable from memory, or just one variable from a list (instead of remove())
• arange(): returns list of numbers in a given range
• random.shuffle(): randomise the elements
• random.choice(): returns random element
• list.pop(index): deletes and returns the element from a list with specified index
• list.remove(index): deletes the element from a list with specified index
• isinstance(): checks data type
• any(): returns True if any item in an iterable is true, otherwise False
• all(): returns True if all items in an iterable are true, otherwise False
• np.hstack(): stack arrays in sequence horizontally
• np.vstack(): stack arrays in sequence vertically

4.1 Working with files

• “with open(„some.txt“) as f” takes care of closing the file
• .write(), .read(), .close(), .readlines()
• For faster loading it is commonly to use numpy package: np.savetxt(‘some.txt’, np.array(), fmt=‘%.2f’, header=‘..’), np.loadtxt(‘some.txt’)

4.2 Working with dataframes

• Combining dataframes in pandas: append() (for horizontal stacking) and concat() (for vertical stacking)
• Identifying missing values: isnull() and isna()
• Handling missing values: fillna()
• Creating missing values for known indexes: pd.DataFrame(index=…, columns=…)
• df.query(): for extracting data with specified conditions
• When you want to create a new df based on a subset of your initial df, it’s best to use the copy() method
• Pandas also has str() method
• Renaming columns: better to use .rename() function
• Setting index to a column: df.set_index([‘some’]) (it removes column automatically)
• DataFrame.plot makes plots of Series or DataFrame. There is various kinds of plot to produce and can be assigned in „kind“ parameter. Options are: ‘line’ (default), ‘bar’, ‘barh’, ‘hist’, ‘box’, ‘kde’/‘density’, ‘area’, ‘pie’, ‘scatter’, ‘hexbin’’