High-performance mathematical paradigms in Python

Comparing the performances of different implementation paradigms in python

Image for post
Image for post

Functions defining mathematical expressions

Dealing with a large array of numbers is always a challenging task and if there is a numerical operation to be run over those elements, the story changes altogether.

n = 1000000
data_num = range(n)
otpt = []
for x in data_list:
otpt.append(f(x))
return [f(x) for x in data_list]
exp = 'abs(cos(x) + sin(x)) ** 0.5'  
return [eval(exp) for x in data_list]
import numpy as np
data_np = np.arange(1000000)
def f4(data_np):
"""using the numpy vectorization
"""
return (np.abs(np.cos(data_np) + sin(data_np)) ** 0.5)
import numexpr as ne
ex = 'abs(cos(a) + sin(a)) ** 0.5'
ne.set_num_threads(5)
return ne.evaluate(ex)
np.allclose(otpt1, otpt2)
np.allclose(otpt1, otpt3)
np.allclose(otpt1, otpt4)
np.allclose(otpt1, otpt5)
function_list = ['f1', 'f2', 'f3', 'f4', 'f5']
data_com_list = ['data_num', 'data_num', 'data_num', 'data_np', 'data_np']
record_comp_time(function_list, data_com_list)
Image for post
Image for post

Conclusion

For every data intensive operation, python has a number of ways to improve the performance of your code. Given a problem, optimality can be worked upon by using the right combination of different paradigms and choice of libraries. Besides paradigms, there are some very useful libraries which improves the execution speed of the python code.

  • Numba,- it dynamically compiles python code for the CPU.
  • Python’s built-in module multiprocessing for parallel processing

Consultant | Web & Data Science Instructor | My YouTube channel on Data Science: https://www.youtube.com/c/DataSciencewithHarshit

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store