Building 3D Arrays In Python To Replace Loops For Optimization

I'm trying to better understand python optimization so this is a dummy case, but hopefully outlines my idea... Say I have a function which takes two variables: def func(param1, par

Solution 1:

Your example:

In [198]: result=[]
In [199]: for p in param1:
   .....:     result.append(p+3*(param2/2))
In [200]: result=np.array(result)

Same result using broadcasting (and np.newaxis):

In [197]: param1[:,None] + 3*(param2[None,:]/2)
Out[197]: 
array([[ 7,  4,  4,  1, 13, 13, 16],
       [ 8,  5,  5,  2, 14, 14, 17],
       [ 9,  6,  6,  3, 15, 15, 18],
       [10,  7,  7,  4, 16, 16, 19],
       [11,  8,  8,  5, 17, 17, 20]])

Details of some_func would determine whether you use some_func(param1[:,None]) or some_func(param1)[:,None].

Solution 2:

Original answer for some_func(param1) x param2

Write the some_func in such a way that it can accept and return numpy arrays. Then use;

numpy.outer(some_func(param1), param2)

This works because in your example, both param1 and param2 are vectors (1D arrays), so you can use outer and the result will be a 2D array, not 3D.

Edit

As long as the operation you want to do is a universal function ("ufunc"), you can use its outer method;

In [1]: import numpy as np

In [2]: a = np.arange(10)

In [3]: a
Out[3]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [4]: b = np.arange(15)

In [5]: np.add.outer(a, b)
Out[5]: 
array([[ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14],
       [ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15],
       [ 2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16],
       [ 3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17],
       [ 4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18],
       [ 5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19],
       [ 6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20],
       [ 7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21],
       [ 8,  9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22],
       [ 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]])

Solution 3:

You can use numpy broadcasting, as hpaulj suggested. This is generally the first line of attack for vectorizing numpy operations. The essence of using broadcasting, is that in doing so, you move your loops from the slow python level down to the C level.

Alternatively, you may choose to take a look at numba, and numexpr, which accomplish vectorization in a more flexible manner