Append Dataframe In Nested Loop

I have the following code that does some calculation in a function depending on 2 variables (bh and frequency). when i run the code without loop with a fixed value of bh i get a pr

Solution 1:

I am not entirely sure if I understand your problem domain, but you seem to be returning the first iteration of the loop. By having return inside the loop you essentially terminate the for loop early. This should be outside of the loop. Also, you do not save your values in the arrays v, w, and i. You are overwriting the variable.

I have done some modifications (maybe not correct according to your problem domain), but it should do what you want to accomplish.

from mpmath import *
import numpy as np
import cmath
import math
import pandas as pd

mp.dps = 15; mp.pretty = True
a = mpf(0.25)
b = mpf(0.25)
z = mpf(0.75)
frequencies = np.arange(1, 50, 10)  # frequency range
bh = np.arange(10e-6, 30e-6, 10e-6) #10e-6 # widthprint(bh)
D = 1e-6#7.8e-4  # diffusivity
gamma = 0.5772# Euler constant
v = []
w = []
i = []
bhs = []
freqs = []
defq(frequencies):
  for frequency in frequencies:
    for i in bh:
      # for f in frequency:
        omega = (((i ** 2) * 2 * math.pi * frequency) / D)  # depends on bh and frequency
        u = ((-j/(math.pi * omega))*meijerg([[1, 3/2], []], [[1, 1], [0.5, 0]], j*omega))
        v.append(np.real(u))
        w.append(np.imag(u))
        bhs.append(i)
        freqs.append(frequency)
  return bhs, freqs, v, w

data = np.array(q(frequencies)).T
# create DataFrame
df1 = pd.DataFrame(data=data, columns=['bh', 'frequency','Re', 'Im'])
df1

output:

    bh  frequency             ReIm01e-0515.86848609615851-0.99937434684573412e-0514.98625732244539-0.9978669870064521e-05114.34298196390882-0.99487554972041832e-05113.46384911041305-0.98355919045486541e-05213.93236459069042-0.9911208623520652e-05213.05626898509369-0.97221239150773261e-05313.68545733552675-0.98769557251336772e-05312.81234917403506-0.96216798959981281e-05413.50849758486341-0.98448793258832392e-05412.63833200578647-0.952979213441469101e-0515.86848609615851-0.999374346845734112e-0514.98625732244539-0.99786698700645121e-05114.34298196390882-0.994875549720418132e-05113.46384911041305-0.983559190454865141e-05213.93236459069042-0.99112086235206152e-05213.05626898509369-0.972212391507732161e-05313.68545733552675-0.987695572513367172e-05312.81234917403506-0.962167989599812181e-05413.50849758486341-0.984487932588323192e-05412.63833200578647-0.952979213441469201e-0515.86848609615851-0.999374346845734212e-0514.98625732244539-0.99786698700645221e-05114.34298196390882-0.994875549720418232e-05113.46384911041305-0.983559190454865241e-05213.93236459069042-0.99112086235206252e-05213.05626898509369-0.972212391507732261e-05313.68545733552675-0.987695572513367272e-05312.81234917403506-0.962167989599812281e-05413.50849758486341-0.984487932588323292e-05412.63833200578647-0.952979213441469

Solution 2:

I'd suggest (1) generating Cartesian product of bh and frequency in advance and (2) vectorize only the part you really need as np.vectorization is known to be costly (i.e., meijerg() which is not a vectorized function). The Cartesian product can be done by pd.MultiIndex.from_product (see this answer).

# run your code until gamma = 0.5772# Cartesian product of input variables
idx = pd.MultiIndex.from_product([bh, frequency], names=["bh", "frequency"])
df = pd.DataFrame(index=idx).reset_index()

# Omega is vectorized naturally.
omega = (df["bh"].values**2 * df["frequency"].values) * (2 * math.pi / D)

# vectorize meijerg() only, so other operations won't interrupt with this
def f_u(omega_elem):
    return (-j/(math.pi * omega_elem)) * meijerg([[1, 3/2], []], [[1, 1], [0.5, 0]], j*omega_elem)

f_u_vec = np.vectorize(f_u, otypes=[np.complex128]) # output complex

u = f_u_vec(omega)  # np.complex128df["Re"] = np.real(u)
df["Im"] = np.imag(u)

# output (please make sure your arange was set correctly)df
Out[35]: 
        bh  frequency        Re        Im
0  0.00001          1  5.868486 -0.999374
1  0.00001         11  4.342982 -0.994876
2  0.00001         21  3.932365 -0.991121
3  0.00001         31  3.685457 -0.987696
4  0.00001         41  3.508498 -0.984488
5  0.00002          1  4.986257 -0.997867
6  0.00002         11  3.463849 -0.983559
7  0.00002         21  3.056269 -0.972212
8  0.00002         31  2.812349 -0.962168
9  0.00002         41  2.638332 -0.952979

If you want to save separate csv files, you can do something like this:

for bh_elem in bh:
    fname = f"bh={bh_elem:.4e}.csv"
    df_save = df[(df["bh"]==bh_elem)]
    df_save.to_csv(fname)

N.B. tested on pandas 1.1.3 and python 3.7, debian 10 64-bit