How Can I Remove Indices Of Non-max Values That Correspond To Duplicate Values Of Separate List From Both Lists?

I have two lists, the first of which represents times of observation and the second of which represents the observed values at those times. I am trying to find the maximum observed

Solution 1:

Without going into the details of how to implement and efficient rolling-window-maximum filter; reducing the duplicate values can be seen as a grouping-problem, which the numpy_indexed package (disclaimer: I am its author) provides efficient and simple solutions to:

import numpy_indexed as npi
unique_time, unique_speed = npi.group_by(time_count).max(linspeed)

For large input datasets (ie, where it matters), this should be a lot faster than any non-vectorized solution. Memory consumption is linear and performance in general NlogN; but since time_count appears to be sorted already, performance should be linear too.

Solution 2:

OK, if you want to do this with numpy, best is to turn both of your lists into arrays:

l = np.array(linspeed)
tc = np.array(time_count)

Now, finding unique times is just an np.unique call:

u, i, c = np.unique(tc, return_inverse =True, return_counts =True)

u
Out[]: array([  4.,   6.,   8.,  10.,  14.,  16.])

i
Out[]: array([0, 1, 2, 2, 3, 3, 3, 4, 5], dtype=int32)

c
Out[]: array([1, 1, 2, 3, 1, 1])

Now you can either build your maximums with a for loop

m = np.array([np.max(l[i==j]) if c[j] >1else l[j] for j inrange(u.size)])

m
Out[]: array([ 280. ,  275. ,  475.2,  400.9,  360.1,  400.9])

Or try some 2d method. This could be faster, but it would need to be optimized. This is just the basic idea.

np.max(np.where(i[None, :] == np.arange(u.size)[:, None], linspeed, 0),axis =1)
Out[]: array([ 280. ,  275. ,  475.2,  400.9,  323.8,  289.7])

Now your m and u vectors are the same length and include the output you want.