Skip to main content

Quick start

Pass a list of evenly sampled numbers in, get lists back, and use SciPy's documentation and examples as they are: the names, parameters and defaults match.

Your first calls​

x = [20.0 + 0.5 * ((k * 37) % 11 - 5) for k in range(600)] # a noisy signal
sos = system.signal.butter(4, 0.05, output="sos") # 4th-order low-pass, cut-off 5% of Nyquist
clean = system.signal.sosfiltfilt(sos, x) # zero-lag filtering
despiked = system.signal.medfilt(x, 5) # remove single-sample spikes
print len(clean), clean[:3]

Data in and out​

TopicRule
SignalsLists, tuples, Java arrays or ArrayLists of numbers; integers are fine
Missing valuesNone, NaN or infinity in a signal raise ValueError, never a silent NaN result
ShapeSignals are one-dimensional; axis accepts -1 and 0
SamplingSamples must be evenly spaced; pass fs (samples per second) where a function takes it
ResultsLists of floats; several results come back as a tuple, as in SciPy (b, a = butter(...))
Complex valuesPython complex numbers: abs(h) and h.real work
find_peaksReturns (peaks, properties): a list of sample indices and a dict of lists
ErrorsPython ValueError and TypeError, where SciPy raises them

Evenly sampled history​

Ask the historian for a fixed interval and fill its gaps (returned as None) before calling a function:

end = system.date.now()
start = system.date.addHours(end, -2)
ds = system.tag.queryTagHistory(paths=["[default]Line1/Flow"], startDate=start, endDate=end,
intervalSeconds=1, aggregationMode="Average", returnFormat="Wide")
values = [ds.getValueAt(r, 1) for r in range(ds.rowCount)]
last = next((v for v in values if v is not None), 0.0)
x = []
for v in values: # carry the last good value across gaps
last = v if v is not None else last
x.append(float(last))

History or real time?​

UseFor history windows (needs later samples)For live values (real time)
Smoothingsosfiltfilt, filtfilt, savgol_filtersosfilt or lfilter with zi
Derivativessavgol_filter(..., deriv=1)savgol_coeffs(..., pos=window_length - 1, use="dot") on the latest window
Spikesmedfiltmedfilt on a rolling buffer (lags by half its window)
Peaks, spectrafind_peaks, welch, spectrogram on the windowthe same, rerun on a sliding window

Zero-lag filters look at samples on both sides, so they suit history. For a live value, filter one sample at a time and carry the filter's state between calls. Filtering sample by sample this way gives exactly the same result as filtering the whole record at once.

A live filtered tag​

A gateway timer script, every 1,000 ms. It needs a Float memory tag for the result and a String memory tag that keeps the filter state:

x = system.tag.readBlocking(["[default]Line1/Flow"])[0].value
if x is not None:
state = system.tag.readBlocking(["[default]Line1/FlowFilterState"])[0].value
sos = system.signal.butter(2, 0.02, output="sos") # 0.01 Hz cut-off at one sample per second
if state:
zi = system.util.jsonDecode(state)
else: # first run: start in steady state
zi = [[z * x for z in row] for row in system.signal.sosfilt_zi(sos)]
y, zf = system.signal.sosfilt(sos, [x], zi=zi)
system.tag.writeBlocking(["[default]Line1/FlowFiltered", "[default]Line1/FlowFilterState"],
[y[0], system.util.jsonEncode(zf)])

A live filter always lags a little. For a second-order Butterworth filter the lag is about 0.225 / cutoff: 22 s at 0.01 Hz. Halving the cutoff doubles the smoothing and the lag.