Quick start
Pass a list of evenly sampled numbers in, get lists back, and use SciPy's documentation and examples as they are: the names, parameters and defaults match.
Your first calls
x = [20.0 + 0.5 * ((k * 37) % 11 - 5) for k in range(600)] # a noisy signal
sos = system.signal.butter(4, 0.05, output="sos") # 4th-order low-pass, cut-off 5% of Nyquist
clean = system.signal.sosfiltfilt(sos, x) # zero-lag filtering
despiked = system.signal.medfilt(x, 5) # remove single-sample spikes
print len(clean), clean[:3]
Data in and out
| Topic | Rule |
|---|---|
| Signals | Lists, tuples, Java arrays or ArrayLists of numbers; integers are fine |
| Missing values | None, NaN or infinity in a signal raise ValueError, never a silent NaN result |
| Shape | Signals are one-dimensional; axis accepts -1 and 0 |
| Sampling | Samples must be evenly spaced; pass fs (samples per second) where a function takes it |
| Results | Lists of floats; several results come back as a tuple, as in SciPy (b, a = butter(...)) |
| Complex values | Python complex numbers: abs(h) and h.real work |
find_peaks | Returns (peaks, properties): a list of sample indices and a dict of lists |
| Errors | Python ValueError and TypeError, where SciPy raises them |
Evenly sampled history
Ask the historian for a fixed interval and fill its gaps (returned as None) before calling a function:
end = system.date.now()
start = system.date.addHours(end, -2)
ds = system.tag.queryTagHistory(paths=["[default]Line1/Flow"], startDate=start, endDate=end,
intervalSeconds=1, aggregationMode="Average", returnFormat="Wide")
values = [ds.getValueAt(r, 1) for r in range(ds.rowCount)]
last = next((v for v in values if v is not None), 0.0)
x = []
for v in values: # carry the last good value across gaps
last = v if v is not None else last
x.append(float(last))
History or real time?
| Use | For history windows (needs later samples) | For live values (real time) |
|---|---|---|
| Smoothing | sosfiltfilt, filtfilt, savgol_filter | sosfilt or lfilter with zi |
| Derivatives | savgol_filter(..., deriv=1) | savgol_coeffs(..., pos=window_length - 1, use="dot") on the latest window |
| Spikes | medfilt | medfilt on a rolling buffer (lags by half its window) |
| Peaks, spectra | find_peaks, welch, spectrogram on the window | the same, rerun on a sliding window |
Zero-lag filters look at samples on both sides, so they suit history. For a live value, filter one sample at a time and carry the filter's state between calls. Filtering sample by sample this way gives exactly the same result as filtering the whole record at once.
A live filtered tag
A gateway timer script, every 1,000 ms. It needs a Float memory tag for the result and a String memory tag that keeps the filter state:
x = system.tag.readBlocking(["[default]Line1/Flow"])[0].value
if x is not None:
state = system.tag.readBlocking(["[default]Line1/FlowFilterState"])[0].value
sos = system.signal.butter(2, 0.02, output="sos") # 0.01 Hz cut-off at one sample per second
if state:
zi = system.util.jsonDecode(state)
else: # first run: start in steady state
zi = [[z * x for z in row] for row in system.signal.sosfilt_zi(sos)]
y, zf = system.signal.sosfilt(sos, [x], zi=zi)
system.tag.writeBlocking(["[default]Line1/FlowFiltered", "[default]Line1/FlowFilterState"],
[y[0], system.util.jsonEncode(zf)])
A live filter always lags a little. For a second-order Butterworth filter the lag is about 0.225 / cutoff: 22 s at 0.01 Hz. Halving the cutoff doubles the smoothing and the lag.