Testing and verification
Every test run checks about 65,000 calls, some 54 million samples of signal, against SciPy 1.18.1, and the signed modules pass the same comparison inside Ignition 8.1 and 8.3. The tests are in the repository, so anyone can rerun them.
What is tested
| Test | What it proves | Scale |
|---|---|---|
| Random comparison with SciPy | Every function gives SciPy's result, on random signals shaped like plant data: offsets, drifts, noise, steps, spikes, quantization and long flat runs | About 65,000 calls and 54 million samples per run; signals up to 100,000 samples |
| Invalid calls | Wrong arguments fail where SciPy fails, with ValueError or TypeError, never a Java error | Thousands of cases per run |
| Property checks | Results behave as they must regardless of SciPy: zero-lag filters keep peaks in place, filter states reach steady state, Parseval's theorem holds, polynomials pass Savitzky-Golay unchanged | Hundreds of cases per function |
| Accuracy inside Ignition | The signed modules give the same results as SciPy in the real Ignition runtime, on Ignition 8.1.53 and 8.3.10 | About 2,600 cases and 64 runtime checks per gateway |
| Gateway scope | Every function works in gateway scripts, the path tag events, timers and Perspective use | 34 checks per gateway |
| Concurrency | 8 threads calling at once reproduce single-threaded results exactly | 250 heavy calls, 0 mismatches |
| Planted bugs | The tests catch real faults: deliberately introduced bugs, from wrong signs to off-by-one edges, must each fail | More than 40 planted bugs |
How results are compared
Where Signal Processing performs SciPy's operations in SciPy's order, results must be identical to the last bit, and they are: filtering, median filtering and peak finding. Where they cannot be identical, as with a different FFT, each result must fall within a bound derived from the rounding involved, not a fixed loose tolerance. For filter designs and IIR decimation, the bound comes from rerunning SciPy on inputs moved by a few roundings, so it reflects how sensitive that particular problem is.
Speed inside Ignition
Measured in the accuracy test on two test gateways, Ignition 8.1.53 on Windows and 8.3.10 on Ubuntu:
| Task on 1,000,000 points | Time |
|---|---|
| Linear detrend | 28 to 46 ms |
Zero-lag filtering, order 8 (sosfiltfilt) | 43 to 84 ms |
| Decimation by 10 | 46 to 61 ms |
| Peak finding with prominence and width (81,000 peaks) | 79 to 139 ms |
| Welch spectrum, 1,024-sample segments | 87 to 122 ms |
Rerun the tests
From the repository's signal-core folder, with Python 3, JDK 17 and Maven:
python -m venv .venv
.venv\Scripts\Activate.ps1
pip install -r src\test\python\requirements.txt
mvn -q test-compile
python src\test\python\random_check.py
It prints one line per function with the cases checked, failures and the worst result as a fraction of its bound, and ends in PASS or FAIL. To test inside Ignition: run random_check.py --export, paste target\accuracy_test.py into the Designer Script Console, then run compare_ignition.py --results <path it prints>. src\test\ignition\gateway_check.py and e2e_concurrency.py carry their own instructions for the gateway-scope and concurrency checks.