Version history

FST4 v4.2 (October 2026) is the first published FST4 version — the last public release before it was FST3 3.1.1. Versions 4.0 and 4.1 were internal development milestones whose changes reach users as part of v4.2; their per-version accounts are kept below because the lineage explains the design. The complete change record ships in the distribution as RELEASE_NOTES.md.

4.2.0 (October 2026 — the first public FST4 release)

The practitioner wrapper expansion.

• Wrapper metrics beyond accuracy (--wrapper-metric): balanced accuracy, F1 (with --f1-positive class choice), macro-F1, and AUC — exact binary AUC with midrank ties on two-class data, one-vs-rest macro AUC on any class count — plus per-class cost weights (--class-weights); R²/negative-RMSE/negative-MAE for the regression wrappers. All computed from deterministic per-sample scores; the accuracy default remains byte-identical to the historical output.
• Five new dependency-free, deterministic wrapper models:
  • Random Forest (wrapper-rf) — CART/Gini trees, bagging, sqrt-mtry node subsampling; training randomness derived purely from (subset, seed, split), so every evaluation is a pure function — bit-identical at any thread count.
  • Linear SVM (wrapper-linsvm) — dual coordinate descent with a fixed cyclic order (RNG-free); retires the long-documented LibSVM LINEAR-kernel freeze for linear-SVM selection.
  • RBF kernel SVM (wrapper-rbfsvm) — C-SVC kernel dual coordinate descent with a penalized-bias kernel offset; native C/gamma tuning via --rbfsvm-optimize.
  • Ridge regression (wrapper-ridge) — closed-form Cholesky-solved L2 least squares, R² by default.
  • Linear discriminant (wrapper-lda) — pooled shared-covariance discriminant, one matrix inverse per training, parameterless.
• External bridges — bring your own component: an external classifier over a persistent-process line protocol (wrapper-extern), an external criterion answering per-subset VALUE requests (extern-value, with a monotonicity assertion admitting Branch & Bound), and pure black-box set-function optimization with no data file at all (--features N --extern-no-data).
• LibSVM demoted to an optional extension — the native SVM wrappers cover both practical kernels, and LibSVM is no longer distributed with the toolbox (fetch instructions ship in the package); builds and the golden harness handle its absence gracefully.
• Threading completed: candidate-level threading for all four Branch & Bound variants and the hybrid/ensemble step evaluators — every search and every evaluator now accepts --threads N with results bit-identical to sequential; a persistent worker pool collapsed the small-batch overhead.
• Research-grade reporting: --report detailed (the criterion-vs-holdout overfitting curve over subset size, tabular ROC/PR curves, confusion matrices, tuned hyperparameters), --dump-scores (per-sample holdout predictions), --curves-svg (self-contained SVG plots), and stdin data input (--data -) completing the Unix-pipe story.
• Data input: the CSV loader — the simplest Excel-to-FST4 path, with delimiter/header auto-detection tuned so European semicolon + decimal-comma exports load with zero flags, missing-value policy flags (--csv-missing-to), and ARFF robustness (unbounded line length, graceful per-column skip of unsupported attribute types).
• Leakage-free normalization: --scaler-fit train fits scaling transforms on each split's training part only, closing a documented FST3-era methodological leakage (the historical whole-dataset fit remains the byte-identical default).
• Reproducible-by-default builds: fixed CPU-independent release flags on every architecture (machine-specific native optimization behind an explicit opt-in), with the golden baselines captured under exactly the default flags on all three platforms.
• Deliberate behavior changes: the kNN decision rule unified with its scoring rule (accuracy and AUC now describe the same classifier), and an Oscillating Search initial-subset shrink fix.
• The deep review: a dedicated post-program review — security review of the process-spawning surface, an adversarially-verified multi-agent bug hunt, a direction-symmetry sweep, performance scouting and a simplification pass — yielded 27 dispositioned findings, among them: backward searches no longer corrupt state across repeated-split trials; threaded runs fail as cleanly as sequential ones; classified samples respect the active scaler; external-bridge robustness hardening; a LibSVM heap overread fix; 32-bit index-arithmetic overflows widened.
• The second review pass: a second full correctness review — the criterion and classifier mathematics, the data layer, the search layer, the CLI pipeline and every user-facing text — found five defects that produced wrong results, hung or failed on realistic input, all fixed: the Bhattacharyya distance no longer over- or underflows on badly scaled data (log-determinants instead of raw determinant products); the multinomial mutual-information ranking uses the proper per-term formula (the FST3 formula ranked one-document terms first); repeated-split trials with a wrapper criterion require a nested split (un-nested, they never ended); sample classification under train-fitted scaling applies each feature's own transform; and threaded repeated-split trials no longer repeat the first trial. Seven scenarios joined the release gate (105 → 112), and the class indices in the load summary are 0-based like every other index.
• The live-code cleanup pass: the oddities noted while the shipped headers were read comment by comment — missing guards against zero split counts and empty cardinality ranges, dead or vacuous statements, output strings that did not say what they meant, and include/const/naming hygiene — fixed under the release gate with no result changed.
• Distribution shape: all 51 demo configs in demos/; sample data sets bundled with full per-file attribution (data/NOTICE); the FST3-era speech data withdrawn and the sonar class split corrected to the official UCI one (correctness over parity); three per-platform golden baselines (Windows x64, macOS arm64, Linux arm64), each 112 scenarios verified green.

4.1.0 (July 2026 — internal milestone)

Very-high-dimensional selection.

• Budgeted stochastic (sampled) sequential step (--evaluator sampled): per-step evaluation cost bounded by a candidate budget independent of dimensionality — softmax/uniform/top-k proposal samplers over online per-feature statistics, exploration floor, forward/backward budgets, warm-start probes. Makes floating-search-style selection feasible at thousands to tens of thousands of features.
• Threaded evaluation made bit-identical to sequential at matched seed, for any thread count: the sampled evaluator (proposals on the master RNG stream, results replayed in proposal order) and the streaming searches (Monte Carlo, exhaustive) alike — making the DAF probe pipeline deterministic under threading.
• Multinomial logistic regression classifier (wrapper-logreg): dependency-free damped-Newton training, internally standardized, RNG-free and deterministic.
• SFFS frontier bound (--sffs-delta): stops the floating frontier at target-d+delta — the knob that makes the classic protocol tractable on very-high-dimensional data.
• DAF ranking consolidated to the published 2011 method (any probe-generating search + the feature-statistics tracker), retiring the internal experimental "new-generation" DAF stack; opt-in pairwise group statistics (--daf-pairs) preserve its useful idea as pure post-processing.
• Cross-platform seed-for-seed reproducibility: the FST-owned PRNG shim — all randomness through one UCRT-compatible generator with thread-local state, identical draw sequences on Windows and macOS; an unseeded run equals --seed 1, which is why the CLI scenarios reproduce the historical demos' unseeded results exactly.
• FST3 demo parity configs: every demo published in the public FST3 as a ready-to-run config file producing the same results through the CLI.
• A latent out-of-bounds access in the sampled-step proposal sampler (complete softmax underflow) found and fixed by the threading validation.

4.0.0 (July 2026 — internal milestone)

The modernization — the largest change to the toolbox since FST3 itself. The algorithmic core — fifteen years of numerically validated feature-selection methods — was preserved and verified byte-exact against the FST3 3.1.1 outputs throughout.

• The fst4 command-line tool: the whole toolbox usable without writing C++ — every released demo scenario expressible as flags or a config file, machine-readable JSON/CSV output, search-progress streaming, seeded reproducible runs, cross-scenario similarity comparison.
• C++17, header-only, dependency-free: the C++98/Boost era ends — smart pointers and threading from the standard library, no compiled translation unit, no external dependencies (LibSVM and Eigen optional). GCC, MinGW-W64 and MSVC supported.
• CMake build with exact per-translation-unit dependency tracking; the historical Makefile retired.
• One implementation per search algorithm: the threaded searches were previously near-copies of their sequential twins — a proven source of drift bugs; sequential vs. thread-pooled execution is now chosen by an injected evaluation engine, not by class choice.
• The clone contract made structural: thread-safe cloning — previously hand-written per class and the library's largest audited defect class — is now implemented uniformly by a CRTP mixin; the defect class can no longer be written.
• Search progress observable through an observer seam (with the historical text output preserved byte-exactly when unused); reentrant iteration ranges replace stateful cursor protocols (old APIs deprecated but functional).
• Numerics cross-validated: an optional Eigen-backed build reproduces the hand-rolled LU/inverse/determinant results to print precision on all normal-model criteria.
• A full-codebase audit, fully adjudicated (~46 recorded findings; every one fixed and regression-gated or explicitly adjudicated with rationale). Among what FST3 users had been exposed to: the normal Bayes classifier never subtracted the class mean and lacked log-domain math (systematic bias toward class 0 on hard patterns); ARFF nominal attributes read as garbage and indented headers silently loading zero records; TRN files without #data shifting all values by one row; an out-of-bounds read in every Subset iterator loop; a lost-wakeup race that could hang any threaded search; five broken clone() implementations crashing threaded searches; demos exiting 0 on fatal errors.
• The golden-output regression harness as the release gate: byte-identical output verification across the full demo and CLI scenario set, on every change.
• Demo reorganization: the 39 compiled demos reduced to library-API exemplars, with every retired demo's scenario shipping as an equivalent CLI config verified byte-exact where deterministic.

A fix credited separately: an incorrect lock in the threaded search infrastructure that could freeze multi-threaded runs after very long computations — a symptom observed in practice years earlier — was found and fixed in a December 2025 audit of the published FST3 codebase, before the 4.0 program began.

---

FST3 (2010–2012 — the record taken over from the FST3 web)

3.1.1.beta (September 2012)

• Fixed a bug in the Sparse ARFF filter that prevented correct Sparse ARFF files from being read.
• The standard ARFF filter made less sensitive to header formatting; accepts more ARFF files straight away.

3.1.0.beta (March 2011)

• Optimal Branch & Bound methods: BBB (Basic), IBB (Improved), BBPP (Partial Prediction), FBB (Fast Branch & Bound).
• DAF — Dependency-Aware Feature Ranking (DAF0 standard, DAF1 normalized): highly efficient very-high-dimensional feature selection that, unlike BIF, does not ignore contextual information; enables wrapper-based selection at dimensionality 10⁵–10⁶.
• SFRS/SBRS — Sequential Retreating Search, related to Floating Search but more thorough; suitable also with a secondary criterion.
• "Generalized" variants of all sequential methods — testing feature g-tuples per step: (G)SFS, (G)SBS, (G)SFFS, (G)SBFS, (G)OS, (G)DOS, (G)SFRS, (G)SBRS.
• All sequential methods allow starting from an arbitrary subset; threaded BIF ranking; Monte Carlo search (sequential and threaded); post-search retrieval of the best result at each subset size; a faster SFFS implementation closer to the original definition; more efficient threading; redirectable and switchable search output (SILENT / NORMAL / DETAILED); improved result trackers; data-part access substitution and a bias-estimating wrapper; missing-value substitution; per-sample classify(); refactored directory structure; many new demos grouped by purpose.

3.0.2.beta (January 2011)

• Exhaustive Search added (sequential and threaded) for optimal feature selection.
• LibSVM 3.0 support; result trackers gained cloning and memory-usage limits; a demo-output verification log.

3.0.1.beta (November 2010)

• ARFF (Weka) data file support; Visual C++ compilation enabled.

3.0.0.beta (October 2010)

• Initial public release: templated C++ (Boost-based). Criteria: wrappers (normal Bayes, kNN over L-distances, SVM via LibSVM), normal-model filters (Bhattacharyya, Divergence, Generalized Mahalanobis), multinomial filters, ensembles and hybrids. Searches: BIF ranking, SFS/SBS, SFFS/SBFS, OS, DOS — threaded, hybrid or ensemble evaluation in any of them. Supporting techniques: subset-size optimization, result regularization, feature acquisition cost, stability and similarity evaluation. Data handling: nested multi-level splitting (resubstitution, cross-validation, hold-out, leave-one-out, random sampling), normalization, the TRN format.

For the toolbox's first installment (1999–2002), see Legacy — FST1.