Feature Selection Toolbox FST4
UTIA · Czech Academy of Sciences

Feature selection references

A selected reading list on feature selection and its sub-topics. Being limited, it cannot cover everything that matters in the field — it aims to give an overview of the various views of the problem. For a gentle introduction to the topic itself, read the FST4 introduction.

Foundations and surveys

  1. Pierre A. Devijver and Josef Kittler. Pattern Recognition: A Statistical Approach. Prentice Hall, 1982. seminal book on statistical pattern recognition with strong feature selection coverage
  2. Anil K. Jain, Robert P. W. Duin, and Jianchang Mao. Statistical Pattern Recognition: A Review. IEEE Trans. PAMI, 22(1):4–37, 2000. concise overview of statistical pattern recognition
  3. Ron Kohavi and George H. John. Wrappers for Feature Subset Selection. Artificial Intelligence, 97(1–2):273–324, 1997. the paper establishing the key terms of feature selection wrappers and filters
  4. Isabelle Guyon and André Elisseeff. An Introduction to Variable and Feature Selection. Journal of Machine Learning Research, 3:1157–1182, 2003. the most-cited entry point to the field
  5. Huan Liu and Lei Yu. Toward Integrating Feature Selection Algorithms for Classification and Clustering. IEEE Trans. on Knowledge and Data Engineering, 17(4):491–502, 2005. concise overview of feature selection
  6. I. Guyon, S. Gunn, M. Nikravesh, and L. A. Zadeh, editors. Feature Extraction — Foundations and Applications. Studies in Fuzziness and Soft Computing vol. 207, Physica/Springer, 2006. book on feature extraction as well as selection
  7. Petr Somol, Jana Novovičová, and Pavel Pudil. Efficient Feature Subset Selection and Subset Size Optimization. pages 75–97, INTECH, 2010. overview of feature selection with focus on sequential sub-optimal methods
  8. Pavel Pudil and Jana Novovičová. Novel Methods for Subset Selection with Respect to Problem Knowledge. IEEE Intelligent Systems, 13:66–74, 1998.
  9. J. Kittler, Pavel Pudil, and Petr Somol. Advances in Statistical Feature Selection. In ICAPR '01, LNCS 2013, pages 425–434, Springer, 2001.
  10. S. Watanabe. Knowing and Guessing. John Wiley and Sons, 1969.
  11. Vladimir N. Vapnik. The Nature of Statistical Learning Theory. John Wiley, New York, 1998.

Method comparisons, information measures, overfitting, stability

  1. Anil Jain and Douglas Zongker. Feature Selection: Evaluation, Application, and Small Sample Performance. IEEE Trans. PAMI, 19(2):153–158, 1997. comparison of feature selection methods
  2. Mineichi Kudo and Jack Sklansky. Comparison of Algorithms that Select Features for Pattern Classifiers. Pattern Recognition, 33(1):25–41, 2000. comparison of feature selection methods
  3. Athina Salappa, Michael Doumpos, and Constantin Zopounidis. Feature Selection Algorithms in Classification Problems: An Experimental Evaluation. Optimization Methods and Software, 22(1):199–212, 2007. comparison of feature selection methods
  4. Hanchuan Peng et al. Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy. IEEE Trans. PAMI, 27(8):1226–1238, 2005. information measures in feature selection (mRMR)
  5. Gavin Brown. A New Perspective for Information Theoretic Feature Selection. In Proc. AISTATS '09, JMLR W&CP vol. 5, pages 49–56, 2009. information measures in feature selection
  6. F. Sebastiani. Machine Learning in Automated Text Categorization. ACM Computing Surveys, 34(1):1–47, 2002. text categorization, incl. feature selection in very-high-dimensional feature spaces
  7. Šarunas J. Raudys. Feature Over-Selection. In Proc. S+SSPR, LNCS 4109, pages 622–631, Springer, 2006. the problem of feature selection over-fitting
  8. Alexandros Kalousis, Julien Prados, and Melanie Hilario. Stability of Feature Selection Algorithms: A Study on High-Dimensional Spaces. Knowledge and Information Systems, 12(1):95–116, 2007. feature selection stability
  9. Ludmila I. Kuncheva. A Stability Index for Feature Selection. In Proc. 25th IASTED AIAP '07, pages 390–395, ACTA Press, 2007. feature selection stability
  10. Petr Somol and Jana Novovičová. Evaluating Stability and Comparing Output of Feature Selectors that Optimize Feature Subset Cardinality. IEEE Trans. PAMI, 32(11):1921–1939, 2010. extensive survey of stability evaluation measures and related techniques

Recent literature (2010–2026)

Extending the classic canon above towards the present. Citations verified July 2026.

  1. Nicolai Meinshausen and Peter Bühlmann. Stability Selection. Journal of the Royal Statistical Society: Series B, 72(4):417–473, 2010. subsampling-based selection with error control — the statistics community's answer to selection instability
  2. Miron B. Kursa and Witold R. Rudnicki. Feature Selection with the Boruta Package. Journal of Statistical Software, 36(11):1–13, 2010. the widely used all-relevant selection method around Random Forest importance
  3. Girish Chandrashekar and Ferat Sahin. A Survey on Feature Selection Methods. Computers & Electrical Engineering, 40(1):16–28, 2014. compact modern survey of filter/wrapper/embedded methods
  4. Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. Feature Selection: A Data Perspective. ACM Computing Surveys, 50(6):94, 2017. the broadest recent survey, organized by data type; companion to the scikit-feature repository
  5. Scott M. Lundberg and Su-In Lee. A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems 30:4766–4777, 2017. SHAP — model-agnostic attribution values now widely used for importance-based selection
  6. Sarah Nogueira, Konstantinos Sechidis, and Gavin Brown. On the Stability of Feature Selection Algorithms. Journal of Machine Learning Research, 18(174):1–54, 2018. a unifying treatment of stability measures with statistical guarantees
  7. Verónica Bolón-Canedo and Amparo Alonso-Betanzos. Ensembles for Feature Selection: A Review and Future Trends. Information Fusion, 52:1–12, 2019. survey of ensemble feature selection
  8. Ismael Lemhadri, Feng Ruan, Louis Abraham, and Robert Tibshirani. LassoNet: A Neural Network with Feature Sparsity. Journal of Machine Learning Research, 22(127):1–29, 2021. feature selection built into deep networks — the embedded approach in the neural era

Embedded-selection classics (complementing the canon)

  1. Robert Tibshirani. Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58(1):267–288, 1996. the L1 route to feature selection — the foundation of embedded methods
  2. Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. Gene Selection for Cancer Classification using Support Vector Machines. Machine Learning, 46:389–422, 2002. recursive feature elimination (RFE)

FST-related publications

Papers describing methods implemented in FST, or reporting results obtained with it, newest first.

  1. Petr Somol and Jiří Grim. Stochastic Sequential Search in Very-High-Dimensional Feature Selection. arXiv:2608.01502 [cs.LG], 2026. arxiv.org/abs/2608.01502 the budgeted stochastic (sampled) sequential search shipped in FST4 (--evaluator sampled; demo36 and demo37t)
  2. Petr Somol and Jiří Grim. Fast Dependency-Aware Feature Selection in Very-High-Dimensional Pattern Recognition Problems. UTIA TR No. 2295, Czech Academy of Sciences, 2011. the DAF ranking method (FST3 3.1, FST4)
  3. Petr Somol, Jiří Grim, and Pavel Pudil. The Problem of Fragile Feature Subset Preference in Feature Selection Methods and a Proposal of Algorithmic Workaround. In ICPR 2010, IEEE Computer Society, 2010.
  4. Petr Somol, Jana Novovičová, and Pavel Pudil. On the Over-Fitting Problem of Complex Feature Selection Methods. In Proc. 5th International Computer Engineering Conference, pages 12–17, Cairo University, 2009.
  5. Jana Novovičová, Petr Somol, and Pavel Pudil. A New Measure of Feature Selection Algorithms' Stability. In Proc. IEEE ICDM Workshops, pages 382–387, IEEE Computer Society, 2009.
  6. P. Somol, J. Grim, and P. Pudil. Criteria Ensembles in Feature Selection. In Proc. MCS, LNCS 5519, pages 304–313, Springer, 2009.
  7. Petr Somol, Jana Novovičová, Jiří Grim, and Pavel Pudil. Dynamic Oscillating Search Algorithm for Feature Selection. In ICPR 2008, IEEE Computer Society, 2008.
  8. Petr Somol and Jana Novovičová. Evaluating the Stability of Feature Selectors that Optimize Feature Subset Cardinality. In Proc. S+SSPR, LNCS 5342, pages 956–966, 2008.
  9. Jana Novovičová, Petr Somol, and Pavel Pudil. Oscillating Feature Subset Search Algorithm for Text Categorization. In Proc. S+SSPR, LNCS 4109, pages 578–587, Springer, 2006.
  10. Petr Somol, Jana Novovičová, and Pavel Pudil. Flexible-Hybrid Sequential Floating Search in Statistical Feature Selection. In Proc. S+SSPR, LNCS 4109, pages 632–639, Springer, 2006.
  11. Petr Somol, Pavel Pudil, and Josef Kittler. Fast Branch & Bound Algorithms for Optimal Feature Selection. IEEE Trans. PAMI, 26(7):900–912, 2004.
  12. Pavel Pudil, Jana Novovičová, and Petr Somol. Recent Feature Selection Methods in Statistical Pattern Recognition. In Pattern Recognition and String Matching, pages 565–616, Springer, 2002.
  13. Petr Somol and Pavel Pudil. Feature Selection Toolbox. Pattern Recognition, 35(12):2749–2759, 2002. the toolbox paper — please cite this when FST contributes to your research
  14. Petr Somol and Pavel Pudil. Oscillating Search Algorithms for Feature Selection. In ICPR 2000, vol. 02, pages 406–409, IEEE Computer Society, 2000.
  15. Helmut A. Mayer, Petr Somol, Reinhold Huber, and Pavel Pudil. Improving Statistical Measures of Feature Subsets by Conventional and Evolutionary Approaches. In Proc. Joint IAPR International Workshops on Advances in Pattern Recognition, LNCS 1876, pages 77–86, Springer, 2000.
  16. Petr Somol, Pavel Pudil, Jana Novovičová, and Pavel Paclík. Adaptive Floating Search Methods in Feature Selection. Pattern Recognition Letters, 20(11–13):1157–1163, 1999.

Software

Feature selection implementations elsewhere. Links verified July 2026; suggestions welcome via Contacts.

  • scikit-learn — feature selection in the standard Python ML stack (univariate filters, RFE, model-based selection).
  • mlxtend — Python implementations of sequential forward/backward and floating selection (SFS/SBS/SFFS/SBFS).
  • Weka — the Waikato data-mining workbench (Java) with its attribute-selection framework.
  • Boruta (R) — all-relevant feature selection around Random Forest importance.
  • feature-engine (Python) — feature engineering and selection transformers.
Copyright © 2000–present ÚTIA, Institute of Information Theory and Automation of the Czech Academy of Sciences · Contacts · This site sets no cookies and uses no analytics.