Dependency-free linear SVM trained by dual coordinate descent (the method of Hsieh et al., ICML 2008, as popularized by LIBLINEAR — implemented from scratch, no external code). More...

#include <classifier_linear_svm.hpp>

Inheritance diagram for FST::Classifier_Linear_SVM< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >:

Public Types
typedef Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >	parent
	the immediate abstract base
typedef Clonable_Impl< Classifier_Linear_SVM, parent >	parent_impl
	implements clone() via the (implicit, member-wise) copy-constructor – all members are value-semantic
typedef std::shared_ptr< DATAACCESSOR >	PDataAccessor
	shared pointer to the data accessor supplying train/test blocks
typedef std::shared_ptr< SUBSET >	PSubset
	shared pointer to the feature subset being trained/evaluated
typedef DATAACCESSOR::PPattern	PPattern
	one data sample as exposed by DATAACCESSOR
Public Types inherited from FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >
typedef std::shared_ptr< DATAACCESSOR >	PDataAccessor
	shared pointer to the data accessor supplying train/test blocks
typedef std::shared_ptr< SUBSET > const	PSubset
	shared pointer to the (fixed, non-reassignable) feature subset being evaluated
typedef DATAACCESSOR::PPattern	PPattern
	one data sample as exposed by DATAACCESSOR (a raw pointer to its feature vector)

Public Member Functions
void	set_parameter_C (const REALTYPE c)
	sets the SVM regularization parameter C (box constraint upper bound U under L1-loss)
void	set_l2_loss (const bool l2)
	true = squared hinge (default), false = hinge
void	set_max_iter (const unsigned int mi)
	cap on full coordinate sweeps
void	set_tolerance (const REALTYPE t)
	max \|projected gradient\| sweep-stopping threshold
void	initialize (const PDataAccessor da)
	Records the class/feature counts of `da` and allocates the subset index buffer. Must be called before the first train(); throws fst_error when `da` has fewer than 2 classes.
virtual bool	classify (DIMTYPE &cls, const PPattern &pattern) override
virtual bool	classify_scores (std::vector< RETURNTYPE > &scores, const PPattern &pattern) override
virtual bool	train (const PDataAccessor da, const PSubset sub) override
	Trains the linear SVM: buffers the subset index, standardizes the design with an appended bias coordinate (dense_train_tools.hpp), then solves one dual coordinate-descent problem per one-vs-rest model (solve_dual(), FIXED cyclic order) into the corresponding row of _W. Returns false only when the split has no training data (N==0); a subset/data shape mismatch is an assertion failure, not a bool return.
virtual std::ostream &	print (std::ostream &os) const override
	writes a human-readable identification of the object; base default writes nothing
Public Member Functions inherited from FST::Clonable_Impl< Classifier_Linear_SVM< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >, Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > >
Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > *	clone () const override
	dispatches to the DEEP_KIND mechanism (default CLONE_DEEP: Derived's copy-constructor)
Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > *	sharing_clone () const override
	dispatches to the SHARING_KIND mechanism (default CLONE_INHERIT: defers to Base, which throws by family convention)
Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > *	stateless_clone () const override
	dispatches to the STATELESS_KIND mechanism (default CLONE_INHERIT: defers to Base, which throws by family convention)
Public Member Functions inherited from FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >
virtual bool	train (const PDataAccessor da, const PSubset sub)=0
	learns from designated training part of data
virtual bool	test (RETURNTYPE &result, const PDataAccessor da)
virtual bool	test_confusion (std::vector< RETURNTYPE > &confusion, DIMTYPE &classes, const PDataAccessor da)
virtual bool	test_scores (std::vector< std::pair< RETURNTYPE, DIMTYPE > > &scores, const PDataAccessor da)
virtual bool	test_scores_matrix (std::vector< RETURNTYPE > &scores, std::vector< DIMTYPE > &labels, DIMTYPE &classes, const PDataAccessor da)
virtual bool	test_metric (RETURNTYPE &result, const PDataAccessor da, const Metric_Spec< RETURNTYPE > &spec)
Public Member Functions inherited from FST::Clonable
bool	is_clone () const
bool	is_sharing () const
	true if this instance is a sharing_clone() result (referencing, not owning, the source's structures)
bool	is_sharing_clone () const
	true if this instance is BOTH a clone and sharing – i.e. specifically a sharing_clone() result, not a clone()/stateless_clone() one

Protected Member Functions
void	solve_dual (REALTYPE *w, const DIMTYPE positive_class)
DIMTYPE	model_count () const
Protected Member Functions inherited from FST::Clonable
void	set_cloned (const bool cloned=true)
	marks/unmarks this instance as a clone; call from clone()/stateless_clone() overrides (Clonable_Impl does this automatically)
void	set_sharing (const bool sharing=true)
	marks/unmarks this instance as sharing (see is_sharing())
void	set_sharing_cloned (const bool cloned=true, const bool sharing=true)
	sets both flags at once; call from sharing_clone() overrides (Clonable_Impl does this automatically)

Protected Attributes
REALTYPE	_C
	regularization parameter, see set_parameter_C()
bool	_l2loss
	true = squared hinge (default), false = hinge
unsigned int	_max_iter
	cap on full coordinate sweeps in solve_dual()
REALTYPE	_tol
	max \|projected gradient\| sweep-stopping threshold
DIMTYPE	_classes
	class count, set by initialize()
DIMTYPE	_features
	raw (full, unnarrowed) feature count, set by initialize()
DIMTYPE	_subfeatures
	d = current subset size
std::vector< DIMTYPE >	_index
	subset position -> raw feature index (first d entries valid)
std::vector< RETURNTYPE >	_cs_sc
	classify() score scratch (member to avoid per-pattern allocation)
std::vector< REALTYPE >	_mean
	standardization shift per subset position
std::vector< REALTYPE >	_scale
	standardization scale per subset position (1 for constant features)
std::vector< REALTYPE >	_W
	models x (d+1) row-major; column d = bias; two-class data holds ONE model
std::vector< REALTYPE >	_Z
	N x (d+1) standardized design incl. bias coordinate.
std::vector< DIMTYPE >	_y
	N training labels.
std::vector< REALTYPE >	_alpha
	N dual variables of the model currently being trained (solve_dual() scratch).
std::vector< REALTYPE >	_qbar
	N diagonal entries x_i^T x_i (+ 1/(2C) for L2-loss).

Detailed Description

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>
class FST::Classifier_Linear_SVM< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >

Dependency-free linear SVM trained by dual coordinate descent (the method of Hsieh et al., ICML 2008, as popularized by LIBLINEAR — implemented from scratch, no external code).

L2-regularized L1-loss (hinge) or L2-loss (squared hinge) dual, solved coordinate-wise with the weight vector maintained incrementally. The coordinate order is FIXED CYCLIC — deliberately no random permutation, so training is fully deterministic and RNG-free: it never touches the fst_rand stream, and results are therefore identical for any seed and at any thread count. The bias enters as an appended constant coordinate (LIBLINEAR's -B 1 convention). Features are standardized internally per split like the logistic-regression classifier; constant features stay inert.

Multiclass = one-vs-rest with K linear models; two-class data trains ONE model (class index 0 = +1 side) and derives both scores from its single decision value. classify() takes the argmax of the per-class decision values (first maximum wins); classify_scores() exposes them for the ranking metrics (AUC).

This wrapper is also the remedy for a known LibSVM weakness: LibSVM's LINEAR-kernel training can fail to converge (appear to freeze) on some data, whereas this solver has no such behaviour — and linear-SVM selection no longer needs LibSVM at all.