FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR > Class Template Reference

Implements Naive-like Bayes classifier based on multinomial model. More...

#include <classifier_multinom_naivebayes.hpp>

Inheritance diagram for FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >:

Public Types
typedef Clonable_Impl< Classifier_Multinomial_NaiveBayes, Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > >	parent_impl
	implements clone(); sharing/stateless_clone throw in Classifier
typedef std::shared_ptr< DATAACCESSOR >	PDataAccessor
	shared pointer to the data accessor supplying train/test blocks
typedef std::shared_ptr< SUBSET > const	PSubset
	shared pointer to the (fixed, non-reassignable) feature subset being trained/evaluated on
typedef DATAACCESSOR::PPattern	PPattern
	one data sample as exposed by DATAACCESSOR (a raw pointer to its feature vector)
Public Types inherited from FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >
typedef std::shared_ptr< DATAACCESSOR >	PDataAccessor
	shared pointer to the data accessor supplying train/test blocks
typedef std::shared_ptr< SUBSET > const	PSubset
	shared pointer to the (fixed, non-reassignable) feature subset being evaluated
typedef DATAACCESSOR::PPattern	PPattern
	one data sample as exposed by DATAACCESSOR (a raw pointer to its feature vector)

Public Member Functions
void	enable_prelearn_mode (const PDataAccessor da)
void	disable_prelearn_mode ()
	reverts to the default (per-train() relearning) mode; see enable_prelearn_mode()
bool	get_prelearn_mode () const
	true if enable_prelearn_mode() is currently in effect
void	initialize (const PDataAccessor da)
virtual bool	classify (DIMTYPE &cls, const PPattern &pattern) override
virtual bool	classify_scores (std::vector< RETURNTYPE > &scores, const PPattern &pattern) override
virtual bool	test_confusion (std::vector< RETURNTYPE > &confusion, DIMTYPE &classes, const PDataAccessor da) override
virtual bool	train (const PDataAccessor da, const PSubset sub) override
	learns from designated training part of data
virtual bool	test (RETURNTYPE &result, const PDataAccessor da) override
DIMTYPE	get_n () const
	the underlying model's full ("raw") learned dimensionality (Model_Multinomial::get_n())
DIMTYPE	get_d () const
	the underlying model's current virtual (narrowed) dimensionality (Model_Multinomial::get_d())
virtual std::ostream &	print (std::ostream &os) const override
	writes a human-readable identification of the object; base default writes nothing
Public Member Functions inherited from FST::Clonable_Impl< Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >, Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > >
Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > *	clone () const override
	dispatches to the DEEP_KIND mechanism (default CLONE_DEEP: Derived's copy-constructor)
Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > *	sharing_clone () const override
	dispatches to the SHARING_KIND mechanism (default CLONE_INHERIT: defers to Base, which throws by family convention)
Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR > *	stateless_clone () const override
	dispatches to the STATELESS_KIND mechanism (default CLONE_INHERIT: defers to Base, which throws by family convention)
Public Member Functions inherited from FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >
virtual bool	test_scores (std::vector< std::pair< RETURNTYPE, DIMTYPE > > &scores, const PDataAccessor da)
virtual bool	test_scores_matrix (std::vector< RETURNTYPE > &scores, std::vector< DIMTYPE > &labels, DIMTYPE &classes, const PDataAccessor da)
virtual bool	test_metric (RETURNTYPE &result, const PDataAccessor da, const Metric_Spec< RETURNTYPE > &spec)
Public Member Functions inherited from FST::Clonable
bool	is_clone () const
bool	is_sharing () const
	true if this instance is a sharing_clone() result (referencing, not owning, the source's structures)
bool	is_sharing_clone () const
	true if this instance is BOTH a clone and sharing – i.e. specifically a sharing_clone() result, not a clone()/stateless_clone() one

Protected Attributes
std::unique_ptr< Model_Multinomial< DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR > >	_model
	the underlying multinomial model, learned/narrowed by train()
DIMTYPE	_classes
	class count, sized by initialize() (dimensions _Pcd)
DIMTYPE	_features
	raw feature count, sized by initialize() (dimensions _index)
std::unique_ptr< REALTYPE[]>	_Pcd
	per-class score of the pattern last passed to classify(): the UNNORMALIZED log-posterior ln P(c) + sum_f x[f]*ln theta(f|c), not a probability (lazy allocation)

Private Member Functions
	Classifier_Multinomial_NaiveBayes (const Classifier_Multinomial_NaiveBayes &cmnb)
	copy-constructor (deep-copies _model, _Pcd and _index, each only if the source had them allocated, i.e. only after initialize())

Private Attributes
friend	parent_impl
	grants Clonable_Impl access to the copy-constructor
bool	_prelearn_mode
	see enable_prelearn_mode()/disable_prelearn_mode()/get_prelearn_mode()
std::unique_ptr< DIMTYPE[]>	_index
	subset position -> raw feature index (first _subfeatures entries valid), filled by train()
DIMTYPE	_subfeatures
	number of features in the currently trained subset (sub->get_d_raw() at the last train() call)

Additional Inherited Members
Protected Member Functions inherited from FST::Clonable
void	set_cloned (const bool cloned=true)
	marks/unmarks this instance as a clone; call from clone()/stateless_clone() overrides (Clonable_Impl does this automatically)
void	set_sharing (const bool sharing=true)
	marks/unmarks this instance as sharing (see is_sharing())
void	set_sharing_cloned (const bool cloned=true, const bool sharing=true)
	sets both flags at once; call from sharing_clone() overrides (Clonable_Impl does this automatically)

Detailed Description

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>
class FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >

Implements Naive-like Bayes classifier based on multinomial model.

The document classification rule of the multinomial model (Model_Multinomial): a pattern is assigned to the class maximizing

ln P(c) + sum over the selected features f of  x[f] * ln theta(f|c)

where x[f] is the pattern's term count and theta(f|c) the class-conditional term probability the model learned. This is the log-posterior up to the constants (the multinomial coefficient and the evidence) that are the same for every class and therefore cannot change the argmax.

"Naive-LIKE" rather than plain Naive Bayes: the per-feature factors are the multinomial term probabilities of one shared word distribution per class, not independent per-feature distributions - the feature contributions are coupled through theta's normalization over the selected features, so the usual conditional-independence reading does not apply literally.

Warning

A ONE-FEATURE subset is degenerate here: with a single feature theta(f|c)==1 in every class, every log term vanishes and the score no longer depends on the data at all. The class treats such subsets as unusable rather than pretending to classify them, and does so consistently: classify() returns false, test() reports accuracy 0, the confusion matrix holds one off-diagonal count (so every confusion-based metric is 0 as well), and classify_scores() returns all-equal scores (AUC 0.5). Feature selection driven by this classifier therefore never settles on a single feature.

Member Function Documentation

classify()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

bool FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::classify ( DIMTYPE & cls, const PPattern & pattern )

overridevirtual

Classifies one pattern; on success cls receives the predicted class index (0-based, in [0,classes)). Returns false when no decision can be made at all - the model has not been trained, or the implementation hit an unrecoverable condition; false is not a "reject" option and aborts the enclosing test loop.

Implements FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >.

classify_scores()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

virtual bool FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::classify_scores ( std::vector< RETURNTYPE > & scores, const PPattern & pattern )

inlineoverridevirtual

per-class real scores for one pattern (any monotone transform of the class posteriors); enables ranking-based metrics (AUC). Default: not supported.

Reimplemented from FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >.

enable_prelearn_mode()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

void FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::enable_prelearn_mode

(

const PDataAccessor

da

)

pre-learning mode: if true, calls once _model->learn() for full set size and later in train() uses only narrow()ing to access submatrixes (makes sense only as long as training data do not change, i.e., within one split) if false (default), learns new model in each train() call

initialize()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

void FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::initialize

(

const PDataAccessor

da

)

(Re)allocates the _Pcd and _index buffers to da's current class/feature counts. train() calls this itself on every invocation, so an explicit call is only needed to size these buffers ahead of train() (the fst4 CLI's classifier factory does this right after construction).

Note

It allocates only; it learns nothing. The multinomial model itself is learned (or narrowed) inside train().

print()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

virtual std::ostream & FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::print ( std::ostream & os ) const

inlineoverridevirtual

writes a human-readable identification of the object; base default writes nothing

Reimplemented from FST::Clonable.

test()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

bool FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::test ( RETURNTYPE & result, const PDataAccessor da )

overridevirtual

Estimates accuracy (fraction of correctly classified test patterns) using the designated test data. The default implementation is the canonical accuracy loop over the virtual classify(); classifiers needing extra preconditions or a batched evaluation override it (Classifier_Normal_Bayes, Classifier_Multinomial_NaiveBayes, Classifier_SVM, Classifier_Extern). Returns false if the test part is empty or classify() failed.

Reimplemented from FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >.

test_confusion()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

virtual bool FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::test_confusion ( std::vector< RETURNTYPE > & confusion, DIMTYPE & classes, const PDataAccessor da )

inlineoverridevirtual

assembles the classes*classes confusion matrix (row-major, row = true class) over the designated test data. Default implementation replicates the canonical test() loop via the virtual classify(); concrete classifiers override only to preserve special conventions (see Classifier_Multinomial_NaiveBayes).

Reimplemented from FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >.

train()

template<class RETURNTYPE, typename DATATYPE, typename REALTYPE, typename IDXTYPE, typename DIMTYPE, class SUBSET, class DATAACCESSOR>

bool FST::Classifier_Multinomial_NaiveBayes< RETURNTYPE, DATATYPE, REALTYPE, IDXTYPE, DIMTYPE, SUBSET, DATAACCESSOR >::train ( const PDataAccessor da, const PSubset sub )

overridevirtual

learns from designated training part of data

Implements FST::Classifier< RETURNTYPE, DIMTYPE, SUBSET, DATAACCESSOR >.

---

The documentation for this class was generated from the following file:

• /Users/psms/dev/devtest1/_src_criteria/classifier_multinom_naivebayes.hpp