Conditional Random Fields › Key Parameters

When working with CRFs using sklearn-crfsuite, the following parameters most significantly affect model performance:

• algorithm: The optimization algorithm for training. Options include "lbfgs" (Limited-memory BFGS, default), "l2sgd" (SGD with L2 regularization), "ap" (Averaged Perceptron), and "pa" (Passive Aggressive). L-BFGS typically converges faster and works well for most tasks. SGD variants may be preferred for very large datasets that don't fit in memory.

• c1 (L1 regularization): Controls feature sparsity. Higher values push more feature weights to exactly zero, producing simpler models with fewer active features. Start with values between 0.0 and 0.5. Useful when you have many features and want automatic feature selection.

• c2 (L2 regularization): Controls weight magnitude. Higher values prevent any single feature from dominating predictions. Start with values between 0.01 and 0.1. Helps prevent overfitting, especially with limited training data.

• max_iterations: Maximum number of optimization iterations. Default is 100, which suffices for most tasks. Increase if the model hasn't converged (check training logs). Very large values rarely help and slow training.

• all_possible_transitions: When True, includes transition weights for all label pairs, even those not observed in training. This allows the model to learn that unseen transitions should have negative weights. Generally recommended for BIO tagging where you want to explicitly penalize invalid transitions like B-PER to I-LOC.

• all_possible_states: When True, includes state features for all labels, even for features not observed with that label in training. Useful when test data may contain feature-label combinations not seen during training.

For neural CRFs, key parameters include the hidden dimension of the LSTM (typically 64-256), dropout rate (0.1-0.5), and learning rate (1e-4 to 1e-3 with Adam optimizer).