Naive Bayes › Disadvantages

The central weakness is the conditional independence assumption. Naive Bayes treats features as independent once the class is known. When features remain dependent within a class, multiplying their marginal likelihoods can count overlapping evidence multiple times. A spam classifier may treat "free" and "win" as independent pieces of evidence. If those words remain dependent within spam, multiplying their marginal likelihoods discards that dependence. The result is a classifier that can rank observations well but produce probabilities that are too extreme.

The second weakness is the distributional assumption. Gaussian naive Bayes assumes each feature is normally distributed within each class. If a feature is skewed, multimodal, or bounded, a single Gaussian may be a poor description. Multinomial naive Bayes assumes that word counts follow a multinomial distribution with class-specific word probabilities. That is a reasonable approximation for some text data, but it ignores word order, grammar, and longer-range dependencies.

The third weakness is the zero-frequency problem. If a word never appears in the training documents of a class, the unsmoothed multinomial estimate for that word is zero. A test document containing that word then receives zero likelihood for that class, regardless of all other evidence. Smoothing fixes the numerical problem, but it does not make the model correct.

Class imbalance is another practical concern. The prior $p(y=k)$ can dominate the score when one class is much more frequent than another. A model that predicts the majority class most of the time can have high accuracy but poor recall for the minority class. We need to evaluate with class-specific metrics and consider prior adjustment or threshold tuning.

Finally, naive Bayes cannot capture feature interactions. If the class depends on a combination of features rather than on each feature separately, naive Bayes may fail even with unlimited data. This is a bias problem, not a variance problem. More data will not fix a model that cannot represent the true decision boundary.