N-Gram Model

Algorithm_Optimization_Audit

Python Pattern Recognition // Bigram & Trigram

Execution_Environment

Python // NumPy // Regex // Jupyter

Optimization_Log

// Data_Cleansing

Analyzed a subset of the 1 Billion Word Dataset. Engineered custom Regex pipelines to tokenize and normalize the corpus, mapping unknown tokens to safely handle data sparsity.

// Probabilistic_Architecture

Developed Bigram and Trigram models from scratch. Resolved zero-probability bottlenecks through Add-K (Laplace) smoothing and Stupid Backoff.

// Final_Optimization

Evaluated accuracy via Perplexity (PP) in log-space. Achieved a 4500x improvement in Trigram performance by migrating to optimized Linear Interpolation.

Primary_Document_Stream