LayerNorm and RMS Norm in Transformer Models
This post is divided into five parts; they are: • Why Normalization is Needed in Transformers • LayerNorm and Its Implementation • Adaptive LayerNorm • RMS Norm and Its Implementation • Using PyTorch's...
View ArticleLinear Layers and Activation Functions in Transformer Models
This post is divided into three parts; they are: • Why Linear Layers and Activations are Needed in Transformers • Typical Design of the Feed-Forward Network • Variations of the Activation Functions The...
View ArticleYour First Local LLM API Project in Python Step-By-Step
Interested in leveraging a large language model (LLM) API locally on your machine using Python and not-too-overwhelming tools frameworks? In this step-by-step article, you will set up a local API where...
View ArticleMixture of Experts Architecture in Transformer Models
This post covers three main areas: • Why Mixture of Experts is Needed in Transformers • How Mixture of Experts Works • Implementation of MoE in Transformer Models The Mixture of Experts (MoE) concept...
View Article5 Advanced RAG Architectures Beyond Traditional Methods
Retrieval-augmented generation (RAG) has shaken up the world of language models by combining the best of two worlds:
View ArticleSkip Connections in Transformer Models
This post is divided into three parts; they are: • Why Skip Connections are Needed in Transformers • Implementation of Skip Connections in Transformer Models • Pre-norm vs Post-norm Transformer...
View ArticleSecuring FastAPI Endpoints for MLOps: An Authentication Guide
In today's AI world, data scientists are not just focused on training and optimizing machine learning models.
View Article10 NumPy One-Liners to Simplify Feature Engineering
When building machine learning models, most developers focus on model architectures and hyperparameter tuning.
View ArticleDecision Trees Aren’t Just for Tabular Data
Versatile, interpretable, and effective for a variety of use cases, decision trees have been among the most well-established machine learning techniques for decades, widely used for classification and...
View Article
