# Code

## Gaussian Distribution Explained Visually

Gaussian distribution appears in various parts of science and engineering. Apart from a distribution often appear in nature, it has got important properties such as its relation to Central Limit Theorem (CLT). The figure above shows one-dimensional Gaussian distributions of various mean and variance values. Libraries like NumPy provide functions that can return Gaussian distribution …

## Coding a Simple Markov Decision Process

A Markov Decision Process (MDP) is a mathematical framework used to model decision-making situations where the outcome of a decision depends on both the current state of the system and the actions taken by the decision maker. In an MDP, the decision maker is represented as an agent, and the system is represented as a …

## An Intuitive Explanation of Naive Bayes Classifier

Introduction In this post, let’s take a look at the intuition behind Naive Bayes Classifier used in machine learning. Naive Bayes classifier is one of the basic algorithms often encountered in machine learning applications. If linear regression was based on concepts from linear algebra and calculus, naive Bayes classifier mostly backed up by probability theory. …

## Animate Matplotlib Graphs

In this post, we will see how to animate matplotlib graphs. In many situations you might wish to create an animated content of some dynamic graph or process. You might want to do this to explain or understand a concept better. It is especially appealing for people like me who cannot think in abstract terms. …

## Open Source Audio Signal Processing Tools

Introduction In the initial years of my signal processing career, I have struggled to find tools and software that can do audio signal processing tasks. I was aware of free software like Audacity and python programming language. But I often felt a curated list of all possible options would have helped me a lot …

## An Intuitive Explanation of Linear Regression

Linear regression is one of the algorithm machine learning enthusiasts start to learn first. In this article, l will walk you through the linear regression intuition. We will implement the basic form of it without using any machine learning packages. The main philosophy of supervised machine learning algorithms is to learn from data. The algorithms …

## Iterative Policy Evaluation for Estimating Value Function

Introduction In this tutorial, I am going to code the iterative policy evaluation algotithm from the book “Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton”. I am going to take psuedo code, image and examples from this text. The example I am taking for this tutorial is the gird world maze from Chapter …

## Discount Factor in Reinforcement Learning

This article shows two key visual intuitions behind the usage of a discount factor in reinforcement learning with image, code, and video. Introduction Most of the advances in science and technology happened in the last 100 years. We can see mind-boggling progress in automotive, medicine, communication, energy, etc. . Among these advances, some technologies shake …

## LaTeX Tutorial

This tutorial is on the typesetting language called LaTeX. LaTeX is useful when it comes to creating complex documents like thesis, project reports, research articles, etc. We will see the basic use-cases in around 22 examples. LaTeX Installation Instructions Windows In Windows, you need to install two programs called MiKTeX and TeXstudio. Please be careful to install …

## Epsilon Greedy Algorithm in Bandit Problems

Introduction Bandit problems are the simplest possible reinforcement learning scenario. Here the bandit machine can have k arms and pulling each arm leaves the user a reward. One of the arms will be giving higher rewards in the long run and moreover this pattern could be changing over a time period. Think of the scenario …