The concept of simplex appears in science and engineering over and over again. For example it appears in optimization, information theory, communication systems, linear algebra, etc. Let’s have a quick look at it from different contexts. In general, simplex can be thought of as a mathematical expression of $\alpha x + (1-\alpha)y$. It can be …
Introduction In this tutorial, I am going to code the iterative policy evaluation algotithm from the book “Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton”. I am going to take psuedo code, image and examples from this text. The example I am taking for this tutorial is the gird world maze from Chapter …
Iterative Policy Evaluation for Estimating Value Function Read More »
This article shows two key visual intuitions behind the usage of a discount factor in reinforcement learning with image, code, and video. Introduction Most of the advances in science and technology happened in the last 100 years. We can see mind-boggling progress in automotive, medicine, communication, energy, etc. . Among these advances, some technologies shake …
In summertime when I go out, I don’t take the raincoat with me. On what basis am I not expecting it to rain during summer?. Isn’t it my past experience gives me that knowledge and estimate the outcome?. The expected value in probability theory leverages such a pattern in data to make a good representation …
Throughout my academics when I come across the term tensor, my brain send reflex that “something complicated, take a break, relax and read later”. Often, I gave a thought, Am I reading it Tensor or Terror!! Physicists use Tensors to describe properties of systems which vary with direction called anisotropic systems whereas for mathematicians Tensors …
I am sure you might have heard the term bootstrapping already. The first time I heard it, I felt it like jargon. Later I came to know that there are four different scenarios the term comes in. All of them have some basic roots but are in different subject domains. The term “bootstrap” has different …
This tutorial is on the typesetting language called LaTeX. LaTeX is useful when it comes to creating complex documents like thesis, project reports, research articles, etc. We will see the basic use-cases in around 22 examples. LaTeX Installation Instructions Windows In Windows, you need to install two programs called MiKTeX and TeXstudio. Please be careful to install …
What is a vector? I know you might be already thinking- “a vector is something which has magnitude and direction”. But that’s old school definition. I also used to think that way, but after I left the academic world, I had to dig deeper. The more I explored, the deeper the concept became. This post …
Introduction Iterative Policy Improvement (IPI) is an algorithm in reinforcement learning to find the optimal course of action given the enviroment conditions. This blog post explains how it is done using a simple grid world navigating example. It works by iteratively improving an initial policy using the policy evaluation and policy improvement steps. Here’s how …
Introduction Bandit problems are the simplest possible reinforcement learning scenario. Here the bandit machine can have k arms and pulling each arm leaves the user a reward. One of the arms will be giving higher rewards in the long run and moreover this pattern could be changing over a time period. Think of the scenario …