Introduction This post is going to be a short and birds-eye view of probability theory. During my undergraduate days, I often wonder what is the use of probability theory provided its fluctuating nature. For example chance of getting ahead by tossing a fair coin is 0.5. Suppose we got the head the first time, there …
Bellman operators come in Reinforcement Learning (RL). When I first encountered it, I had many questions regarding it. I often feel it is interesting to observe that what feels intriguing to someone. Many questions surface out in our minds. Why is it called an operator?. What are the inputs to it and output from it?. …
In this post, we will see how to animate matplotlib graphs. In many situations you might wish to create an animated content of some dynamic graph or process. You might want to do this to explain or understand a concept better. It is especially appealing for people like me who cannot think in abstract terms. …
Overview In this note about distributional reinforcement learning, I am going to reflect on the paper titled A Distributional Perspective on Reinforcement Learning. I will try to give an overview of the underlying ideas behind this paper. Please keep in mind that a research contribution is often a culmination of multiple ideas/components. This makes from …
Summary of research paper “A Distributional Perspective on Reinforcement Learning” Read More »
Introduction In the initial years of my signal processing career, I have struggled to find tools and software that can do audio signal processing tasks. I was aware of free software like Audacity and python programming language. But I often felt a curated list of all possible options would have helped me a lot …
Linear regression is one of the algorithm machine learning enthusiasts start to learn first. In this article, l will walk you through the linear regression intuition. We will implement the basic form of it without using any machine learning packages. The main philosophy of supervised machine learning algorithms is to learn from data. The algorithms …
The concept of simplex appears in science and engineering over and over again. For example it appears in optimization, information theory, communication systems, linear algebra, etc. Let’s have a quick look at it from different contexts. In general, simplex can be thought of as a mathematical expression of $\alpha x + (1-\alpha)y$. It can be …
Introduction In this tutorial, I am going to code the iterative policy evaluation algotithm from the book “Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton”. I am going to take psuedo code, image and examples from this text. The example I am taking for this tutorial is the gird world maze from Chapter …
Iterative Policy Evaluation for Estimating Value Function Read More »
This article shows two key visual intuitions behind the usage of a discount factor in reinforcement learning with image, code, and video. Introduction Most of the advances in science and technology happened in the last 100 years. We can see mind-boggling progress in automotive, medicine, communication, energy, etc. . Among these advances, some technologies shake …
In summertime when I go out, I don’t take the raincoat with me. On what basis am I not expecting it to rain during summer?. Isn’t it my past experience gives me that knowledge and estimate the outcome?. The expected value in probability theory leverages such a pattern in data to make a good representation …