# Sajil C. K.

I am an inquisitive person with interests falling at the intersection of mathematics, programming, and research. I like to understand concepts to the core by building things from scratch.  ## A conceptual look at Bellman operator

Bellman operators come in Reinforcement Learning (RL). When I first encountered it, I had many questions regarding it. I often feel it is interesting to observe that what feels intriguing to someone. Many questions surface out in our minds. Why is it called an operator?. What are the inputs to it and output from it?. … ## Summary of research paper “A Distributional Perspective on Reinforcement Learning”

Overview In this note about distributional reinforcement learning, I am going to reflect on the paper titled A Distributional Perspective on Reinforcement Learning. I will try to give an overview of the underlying ideas behind this paper. Please keep in mind that a research contribution is often a culmination of multiple ideas/components. This makes from … ## An Intuitive Explanation of Linear Regression

Linear regression is one of the algorithm machine learning enthusiasts start to learn first. In this article, l will walk you through the linear regression intuition. We will implement the basic form of it without using any machine learning packages. The main philosophy of supervised machine learning algorithms is to learn from data. The algorithms … ## Simplex explained in one minute

The concept of simplex appears in science and engineering over and over again. For example it appears in optimization, information theory, communication systems, linear algebra, etc. Let’s have a quick look at it from different contexts. In general, simplex can be thought of as a mathematical expression of $\alpha x + (1-\alpha)y$. It can be … ## Iterative Policy Evaluation for Estimating Value Function

Introduction In this tutorial, I am going to code the iterative policy evaluation algotithm from the book “Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton”. I am going to take psuedo code, image and examples from this text. The example I am taking for this tutorial is the gird world maze from Chapter … ## Discount Factor in Reinforcement Learning

This article shows two key visual intuitions behind the usage of a discount factor in reinforcement learning with image, code, and video. Introduction Most of the advances in science and technology happened in the last 100 years. We can see mind-boggling progress in automotive, medicine, communication, energy, etc. . Among these advances, some technologies shake …