# Conceptual

## A conceptual look at Bellman operator

Bellman operators come in Reinforcement Learning (RL). When I first encountered it, I had many questions regarding it. I often feel it is interesting to observe that what feels intriguing to someone. Many questions surface out in our minds. Why is it called an operator?. What are the inputs to it and output from it?. …

## Paper Note: A Distributional Perspective on Reinforcement Learning

Overview In this note about distributional reinforcement learning, I am going to reflect on the paper titled A Distributional Perspective on Reinforcement Learning. I will try to give an overview of the underlying ideas behind this paper. Please keep in mind that a research contribution is often a culmination of multiple ideas/components. This makes from …

## An Intuitive Explanation of Linear Regression

Linear regression is one of the algorithm machine learning enthusiasts start to learn first. In this article, l will walk you through the linear regression intuition. We will implement the basic form of it without using any machine learning packages. The main philosophy of supervised machine learning algorithms is to learn from data. The algorithms …

## Simplex explained in one minute

The concept of simplex appears in science and engineering over and over again. For example it appears in optimization, information theory, communication systems, linear algebra, etc. Let’s have a quick look at it from different contexts. In general, simplex can be thought of as a mathematical expression of $\alpha x + (1-\alpha)y$. It can be …

## Discount Factor in Reinforcement Learning

This article shows two key visual intuitions behind the usage of a discount factor in reinforcement learning with image, code, and video. Introduction Most of the advances in science and technology happened in the last 100 years. We can see mind-boggling progress in automotive, medicine, communication, energy, etc. . Among these advances, some technologies shake …

## Expected Value

In summertime when I go out, I don’t take the raincoat with me. On what basis am I not expecting it to rain during summer?. Isn’t it my past experience gives me that knowledge and estimate the outcome?. The expected value in probability theory leverages such a pattern in data to make a good representation …

## Tensors, Simple yet Complicated!!

Throughout my academics when I come across the term tensor, my brain send reflex that “something complicated, take a break, relax and read later”. Often, I gave a thought, Am I reading it Tensor or Terror!! Physicists use Tensors to describe properties of systems which vary with direction called anisotropic systems whereas for mathematicians Tensors …

## Bootstrapping: four types of it

I am sure you might have heard the term bootstrapping already. The first time I heard it, I felt it like jargon. Later I came to know that there are four different scenarios the term comes in. All of them have some basic roots but are in different subject domains. The boot-strapping We all know …

## A Short Intuitive Explanation of Vectors

What is a vector? I know you might be already thinking- “a vector is something which has magnitude and direction”. But that’s old school definition. I also used to think that way, but after I left the academic world, I had to dig deeper. The more I explored, the deeper the concept became. This post …