Probability Primer


This post is going to be a short and birds-eye view of probability theory. During my undergraduate days, I often wonder what is the use of probability theory provided its fluctuating nature.

For example chance of getting ahead by tossing a fair coin is 0.5. Suppose we got the head the first time, there is no guarantee that we will get a tail next time.

Then why are we going after/what is the use of such a discipline?. Probability theory has utility and that is why it is used in so many disciplines.

tossing coin
Tossing coin

The birth of probability theory was to deal with the uncertainty of events. Many things in real-life are uncertain, and only a few of them are predictable. Predictable things in the universe are studied through physical sciences.

But there are situations where we cannot explain things using physical sciences. We may know the relation between the time taken for a ball (of a certain mass) to hit the ground from a certain height. We can formulate a specific equation for this. But we cannot formulate an equation to predict how much rainfall we will get tomorrow.

This brings up two questions primarily. First, is why we can understand some processes but some cannot?. Second is what would be our strategy to deal with such uncertain events?.

Bring back the original thinker in you

original thoughts
Think like baby, bring your original thoughts

For a moment let’s forgot all the education we got and put some original thoughts.

Uncertain events are uncertain to us because it involves complex processes and it is very difficult for us to decipher the relationship between the parameters involved. In the ball falling example, we can identify the major factors like the mass of the ball, the height from which it is dropped, acceleration due to gravity, etc

Of course, there are other factors like the air resistance, shape of the ball (if you agree with me on the fact that there is no perfect sphere in real world), direction, and strength of any wind blowing, etc. But compared to the major factors, the effects produced by these are negligible and we were able to formulate a mathematical representation for the process that describes it fairly accurately.

But when it comes to phenomena like whether it will rain tomorrow or not. The sheer volume of factors involved makes it not decipherable to use. Now the question is what will be our strategy in such a situation?.

My uneducated mind tells me that if you can’t figure it out then observe whether there is any pattern. In complex phenomena, all we can do is to observe data, look for patterns, and make predictions that may be wrong quite often. But still, we are better at predicting than never. The only requirement is that we need to have access to a large pool of historical data.

We apply probability in our everyday life often without realizing it. To decide whether to buy a lottery or not, when to start our journey from home to office, choosing a career path with less competition, etc. Aren’t these all evidence of the fact that we know/use probability inherent in our thoughts?

The concept of limit and its relation to probability

If we toss a fairs coin, the chance to get heads is 0.5. This doesn’t guarantee that if we toss 10 times in a row, we will get 5 heads. The idea is as the number of trials goes infinity the ratio number of heads/ total number of trials approaches 0.5.

The concept of infinity appears here. If this is the case then how can we trust probability theory if we are not talking about/dealing with an infinite number of experiments?. Between, the concepts of limit and infinity are other mind-twisting ones for me. It is certainly not an obvious concept and somewhat beyond our physical reality. We will have a look at them in other posts.

Venn diagrams and dice example

Dice and venn diagram

Probability theory has sample space, events, and probability values. Here there is a connection between ratios and areas. We usually pictorially represent sample space as a 2D geometrical shape like a circle or rectangle. We can relate the ratio of areas to the final probability value.

But is pictorial explanations always going to be accurate? What if the sample space pictorial representation goes wrong? won’t it pollute the answer also?

For example, consider the probability of getting 2 if we roll a fair die. When I say event here a rectangle with six equal parts comes to my mind. I already assumed all numbers (1,2,3,4,5,6) are equally likely and assigned areas according to their respective probability values. Is it possible to have such representations for all events?

Relation to statistics

How do probability and statistics relate to each other?. Are they complement each other?. Probability looks at all possibilities and predicts behaviors. Statistics looks at behaviors/outcomes and try to infer the underlying phenomenon.

Continuous and discrete probability

Probability theory has two tracks Continuous and Discrete. For me, there is only mathematical analysis possible with continuous probability. And it lives only in the abstract mathematical world. Because we humans certainly can’t deal with infinite items.

Conditional or marginal probability

The conditional probability is also known as the marginal probability. Written in table form, it often limits our calculation to marginal columns hence the historical etymology. The condition here gives us additional information and reduces our sample space. This helps us to get to update our estimates.

Closing thoughts

It is also important to mention that probability theory has mainly two schools of thought: Bayesian and Frequentist. This requires deep thoughts. We will explore it in another post.

We barely scratched some very basic questions on this topic. I often feel in traditional education we often jumps too quickly to advanced concepts without such a philosophical discussion. This was my humble attempt to fill that gap. I am pretty sure that there could be flaws or errors in my arguments. Please feel free to share your thought on this.

1 thought on “Probability Primer”

  1. Pingback: Machine Learning Research Advice - Summary of Reddit Discussion

Leave a Comment

Your email address will not be published.