Probability theory

Probability theory is a branch of mathematics concerned with determining the long run frequency or chance that a given event will occur. This chance is determined by dividing the number of selected events by the number of total events possible. For example, each of the six faces of a die has one in six probability on a single toss. Inspired by problems encountered by seventeenth century gamblers, probability theory has developed into one of the most respected and useful branches of mathematics with applications in many different industries. Perhaps what makes probability theory most valuable is that it can be used to determine the expected outcome in any situation from the chances that a plane will crash to the probability that a person will win the lottery.

History of probability theory

The branch of mathematics known as probability theory was inspired by gambling problems. The earliest work was performed by Girolamo Cardano (1501-1576) an Italian mathematician, physician, and gambler. In his manual Liber de Ludo Aleae, Cardano discusses many of the basic concepts of probability complete with a systematic analysis of gambling problems. Unfortunately, Cardano's work had little effect on the development of probability because his manual, which did not appeared in print until 1663, received little attention.

In 1654, another gambler named Chevalier de Méré created a dice proposition which he believed would make money. He would bet even money that he could roll at least one 12 in 24 rolls of two dice. However, when the Chevalier began losing money, he asked his mathematician friend Blaise Pascal (1623-1662) to analyze the proposition. Pascal determined that this proposition will lose about 51% of the time. Inspired by this proposition, Pascal began studying more of these types of problems. He discussed them with another famous mathematician, Pierre de Fermat (1601-1665) and together they laid the foundation of probability theory.

Probability theory is concerned with determining the relationship between the number of times a certain event occurs and the number of times any event occurs. For example, the number of times a head will appear when a coin is flipped 100 times. Determining probabilities can be done in two ways; theoretically and empirically. The example of a coin toss helps illustrate the difference between the two approaches. Using a theoretical approach, we reason that in every flip there are two possibilities, a head or a tail. By assuming each event is equally likely, the probability that the coin will end up heads is 1/2 or 0.5. The empirical approach does not use assumption of equal likeliness. Instead, an actual coin flipping experiment is performed and the number of heads is counted. The probability is then equal to the number of heads divided by the total number of flips.

Counting

A theoretical approach to determine probabilities requires the ability to count the number of ways certain events can occur. In some cases, counting is simple because there is only one way for an event to occur. For example, there is only one way in which a 4 will show up on a single roll of a die. In most cases, however, counting is not always an easy matter. Imagine trying to count the number of ways of being dealt a pair in 5 card poker.

The fundamental principle of counting is often used when many selections are made from the same set of objects. Suppose we want to know the number of different ways four people can line up in a carnival line. The first spot in line can be occupied by any of the four people. The second can be occupied any of the three people who are left. The third spot can be filled by either of the two remaining people, and the fourth spot is filled by the last person. So, the total number of ways four people can create a line is equal to the product 4 × 3 × 2 × 1 = 24. This product can be abbreviated as 4! (read "4 factorial"). In general, the product of the positive integers from 1 to n can be denoted by n! which equals n × (n-1) × (n-2) ×...2 × 1. It should be noted that 0! is by definition equal to 1.

The example of the carnival line given above illustrates a situation involving permutations. A permutation is any arrangement of n objects in a definite order. Generally, the number of permutations of n objects is n. Now, suppose we want to make a line using only two of the four people. In this case, any of the four people can occupy the first space and any of the three remaining people can occupy the second space. Therefore, the number of possible arrangements, or permutations, of two people from a group of four, denoted as P4,2 is equal to 4 × 3 = 12. In general, the number of permutations of n objects taken r at a time is

This can be written more compactly as Pn,r = n!/(n-r)!

Many times the order in which objects are selected from a group does not matter. For instance, we may want to know how many different 3 person clubs can be formed from a student body of 125. By using permutations, some of the clubs will have the same people, just arranged in a different order. We only want to count then number of clubs that have different people. In these cases, when order is not important, we use what is known as a combination. In general, the number of combinations denoted as Cn,r or is equal to Pn,r /r! or Cn,r = n!/r! × (n-r)! For our club example, the number of different three person clubs that can be formed from a student body of 125 is C125,3 or 125!/3! × 122! = 317,750.

Experiments

Probability theory is concerned with determining the likelihood that a certain event will occur during a given random experiment. In this sense, an experiment is any situation which involves observation or measurement. Random experiments are those which can have different outcomes regardless of the initial conditions and will be heretofore referred to simply as experiments.

The results obtained from an experiment are known as the outcomes. When a die is rolled, the outcome is the number found on the topside. For any experiment, the set of all outcomes is called thesample space. The sample space, S, of the die example, is denoted by S= which represents all of the possible numbers that can result from the roll of a die. We usually consider sample spaces in which all outcomes are equally likely.

The sample space of an experiment is classified as finite or infinite. When there is a limit to the number of outcomes in an experiment, such as choosing a single card from a deck of cards, the sample space is finite. On the other hand, an infinite sample space occurs when there is no limit to the number of outcomes, such as when a dart is thrown at a target with a continuum of points.

While a sample space describes the set of every possible outcome for an experiment, an event is any subset of the sample space. When two dice are rolled, the set of outcomes for an event such as a sum of 4 on two dice is represented by E =.

In some experiments, multiple events are evaluated and set theory is needed to describe the relationship between them. Events can be compounded forming unions, intersections, and complements. The union of two events A and B is an event which contains all of the outcomes contained in event A and B. It is mathematically represented as A ∪ B. The intersection of the same two events is an event which contains only outcomes present in both A and B, and is denoted A ∩ B. The complement of event A, represented by A', is an event which contains all of the outcomes of the sample space not found in A.

Looking back at the table we can see how set theory is used to mathematically describe the outcome of real world experiments. Suppose A represents the event in which a 4 is obtained on the first roll and B represents an event in which the total number on the dice is 5.

The compound set A ∪ B includes all of the outcomes from both sets,

The compound set A ∩ B includes only events common to both sets,. Finally, the complement of event A would include all of the events in which a 4 was not rolled first.

Rules of probability

By assuming that every outcome in a sample space is equally likely, the probability of event A is then equal to the number of ways the event can occur, m, divided by the total number of outcomes that can occur, n. Symbolically, we denote the probability of event A as P(A) = m/n. An example of this is illustrated by drawing from a deck of cards. To find the probability of an event such as getting an ace when drawing a single card from a deck of cards, we must know the number of aces and the total number of cards. Each of the 4 aces represent an occupance of an event while all of the 52 cards represent the sample space. The probability of this event is then 4/52 or.08.

Using the characteristics of the sets of the sample space and an event, basic rules for probability can be created. First, since m is always equal to or less than n, the probability of any event will always be a number from 0 to 1. Second, if an event is certain to happen, its probability is 1. If it is certain not to occur, its probability is 0. Third, if two events are mutually exclusive, that is they can not occur at the same time, then the probability that either will occur is equal to the sum of their probabilities. For instance, if event A represents rolling a 6 on a die and event B represents rolling a 4, the probability that either will occur is 1/6 + 1/6 = 2/6 or 0.33. Finally, the sum of the probability that an event will occur and that it will not occur is 1.

The third rule above represents a special case of adding probabilities. In many cases, two events are not mutually exclusive. Suppose we wanted to know the probability of either picking a red card or a king. These events are not mutually exclusive because we could pick a red card that is also a king. The probability of either of these events in this case is equal to the sum of the individual probabilities minus the sum of the combined probabilities. In this example, the probability of getting a king is 4/52, the probability of getting a red card is 26/52, and the probability of getting a red king is 2/52. Therefore, the chances of drawing a red card or a king is 4/52 + 26/52 - 2/52 = 0.54.

Often the probability of one event is dependant on the occupance of another event. If we choose a person at random, the probability that they own a yacht is low. However, if we find out this person is rich, the probability would certainly be higher. Events such as these in which the probability of one event is dependant on another are known as conditional probabilities. Mathematically, if event A is dependant on another event B, then the conditional probability is denoted as P(A|B) and equal to P(A ∩ B)/P(B) when P(B) ≠ 0. Conditional probabilities are useful whenever we want to restrict our probability calculation to only those cases in which both event A and event B occur.

Events are not always dependant on each other. These independent events have the same probability regardless of whether the other event occurs. For example, probability of passing a math test is not dependent on the probability that it will rain.

Using the ideas of dependent and independent events, a rule for determining probabilities of multiple events can be developed. In general, given dependent events A and B, the probability that both events occur is P(A ∩ B) = P(B) × P(A|B). If events A and B are independent, P(A ∩ B) = P(A) × P(B). Suppose we ran an experiment in which we rolled a die and flipped a coin. These events are independent so the probability of getting a 6 and a tail would be (1/6) × 1/2 = 0.08.

The theoretical approach to determining probabilities has certain advantages; probabilities can be calculated exactly, and experiments with numerous trials are not needed. However, it depends on the classical notion that all the events in a situation are equally possible, and there are many instances in which this is not true. Predicting the weather is an example of such a situation. On any given day, it will be sunny or cloudy. By assuming every possibility is equally likely, the probability of a sunny day would then be 1/2 and clearly, this is nonsense.

Empirical probability

The empirical approach to determining probabilities relies on data from actual experiments to determine approximate probabilities instead of the assumption of equal likeliness. Probabilities in these experiments are defined as the ratio of the frequency of the occupance of an event, f(E), to the number of trials in the experiment, n, written symbolically as P(E) = f(E)/n. If our experiment involves flipping a coin, the empirical probability of heads is the number of heads divided by the total number of flips.

The relationship between these empirical probabilities and the theoretical probabilities is suggested by the Law of Large Numbers. It states that as the number of trials of an experiment increases, the empirical probability approaches the theoretical probability. This makes sense as we would expect that if we roll a die numerous times, each number would come up approximately 1/6 of the time. The study of empirical probabilities is known as statistics .

Using probabilities

Probability theory was originally developed to help gamblers determine the best bet to make in a given situation. Suppose a gambler had a choice between two bets; she could either wager $4 on a coin toss in which she would make $8 if it came up heads or she could bet $4 on the roll of a die and make $8 if it lands on a 6. By using the idea of mathematical expectation she could determine which is the better bet. Mathematical expectation is defined as the average outcome anticipated when an experiment, or bet, is repeated a large number of times. In its simplest form, it is equal to the product of the amount a player stands to win and the probability of the event. In our example, the gambler will expect to win $8 × 0.5 = $4 on the coin flip and $8 × 0.17 = $1.33 on the roll of the die. Since the expectation is higher for the coin toss, this bet is better.

When more than one winning combination is possible, the expectation is equal to the sum of the individual expectations. Consider the situation in which a person can purchase one of 500 lottery tickets where first prize is $1000 and second prize is $500. In this case, his or her expectation is $1000 × (1/500) + $500 × (1/500) = $3. This means that if the same lottery was repeated many times, one would expect to win an average of $3 on every ticket purchased.

Resources

books

Freund, John E., and Richard Smith. Statistics: A First Course. Englewood Cliffs, NJ: Prentice Hall Inc., 1986.

McGervey, John D. Probabilities in Everyday Life. New York: Ivy Books, 1986.

Perry Romanowski

KEY TERMS

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Combination

—A method of counting events in which order does not matter.

Conditional probabilities

—The chances of the occupance of an event given the occupance of a related second event.

Empirical approach

—A method for determining probabilities based on experimentation.

Event

—A set of occurrences which satisfy a desired condition.

Independent probabilities

—The chances of the occupance of one event is not affected by the occupance or non occupance of another event.

Law of large numbers

—A mathematical notion which states that as the number of trials of an empirical experiment increases, the frequency of an event divided by the total number of trials approaches the theoretical probability.

Mathematical expectation

—The average outcome anticipated when an experiment, or bet, is repeated a large number of times.

Mutually exclusive

—Refers to events which can not happen at the same time.

Outcomes

—The result of a single experiment trial.

Permutation

—Any arrangement of objects in a definite order.

Sample space

—The set of all possible outcomes for any experiment.

Theoretical approach

—A method of determining probabilities by mathematically calculating the number of times an event can occur.