Lecture 1: Tutorial on probability and estimation
TTIC 31020: Introduction to Machine Learning
Instructor: Greg Shakhnarovich,
Lecture by Dhruv Batra
TTI–Chicago
September 27, 2010
Lecture 1: Tutorial on probability and estimation TTIC 31020
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community
Ask the community for help and clear up your study doubts
University Rankings
Discover the best universities in your country according to Docsity users
Free resources
Our save-the-student-ebooks!
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Tutorial on Probability and Estimation-Introduction to Machine Learning-Lecture 01-Computer Science, Lecture notes of Introduction to Machine Learning
Tutorial on Probability and Estimation, Dhruv Batra, Probability, Continuous Random Variables, Bias, Probabilistic Model, Probability Distributions, Sequence Probability, Parameter Estimation Problem, Maximum Likelihood Estimator, Bernoulli, Bayes Rule, MAP Estimation, Discrete Vs. Continuous RV, Beta Prior for Bernoulli, Conjugate Prior, MAP Estimate, Bernoulli, Beta Distributions, Gaussian Distributions, Greg Shakhnarovich, Lecture Slides, Introduction to Machine Learning, Computer Science, To
Typology: Lecture notes
2011/2012
1 / 89
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Tutorial on Probability and Estimation-Introduction to Machine Learning-Lecture 01-Computer Science and more Lecture notes Introduction to Machine Learning in PDF only on Docsity!
Lecture 1: Tutorial on probability and estimation
TTIC 31020: Introduction to Machine Learning
Instructor: Greg Shakhnarovich, Lecture by Dhruv Batra
TTI–Chicago
September 27, 2010
Welcome
TTIC 31020, Introduction to Machine Learning
MWF 9:30-10:20am
Instructor: Greg Shakhnarovich, greg@ttic.edu
TA: Feng Zhao, lfzhao@ttic.edu
Greg is traveling this week; all administrative details will be discussed next Monday.
Why probability?
This class is mostly about statistical methods and models in machine learning
Probability is fundamental in dealing with uncertainty inherent in real world problems
Statistics leverages laws of probability to evaluate important properties of the world from data, and make intelligent predictions about future
Background
Things you should have seen before
Background
Things you should have seen before
- Discrete vs. Continuous Random Variables
- (^) pmf vs pdf
Background
Things you should have seen before
- Discrete vs. Continuous Random Variables
- (^) pmf vs pdf
- (^) Joint vs Marginal vs Conditional Distributions
Background
Things you should have seen before
- Discrete vs. Continuous Random Variables
- (^) pmf vs pdf
- (^) Joint vs Marginal vs Conditional Distributions
- IID: Independant Identically Distributed
- (^) Bayes Rule and Priors
Background
Things you should have seen before
- Discrete vs. Continuous Random Variables
- (^) pmf vs pdf
- (^) Joint vs Marginal vs Conditional Distributions
- IID: Independant Identically Distributed
- (^) Bayes Rule and Priors
This refresher WILL revise these topics.
Problem: estimating bias in coin toss
A single coin toss produces H or T.
A sequence of n coin tosses produces a sequence of values; n = 4 T ,H,T ,H
Problem: estimating bias in coin toss
A single coin toss produces H or T.
A sequence of n coin tosses produces a sequence of values; n = 4 T ,H,T ,H H,H,T ,T
Problem: estimating bias in coin toss
A single coin toss produces H or T.
A sequence of n coin tosses produces a sequence of values; n = 4 T ,H,T ,H H,H,T ,T T ,T ,T ,H
A probabilistic model allows us to model the uncertainly inherent in the process (randomness in tossing a coin), as well as our uncertainty about the properties of the source (fairness of the coin).
Probabilistic model
First, for convenience, convert H → 1, T → 0.
- We have a random variable X taking values in { 0 , 1 }
Probabilistic model
First, for convenience, convert H → 1, T → 0.
- We have a random variable X taking values in { 0 , 1 }
Bernoulli distribution with parameter μ:
Pr(X = 1; μ) = μ.
We will write for simplicity p(x) or p(x; μ) instead of Pr(X = x; μ)
Probabilistic model
First, for convenience, convert H → 1, T → 0.
- We have a random variable X taking values in { 0 , 1 }
Bernoulli distribution with parameter μ:
Pr(X = 1; μ) = μ.
We will write for simplicity p(x) or p(x; μ) instead of Pr(X = x; μ)
The parameter μ ∈ [0, 1] specifies the bias of the coin
- Coin is fair if μ = (^12)