Part 1: Linear Regression and OLS

Explain Ordinary Least Squares (OLS) linear regression. Your explanation should cover:

• The model form.
• The loss function minimized by OLS.
• The closed-form estimator, if it exists.
• Key assumptions commonly used for statistical inference.
• How to interpret the coefficients.
• Potential issues arising from multicollinearity, outliers, heteroskedasticity, or omitted variables.
• Methods for evaluating the model's performance and suitability.

Part 2: Law of Large Numbers and Central Limit Theorem

Explain the fundamental difference between the Law of Large Numbers (LLN) and the Central Limit Theorem (CLT).

Then, apply these concepts to the following scenario:

Let X_1, X_2, ..., X_n be independent and identically distributed (IID) random variables with a mean mu and variance sigma^2. Define the sample mean as X_bar = (X_1 + X_2 + ... + X_n) / n.

1. Describe what happens to X_bar as n becomes very large.
2. What is the approximate distribution of X_bar for large n?
3. How would you approximate P(X_bar > a) for a given threshold a?

Part 3: Three-Person Hat Strategy

Three participants are involved in a game where each is randomly assigned a hat, which is independently either black or white with equal probability. Each participant can see the hat colors of the other two but cannot see their own.

Simultaneously, each participant must decide to either guess their own hat color or pass.

Rules for Winning/Losing:

• Win: The team wins if at least one participant guesses correctly AND no participant guesses incorrectly.
• Lose: The team loses if anyone guesses incorrectly OR if everyone passes.

Before the hats are assigned, the participants are allowed to agree on a strategy. Your task is to determine the strategy that maximizes their probability of winning, and to state what that maximum probability is.