Coursera ML(10)-机器学习诊断法
in Coursera ML with 0 comment

# 目前已有的方法：

• Getting more training examples
• Trying smaller sets of features
• Trying polynomial features
• Increasing or decreasing λ

# Evaluating a Hypothesis

• For linear regression:
$$J_{test}(\Theta) = \dfrac{1}{2m_{test}} \sum_{i=1}^{m_{test}}(h_\Theta(x^{(i)}_{test}) - y^{(i)}_{test})^2$$
• For classification :
误分类的比例，对于每一个测试实例，计算：

$$err(h_\Theta(x),y) = \begin{matrix} 1 & \mbox{if } h_\Theta(x) \geq 0.5\ and\ y = 0\ or\ h_\Theta(x) < 0.5\ and\ y = 1\newline 0 & \mbox otherwise \end{matrix}$$

$$\text{Test Error} = \dfrac{1}{m_{test}} \sum^{m_{test}}_{i=1} err(h_\Theta(x^{(i)}_{test}), y^{(i)}_{test})$$

# Model Selection and Train/Validation/Test Sets(交叉验证机)

• Optimize the parameters in Θ using the training set for each polynomial degree.
• Find the polynomial degree d with the least error using the cross validation set.
• Estimate the generalization error using the test set with $J_{test}(\Theta^{(d)})$, (d = theta from polynomial with lower error);
简单来讲：

# Diagnosing Bias vs. Variance

• High bias (underfitting): both $J_{train}(\Theta)$ and $J_{CV}(\Theta)$ will be high. Also, $J_{CV}(\Theta) \approx J_{train}(\Theta)$.
• High variance (overfitting): $J_{train}(\Theta)$ will be low and $J_{CV}(\Theta)$ will be much greater than $J_{train}(\Theta)$.

## Decide Bias or Variance

• 训练集误差和交叉验证集误差近似时：偏差/欠拟合
• 交叉训练集误差 >> 训练集误差时：方法/过拟合

# Regularization and Bias/Variance

• Create a list of lambdas (i.e. λ∈{0,0.01,0.02,0.04,0.08,0.16,0.32,0.64,1.28,2.56,5.12,10.24});
• Create a set of models with different degrees or any other variants.
• Iterate through the $\lambda$s and for each $\lambda$ go through all the models to learn some $\Theta$.
• Compute the cross validation error using the learned Θ (computed with λ) on the $J_{CV}(\Theta)$ without regularization or λ = 0.
• Select the best combo that produces the lowest error on the cross validation set.
• Using the best combo Θ and λ, apply it on $J_{test}(\Theta)$ to see if it has a good generalization of the problem.
简单说：

## Regularization 相关结论

• 当$\lambda$较小时，训练集误差较小（过拟合）而交叉验证集误差较大。
• 随着$\lambda$增加，训练集误差不断增加（欠拟合），而交叉验证集误差则是先减小后增大。

# Learning Curves

• 学习曲线是一个很好的工具，我们会经常使用学习曲线来判断某一个学习算法是否处于偏差、方差问题。
• 学习曲线试讲训练集误差和交叉验证集误差作为训练集实例数量（m）的函数绘制的图表。

## Experiencing high bias:

• Low training set size: causes $J_{train}(\Theta)$ to be low and $J_{CV}(\Theta)$ to be high.
• Large training set size: causes both $J_{train}(\Theta)$ and $J_{CV}(\Theta)$ to be high with $J_{train}(\Theta)$≈$J_{CV}(\Theta)$
因此在高偏差（欠拟合）的情况下，增加训练集数量并不是一个好办法。此时，我们应当增加features。

## Experiencing high variance:

• Low training set size: $J_{train}(\Theta)$ will be low and $J_{CV}(\Theta)$ will be high.
• Large training set size: $J_{train}(\Theta)$ increases with training set size and $J_{CV}(\Theta)$ continues to decrease without leveling off. Also, $J_{train}(\Theta)$ < $J_{CV}(\Theta)$ but the difference between them remains significant.
对比之下，如果在高方差（过拟合）的情况下，增加训练集数量可以明显降低误差，提高算法效果。

# 决定下一步做什么

• 获得更多的训练实例——解决高方差
• 尝试减少特征的数量——解决高方差
• 尝试获得更多的特征——解决高偏差
• 尝试增加多项式特征——解决高偏差
• 尝试减少归一化程度 λ--->提高拟合准确度--->解决高偏差
• 尝试增加归一化程度 λ--->防止过拟合--->解决高方差
Responses

From now on, bravely dream and run toward that dream.