Week 2 Comprehensive

Week 2 Comprehensive >> Introduction to Machine Learning

Mathematically define network outputs

Reward indecision

Ignore historical statistical developments

Penalize overconfidence

Model complexity is perfectly matched to the data.

Overfitting refers to the fact that more complexity is always better, which is why deep learning works.

Model complexity fits too well to training data and will not generalize in the real-world.

Model complexity is not enough to capture the nuance of the data and will under-perform in the real-world.

More than one use can lead to overfitting.

The model cannot learn anything new from subsequent uses.

It is expensive to use more than once.

More than one use can lead to bias.

To estimate the performance of a model.

To pick the best performing model.

To test the performance in lieu of real-world data.

To learn the model parameters.

Monte Carlo simulation

Downhill skiing

Gradient descent

Analytically determine global minimum

6.What technique is used to minimize loss for a large data set?

Newton’s method

Gradient descent

Stochastic gradient descent

Taylor series expansion

With stochastic gradient descent, the update time does not scale with data size.

Stochastic gradient descent gets near the solution quickly.

Stochastic gradient descent finds the solution more accurately.

Stochastic gradient descent finds a more exact gradient than gradient descent.

Stochastic gradient descent can update many more times than gradient descent.

Calculating the gradient requires looking at every single data point.

Large data sets require deeper models, which have more parameters.

There are too many local minima for an algorithm to find.

Large data sets do not permit computing the loss function, so a more expensive measure is used.

It helps save computation cost.

It performs better in the real world.

It improves the training loss.

There is rigorous statistical theory on it.

Optimization seeks to do as well as possible on a training set, while validation seeks to do as well as possible on a validation set.

Optimization seeks to generalize to the real world, while validation seeks to do as well as possible on a validation set.

Optimization seeks to do as well as possible on a training set, while validation seeks to generalize to the real world.

They are not at odds—they have the same goal.

Week 2 Comprehensive >> Introduction to Machine Learning