Section 5 Introduction to MLE

This section will provide an overview of MLE. There are a few general connections we can make between the things we know (the linear least squares model) and the concepts introduced here.

  • Practical Uses: Going from nice continuous outcome data to outcome data generated differently
  • Estimation: Going from minimizing squared error to maximizing likelihood.
    • Both involve optimization. In likelihood, we are trying to find the optimal values of parameters for a distribution given observed data.
  • Formulation: Going from Linear Model to Generalized Linear Model
  • Mechanics in R: Going from lm() to glm() and its friends.

The first sections will focus on drawing out these connections. We will then get into the details on the derivations for common methods.

This video is an overview of some of the concepts we discuss. Don’t worry about the mathematical details in the video. For a given model, we will go into greater depth in the future. Focus on the general concepts and process. The notes to follow in sections 5.1 and 5.2 elaborate on the concepts in the video.

I also encourage you to watch this video from StatQuest to get an initial understanding of likelihood and what it means to “choose the maximum likelihood” using a visual example.

These concepts are also addressed in: King, Gary. 1998. Unifying political methodology: The likelihood theory of statistical inference. University of Michigan Press. Note: Available as an electronic resource through the Rutgers library. Chapters 1, 2, 3, 4.6-4.8. (Available online through Rutgers University)