Entropy (for data science) Clearly Explained

NOTE: This StatQuest was supported by these awesome people who support StatQuest at the Double BAM level: Z. Rosenberg, S. Shah, J. N., J. Horn, J. Wong, I. Galic, H-S. Ming, D. Greene, D. Schioberg, C. Walker, G. Singh, L. Cisterna, J. Alexander, J. Varghese, K. Manickam, N. Fleming, F. Prado, J. Malone-Lee

7 thoughts on “Entropy (for data science) Clearly Explained

  1. Hi Josh,
    I am dissatisfied with your explanation for surprise being log(1/p) on the basis that it is 0 when p is 1 and infinity when p is 0. There are a zillion functions of p with those limits. Why is log(1/p) preferred over all of the others? Is it just a convention chosen to match the physics definition and because of the convenient mathematical properties of the log? Or is there something more to it?

Leave a Reply

Your email address will not be published. Required fields are marked *