Lectures 2018

  1. Lecture September 24, 2018 -  Introduction to the course. Questionnaire for checking background. Generalities on AutomaticLearning. (Ref. slide 1st-lecture)
  2. Lecture September 25, 2018 - The learning process. Data, learning machine. Minimization of the expected risk. Over and under fitting.  (Ref. slide 2nd lecture, Bishop – Pattern Recognition and Machine Learning, Springer, 2006 (Chap 1), V. Cherlassky, F. Mulier - Learning from Data, John Wiley and Sons, 2007 (chap.1 and 2) ,   Bottou, Curtis, and Nocedal - Optimization Methods for LargeScale Machine Learning (section 1 and 2), Teaching Notes chapt. 1)
  3. Lecture October 1, 2018 - Hint on Vapnik Chervonenkis theory and Structural risk minimization principle.  VC dimension of oriented hyperplanes (theorem and proof)  (Ref. slide 3rd lecture, Teaching Notes chapt. 1)
  4. Lecture October 2, 2018 -  Hyperplanes with margin - (Ref. Slide 4th lecture, Teching notes chapt.1, P. Domingos - A few useful things to know about Machine Learning). Brief review on optimization problems: basic defintions.
  5. Lecture October 8, 2018 -   The perceptron algorithm. Example: the logical OR. (Ref. Slide 5th lecture, Teching notes chapt.2). Presentation of the Brdgestone Hackathon (deadline December 31th, 2018)
  6. Lecture October 9, 2018 - The perceptron algorithm (proof). The voting and the average perceptron (Ref.  Slide 6th lecture, (Authors' faces), Teching notes chapt.2). Project (not mandatory) on the perceptron.
  7. Lecture October 15, 2018 - Beyond perceptron. Feedforward Neural Network: Multiplayer perceptron (Ref. Slide 7th-8th Lecture, Palagi - Global optimization issues in deep network regression: an overview -Section 4, Teaching notes chapt. 3)
  8. Lecture October 16, 2018 - Hyperparameters and weights problems (Ref. Slide 7th-8th Lecture). Python + libraries (Numpy, SkLearn, pandas, scipy.optimize) by Ruggiero Seccia (Ref. Pitone1)
  9. Lecture October 22, 2018 - Deep Network - Brief review of optimization algorithms. Stopping criterion
  10. Lecture October 24, 2018 - Backpropagation rule (slide 10th-11th Lecture)
  11. Lecture November 5, 2018 - Backpropagation rule - Convergence of the batch BP (learning rate) - Momentum term (Ref. Slide of 10-11th Lecture, teaching notes chapt. 3)
  12. Lecture November 6, 2018 -  Online methods for MLP: incremental, stochastic gradient, batch method (Ref. Slide of the 12th lecture; in depth reading:  Optimization methods for large-scale machine learning by L. Bottou, FE Curtis, J Nocedal)
  13. Lecture November 9, 2018 - Decomposition methods for MLP: Extreme learning (Ref. Slide of the 13th lecture)
  14. Lecture November 12, 2018 - Decomposition methods for MLP (Ref. Slide of the 14th lecture)
  15. Lecture November 13, 2018 - Regularized RBF; generalized RBF network; learning paradigm for RBF network: unsupervised versus supervised selection of the centers (Ref. Slide of the 15th lecture; Teaching notes chapt. 4)
  16. Lecture November 13, 2018 - Supervised selection of the centers Full optimization. Two block decompostion methods, exact and inexact solution of subproblems, convergence properties. Decomposition methods: block learning of centers (Ref. Slide of the 16-17 th lecture; Teaching notes chapt. 4)
  17. Lecture November 16, 2018 - Two block decompostion methods, exact and inexact solution of subproblems, convergence properties. Decomposition methods: block learning of centers (Ref. Slide of the 16-17th lecture; Teaching notes chapt. 4)
  18. MIDTERM November 20, 2018
  19. Lecture November 23 2018 - Hard SVM: generalities, defintion of margin, defintion of the max margin problem (Ref. Slide 19th lecture). The primal hard SVM problem
  20. Lecture November 26 2018 - Convex optimization: KKT conditions for linearly constrained porblems. Feasible and descent directions. Frank-wolfe conditional gradient
  21. Lecture November 27 2018 - Duality in convex quadratic programming. The Wolfe dual. The weak duality theorem. Construction of the dual Hard SVM problem
  22. Lecture December 3, 2018 - Soft SVM. The dual problem of C-SVM
  23. Lecture December 4, 2018 -
  24. Lecture December 10, 2018 - Dual problem optimality conditions (KKT and feasible&descent directions)
  25. Lecture December 11, 2018 - Definition of kernel (non linear SVM). Analytic solution of QP in two dimension.  Use of Quadrprogr. Tutorial on TensorFlow and Keras (Scipy, class1, class2)
  26. Lecture December 14, 2018
  27. Lecture December 17, 2018
  28. Lecture December 18, 2018 - Final Term