Statistical Machine Learning

Course Presentation

Prof. Jodavid Ferreira

UFPE

Course Structure


  • Workload: 72 hours

  • Course format:

    • Lectures;
    • Practical classes (computer labs);
    • Assessment:
      • 2 Exams:
        • 1st Exam - 24.04.2026
        • 2nd Exam - 19.06.2026
      • 1 Project - 26.06.2026:
        • Report
        • Presentation

Course Topics


  1. Programming Languages for Machine Learning: Introduction to R and Python: basic concepts and applications.

  2. Fundamentals of Data Modeling: Prediction vs. Inference and model accuracy evaluation.

  3. Continuous Optimization and Risk Function: Concepts of continuous optimization and its relationship with the risk function.

  4. Resampling Methods and Parametric Techniques: Resampling methods and high-dimensional parametric approaches.

  5. Non-Parametric Methods and Supervised Learning: kNN, and tree-based methods.

  6. Support Vector Machines: SVM for regression and classification

  7. Unsupervised Learning: Dimensionality reduction, clustering.

Programming Languages


What is High-Level programming and Low-Level programming?

- High-Level Programming:

A high-level programming language provides commands (keywords) that are very close to natural language. As a result, the process of “talking” to the computer is made easier, because these keywords provide greater clarity on how to orchestrate what the computer should do for us.


- Low-Level Programming:

A low-level programming language is closer to machine language, that is, closer to what the computer actually understands. Because of this, the process of “talking” to the computer is more difficult, as it is necessary to understand how the computer works in order to program it. An example of a low-level language is Assembly, which is a programming language that uses machine code to program.

Programming Languages

R:

  • It is a programming language focused on statistical analysis and graphics;

  • It is a high-level and open-source programming language (R is Free Software);

  • It is one of the most widely used languages in statistics and data analysis, and it has a large number of packages for such purposes.

Python:

  • It is a high-level and open-source programming language (Python is Open Source);

  • It is one of the most widely used languages in software development and data analysis;

  • It is a programming language that also has a large number of packages for data analysis.

Programming Languages


  • Example in R
# Code example in R
cat('Hello, World!\n')
  • Example in Python
# Code example in Python
print('Hello, World!')
  • Example in Assembly
section .data
    msg db 'Hello, World!', 0xA  ; Message to be printed followed by a new line
    len equ $ - msg              ; Calculates string length

section .text
    global _start                ; Defines the entry point of the program

_start:
    mov edx, len                ; Sets the third argument: message length
    mov ecx, msg                ; Sets the second argument: pointer to the message
    mov ebx, 1                  ; Sets the first argument: file descriptor (stdout)
    mov eax, 4                  ; Sets the system call number for sys_write
    int 0x80                    ; Calls the kernel

    mov eax, 1                  ; Sets the system call number for sys_exit
    xor ebx, ebx                ; Sets the argument for sys_exit: 0
    int 0x80                    ; Calls the kernel

IDEs


What are IDEs?


An Integrated Development Environment (IDE) is a software application that combines various necessary functions to create an environment for coding and programming. As previously mentioned, R has versions compatible with various operating systems, such as Linux, Windows, and macOS. For each system, there are several IDEs that can be used to program in R, and some of these IDEs also have versions for all operating systems. The best IDE choice is the one the user feels most comfortable programming in; however, here are some recommended IDEs to use, especially for new R users and programmers:

References


Basics

  • Aprendizado de Máquina: uma abordagem estatística, Izibicki, R. and Santos, T. M., 2020, link: https://rafaelizbicki.com/AME.pdf.

  • An Introduction to Statistical Learning: with Applications in R, James, G., Witten, D., Hastie, T. and Tibshirani, R., Springer, 2013, link: https://www.statlearning.com/.

  • Mathematics for Machine Learning, Deisenroth, M. P., Faisal. A. F., Ong, C. S., Cambridge University Press, 2020, link: https://mml-book.com.

References


Complementary