Data Science

Data Science Phases

Data exploration phase

Visualize the data

• Dataset Visualisation or Exploratory Data Analysis
  • Matplotlib tutorial (2D)
    • flashcards, matplotlib tutorial 2d
  • Matplotlib tutorial (3D)

Data Preprocessing/Loading phase

• Handle missing values

• one-hot encode

• Data Normalization + Centering

  • standard scaler, z-score normalization
  • robust scaler, normalization
  • Max absolute scaler

• Outlier Handling

• Data Augmentation

• Dataloader and Dataset

  • flashcards, dataloader and dataset

Model selection/building phase

• Model Selection
• Loss functions
• Optimizer
• Model Implementation (Pytorch)
• python, ROC Curve, AUC

Multimodal input

When the input data has different modalities (most likely different sources). Example: Image data + location of the images.

• Late Fusion

Training phase

• Pytorch Model training
• Model Training Acceleration
• Ray Clustering

Hyperparameters

• Hyperparameter Search Optuna
  • flashcards, Hyperparameter search optuna
• Early Stopping (patience)

Validation, measurement

• Model classification metrics

  • flashcards coding, model classification metrics
  • flashcards concepts, model classification metrics

• K-Fold cross validation

  • Nested cross validation

model progress visualizations

visualize epoch training logs

Debugging phase

• Model Debugging (pytorch))

Final Model

• How to train your final model

Inference phase

• Inference (meaning)

Model Types

• How to think about model architecture

  • Scale separation, ml architecture context

• U-Net

• Graph neural networks

• convolutional neural network

• 1D convolutional model

• Siamese neural network

Transformers

• Transformer, theory

• Attention mechanism

• Transformers implementation (pytorch)
  - Embeddings
  - Self-Attention Mechanism
  - Positional Encoding
  - Transformer final layers NOT WRITTEN YET. Combine with Model selection?

Layer Types

Deep Neural Network Layer types overview

Activation Functions
Relu activation function

Normalization between layers
- BatchNorm
- LayerNorms
- Instance Normalization

Pooling layers
- Max Pooling Layer
- Average Pooling Layer
- Global Pooling Layers

Dropout Layers

Data Science related topics

Model components and properties

• Normalization between layers
• Activation Functions

Data

• Statistical hypothesis test, p value
• Flashcards, pandas - Series - Dataframes
• Flashcards, python dictionaries

Terminology

• Symbolic AI vs Non-symbolic AI

XAI

• Sensitivity Analysis (XAI)
• Layerwise Relevance Propagation

Mathematical concepts

• similarity based approaches in high dimensional space

• Curse of dimensionality, model training

• Bootstrap method

• Regression

• Regularisation

• Decomposing Model Error

• Statistical hypothesis test, p value

• Monte Carlo Simulations

Uncertainty in ML

• Uncertainty in ML Knowledge Base

• Sources of Uncertainty

• Bayesian Neural Networks

• Piecewise affine functions, relation to deep neural networks and overconfidence.

Vectorization, accellerations

Vectorize everything, get rid of loops

• numpy accelerations
• masking

Divers

• How to use models from Huggingface
• RAG, retrieval augmented generation

Singular Topics

Devops/Environment

• Python version management

• Virtual environments

• Global CLI tool installation

• Python packaging

• 01. Main File Docker

Disk operations

• Read a file in python

  • flashcards, read a file in python

• Pickling, saving objects to disk

  • Flashcards, pickling, saving objects to disk

• flashcards, path handling and navigating, python

Fancy code

• python decorators

• **kwargs, python arbitrary keyword arguments

• property decorator

  • Flash Cards, property decorator

• Time decorator

• Memory decorator

• Context Managers

• numpy accelerations

• mapping

• array manipulations

Radom numbers

• Random Number Generation
  • Flashcards, Random Number Generation
• (Non random) number generation
  • Flashcards, non random number generation

Visualisations

• Matplotlib tutorial (2D)
  • flashcards, matplotlib tutorial 2d
• Matplotlib tutorial (3D)

Namespaces

Created to properly understand how to do imports.

• 01. Namespaces main file
• Environment Variables
• Pythonpath

UNIX

• Permission management

• Systemd

• nginx

• send huge amounts of big files

• Background terminal execution

Other

• In memory buffers

• python, image handling

• python, graph handling

• Data Binarization

• python, printing tricks

• Caching

• Bit manipulations in python

• Multiprocessing, python

  • flashcards, multiprocessing in python

• copy, deepcopy

• flashcards, datetime, python

• flashcards, sorting, python

• flashcards, regex filtering python

General Coding Tipps

• Theory testing setup
• Jupyter Notebook Tipps
• Interactive python window tipps and tricks
• VS Code Tipps and Tricks
• ssh
• git (and github) tips and tricks
• Virtual environments
• Python version management

Algorithms

Trie Data Structure

Is this still CS?

• Edit distance

• Hamming distance

• Statistical hypothesis test, p value