Course overview

The course is formed by several modules, starting with introductory notions and ending with advanced ones. The modules are described below. Note that a module is (typically) not 1-to-1 equivalent with a lecture.

Introduction

• what is multimedia retrieval (MR)
• types of MR data
• the MR pipeline
• example applications

Data representation

• how MR data is represented
• sampling and reconstruction
• basis functions, grids, interpolation
• types of MR attributes (scalar, vector, color, tensor)

Perception in MR

• why human perception is important in MR
• challenges of modeling perception
• Gestalt principles
• visual illusions

Feature extraction

• concept of features and descriptors
• taxonomy of features
• features for image, 2D shape, and 3D shape data
• feature extraction challenges and good practice

Matching

• concepts of similarity and distance
• different types of distance functions
• distance transforms

Scalability

• scalability challenge in MR
• solutions: indexing, k-nearest neighbors, clustering
• relation of MR with Machine Learning

Presentation

• how to query and visually explore MR data
• dimensionality reduction for exploration
• dimensionality reduction for querying

Evaluation

• quality concept in MR
• quality metrics (FP, TP, F-score, AUC)
• quality measurement in practice