8.2 BoW Classification

     In this lecture, we are going to learn about BoW Classification.

 

 

Once again, let's see how Classical Image Classification Pipeline looks.

Image Classification Pipeline

Among 4 steps, today we are going to focus on 'encoder', 'bag of visual words' encoder.

 

 

● BoW encoder

In BoW encoder view, we can define 'encode' and 'decode' as below.

 

⊙ Encode

1. Dictionary Learning

 

a. Extract features from image

extract features from image

* Regular grid

- Vogel & Schiele, 2003

- Fei-Fei & Perona, 2005

 

* Interest point detector

- Csurka et al. 2004

- Fei-Fei & Perona, 2005

- Sivic et al. 2005

 

* Other methods

- Random sampling (Vidal-Naquet & Ullman, 2002)

- Segmentation-based patches (Barnard et al. 2003)

 

b. Unsupervised Clustering

Unsupervised Clustering

In Unsupervised Clustering, the answers are not given. 

 

* 4 steps in Unsupervised Clustering

① Select initial centroids at random

② Assign each object to the cluster with the nearest centroid

③ Compute each centroid as the mean of the objects assigned to it (go to 2)

② Assign each object to the cluster with the nearest centroid (again)

Repeat previous 2 steps until no change.

 

* K-means Clustering

given k:

① Select initial centroids at random.

② Assign each object to the cluster with the nearest centroid.

③ Compute each centroid as the mean of the objects assigned to it.

④ Repeat previous 2 steps until no change.

 

 

2. Build Bags-of-Words (BoW) descriptor

After learning a visual dictionary, an image representation is needed, so we have to use a bag-of-words descriptor.

 

a. Quantize feature

Quantize feature

- Associate each feature to nearest 'word' cluster.

 

b. Build histogram (for each image)

 

 

⊙ Decode

1. Train classifier

Train classifier

- Given the bag-of-features representations of images from different classes, train some fancy machine learning model.

 

 

 

2. Classify using BoW descriptor

Classify using BoW descriptor

- Classify using BoW descriptor

 

 

In next lecture, we will learn about decoder, optimizer.