Segmentation via Maximum Entropy Model

1. Segmentation via Maximum Entropy Model

2. Goals • Is it possible to learn the segmentation problem automatically? • Using a model which is frequently used in NLP tasks - the Maximum Entropy model • Finding features compatible to the segmentation task

3. Segmentation as a tagging problem • Attach each pixel one of two tags: is it an edge between segments or not. • Use a tagged set of images to build a statistical model. • Given a new image we want to find the most probable tag for each pixel using this model

4. Maximum Entropy • We want to build a statistical model which makes as few assumptions as possible. • Entropy – the uncertainty of a distribution. The level of “surprise”. • So, we need a statistical model with the highestentropy possible.

5. Maximum Entropy - Definitions • Event – a couple (p, t) where p is a pixel and t is its tag. • Feature – elementary piece of evidence that link p to t. A feature has a real value. • The decision about a pixel is based only on the features active at that point. • Looking for conditional probability P(p|t,f)

6. Maximum Entropy – building the model • We use a tagged training set to build the statistical distribution. • Constrains on the distribution: • The expectation for each feature should be equal to its expectation in the training set. • Maximizing the Entropy over all possible distributions. • We get an optimization problem which can be solved using Lagrange multipliers (weights).

7. opennlp maxent • Machine learning package compatible for NLP tasks. • Used for training maximum entropy models. • All is left is adjusting it to work with images and finding a set of features compatible for the segmentation task.

8. The Berkeley segmentation database • Consists of 300 images and the corresponding segmentation maps. • The segmentation maps where constructed by humans. • The database is divided into a train set (200) and a test set (100).

9. The Berkeley segmentation database - example

10. Features • I focused on brightness and texture features (technical problems): • Brightness difference with surrounding pixels • Gradient magnitude using the Prewitt operators. • Canny edge detection results

11. 1 2 3 100101103 405080 506090 4 5 1 1 1 1 1 1 0 0 8 7 6 Features – cont. Local Binary Partition Measure: For each pixel p, create an 8-bit number b1 b2 b3 b4 b5 b6 b7 b8 where bi=0 if neighbor i has value less than or equal to p’s value and 1 otherwise. For example:

12. Features – cont. Laws Texture Measure: • Basic texture matrices: L5 = [ 1 4 6 4 1 ] E5 = [ -1 -2 0 2 1 ] S5 = [ -1 0 2 0 -1 ] R5 = [ 1 -4 6 -4 1 ] • Creating a 9 feature vector for each pixel: L5*E5/E5*L5L5*R5/R5*L5E5*S5/S5*E5 S5*R5/R5*S5L5*S5/S5*L5E5*R5/R5*E5 R5R5 S5S5 E5E5 • For the current pixel calculating the distance to surrounding pixels

13. Results

14. Results

15. Results

16. Results

17. Discussion • The model is basic but it has managed to find segments at some level. • Best result are found where there is an object with a contrasting background or radical difference of texture. • The model had succeeded in filtering edges which are not segments. For example:

18. Discussion

19. Feature Work • More features • A compatible package for images • Search algorithm for finding the best sequence • Working with color images • Special learning: by one person, by category etc.