When building vision systems that predict structured objects
such as image segmentations or human poses, a crucial
concern is performance under task-specific evaluation
measures (e.g. Jaccard Index or Average Precision). An ongoing
research challenge is to optimize predictions so as to
maximize performance on such complex measures. In this
work, we present a simple meta-algorithm that is surprisingly
effective - Empirical Min Bayes Risk. EMBR takes as
input a pre-trained model that would normally be the final
product and learns three additional parameters so as to optimize
performance on the complex high-order task-specific
measure. We demonstrate EMBR in several domains, taking
existing state-of-the-art algorithms and improving performance
up to ~7%, simply with three extra parameters.
We tested EMBR on three different applications and saw similar trends.