It is usual in machine learning theory to assume that the training and testing sets comprise of draws from the same distributions.
This is rarely, true if, ever true and one must admit the presence of corruption. There are many different types of corruption that
can arise, and as of yet there is no general means to compare the relative ease of learning in these settings. Such results are necessary if we are to make informed economic decisions regarding the acquisition of data.
Here we begin to develop an abstract framework for tackling such problems. We present a generic method for learning from a fixed,
known, reconstructible corruption, along with an analyses of its statistical properties.
We demonstrate the utility of our framework via concrete novel results in solving supervised learning problems wherein the
labels are corrupted, such as learning with noisy labels, semi-supervised learning and learning with partial labels.
Many leading classification algorithms output a classifier that is a weighted average of kernel evaluations.
Optimizing these weights is a non-trivial problem that still attracts much research effort. Furthermore, explaining these methods to the uninitiated is a difficult task. Letting all the weights be equal leads to a conceptually simpler classification rule, one that requires little effort to motivate or explain, the mean.
Here we explore the consistency, robustness and sparsification of this simple classification rule.
The scientific process is a means for turning the results of experiments into knowledge about the world in which we live. Much recent research effort has been directed toward automating this process. To do this, one needs to formulate the scientific process in a precise mathematical language. This paper outlines one such language.