Posts Tagged statistics

battle

When it comes to inference in computer science, there is a holy war between the good guys (Bayesian computer scientists) and the bad guys (the notorious frequentist gang). Bayesians try to keep track of the possibility of all different scenarios and how likely they are according to the observations and prior knowledge, and then check to see what the future is going to look like under each scenario (usually in a compact mathematical formulation but with concrete probabilistic interpretation). The frequentist gang, on the other hand, usually take the most likely scenario (or try to accept/reject different scenarios) and use heuristics all around the place, often times without concrete reasoning as to why these heuristics work.

Bayesian inference is, of course, computationally expensive. So the bad guys keep mocking the good guys that their methods are only theoritical and suitable for textbooks rather than real life. Now guess what… They are right! Exact Bayesian inference is not an option in most cases. This means that you need to do approximations. The funny thing is that these approximations usually lead to the exact same heuristics that are used by frequentists. But the good part of going this way is that you know what you are doing and why these methods work. As the great spritual leader Mahdi Milani Fard XIII once said:

Think Bayesian
Act frequentist
And live free

They should write that in gold and put it over the main enterance of all CS departments.

P.S.: This view is of course not exclusively mine. Many CS researchers like to think this way.