Archive for February, 2009

A brief note on experimental design

February 26, 2009

On the Metro ride home I was just starting into an article (on matching estimators, if you must know) when I had a moment of inspiration that got me around a major blockade standing in the way of my hopes for a research project.

I won’t get into specifics on the off-chance that the solution I “discovered” gives me a competitive edge in the short run, however:

Experiments in social sciences are difficult because you can rarely control all of your variables – i.e. the humans whose behavior you are trying to understand. Nonetheless, we often run across situations that are very nearly controllable experiments except for some minor detail which, lacking the means to control for it in some way, renders our scientific foundations too shaky to merit the pursuit (or, at least, the avenue of that pursuit). The way around this problem (if such a way exists) often eludes us because we focus on what we see in our frame. Stuck in that state of mind, one is prone to forget that a subject can not affect an experiment if it is demonstrably absent from the experimental frame while measurements are being recorded.

It’s ceteris paribus in action. Except sometimes you realize that you don’t need to hold everything else constant by sheer statistical will and force of data. If you’re paying attention, you will see that sometimes things will hold themselves constant for you, and you just have to show how they did that.