For the past month and a half, I have been in the process of constructing a survivorship free database for the NDX 100. I am currently about 90% done but I have to say this is no easy task.
According to wikipedia:
“Survivorship bias is the logical error of concentrating on the people or things that “survived” some process and inadvertently overlooking those that didn’t because of their lack of visibility. This can lead to false conclusions in several different ways. The survivors may literally be people, as in a medical study, or could be companies or research subjects or applicants for a job, or anything that must make it past some selection process to be considered further.”
Coming from testing futures trading systems, I initially failed to grasp on the need to test trading systems survivorship bias free database. But from numerous sources and evidence, I found that testing systems on biased data over state results significantly. I suggest that the trader/ researcher give a good thorough thought on this topic and check if they are employing survivorship free database.
For those who don’t want to create their own, Frank Hasslers’s blog, engineering returns offer two database for some cash.