Couple of months ago I mentioned the importance of survivor-ship bias when testing strategies that trade on equities. I thought it will help traders to achieve better testing results if they could build them themselves. I got from my university a list of NDX100 constituents all the way back to 1995. Those of you interested can email me if you want a copy of it. I would of upload it but I cant with my current wordpress account.
For the past month and a half, I have been in the process of constructing a survivorship free database for the NDX 100. I am currently about 90% done but I have to say this is no easy task.
According to wikipedia:
“Survivorship bias is the logical error of concentrating on the people or things that “survived” some process and inadvertently overlooking those that didn’t because of their lack of visibility. This can lead to false conclusions in several different ways. The survivors may literally be people, as in a medical study, or could be companies or research subjects or applicants for a job, or anything that must make it past some selection process to be considered further.”
Coming from testing futures trading systems, I initially failed to grasp on the need to test trading systems survivorship bias free database. But from numerous sources and evidence, I found that testing systems on biased data over state results significantly. I suggest that the trader/ researcher give a good thorough thought on this topic and check if they are employing survivorship free database.
For those who don’t want to create their own, Frank Hasslers’s blog, engineering returns offer two database for some cash.
For those of you who are looking for data, I suggest you go to the following webpage>
There is data back to 1995 till present. It is update daily.
i just got myself data from all the way back to 1989 for backtesting
CSI offers end of the day (EOD) data at a cost. Ninja trader paired with Kinetick is free. Now I have access to end of the day data plus charting for literally all futures markets at the tips of my finger tips for free 😀