Handling Outliers
By Dave Mabe
Yesterday, I discussed how to identify outliers in your backtest.
Now let's see what you can actually do about them.
To be clear, this assumes the outliers are due to weird position-sizing situations.
(For more info on handling outliers in P&L from a well-calibrated position sizing approach, see this post.)
Once you identify the outliers, it's important to understand why the sizes are so out of whack.
The backtest simply follows your logic, so make sure you understand why your code can generate the large (or small) share sizes.
Here's the screenshot from Amibroker, showing trades with share sizes in the 200k range:
The real issue here is that the profit for these trades will not reflect reality.
That is, if you were to trade this strategy as-is, the fills you get would have an overwhelming amount of slippage.
The bottom line is your backtest is way too optimistic - which is always bad!
This is more common when you size your position based on the distance to the stop price.
What to do about it
The simplest approach is to use a floor for how far the stop can be from the entry price.
For example, if your stop is the low of the day and the entry price is one cent away, you're going to end up with a large position size.
You can set a minimum distance that gets enforced in your code, so the default would be the low of the day, unless it's too close to your minimum.
You can use a distance in terms of dollars, but that can have problems as well.
Think of a high-priced stock (like SNDK currently) and using a tiny dollar-based stop.
In some sense, this papers over the real problem.
It's good to zoom out, think about your strategy, and consider what a more calibrated stop would look like.
Some ideas:
Stop is at the LOD, plus a percentage of daily ATR
Stop is at the LOD, plus a percentage of a recent intraday ATR
Stop ignores LOD entirely and uses a percentage of ATR
All these approaches have the advantage of dynamically adjusting based on price and/or ATR, so you don't end up with, say, a strategy for stocks between $10 and $20 and another for stocks between $20 and $50, etc.
So consider quick ways to fix your sizing, but realize a more comprehensive approach could be best.
Also, beware: it's easy to get stuck in "backtest world" here, so don't get stuck on trying to be perfect.
-Dave
P.S. Are you in a drawdown and ready to quit? Join the other traders who reversed their drawdowns and are printing money with MabeKit. Get Instant Access