Stop Analysis and Position Sizing
By Dave Mabe
Here's a question from a long-time list member, Rohit A. (used with permission, lightly edited for clarity):
Rohit A:
You mentioned that you do stop analysis in the Cruncher. What's the best way to compare strategy metrics across different stop sizes? For example, if we compare a tighter stop to a looser one, we should technically be sizing the trade larger with the tighter stop to keep risk constant. But what I’ve found is that this can make the few trades that work out look massively outsized in a backtest, even if the result isn’t particularly realistic. When you analyze different stop sizes, do you resize the trades at all, or do you keep position sizing constant for the comparison?
Dave:
Great question, and this is one of the dangers of sizing based on distance to the exit stop price.
If you're not careful, you'll end up with huge outliers hidden in your backtest.
Then, when you use the Cruncher, you'll unknowingly be optimizing to those outliers.
And just a few outliers can make an enormous difference.
(I remember one backtest I ran where a single outlier changed the direction of the edge from long to short - yikes!)
Luckily, there's a quick way to identify the outliers.
In the Amibroker trades list, click on the column header to sort by these fields:
Profit
Shares
Position value
Sort them in both descending and ascending order, and look for outliers: trades with much larger or smaller profit or a much larger position size relative to the other trades.
Here's an example from a backtest I just ran:
This situation is so common that I've created a warning in the Strategy Cruncher for it.
When you try to optimize a strategy with outliers, you'll get a clear warning at the top of the report:
This is one of a growing list of warnings in the Cruncher to point traders to issues early on in their process.
The warning has saved a lot of heartache for traders who get their hopes up only to be disappointed when they go live and discover the issue too late.
Tomorrow, I'll share what you can do to clean up the outliers in a reasonable way.
-Dave
P.S. Here's a quote from MabeKit user Scott J., who recently went live with some strategies he created with the Cruncher: "The Cruncher gives answers quickly and can run a lot of different scenarios. It would often push the profit factor into the 1.6 to 2.0 range. These are really simple strategies. It’s nothing complex. The Strategy Cruncher added the edge.” - Scott J. Get Instant Access