"The Backtest Filter I'm Applying Doesn't Get Rid of All the Trades. Why?
By Dave Mabe
Here's a question from Brad (name used with permission, lightly edited for clarity).
Brad:
I recently purchased your AFL course and template and have been working with it extensively. It has been an excellent foundation for building strategies and learning AmiBroker.
I am currently working with a smaller database of approximately 1.5 to 2 years of data to learn proper backtesting techniques more efficiently. I have been using the strategy in your template and adding my own columns over time.
My process involves exporting the results from AmiBroker into Excel, applying filters to the columns, and identifying results that show potential. However, I have encountered an issue: when I apply those same filters within the AmiBroker backtester and re-run the test, I get completely different and almost consistently worse results.
I have reviewed my database settings and code, but I cannot find the source of the discrepancy. Are there some specific settings you know of to check or concepts I might be overlooking that would cause such a difference between the exported data and the backtester results?
Dave:
I can understand your frustration because I've been there!
I can't see Brad's code, but I'm reasonably certain he's running into what I call the "peek-a-boo" problem.
I recently gave a talk on SMB Capital's Bionic Trader show on this exact topic, so it's fresh in my mind.
The issue is that unless your code is set up to handle this, you're likely just postponing trades rather than filtering them from your backtest.
Take this hypothetical example. Your strategy's signal is when a stock reaches a new high. You go long.
You run the backtest and one of your columns is Volume Today.
In your Excel analysis, it tells you that a minimum value of, say, 500k will improve your strategy by removing 1/3 of the trades.
You add the filter and run the backtest, but find it did NOT remove 1/3 of the trades. Why?
Well, think about your signal and the Volume Today filter.
New highs can occur multiple times per day for a single symbol.
And the Volume Today continues to increase throughout the trading day.
The earlier trade is gone (when the Volume Today was less than 500k), but you've replaced it with a trade in the same symbol later in the day when that Volume Today minimum is finally met.
How do you handle this?
You should handle this like the MabeKit AFL Generator handles this, by having a way to invalidate the symbol for the remainder of the trading day when a signal is found that DOESN'T match your filter set.
Here's what it looks like in the code that the generator produces:
Invalid = VolumeToday < 500000;
This makes the backtest filter out the earlier trade AND put it on a list that knows to skip any futures signals that might occur for that symbol for the remainder of the day.
More on this topic in the coming days - it's an important topic to understand!
Great question, Brad.
-Dave
P.S. Are you struggling with writing AFL code for backtests in Amibroker? Generate working backtests in seconds with MabeKit.