Your Backtest Looks Perfect... Then Reality Hits

[00:03] trade a strategy without having back tested it before. But, what if you do the back test wrong? How much trouble can you get yourself >> A back test is a means to an end. What we want to do is build a robust back

[00:18] test that allows us to put on a strategy with confidence. And if we build a back test, a back test is a tool. It's like a hammer. You can take a hammer and you can build a house and do something productive, or you can take a hammer and

[00:31] damage. So, it's really important to have a robust back test. >> Today, we will help you not to mess up your back test. I'm joined by co-founder Matt Simon in Option Omega, who specializes in back testing and trade

[00:47] >> Hello. >> And full disclosure, this video is produced in cooperation with Option Omega, which is an affiliate partner of Theta Profits. But, let's get to the topic. Matt,

[01:02] what do we mean when we talk about back testing an an option strategy? testing an an option strategy? >> Yeah, so back testing is running a trade >> Yeah, so back testing is running a trade setup through a simulation of past

[01:15] events. So, using historical market data, we can build a strategy and we can back test it and we can do this going back in time many years. So, we can see how a certain straight a certain strategy, a certain trade setup will

[01:30] have performed during different market events, different geopolitical times, different VIX environments, different trading circumstances. And again, the goal of this is to build something that we have confidence in and that will

[01:45] allow us to craft a trading strategy that we can put on and feel confident that we've seen it experience different things historically. >> Give us a very short summary of what is considered best practice when you do a

[01:59] back test. >> Yeah, so just at a high level, what you want is you want a back test that's going to be realistic. So, you want to have a good data sample. What most people do is they try to get as many

[02:13] possible trades in their back test as possible. The more data points you have, that will give you higher degrees of confidence in the data. You also just want to make sure your back test is robust and true to

[02:27] life. And we're going to talk about a couple ways that you can do that, or even better, a couple mistakes that you can easily avoid so that you can build these best practices in to your back testing.

[02:42] >> And I know that you have talked to thousands of options traders about their back testing. You see all the mistakes that they do and all the what they do right. So, I want to spend some time on what

[02:56] are the most typical mistakes you see out there that people are doing. >> Well, I would say the first one is >> Well, I would say the first one is not building a robust back test. And

[03:09] there's a lot of things to consider when you're looking at historical data. And so we're going to look at an example here. Some a very simple strategy. So this is a iron condor that would be considered an MEIC, a multiple entry

[03:23] iron condor. I'm not going to spend a lot of time talking about the strategy. this back test. And real briefly, we're just selling a zero DTE condor many times a day. And if you showed somebody this back test, they

[03:38] would look at it and probably be pretty excited, right? It's got almost 2,000 data points, and it wins most of the time, and you know,

[03:51] there's a lot to get excited about. The problem is this is not a back test that I would feel comfortable putting on. It's missing a couple things. And if we just do something very simple like add opening fees and closing fees and and

[04:06] opening fees and closing fees and and slippage, that changes this back test from a big winner to something that nobody would want to trade. And so all I did was I added some opening fees and some closing fees.

[04:19] And then added some slippage. Some entry slippage and some stop loss slippage. And we can play with the slippage and say even if you say, "Matt, that that bad." And you take it down to 20 cents,

[04:31] you can see that this changes the back test still to something that looked very good to something that's losing money. So there's a lot of different ways to make your back test robust. In the scope of this video, we won't talk about all

[04:44] of them, but that is the concept. You want to make a robust back test. So accounting for things like fees and commissions and especially slippage is very, very important. >> What what what do you mean by slippage?

[04:57] mean by slippage? Let's go back to this example. So most back testers are going to use a historical mid price, right? And that is a big assumption that

[05:09] you will get filled at a historical mid price. So what we suggest as a best practice is is having slippage on both entry and exit. So in this example on the screen, we have 5 cents entry slippage that would apply to the

[05:24] spreads. And then also if one of the spreads gets stopped out, it has 20 cents of stop loss slippage. Now the thing is people who trade regularly can dial these slippage numbers in extremely accurately. So they can use their their

[05:41] forward testing to inform their back testing and really dial in the slippage robust. >> So, I guess the message here here is to make sure that this is as close to reality as possible and that you get all

[05:55] those costs we don't want to think about when we trade. >> That's right. Because those costs now I picked an example here where it's it's an extreme difference, but this is a very common type of trade setup that a

[06:09] lot of people would do. And simply forgetting to put in the fees and slippage, if it turns a a trade setup from very red from very green to very red, that's something that you really want to consider in your trade setup as

[06:23] well. But, the overarching point is yes, you want to make sure that your back test is robust and that it's true to life as close as you can make it. And the good thing is there's so many people that have traded and automated back

[06:36] tested strategies that there's an enormous community of people if you're trying to set up something that you've never traded before, it's fairly easy to get help and say, "Hey, is this a realistic set of assumptions that I'm

[06:48] making in my historical back test?" >> Are there are there typical things people forget when they don't make a robust check? >> Absolutely. Absolutely. And I'm going to show you another extreme example. And

[07:00] so, this one would be something like I would call it polishing dirt. And so, would call it polishing dirt. And so, what I have here is just a 21 DTE iron what I have here is just a 21 DTE iron condor. It's not very good. And it

[07:13] it you know, it's got 300 data points. And what I'm just going to show you is And what I'm just going to show you is all this has is a min VIX for the entry. So, the second mistake that I see a lot is over fitting, okay? So, this is not a

[07:28] good back test in the sense of this is not something you would want to trade, right? But, it's not over fitted. It's entering every day of the week at a set time. And the only real qualification it has is that it makes

[07:42] sure that the the the the days and the strikes are available and then it's doing a minimum of a VIX 14. So, you're not going to put it on when VIX is below not going to put it on when VIX is below 14. And if I add start adding to that, I

[07:55] can take this back test that went 60% of the time and make it look really good. And so, if I start to add a very narrow VIX regime, what happens is it doesn't look good, but it looks a little bit better and we've taken most of the

[08:10] trades out. And I can keep going and add all of these filters like an RSI, an EMA, and SMA and I can get a back test that has only lost one time. But the problem is, John, this isn't a terrible

[08:25] back test. You have so many filters on here and so many different qualifications that it takes this from something that was, you know, not a good back test, but a more robust back test and you just add

[08:39] these filters until it's something unrecognizable that barely trades and the problem is is I've seen many examples of people doing this. Let me show you one more. Here's another one. This is This is a a pretty

[08:53] good back test. It has three entry filters and what I'm going to show you is not going to be as dramatic, but you're going to see the point and what a lot of folks do. So, we have three entry filters. I can

[09:06] play with those entry filters and change this back test has a 3.6 margin, almost a 24% capture rate. Well, I can juice those numbers just by tweaking these and making these very specific. And so, here's the question. Is there really a

[09:22] difference in taking the max premium from 15 down to 14.85? Is there really a big difference in taking the min VIX move down from 1 to 1.12? Is there really a big difference from

[09:36] going from 0.01 to 0.03? You know, you're making your backtest better, but are you really making your backtest better? That's a question of overfitting. And so, this is the kind of thing There's two examples that you just

[09:52] saw of some, you know, one really extreme and one less extreme, but that's the kind of thing that people can do with overfitting. This is when I want to come back to that point of we're not trying to game a

[10:04] point of we're not trying to game a backtest. We're trying to build a robust strategy that we can trade moving forward. So, there's nothing magical about picking Vix 1.12 over Vix 1. That may be okay,

[10:19] right? Like that may be fine. What I want to talk about is the mindset. And if you view yourself as a technician behind a pat panel tweaking every little parameter thinking that you're going to eliminate all these bad setups and all

[10:32] these bad trades by just going through the historical data and picking out losers based on tweaking numbers, that's probably not the kind of mindset that is going to result in a strategy that you're going to feel comfortable trading

[10:45] moving forward. And that's the whole Again, that's kind of the whole point of backtesting. >> It seems to me that both these two first mistakes that you have walked us through are about removing

[10:57] yourself from reality. In the first case, about just not ignoring that there is a slippage in the real world. And in the second case, you try to design such a perfect conditions for your trade that you're never going to be in that

[11:11] situation, basically. >> Right. You're You're You're trying to You're trying to again, I think people just kind of lose sight of what they're trying to do. They They're trying to like almost game the backtester. And

[11:24] that kind of leads me to the third point. Not to jump ahead, but something that we have seen an increasing amount is that people, for the third mistake I would say people make, people are trading things without a clear thesis,

[11:40] without a clear understanding of their strategy. And AI tools are so prevalent now, automating options strategies, backtesting options strategies, modeling options strategies, those things are a

[11:53] lot more commonplace now than they were a couple years ago. And that's great. The tricky thing is, is that it also in having those tools available, it it's

[12:05] easy to go on and automate your option strategy. It has removed a level of understanding that people used to have more typically when they used to have more typically when they were trading. And so people now are

[12:19] trading strategies with no understanding of why they work, what the thesis is behind them. And that can be a huge problem. Obviously, options are leveraged inherently. So

[12:34] mistakes you can make? Well, if you're trading something that you have no clear understanding of, you're making you're setting yourself up to make mistakes, and you're setting yourself up to make mistakes with big leverage behind it,

[12:47] which is terrible. >> So basically, with all the tools, you have the backtesting tool, you have the trade automation tool, you can set it all up, you can go to AI, use the different

[13:00] LLMs to get all the ideas for how you could set up a strategy, but if you don't understand options trading, how the options are priced, how the price develops, etc., you will eventually fail.

[13:15] Yeah, you're setting yourself up for a disaster because you can get in a situation where you have positions on that you have no understanding. And then if there's a big market move, and let's be honest, we're in an era with

[13:28] algorithms and social media and tweeting when we've had a lot of moves and not just overnight but intraday as well. And so, nobody wants to be in a position they don't really understand what their risk is.

[13:43] >> So, I said you have spoken to thousands of traders and you know a lot about their mistakes. But, what about the best factors that you see in your community? What are they doing that is unique?

[13:57] >> Yeah, so this is a this is a great question. So, a couple practices that are kind of best practices that people do is number one, I'm going to go in reverse order here, they can clearly articulate their edge and what they're

[14:11] trying to backtest. They can present an idea and say, "This is why this idea has worked historically. This is why it makes sense." feedback from other people. You know, there's so many

[14:25] different trade setups, but you don't have to guess. If you're part of an active community where people have a lot of trading experience and hopefully who've traded more than you and who are smarter than you, who can give you

[14:39] feedback, that can really shorten your learning curve to learning the craft of options trading. Because even with automation and with backtesting and with these these tools now, it's still there's still an element of learning,

[14:52] fundamental information that you want to have to set yourself up to be as successful as possible. >> So, how will you sum up what we have been through so far? And especially, what would be your two or three most

[15:05] important takeaways for the audience to remember from this interview? >> Again, I would say that the couple things are that you want to make your things are that you want to make your backtest as real to life as possible.

[15:18] Keep your end goal in mind. The goal is to design a trading strategy that you can have confidence in. Gaming a backtest does not pay you any money, does not serve any purpose. You want to build a robust strategy. And I would say

[15:32] the other thing is that you can really kind of shortcut that process and speed up that process and speed up your own learning if you're not doing that in isolation. So, John, you've done a fantastic job of highlighting a lot of

[15:46] different traders in the community, but I think finding people that you can talk about your execution with, your ideas with is a really helpful thing. So, I would put joining a community, being part of a group,

[16:01] building a network, those are all really helpful things as well for traders. >> And as I said in the beginning, this video is produced in cooperation with Option Omega. So, Matt, I give you 20 seconds to tell what Option Omega is

[16:14] >> Sure. Well, we have a platform that allows users to design, craft, and automate their own strategies. So, you can model, backtest, and automate your

[16:26] community. People have been doing this for for years now and have a great group. So, if you're looking to take kind of control of your options journey, do it all in one place with the

[16:38] backtesting, the modeling, and the automation, optionomega.com. >> And if you want to explore Option Omega, you will find a link on the screen and in the description to this video. They do offer a 50% discount for the first

[16:52] year if you use the link that you find here on Theta Profits. here on Theta Profits. Matt, thank you very much for sharing how we can avoid making bad backtests. >> You're welcome. Thank you for having us.