Fix Backtest Failures in Live Trading — Step-by-Step Guide & Transcript

[00:00] To help you build out a framework for someone watching specifically who's learned things online, they've learned strategies online, but they are not finding success, but they find it in backtesting. In live markets, they don't. It can only be a psychological gap.

[00:13] How can they go about fixing it now and say, this is how I find a more congruent strategy to my congruence? So the first question you have to ask yourself is, are your backtests valid? Because frequently your backtest is not valid, even if it shows a very large rate of return.

[00:28] So I'll give you the classic example of this, which many people still do, which is surprising. So people will say, I want to trade a moving average crossover strategy. And then they'll find, you know, some asset that they want to trade.

[00:40] And then they'll find that exact combination of two moving averages that produce the highest backtest. And then they'll say, well, I need to trade this. The chances that that's going to work in the future are very low. Why? Because you've almost certainly hit upon some peak in the backtest.

[00:55] and that just happened to be at random at that specific spot and the future returns are going to be substantially lower if not negative. What you really need to do if you're going to do that is you want to find a stable set of parameters.

[01:08] You're not looking for the highest rate of return. You're looking for the most stable rate of return. Did you find the plateau? Can you vary your parameters and still get basically the same rate of return? And as I told you last time, can you add noise to the prices that you're using

[01:20] and by adding noise your return slowly degrades? Is it doing that? That's a good sign too that you've found a good backtest. So that's the first thing you've got to do. You've got to get a good backtest. How do you do it?

[01:33] A good backtest? Yes. I mean, myself and most, we go on trading gear and we rewind and we play price action and we see what we would do. So you have to first, it's a good question, you have to first ask yourself, you don't have supercomputers, you don't have 100 PhDs working for you.

[01:49] So you can't play the renaissance game of let's find every pattern and trade them all. So you have to do the opposite. it. You have to really drill down. You have to ask yourself, what is my thesis? What am I saying? What is it that I know or understand that is allowing me to make this excess money

[02:05] in the market? Once you can define that and you can actually state it precisely and clearly, I think my thesis is the following, then you can put together a backtest that's of some use. So I'll give you an example. Recently, I started backtesting a couple of things that

[02:23] basically were funnel. I haven't started using them yet. One thing that I read from a bunch of commentators is that the US has lots of debt and its debt keeps going up and so

[02:35] therefore the dollar is going to collapse and if the dollar collapses, that's going to be terrible for the market. So I said, okay, that seems reasonable to me. I don't know if it's true or it's false. Why don't I look at DXY and ask myself

[02:47] do extreme moves in the DXY which is the dollar index in the past have they ever told me anything useful about the market The answer is no So maybe the dollar will collapse maybe

[03:00] it will be bad for the market, but is there any evidence for this assertion? None that I can find. So, can you trade that? I wouldn't. On the other hand, you can look at, for example,

[03:12] the high-yield option-adjusted spread, right, for high-yield bonds. There is some evidence to suggest that when there's stress in the high-yield bond market, that the stock market is about to have significant risk.

[03:27] Correlation or causation? Good question. But it is fairly consistent, which suggests that it's causation, or at least an early warning canary, if you want to call it that. So, in other words, there is stress in the system.

[03:39] the stock market hasn't seen it yet because it hasn't quite affected stocks and so you could treat the high yield option adjusted spread however you want to do it as a

[03:51] leading indicator of something and then you have to think about it but that's the dichotomy so you're always trying to find a reason a good sensible reason for why this might be true you know the old joke is that you know

[04:05] butter production in Bangladesh is heavily correlated to stocks that begin with A or something like that. I forget the exact term. Something like that. Okay. I mean, I'm giving you a ridiculous example because it is ridiculous.

[04:18] But you have to be very careful with what it is that you're actually backtesting because there's no other way of finding a signal. This begs the question, or two questions. Number one, most people watching, especially new traders,

[04:30] the arena when we see how can we estimate price is fundamental, is it sentiment, is it trying to understand people's behaviour, psychological, and then there's also technicals price. Most people end up at price because it's easy to open

[04:43] up a chart and draw lines. What would you argue in the case of can technicals alone be a way to model and trade price? If you use technicals because you're trying

[04:56] to control your risk, there's a decent chance it will work. If you're using technicals with With some exceptions, we can get into that. To try to pick assets or stocks or whatever it is that are going to go up, say,

[05:13] there's a decent chance, with some exceptions, that that's wrong. That's the best thing that I can tell you because it's a question of base rates. I mean, you know medicine as well as I do. The base rates are in your favor if you're talking about risk.

[05:28] The base rates are against you if you're talking about return, with one exception. Okay. The reason I was asking about technical is because the premise has to be to them believe it or not is can past price predict future price Yes to some extent But the question is does it predict it enough that you can make money

[05:52] Is there a variance on time horizon? Shorter versus longer time horizon? Yes, there is. So if you are a small investor and you decide that you want to day trade, you actually have a chance of making some money

[06:04] with small positions. And I can even explain how. If you are a larger than small investor, then it's quite difficult to deploy enough capital on a very short-term time horizon as an individual investor.

[06:19] You don't have the nanosecond execution. The exception that I was mentioning is that the one thing that seems to work in almost every market, and almost, not quite, every time frame is momentum.

[06:33] So you can certainly trade, for example, breakout systems if you're willing to live with the risk. So essentially, you have a channel, draw it however you want doesn't matter Bollinger Bands lines makes no difference and when the

[06:45] market breaks above you trade one way market breaks below another way you have to have strict risk control you have to worry about stop loss you have to worry about all kinds of things but actually you have a chance of trying to make some money there's a limited amount of

[06:57] capital you can deploy but you can't do it so momentum is the one exception to that statement I assume the reason you say limited capital can be put through that needle of alpha yes is because we'll move to market

[07:09] No, you'll move the market. Because you've only got so much liquidity at that point and you need to enter at that point because if you don't enter at that point, you're no longer profitable.

[07:21] I remember last time you told me that there's a lot of liquidity in the market, but at the instance, it's very finite or limited. Yes. Even in something like the S&P, which I found pretty cool. Exactly. Okay, so let's say my goal is now to find

[07:34] something like this, a technical pattern, like a breakout, and I'm trading momentum. and then there is a certain limit that I could put into it before I move the market and I miss the opportunity. Is that something a retail trader should be concerned about

[07:47] or is it a pretty big limit? It's a pretty big limit. For a retail trader, you don't have to worry about it too much. And then, again, for a retail trader, there are some other very clever things that you can do. One of them is there are three kinds of momentum.

[08:01] And so as a retail trader, you should look at all three kinds of momentum and see if any of them are to your liking, to your personality, and you want to trade them. and those three kinds of momentum are time series momentum so that is to say

[08:13] the momentum of the thing itself right so like if you decide to trade let's say you take a moving average and you buy it when it's above and sell it when it's below that's time series momentum. Your sell series momentum is effectively how much it

[08:28] moved in a set amount of time. Well the reason it's called time series momentum is that it just calculated from the time series of the thing that you trading So you just moving average is the last 10 days average of right That why it called time series momentum So that one kind of momentum The second kind of momentum

[08:44] is called cross-sectional momentum. So in cross-sectional momentum, you don't look at how the asset did by itself. You put the asset in a basket of relatively similar assets, let's say all stocks or something or the

[08:56] other. And then you say, okay, I'm going to buy the top X percent of the performers in this market. That's cross-sectional momentum. The reason it's cross-sectional is that you have more than one asset in that basket.

[09:08] And peculiarly, you can even take unrelated assets and throw them in there and do the same thing and it'll work. Don't ask me why, but it does work. So, that's two kinds of momentum. But there's a third kind of momentum, which

[09:20] is a way of taking advantage of the stupidity of academic finance. Okay? So, you remember I was telling you earlier that alpha may not be defined. it might be

[09:33] might not be I just wish people would realize that it's not as clear cut as people think it is so because because it's unstable there's something clever that you can do you can take any given stock

[09:46] or even portfolio like an ETF or whatever and you can quote decompose it into factors so you can ask how much of its return comes from size and how much comes from value and how much comes from growth and how much those are factors

[09:59] now the fact is, my pun, that factors are themselves unstable. So this entire statement that you can decompose a set of returns or a portfolio into factors

[10:12] is, to put it politely, the technical term is crap. But, never mind, that gives you something that you can use. And the something that you can use is use factor momentum.

[10:25] So let's say that you've got, you're looking at, oh I don't know some collection of stocks that you decide to break out in a certain way. You can ask which factors are working now

[10:38] and then you can invest in the factors that are working now. And because they are unstable for a short period of time at least that factor is likely to outperform the other factors.

[10:51] So you can make a bit of extra return from being in those factors and not in the other factors. Now you have to keep watching this and so when that changes, you have to change the other factors and you keep hopping from factor to factor to factor to factor. This works because of the instability.

[11:05] But with all three of these types of momentum, especially the factors one, could be condensed down into just reading the price. It would reflect on the chart.

[11:18] No, because two of them are relative. So you'd actually have to put a few hundred stops on the same chart. say and then try to figure out which are the strongest and which are the weakest

[11:30] so then you're better off not doing it that way.