NOTE: at 4:20, pandas data reader has been deprecated!! So, the logic when obtaining your data will change. Thus, I have updated the Jupyter notebook here: github.com/SpencerPao/Pairs_Trading At 9:40 2 standard deviations are 95%. 1.25 standard deviations is ~90%
Should note: you can have a cointegrated pair with zero correlation. Correlation does not imply cointegration or rather an underlying relationship which influences the process of each time series in a distinct way. The relationship is what we are trading, or in other more specific terms, the statistical deviation from a cointegrated spread.
Yes, people do use returns for calculating hedging ratios in real life.. but I guess the reason he used "Price" is it is easier to create visualizations using Pricing data.
What is your rationale for doing CZcams instead of developing a trading model? Do you view career or CZcams as a more predictable way to increase earnings or do you doubt that it's possible to develop such model? Ignore question confines if needed, any thoughts appreciated.
An excellent question -- I actually never really put that much thought into creating quant models; I am more into Long only trades with longer time horizons. It may be an interesting venue to explore. It essentially boils down to time -- I only have so many hours in the day, and I recently became very, very busy. There's a lot more to that story but those are the high level points.
Hi Spencer, this is a fantastic channel! Your explanations are some of the clearest I've seen. When you have the buying and selling signals, that's on the specific asset you declared you were going to be long or short one, correct? I would love a video on how to implement the actual buying/selling part of the strategy and backtesting if you're taking recommendations! Thanks again Spencer
To answer your first question -- Yes indeed! I'm glad that you enjoyed this video. :) I'll dabble in the back testing / actual implementation of the algorithm via a brokerage. and see if I can make a video quality walkthrough. Note that the back testing is an art form; many of the testing algorithms are in house. I'm sure that there are some open source backtesting algo's to use.
Spencer, very interesting video. Please note that there is an “overfitting” problem in your concept. You are using a prior knowledge regarding the correlation of these two instruments. In other words, the fact that these two stocks are correlated in the period of time that you are analyzing is not necessarily going to be true in the future (and you are analyzing the performance of your method only for the past signal… that was used to determine if the stocks are correlated in the first place). Take care
You're absolutley correct! This is an assumption that there is an inherent dependency in this method and your positions will inherently become more risky the longer you hold onto a specific position.
Hey Spencer, thankyou so much for the video!! This is my first video on pairstrading and I finally understood the idea. However I have a few doubts especially at 8:55 , when you calculate the mean and then find your price signals. When you do that does that not cause a look ahead bias for older signals? Like we really never knew it was the mean at the beginning of the datapoint right? I'm sorry for the dumb question, but if you can clarify it, that would help me a lot! Thankyou!
There are no dumb questions! :) There are inherent assumptions when you go and utilize the pair trading algorithm. (i.e what time horizon you want to choose) Bias is an inherent factor with most if not all time-series particular model. There are some mays to mitigate that bias: 1) you can have normalized models in relation to the market over differing economic trends (recession, depression, growth etc..) and see how that compares. 2) Have additional features that is more representative of the data you are seeing now 3) Data augmentation etc. Hopefully that answered your question?
Sir, In pars trading can one trade spread between SP500 & Nasdq 100 based just on market sentiment of broad market/major indices.When sentiment is bullish,Nasdq100 would outperform SP 500 index.And in bearish scene vice versa.Same may be applied for pairs trading SP500 & Dow 30.In weak market Dow 30 would hold better & go down less compared to SP500/nasdq100.We seek your wisdom.Thank you.
I talk with others who are in the field. Honestly, the 'best' way to become a quant trader is to do it yourself. And, if you want to join a huge fund, you will have an edge over everyone else because you were an actual practioner. You learn by failing over and over.
Sir, Have you tried pair trading forex using rsi7 ,rsi14,rsi30 (add them up for comparison) say on hourly chart & selling strong pair & buying weak pair--pairs have to be highly correlated(eg aud/usd and nzd/usd OR dow30 & sp500).One can do this on any correlating underlying stocks/commodities/futures/crypto/bonds. Trading on hourly charts there would be tons of opportunities all year around.
I tried to run it but sadly, the signal is not correct. As you can see on the graph u presented, the signal is false as well. Any other way to detect peak and bottom of a graph ?
Any thoughts on rolling z-scores or z-z-scores (z-scores of z-scores when distributions may be not as mean reverting) and testing for effective lookback periods without data mining and information creep?
You most definitley can do a rolling z-score of z scores approach. (In fact that approach can be an extension of what I did in this video.) In terms of back testing, that is definitley more of an artform that I am not 100% super familiar with. Many companies have their own built in backtesting. But, you can look at it from the point of view of specific time horizons in the stock exchange to judge how well your model will fare in economic upturns or dowturns. You don't have to have real time artificially generated data that represents a specific trend. At the end of the day, if you get good predictions, then by all means go for it! :)
Goodnight ! Do you have any video on your channel about the Pair Trading strategy but instead of long x short, does it work in the form long & long or short & short? Note: congratulations for the channel.
Hello! I do not have a video on that topic. However, what you are stating is quite similar to the pair trading algorithm I discussed. Just switch the logic. I.e if asset A and asset B were to decrease (high correlation), you can spread your bets between the 2 asset classes.
Ah LQE? Few folks are aware of that haha. Estimating position(s) is actually incredibly interesting based on uncertain features. That would be an entirely different video altogether (and not necessarily related to Pairs Trading) I have thought about using Kalman filter on an ARIMA model, and after reading a paper, the filter does provide an incremental improvement. I might do a video on the topic but not in the near future.
Hey great video Spencer. I have a doubt at the end when we get the buy and sell signals, so say the buy signal is the trigger we got, now do we buy BRK or buy MSFT? how do we decide? Also how is it a pair strategy, we are only buying or selling only one stock at a time?
Mathematically and theoretically, you can go long on X and short Y. The direction of either one depends on what the model suggests. Also, from the pairs strategy (X,Y), you would typically standardize the pair i.e X would be buy and Y would be sell
Sir, In pair trading correlated currency pairs(say AUD/USD & NZD/USD) do you compare RSI value & sell overpriced pair & buy the weaker? OR do you compare CCI of each pair? Do you suggest using correlated cross currencies as pair trades(eg EUR/JPY & GBP/JPY)? On 4H chart sum up RSI 3, RSI 7,RSI 14, & compare added value on each pair & go. Thank you.
You can try any of those -- whichever has the best signals, go for it. Regardless of which relationships you seek, the pairs trading principle applies to any.
@@ronaknathwani5142 Hmm. Those are ticker symbols I assume? You can run any tool that you are comfortable with. (by tools do you mean software?) Check out code in C. Those are typically the fastest language in the quant world. If you mean which theories to use, you should test whichever one in your overall strategy to get your edge. Perhaps even use a combination of them or some other strategy that I am unfamiliar with.
@@ronaknathwani5142 Sir, On one chart plot RSI 3, RSI 7, RSI 14 ( say using MT4) & write values of each & add the numbers for ES NQ & YM--and compare.Please sell overpriced & buy the underpriced. System Programs use RSI 14, but as you know RSI 3 is the best guide to price & RSI 7 is more recent than RSI 14--so we have better picture of strength/weakness mtf.You need no program--just paper & pencil.
Hi Spencer thanks for the video. why did you choose 1.25 sd ? 1 sd from the mean covers 68% of the data I agree, but If you want to cover 95% of the data, you should choose 2 sd, considering a normal distribution. Thnaks
Hi ! I tried to use your code however there is an error. From what I understand, it can't access the data from yahoo finance correctly. Are you experiencing the same issue ?
Yes indeed! Thank you for letting me know! Yahoo finance has changed their backend so the functions I go over in this video are deprecated. However, I have updated my notebook to adjust for this. You can check it out on my Github here: github.com/SpencerPao/Pairs_Trading
I am curious about something, aren't we having a lookahead bias here when doing the tests to determine the pairs to trade, like say in one 1, we wont have info from the future right to tell us if this pair is cointegrated. How do you think we can tackle this?
Bias is huge. And, I don't talk about penalization. One way to do this is through continuous backtesting and revalidating the model (reweighting and reupdating the model) OR even having a different model altogether.
Yes! I was essentially using the ratio between two different assets based on the historical data. And, based on the base model: rolling averages, I depend on the historical data to have a prediction in the future. (In essence, using in sample data for predictions) It is a basic model where the typical machine learning principles are not at play.
It depends on your use case. If speed is not an issue, you can resort to scraping data from Yahoo Finance with Python/R. Those languages should suffice.
Hey, thanks for the amazing content, great video. I have one python-related technical question: by using pandas 1.2, the statsmodels doesn't work. I get this error when trying to import statsmodels.tsa.stattools: "module 'pandas' has no attribute 'Panel'". It seems the panel has been removed from Pandas module 0.25.0 onwards. How did you overcome this problem? Thanks in advance. Keep up the good work!
Most likely. If you are able to query data from the Binance APi without much limit, you can do the pairs trading algorithm. It's just a matter of latency at that point.
Algo system? Not really. For deployments for trading algorithms, you can look into using interactive brokers via their api. Plug in your model and stream their data. They are quite hefty in costs though.
@@amriksingh5003 Hmm. Well, unfortunatley everything has some elment of code. Though, you could just run my notebook locally from github and plug in your pair of public companies (for example) manually purchase/sell equity on some exchange.
@@user-pb7qd2fo5z ooh. I think you're referring to scraping financial asset data? I have not attempted to scrape from binance, so I am not familiar with the tooling, but you can set up price scraping mechanisms on Yahoo Finance's API. I did a similar video on this topic: czcams.com/video/GGgNM7WanK8/video.html Or this video (includes Yahoo finance data): czcams.com/video/wlaLmM_LvWg/video.html
dude, you gotta get rid of this line in your code "Cointegration test: A technique used to find a potential correlation in a time series (long term)". This is *absolutely* not what it is. But you have mistaken correlation for causality the entire way through this video so I'm not surprised. Cointegration actually establishes a causal link from one time series to another" Peace bro. But please don't say the purpose of cointegration is to find correlations. Nothing is further from the truth :-)
Hi Spencer, this is pure gold. Thank you for sharing so much knowledge with us for free :)
NOTE: at 4:20, pandas data reader has been deprecated!! So, the logic when obtaining your data will change. Thus, I have updated the Jupyter notebook here: github.com/SpencerPao/Pairs_Trading
At 9:40 2 standard deviations are 95%. 1.25 standard deviations is ~90%
A rather complicated concept made concise and very comprehensive. Excellent!!
Great video, and with Python code too!!
Thank you for making this clear explanation, and for making it available to your viewers
i think i found a rare gem, thanks for the video, keep up the good work
Should note: you can have a cointegrated pair with zero correlation.
Correlation does not imply cointegration or rather an underlying relationship which influences the process of each time series in a distinct way.
The relationship is what we are trading, or in other more specific terms, the statistical deviation from a cointegrated spread.
Very well explained thanks!
You could use the returns instead of price. And then the ratio between retturns of both assets
Yes, people do use returns for calculating hedging ratios in real life.. but I guess the reason he used "Price" is it is easier to create visualizations using Pricing data.
Highly underrated channel!
I love the support :')
What is your rationale for doing CZcams instead of developing a trading model? Do you view career or CZcams as a more predictable way to increase earnings or do you doubt that it's possible to develop such model? Ignore question confines if needed, any thoughts appreciated.
An excellent question -- I actually never really put that much thought into creating quant models; I am more into Long only trades with longer time horizons. It may be an interesting venue to explore. It essentially boils down to time -- I only have so many hours in the day, and I recently became very, very busy. There's a lot more to that story but those are the high level points.
I would also look at cash indices, i quite like DAX vs Nasdaq. Especially as a spreadbet in which u are not exposed to CCY flutuations.
i love this video Spencer!
Hi Spencer, this is a fantastic channel! Your explanations are some of the clearest I've seen. When you have the buying and selling signals, that's on the specific asset you declared you were going to be long or short one, correct? I would love a video on how to implement the actual buying/selling part of the strategy and backtesting if you're taking recommendations! Thanks again Spencer
To answer your first question -- Yes indeed!
I'm glad that you enjoyed this video. :)
I'll dabble in the back testing / actual implementation of the algorithm via a brokerage. and see if I can make a video quality walkthrough.
Note that the back testing is an art form; many of the testing algorithms are in house. I'm sure that there are some open source backtesting algo's to use.
@@SpencerPaoHere that would be fantastic, thank you! Thanks again for the great content - really appreciate all the effort
Brilliant!
You are making it sound sooo easy hahah
great work man! great work!
Amazing Video!!!
Michael, your tactics are incredible Which one are you recommend for the beginners?
Spencer, very interesting video. Please note that there is an “overfitting” problem in your concept. You are using a prior knowledge regarding the correlation of these two instruments. In other words, the fact that these two stocks are correlated in the period of time that you are analyzing is not necessarily going to be true in the future (and you are analyzing the performance of your method only for the past signal… that was used to determine if the stocks are correlated in the first place). Take care
You're absolutley correct! This is an assumption that there is an inherent dependency in this method and your positions will inherently become more risky the longer you hold onto a specific position.
Hey Spencer, thankyou so much for the video!! This is my first video on pairstrading and I finally understood the idea. However I have a few doubts especially at 8:55 , when you calculate the mean and then find your price signals. When you do that does that not cause a look ahead bias for older signals? Like we really never knew it was the mean at the beginning of the datapoint right? I'm sorry for the dumb question, but if you can clarify it, that would help me a lot! Thankyou!
There are no dumb questions! :)
There are inherent assumptions when you go and utilize the pair trading algorithm. (i.e what time horizon you want to choose)
Bias is an inherent factor with most if not all time-series particular model. There are some mays to mitigate that bias: 1) you can have normalized models in relation to the market over differing economic trends (recession, depression, growth etc..) and see how that compares. 2) Have additional features that is more representative of the data you are seeing now 3) Data augmentation etc.
Hopefully that answered your question?
Awesome video l. Thanks
Sir, In pars trading can one trade spread between SP500 & Nasdq 100 based just on market sentiment of broad market/major indices.When sentiment is bullish,Nasdq100 would outperform SP 500 index.And in bearish scene vice versa.Same may be applied for pairs trading SP500 & Dow 30.In weak market Dow 30 would hold better & go down less compared to SP500/nasdq100.We seek your wisdom.Thank you.
Awesome Video! How did you gain the knowledge that you have? What's the best way for an aspiring Quant Trader to gain all these knowledge from basics?
I talk with others who are in the field.
Honestly, the 'best' way to become a quant trader is to do it yourself. And, if you want to join a huge fund, you will have an edge over everyone else because you were an actual practioner. You learn by failing over and over.
Rich in content
Conceptually wrong. You should check if the pairs are cointegrated or not. Correlation does not imply cointegration.
Can you explain it more
very good video
Sir, Have you tried pair trading forex using rsi7 ,rsi14,rsi30 (add them up for comparison) say on hourly chart & selling strong pair & buying weak pair--pairs have to be highly correlated(eg aud/usd and nzd/usd OR dow30 & sp500).One can do this on any correlating underlying stocks/commodities/futures/crypto/bonds. Trading on hourly charts there would be tons of opportunities all year around.
I have not! You can definitely find many correlations -- the trick perhaps is to find causailty.
thank you alot
Buying what ? And selling what ??
Any equities !
I tried to run it but sadly, the signal is not correct. As you can see on the graph u presented, the signal is false as well. Any other way to detect peak and bottom of a graph ?
Are trading past if you have not rolled Z scores. You may have false signal with heteroskedasticity even in a pair of two non homoskedastic series.
Any thoughts on rolling z-scores or z-z-scores (z-scores of z-scores when distributions may be not as mean reverting) and testing for effective lookback periods without data mining and information creep?
You most definitley can do a rolling z-score of z scores approach. (In fact that approach can be an extension of what I did in this video.)
In terms of back testing, that is definitley more of an artform that I am not 100% super familiar with. Many companies have their own built in backtesting. But, you can look at it from the point of view of specific time horizons in the stock exchange to judge how well your model will fare in economic upturns or dowturns. You don't have to have real time artificially generated data that represents a specific trend.
At the end of the day, if you get good predictions, then by all means go for it! :)
Goodnight ! Do you have any video on your channel about the Pair Trading strategy but instead of long x short, does it work in the form long & long or short & short?
Note: congratulations for the channel.
Hello! I do not have a video on that topic. However, what you are stating is quite similar to the pair trading algorithm I discussed. Just switch the logic. I.e if asset A and asset B were to decrease (high correlation), you can spread your bets between the 2 asset classes.
are these trading algorithms "a nice to have" or something that you would ned to properly lidentify pairs?
Thx for the content!
Do you plan by any means to make a video of pair trading but with the use of kalman filter instead of moving averages?
Ah LQE? Few folks are aware of that haha.
Estimating position(s) is actually incredibly interesting based on uncertain features.
That would be an entirely different video altogether (and not necessarily related to Pairs Trading)
I have thought about using Kalman filter on an ARIMA model, and after reading a paper, the filter does provide an incremental improvement.
I might do a video on the topic but not in the near future.
Hey great video Spencer. I have a doubt at the end when we get the buy and sell signals, so say the buy signal is the trigger we got, now do we buy BRK or buy MSFT? how do we decide? Also how is it a pair strategy, we are only buying or selling only one stock at a time?
Mathematically and theoretically, you can go long on X and short Y. The direction of either one depends on what the model suggests.
Also, from the pairs strategy (X,Y), you would typically standardize the pair i.e X would be buy and Y would be sell
Sir, In pair trading correlated currency pairs(say AUD/USD & NZD/USD) do you compare RSI value & sell overpriced pair & buy the weaker? OR do you compare CCI of each pair? Do you suggest using correlated cross currencies as pair trades(eg EUR/JPY & GBP/JPY)? On 4H chart sum up RSI 3, RSI 7,RSI 14, & compare added value on each pair & go. Thank you.
You can try any of those -- whichever has the best signals, go for it. Regardless of which relationships you seek, the pairs trading principle applies to any.
Hi Spencer what tools do I need to run the similar test as above , I’m trading ES NQ and YM correlated pairs and like to see the stats for divergence?
@@ronaknathwani5142 Hmm. Those are ticker symbols I assume? You can run any tool that you are comfortable with. (by tools do you mean software?) Check out code in C. Those are typically the fastest language in the quant world.
If you mean which theories to use, you should test whichever one in your overall strategy to get your edge. Perhaps even use a combination of them or some other strategy that I am unfamiliar with.
@@ronaknathwani5142 Sir, On one chart plot RSI 3, RSI 7, RSI 14 ( say using MT4) & write values of each & add the numbers for ES NQ & YM--and compare.Please sell overpriced & buy the underpriced. System Programs use RSI 14, but as you know RSI 3 is the best guide to price & RSI 7 is more recent than RSI 14--so we have better picture of strength/weakness mtf.You need no program--just paper & pencil.
Hi Spencer thanks for the video. why did you choose 1.25 sd ? 1 sd from the mean covers 68% of the data I agree, but If you want to cover 95% of the data, you should choose 2 sd, considering a normal distribution. Thnaks
Hi! Yeah me choosing 1.25 sd was arbitrary. It does make a lot of sense to do 2 sd.
@@SpencerPaoHere ok :) and a z-score of 1.25 is at the ~90th percentile right, not 95th, as you mentioned. Thats why I got confused.
Hi ! I tried to use your code however there is an error. From what I understand, it can't access the data from yahoo finance correctly. Are you experiencing the same issue ?
Yes indeed! Thank you for letting me know!
Yahoo finance has changed their backend so the functions I go over in this video are deprecated. However, I have updated my notebook to adjust for this. You can check it out on my Github here: github.com/SpencerPao/Pairs_Trading
@@SpencerPaoHere that’s great, thank you !!!
Sir can you help me with adftest for pair trading
Hello, just wanted to know how you calculate the spread mine is all weird. Great video
The spread (which I believe your are mentioning at 6:17) is calculated by BRK_B - MSFT which is the difference between adjusted closing prices.
I am curious about something, aren't we having a lookahead bias here when doing the tests to determine the pairs to trade, like say in one 1, we wont have info from the future right to tell us if this pair is cointegrated.
How do you think we can tackle this?
Bias is huge. And, I don't talk about penalization. One way to do this is through continuous backtesting and revalidating the model (reweighting and reupdating the model) OR even having a different model altogether.
Good explanation ... but you use In Sample data? Really more complex to apply in real world trading when you need to use Out Sample data.
Yes! I was essentially using the ratio between two different assets based on the historical data. And, based on the base model: rolling averages, I depend on the historical data to have a prediction in the future. (In essence, using in sample data for predictions)
It is a basic model where the typical machine learning principles are not at play.
Thanks. Any good tool or means to check stock co-integration?
It depends on your use case. If speed is not an issue, you can resort to scraping data from Yahoo Finance with Python/R. Those languages should suffice.
At 9:40, doesn't 95% of the data lie between 2 standard deviations, not 1.25?
Yes. You are correct. I've updated my pinned comment and it was also asked in an earlier comment as well.
audio quality is not good
Hey, thanks for the amazing content, great video.
I have one python-related technical question: by using pandas 1.2, the statsmodels doesn't work. I get this error when trying to import statsmodels.tsa.stattools: "module 'pandas' has no attribute 'Panel'". It seems the panel has been removed from Pandas module 0.25.0 onwards. How did you overcome this problem?
Thanks in advance.
Keep up the good work!
Yeah. I'd recommend setting up a separate environment for that specific version of pandas and other dependencies for this specific overview.
Update: with the latest anaconda version (released in May 2022), everything works perfectly, there was no need to install a specific pandas version.
Any chance to have the same written for crypto In Binance API?
Most likely. If you are able to query data from the Binance APi without much limit, you can do the pairs trading algorithm. It's just a matter of latency at that point.
can get this code?
I uploaded the notebook here: github.com/SpencerPao/Pairs_Trading
Do you think this can be done using AUDUSD vs. EURAUD?
Hi! Most definitely! Although, there are a lot of other features that are associated with forex, Pairs Trading can apply to any sort of asset classes!
Pairs Trading belongs to Mean Reversion or Statistical Arbitrage?!!!
Mean reversion.
This is more in line with stat arb. czcams.com/video/TNJNCJe9fDY/video.html
Hi do you have the algo system for pair trading I can use. How much it costs ?
Algo system? Not really.
For deployments for trading algorithms, you can look into using interactive brokers via their api. Plug in your model and stream their data.
They are quite hefty in costs though.
@@SpencerPaoHere thanks for your prompt response. I m not programming guy.
@@amriksingh5003 Hmm. Well, unfortunatley everything has some elment of code. Though, you could just run my notebook locally from github and plug in your pair of public companies (for example) manually purchase/sell equity on some exchange.
Wouldnt this work better with forex pairs?
Not sure. I haven't tried. Though, I bet a lot of folks may use a version of pairs trading in any type of financial market.
hi tell me how to do it for the cryptocurrency market?
You could think of it in terms of replacing the assets with coins. Same process.
@@SpencerPaoHere I can’t load data market from binance
it doesn't work there
@@user-pb7qd2fo5z ooh. I think you're referring to scraping financial asset data? I have not attempted to scrape from binance, so I am not familiar with the tooling, but you can set up price scraping mechanisms on Yahoo Finance's API. I did a similar video on this topic: czcams.com/video/GGgNM7WanK8/video.html
Or this video (includes Yahoo finance data): czcams.com/video/wlaLmM_LvWg/video.html
@@SpencerPaoHere oh thanks, I think I found it
dude, you gotta get rid of this line in your code "Cointegration test: A technique used to find a potential correlation in a time series (long term)". This is *absolutely* not what it is. But you have mistaken correlation for causality the entire way through this video so I'm not surprised. Cointegration actually establishes a causal link from one time series to another" Peace bro. But please don't say the purpose of cointegration is to find correlations. Nothing is further from the truth :-)
Why so complicated ? Make money is easy .its simpl .
There's a reason you make CZcams videos instead of working in the industry. Your implementation and programming is garbarge.