Algorithmic Trading - Machine Learning & Quant Strategies Course with Python
Vložit
- čas přidán 3. 06. 2024
- In this comprehensive course on algorithmic trading, you will learn about three cutting-edge trading strategies to enhance your financial toolkit. In the first module, you'll explore the Unsupervised Learning Trading Strategy, utilizing S&P 500 stocks data to master features, indicators, and portfolio optimization.
Next, you'll leverage the power of social media with the Twitter Sentiment Investing Strategy, ranking NASDAQ stocks based on engagement and evaluating performance against the QQQ return. Lastly, the Intraday Strategy will introduce you to the GARCH model, combining it with technical indicators to capture both daily and intraday signals for potential lucrative positions.
✏️ Course developed by @lachone_
💻 Code and course resources: github.com/Luchkata/Algorithm...
🔗 You can sign up for the data API used here: intelligence.financialmodelin...
🔗 Learn more about Lachezar and Quantitative Trading with Python here: www.quantfactory.ai/p/become-...
⭐️ Contents ⭐️
0:00:00 - Algorithmic Trading & Machine Learning Fundamentals
0:15:25 - Building An Unsupervised Learning Trading Strategy
2:05:08 - Building A Twitter Sentiment Investing Strategy
2:28:08 - Building An Intraday Strategy Using GARCH Model
🎉 Thanks to our Champion and Sponsor supporters:
👾 davthecoder
👾 jedi-or-sith
👾 南宮千影
👾 Agustín Kussrow
👾 Nattira Maneerat
👾 Heather Wcislo
👾 Serhiy Kalinets
👾 Justin Hual
👾 Otis Morgan
👾 Oscar Rahnama
--
Learn to code for free and get a developer job: www.freecodecamp.org
Read hundreds of articles on programming: freecodecamp.org/news
One thing I like about trading, everyone has an opinion n yet few are making money 💵
well there are some making money, else there won't be big hedge funds
@@nobodyknows3560 the big hedge funds won't share their opinions clearly though, only vaguely
he failed
This has been a privilege. I worked in finance for a long time but never had the chance to see what goes on behind the scenes at fund management. I was only exposed to traditional portfolio managers who's strategies were thematic in nature and qualitatively driven. I'll probably never use any of these techniques or strategies for personal finance but it really gets you thinking of how things can be approached from a large portfolio perspective. GARCH model is daunting but this inspires me to finally understand what my boss was talking about all those years ago 😅
Turns out I can't even get the arch package to work. After pip install arch, can't even import it in Jupyter NB. 🤷🏻♂️
GARCH is only daunting if you don't understand it.
A fellow Bulgarian! Congratulations on your success, Lachezar!!
Thank you mate, cheers!
Bulgarians don't play around, you know you stopped caring what people think when you don't code in dark mode.
Awesome 😎. I have been looking and waiting for this topic. Good job 👌❤️
I love your analysis, always on the point. And yes, Xeventy is a game changer. Great project!
If you are concerned about stocks that fall out of tne S&P500 (survivorship bias), you could have just checked the top 150 against those that were always present in the S&P500 and then dropped those that werent from your dataset. (Or set some cut-off where it had to be present at least x% of the time)
Can you explain more about this?
Really interesting video. Thanks there are still people with the goal to teach. While I continue with my learning I have bought two bots for ES and NQ just to see how they work and they are giving me nice extra money for the last months. This also gives me the conviction that the authomated strategies could really work fine.
I am amazed to see algo videos rarely on CZcams. I appreciate your work. Could you please make a similar kind of video on price action algo trading with Wyckoff multi-timeframe analysis, without using any indicators, and generate trades live through your machine learning or AI models?
Many people are eagerly waiting to see these kinds of videos, but no one has ever made any price action trading videos. I request you to do so at the earliest.
Thank you,
Muni Babu
Are you sure successful quant funds really trade Wyckoff, or what's worth - Fibonacci retracements? 😅
@@flexorx what ever they do, price action plays right, that enough for us...😏
It's not about beating the market, its about having an unsolvable problem that you can always use to learn against.
Lol. It's about beating the market.
Your KMeans clustering is totally useless when you're not normalizing the RSI values. In essence all the other features are just small noise in the clustering and you end up making the wrong conclusion that RSI is the main feature driving the clustering in your data.
this is correct. upvoted
Yep, the clustering part was only to make the "algo" more appealing. I thought he would have actually analyze each cluster trying to find useful features...
This example is for intra-day and day-trading as specified at the beginning of the video. After a quick research the best indicator for this kind of trading is the RSi. Due to this the main feature chosen by him is the RSI because it is the best indicator for this kind of trading strategies.
Based comment
@@TyTy-gm8yb RSI is not the best indicator. It shows basically what has already happened in the past. It is by no means usable in a forecast model. Should you want to have useful data you could try using Money Flow, or simple price action with Volume. Or if possible, you can create a model using real time agression.
Excellent video. Thank you!
the .loc doesn't fix the error encountered at 51:30 with values starting from 2020. It is actually the rolling average calc parameter min_periods=12 that you set that fixes the issue...
Thank you! Exactly what I've been looking for :3
Are u a computer scientist?
Thanks for this. I've been looking forward to it. 🙏
Are u a computer scientist?
Thanks for sharing! For the 1st project, I wonder how the clustering is different from selecting top RSI stocks in each month given RSI is not normalized? Or is it just for demoing the possibility of using the clustering algo?
Thank you for this treasure! Just bought your other course. Can’t wait to finish it🎉
Hi, very good video and good job! But the daily signal volatility definition needs to be more conservative i.e. 1 std and not 1.5 std, due to the high CV of the signal ( to improve the results)
Can you clarify whether survivorship bias occurs when including data from a company that has gone bankrupt, or when excluding data from a company that has faced bankruptcy?
A good future use case might be the production, storage, use and trade of electricity in a household. Solar panels are so cheap, electricity is quite expensive/ volatile due to renewables at times. Electrification/ automatisation of households, variable prizes from providers, self production (photovoltaic cells), storage (e-cars, warm water, iot, refurbished battery cells,...), weather forecast,...
u made everything look so easy!!! thank you for the lesson, I am trying to duplicate this lesson and go a little beyond in terms of generating a SQL dataset and making a potential web page out of it
spectacular free content - thank you SO much
Thank you so much for this Free Code Camp
Please 🙏 🥺 do same for real estate
I think that you will win " The Python One Liner Award" !
8:08 Workflow Process: 1. Collect and prepare the data, 2. Develop a hypothesis for a strategy, 3. Coding the model, 4. Backtest the strategy
9:19 Unsupervised Learning Project- uses ML strats without a labeled or predefined target variable. Unlike supervised learning because the model is not trained to make predictions, but to extract insights from data
12:45 Sentiment Investing Project - how people feel about stocks can impact stock prices/industries/overall market
14:04 Intraday Strategy Project- intraday approach means to buy/sell financial assets in same trady day to profit from short term price movements. Traders use real time data and risk management to make quick decisions and capitalize on market volatility
2:10:51
Thank you, I was looking for it!
Good day , I would like to work with you on FOREX trading strategies
Tested with data until today (feb-22-2024) and a 1:18:00 there were different clusters 3, on dec-31 there was a cluster with rsi = 80 and other where the top rsi was around 60
Watching your video about trading help more to understand the beauty and benifits of trading, thank you for sharing.
ugliness of trading in market??
Very valuable information, thanks for your work!
MIL GRACIAS POR LOS SUBTÍTULOS EN ESPAÑOL❤❤❤❤❤
Thanks 🎉
I'm a former quant. Back in my days we used to write all the algorithms ourselves on c++, including the Greeks to price options, multidimensional volatility surface(48) calculations per trad, etc.. Now days being a "data scientist" and a "quant" its about being just popular on social media.
Hey,since you are a former quant ,I found it really intriguing .I'm currently working on a script for currency analysis and since I'm relatively new to the quant world , I'd love to connect and chat about it .Maybe we could share some insights and experiences .Let me know if you're open to it. Cheers!
@@thenoblegodplease let me know when he messages you
I’m so glad I have a day job!
Pretty cool! Thanks a lot!😁
Thanks!
Really good video iam curious can we apply this algoritmic trading to Forex markets?
Don't miss the chance to be part of the Xeventy presale guys! Easy 50x till the end of the year, and I´m a very conservative
Perfect for my Quant journey
Are you following quant?
your quant journy will end soon when you realize it is just not possible, at all, you live in a dream if you think it is.
There are just a small number of large organizations who can do this efficiently, it requires much more than a laptop.
Very good content!
Great video! Unfortunately normalization procedure in it introduces lookahead bias. You can't use the full range of values to calculate mean and stdev, you can only use the data from earlier dates at any given point of time.
He collapses it to the end of the month. The number your swing consist of days before
@@loganbishop4619 upon further review, the normalization is applied per security using the entire time series so I think this is an issue raised by @johnsmith-qc8bud ... wouldn't the correct way to do this would be to apply normalization on a rolling basis?
Thank you!
Well, good luck making this profitable.... 😄There are exactly zero profitable strategies out in the open like this, for the simple reason that everyone who actually comes up with something profitable will keep is to themselves so as not to lose their edge on the market.
Please could you recommend books on algorithmic trading with python or machine learning???
Thanks ❤
First 2 hours was ok, mainly about getting some basics about panda, and some clustering idea, though what I miss that if I understand correctly re-balancing/weigthing happens on montly basis, which at 150 security (worst case) it takes big amount of money as most of trading platform has minimum fees, as a consequences probably your extra strategy profit would be eaten by cost. It was fun, but I have doubts that's the way to get rich :)
Nobody knows what's going to happen in the stock market. The tech stack is interesting though.
underrated
Thanks for sharing the video. Can anyone help me understand the daily signal part in the GARCH modeling? Is the belief that the higher predicted return volatility leads to a higher projected return? Doesn't higher volatility mean higher risk?
NVM, the logic is reversed in the later code
thanks mate you are a champ
I love this so much ahahaha lets go!!!
supperrrbzz...amazing..
Is quant for only stocks ? I haven't come across the major currencies.
thanks
does yfinance has survivorship -bias free data?
For number 2, an error says "AttributeError: 'NoneType' object has no attribute 'iloc'". What shoud I do with this?
I am getting the same error and cannot find a workaround.
Such an amazing video an content. You are magic, this content opens the door to explore different quantum trading strategies assited with artificial intelligence. For those who put destructive opinions , please fck yourselves and criticised when you contribute something. This is an awesome approach for beginners in the algorithm trading and machine learning field. This is the main stone for my project to develop an algorithmic trading strategy using quantitative methods, machine learning and artificial intelligence to create profitable long ansd short entry positions
Bookmark: 15:00
Can you make for nse(india) trading. Because there is lot of freelancing job opportunity, but I can't find any good material or course to learn the algorithmic trading specifically for nse.
Is there trading based freelancing in India?
I think he means portfolio management.
Don't bother with Indian clients on freelancing website for trading.
They suck your time and not worth it.
And trading on nse is no different than any other market if you get the fundamentals right.
thanks, now i can buy item in Jita
How to get the current twitter sentiment data for real time trading?
this thing is for students of python code. Not traders. But good video non the less
got an error on Calculate Rolling Factor Betas (58:04) . error message indicate : min_nobs must be larger than the number of regressors in the model and less than window.
May i ask how can i resolve it ?
I think Xeventy´s XVT will be one of the top performers in bullrun 2024/2025.
Youve went from coding to trading lol, tganks for everything
How do we generate the twitter sentiment dataset by ourself?
little confused if you took stocks with RSI around 70 of the previous month what was the logic to use a lot of features
I am on the same page with you. Have you got any response regarding that?
does anyone else having error in the part where calculating rolling five year average of dollar_volumn and reassigning to the dollar_volumn column which converts every values to Nan
code:- data['dollar_volume'] = data['dollar_volume'].unstack('Ticker').rolling(5*12).mean().stack()
This algorithmic trading can be applied to any liquid traded financial market like forex,stocks,and crypto , right?
yes
Good day , I would like to work with you on FOREX trading strategies
i am confused. so you use kmeans clustering to select stocks. then you use 1 year of stock prices and efficient frontier to calculate weight of component stocks. you use 18 features to fit a regression model to get the beta coefficients, which basically tells you the weights of the features on each stock. however, you never use the regression model to make predictions. you only use the betas for clustering? so your 'model' does not have forecasting ability? or does it? and where is the bollinger bands and macd used?
4 hours into the course already, such a great mentor and way of teaching
But the Total Video length as shown above is 2:59 hrs only
Where is the additional hours from?
bro is lying
Very Funny.
In the Project 1 in the point 6 when clustering, there is an error alert saying that: KeyError: "['cluster'] not found in axis" - Any advice? thanks
I had the same , think the variable is not set and you need to drop it as its set only later
if "cluster" in data.columns:
data = data.drop('cluster', axis=1)
@@philipp2117 Here you go:
from sklearn.cluster import KMeans
if 'cluster' in data.columns:
data = data.drop('cluster', axis=1)
def get_clusters(df):
df['cluster'] = KMeans(n_clusters=4,
random_state=42,
init=initial_centroids).fit(df).labels_
return df
data = data.dropna().groupby('date', group_keys=False).apply(get_clusters)
data
Is anyone else having trouble with the following line of code:
df['bb_low'] = df.groupby(level=1)['adj close'].transform(lambda x: pandas_ta.bbands(close=np.log1p(x), length=20).iloc[:,0])
Apparently, pandas.tt_bands() returns none, but that doesn't seem to be the case in the video
If you are pulling current price data, VLTO does not have enough price history. Try removing it from the symbols_list and see if that fixes the issue.
I have same issue, complety stop.
same issue!
I changed 'length 20' to 'length 10'. Like this: df['bb_low'] = df.groupby(level=1)['adj close'].transform(lambda x: pandas_ta.bbands(close=np.log1p(x), length=10).iloc[:,0]). It worked, but I don't know why.
@@oneouthere this is the correct answer.
The api 'Sandbox api' you used in your previous video has stopped working.Any fix for this ?
Nice!
2:27:59 LOL, the twitter sentiment trading strategy was killing it right up until Elon Musk bought twitter. Hahah
I am getting error in this line df.unstack('ticker')['dollar_volume'].resample('M') can anyone tell me how to solve this error?
same here, how did you solve it?
last_cols = [c for c in df.columns.unique(0) if c not in ['dollar_volume', 'volume', 'open',
'high', 'low', 'close']]
# data = (pd.concat([df.unstack('ticker')['dollar_volume'].resample('M').mean().stack('ticker').to_frame('dollar_volume'),
# df.unstack()[last_cols].resample('M').last().stack('ticker')],
# axis=1)).dropna()
#df.unstack(['ticker','date'])['dollar_volume'].reset_index('date').set_index('date').resample('M').mean()
data = (pd.concat([df.unstack(['ticker'])['dollar_volume'].reset_index('date').set_index('date').resample('M').mean().stack('ticker').to_frame('dollar_volume'),
df.unstack()[last_cols].reset_index('date').set_index('date').resample('M').last().stack('ticker')],
axis=1)).dropna()
data
01:04:25 "From this point on things can get really complicated." So, the previous hour was just the warm-up? *gulps nervously*
Doesn't it bother anyone that clustering is done on the ENTIRE dataset? and further asset selection is done month by month using cluster label that was computed using future data?
after doing this for long time, with all sorts of advanced deep learning, math models, and whats not, i can surely say - this is not possible for the simple retail investor to have any alpha (or advantage) over the serious big organizations that do that. It is just not possible, just think that if it was, the biggest AI experts would already do finance and get rich. they aren't, they are making courses like this - because views on youtube pays more than algo trading. It is easy to sell programmers the idea that they are smarter and they can use special coding to make money. they cant.
after a while, I figured this was the case also
What's your expertise? I'm pretty sure you don't have even 1% the level of expertise of this guy.
More Algo videos please
Do you build trading bots for fee?
Could somebody please help out i have an issue with the chapter 3 starting from timestamp 34:02 saying a TypeError and after that the whole chapter doesn't work its affected the other chapters also
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index' this is what i get i even copied the code from the github but still gives me this problem. Could i get some help
Hello thanks a lot for this tutorial, is there any way to download the code please ?
Is any one else getting "None of ['index'] are in the columns" error about 1:57:00 when calculating wights?
change 'index' to "Ticker".
@@hammam92 pls tell where
@@pranavsingh1947 bro give me the code that gives the error.
@@hammam92 ....
returns_dataframe = np.log(new_df['Adj Close']).diff()
portfolio_df = pd.DataFrame()
for start_date in fixed_dates.keys():
try:
end_date = (pd.to_datetime(start_date)+pd.offsets.MonthEnd(0)).strftime('%Y-%m-%d')
cols = fixed_dates[start_date]
optimization_start_date = (pd.to_datetime(start_date)-pd.DateOffset(months=12)).strftime('%Y-%m-%d')
optimization_end_date = (pd.to_datetime(start_date)-pd.DateOffset(days=1)).strftime('%Y-%m-%d')
optimization_df = new_df[optimization_start_date:optimization_end_date]['Adj Close'][cols]
success = False
try:
weights = optimize_weights(prices=optimization_df,
lower_bound=round(1/(len(optimization_df.columns)*2),3))
weights = pd.DataFrame(weights, index=pd.Series(0))
success = True
except:
print(f'Max Sharpe Optimization failed for {start_date}, Continuing with Equal-Weights')
if success==False:
weights = pd.DataFrame([1/len(optimization_df.columns) for i in range(len(optimization_df.columns))],
index=optimization_df.columns.tolist(),
columns=pd.Series(0)).T
temp_df = returns_dataframe[start_date:end_date]
temp_df = temp_df.stack().to_frame('return').reset_index(level=0)\
.merge(weights.stack().to_frame('weight').reset_index(level=0, drop=True),
left_index=True,
right_index=True)\
.reset_index().set_index(['Date', 'index']).unstack().stack()
temp_df.index.names = ['date', 'ticker']
temp_df['weighted_return'] = temp_df['return']*temp_df['weight']
temp_df = temp_df.groupby(level=0)['weighted_return'].sum().to_frame('Strategy Return')
portfolio_df = pd.concat([portfolio_df, temp_df], axis=0)
except Exception as e:
print(e)
portfolio_df = portfolio_df.drop_duplicates()
portfolio_df
...
will print
...
"None of ['index'] are in the columns"
дедок лучший!!!
how do i install the anaconda prompt? i have python and vscode on my computer. thank you
1:34:23 Why do you take the log of the returns?
usually the log returns supposed to follow a normal distribution more closely. well it still doesn't. you would need tick data and use volume instead of time to get really close to a normal distribution.
Is there any tutorial available which can reach how to get real-time data offered by brokers and place trade using api provided by broker. All these are in jupyter notebooks. I'm looking for something in real code which can run continuously
Nice
how did you obtain the sentiment data sir ?
I have trading model that’s proven and I’ve broken down. Would there be anyone with quant experience willing to take my parameters and incorporate into a systematic trading algorithm?
Great tutorial. Stuck on macd function though. Doesn't seem to work as is.def compute_macd(close):
macd = pandas_ta.macd(close=close, length=20).iloc[:,0]
return macd.sub(macd.mean()).div(macd.std())
Good day , I would like to work with you on FOREX trading strategies
same error
23:50 I guess ln is different from log in math which you used it in place of it.
Can anyone explain what was the reason for choosing length of 20 for rsi and 14 for atr ?
I got an error in the following line of code:
Code:
df.unstack()[last_cols].resample('M').last().stack('ticker')
Error:
KeyError: 'Level ticker not found'
Anybody can help?
this is what I could make of it:
last_cols = [c for c in df.columns.unique(0) if c not in ['dollar_volume', 'volume', 'open',
'high', 'low', 'close']]
# data = (pd.concat([df.unstack('ticker')['dollar_volume'].resample('M').mean().stack('ticker').to_frame('dollar_volume'),
# df.unstack()[last_cols].resample('M').last().stack('ticker')],
# axis=1)).dropna()
#df.unstack(['ticker','date'])['dollar_volume'].reset_index('date').set_index('date').resample('M').mean()
data = (pd.concat([df.unstack(['ticker'])['dollar_volume'].reset_index('date').set_index('date').resample('M').mean().stack('ticker').to_frame('dollar_volume'),
df.unstack()[last_cols].reset_index('date').set_index('date').resample('M').last().stack('ticker')],
axis=1)).dropna()
data
how do you present something like this in a portfolio?
Is this transferable to forex trading?
Anyone else running into issues trying to install the pip packages or is it just me? Any solutions?
How much Profit did You do with this ? Thanks for the Info!
why when i run the code i get such a fcked up plot at k-means? it doesn t. reseamble the one in the video
I'm getting this error in the concatenation part:
last_cols = [c for c in df.columns.unique(0) if c not in ['dollar_volume', 'volume', 'open',
'high', 'low', 'close']]
data = (pd.concat([df.unstack('ticker')['dollar_volume'].resample('M').mean().stack('ticker').to_frame('dollar_volume'),
df.unstack()[last_cols].resample('M').last().stack('ticker')],
axis=1)).dropna()
data
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index'
same here, have you solved it?
same error, did you get the solution?
@@madhurshinde4988 So far I did the following:
last_cols = [c for c in df.columns.unique(0) if c not in ['dollar_volume', 'volume', 'open',
'high', 'low', 'close']]
test = df.unstack('ticker')['dollar_volume']
test.index = pd.to_datetime(test.index)
test2 = df.unstack()[last_cols]
test2.index = pd.to_datetime(test2.index)
data = (pd.concat([test.resample('M').mean().stack('ticker').to_frame('dollar_volume'),
test2.resample('M').last().stack('ticker')],
axis=1)).dropna()
data
be aware however that this method drops certain stocks in my case thus I have an empty dataframe after a few transformations and can't continue, I believe it's because if any of the tickers don't have data for the entire period of time covered by the DataFrame, they will be dropped during the resampling process.
same problem
last_cols = [c for c in df.columns.unique(0) if c not in ['dollar_volume', 'volume', 'open',
'high', 'low', 'close']]
# data = (pd.concat([df.unstack('ticker')['dollar_volume'].resample('M').mean().stack('ticker').to_frame('dollar_volume'),
# df.unstack()[last_cols].resample('M').last().stack('ticker')],
# axis=1)).dropna()
#df.unstack(['ticker','date'])['dollar_volume'].reset_index('date').set_index('date').resample('M').mean()
data = (pd.concat([df.unstack(['ticker'])['dollar_volume'].reset_index('date').set_index('date').resample('M').mean().stack('ticker').to_frame('dollar_volume'),
df.unstack()[last_cols].reset_index('date').set_index('date').resample('M').last().stack('ticker')],
axis=1)).dropna()
data