Forecasting with the FB Prophet Model
Vložit
- čas přidán 3. 07. 2024
- In this video I show how you can use facebook's prophet model to easily do time series forecasting in python. This model is very powerful because it uses an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects.
Kaggle Notebook: www.kaggle.com/robikscube/tim...
Forecasting with Machine Learning Part 1: • Time Series Forecastin...
Forecasting with Machine Learning Part 2: • Time Series Forecastin...
Timeline:
00:00 Intro
00:50 Data and Imports
03:43 Features and EDA
05:54 Test Split
06:49 Train and Predict
10:01 Evaluate Forecast
15:01 Adding Holidays
19:19 Make Future Dataframe
Follow me on twitch for live coding streams: / medallionstallion_
My other videos:
Speed Up Your Pandas Code: • Make Your Pandas Code ...
Speed up Pandas Code: • Make Your Pandas Code ...
Intro to Pandas video: • A Gentle Introduction ...
Exploratory Data Analysis Video: • Exploratory Data Analy...
Working with Audio data in Python: • Audio Data Processing ...
Efficient Pandas Dataframes: • Speed Up Your Pandas D...
* CZcams: / @robmulla
* Twitch: / medallionstallion_
* Twitter: / rob_mulla
* Kaggle: www.kaggle.com/robikscube
#fbprophet #python #machinelearning - Věda a technologie
Your videos are pure gold, thank you.
Thanks a lot, I love your forecasting content, especially with that dataset!
Would love to see some more models in the future. Wish you the best!
Thanks! I plan to do more at some point. Glad you've been enjoying the ones I have so far.
Hey Rob, another amazing video, and great overview of Prophet. There are a couple of time series projects i want to tackle and I am going to take a crack and them using Prophet. Just want to say that I think this video is going to help me out a lot. Thanks for the effort you put into making these. Please keep them coming, they are more useful than you know.
Thanks! As many people mentioned there may be better alternatives to prophet but hopefully this tutorial gives you a good idea of where to start.
Great work... I am actually working on a comparative analysis of ARIMA, prophet, and xgboost in time series forecasting for my project
I'm learning a ton, thanks for the great content!
Glad you are learning from them! Thanks for watching.
Surprisingly close. I watch your videos and everything is clear.
Glad you found it clear and helpful.
Amazing video Rob. I hope you continue making these videos and sharing your knowledge. You are also a great teacher.
Glad you learned from the video. I plan to keep making videos when I have time. Hopefully more time soon.
@@robmulla WHAT DOES model.make_future_dataframe(periods=3, freq='H') mean ? does it mean i am doing forecasting for the next 3 hours
Thank you for sharing the knowledge, this helps me a lot
Glad it was helpful. Thanks for watching.
Me encantan tus vídeos. Eres un excelente profesional en tu campo. Gracias por compartir tus conocimientos con la comunidad de CZcams :)
¡Muchas gracias! No hablo español pero estoy usando un programa de traducción. Me alegro de que puedas entender mis videos a pesar de que está en inglés.
I see Rob has new video about time series forecasting. I put like and comment automatically. That's how it works! 😉
Thanks Maqsud!
Great stuff Rob ! thanks
Glad you like it Badro!
thanks! great content as usual :)
Glad you liked it!
thanks teacher , greettings from Chile, gracias profesor
Excellent video!
I wish I would have come across this channel months ago, huge fan, amazing work.
Awesome work!
You made me interested in ML.
I'm PhD in Physics, and I always preferred to avoid ML during University days, but you, and your work made it so interesting for me!
Thanks for your work.
The best thing about machine learning is this... Once you've built your models, you can just train them on different data. I have a single notebook which loads in my data and then the rest of the notebook contains all the models I have learned along with their cross validation and evaluation metrics. I can just change the data and run the entire notebook and then check on it occasionally to see the results. It can take an hour or two to build each model but once you have them all built, you're good to go and you can just add to the notebook as you learn more. You may even want to split your models into separate notebooks but i prefer to run them all in one as i then only need to alter the data in that one.
Thank you very much this is great work and very useful i hope just make new video about LSTM model
Thanks a lot from Bangladesh
great tutorial
Simply love..
Thanks 😊
Like the explanations
Thanks!
Nice!!! Thanks!!
Thanks for watching Kavin!
Video is awesome as always! Quick, easy to follow and interesting. Thank you! btw, where did you get that t-shirt?
Glad you liked the video. The Tshirt is actually from Marques Brownlee's merch! I saw him wearing it in one of his videos and had to get it.
Hey Rob, Awesome Video.
I think you need to take note that if you try to run your kagle notebook with a different data set, it's impossible to evaluate the error metrics because the y_pred array is the same size as the data set. I had to write a few more lines to extract just the last x values I needed.
Thanks again
did you split your data right? Your X_train and y_train should be the same length. and your X_test and y_test should be the same length. And if that's not the problem, then ensure you are evaluating the metrics using y_test and y_pred and not y_train and y_pred?
THE KING IS BACK!
Haha. Thanks Pedro.
Good job! Nive vid!👏
Extra like for the “Model Train” meme ❤
It’s a good one 😃
Hey rob nice video!!
Could you please have another video about using panel datasets and implement AB testing as well.
Great video, thanks for letting us learn this model!
I have a question. At minute 10:13, the forecast interval includes negative values for the dependent variable: how is it possible? and how can we prevent it?
thx!
Hello Rob, great video, thanks for sharing! This was my introduction to the Prophet model, and loved your explanation, crystal clear sa always!
I have a quick question though: if I understand correctly, Prophet should work as a regression model given this very problem, and hence we could calculate R2 as well as metric along with RSE, MAE and MAPE. Is it recommendable to do it, or it makes no sense for this particular model? What's your take?
Thanks a lot!
Thanks for the explanation. When using the xgboost model, can we include holidays in the model using the method you showed in this video?
What is the default number of periods predicted?
very nice new model an great presentation! How long did it take you to programm the nb from top to bottom (roundabout)?
Thanks! It didn't take too long, maybe an hour or so, but I had already created a similar one a few years back.
Hello, in which python version can we run the prophet library? I'm developing on Anaconda-Spyder but I get different errors each time.
Hi Rob, thnks fro great video. Why did you add exrta features to data like hour, day of week? You did not use them as a regressor. They are already embeded in date. I didn't understand why you did add like that.
I am preparing for Quant researcher (new hire) this help a lot . Can we get series of vedios for preparation for quantitative research interviews it will help .
Glad you are finding these helpful. I'm not a quant myself so I might not have the best insight into interview questions but if you have some resources you can point me to I might be able to make a video out of them! Thanks for watching.
Have you published the LSTM video? It is surprising that the initial XGBoost model with held-out test dataset performed the best given the simplicity and minimum effort.
Well I'm curious to see LSTM with multi variant and then we can actually do understand more about how other factors impact, example temperature of the day and climate change anakysys
Hey Rob,
Which algorithm you think is good for forecasting power Hourly data?
XGBoost performs best on this dataset (check out my other video here: czcams.com/video/vV12dGe_Fho/video.html) But Prophet would probably perform better on something like sales or exchange rates.
Hey rob, in general, which TS model would you recommend as best for forecasting(not considering the MAPE values) for univariate dataset?
Something that is consistently reliable.
Thanks for the question. I think it really depends more on the data and the information around it than the model. No model is perfect, but picking the correct one depends a lot on what you are trying to achieve- explainability, accuracy, etc. Sorry for the non-answer but if it was that simple everyone would just use one model.
@@robmulla Sure, but what if one model outperforms and older better performing model with new data being added?
I’m new into TS forecasting and trying to build an approach towards how to make the process efficient.
Would you suggest that one always runs all known models to assess which one is most accurate in the latest test or settle with one model which is consistently more accurate?
Need your help. I do not have time components in my datasets, i only have date and the consumption data...how to choose the right model for this.. should i choose Xgboost or prophet or Arima...
Is it possible to give a neural prophet lecture?
First of all, thank you for this nice video!
I have a question why you created the features_and_target dataframe. I know that the features are helpful when we train models like XGBoost, but here we only used the ds and y column. I'd just like to understand what was the reason why you created it and if there are any ways to use this dataframe with Prophet?
Thank you for the informative video. however there are two aspects which are puzzling ( am new to pandas and prophet, so please be indulgent) first, once you trained your model you perform: pjme_test_fcst = model.predict(pjme_test_prophet). And it is puzzling why you are passing pjme_test_prophet. Test data are not used to perform any error measurement against predicted data, my speculation is the only column taken into consideration is the 'ds' column, used to specify what predictions we want and not the 'Y' column. am I correct?
Moreover looking at yhat_lower and yhat_upper for 2015-01-01 01:00:00, meaning one hour after our training data finish, the predicted values are 23737.947439 32670.335550, a range of about 10K, which is about 50% of yhat_lower and about 30% of yhat_upper which would make for a pretty bad model if it needs such a big interval of confidence; if I understood correctly, did I?
is it necesary to make circular the time variables? as a loop for the 24 hours, or the 7 days of the week, the 12 months and the 365 days of the year
Wonderful. Can you do one tutorial one Neural Prophet as well, lots of thanks
Hey Rob! I am getting an error when fitting the model. It appears that Prophet wants the 'ds' column to be in the datetime format, but it's in the datetime64[ns] format .... I can't figure out how to correct this - any ideas?
I found this stackoverflow post about it. It looks like it could be related to your datetime formatting: "Solution: Change the date format in training data to "%Y-%m-%d""
stackoverflow.com/questions/64427395/facebook-prophet-future-dataframe
Hi Rob, I wanna ask you about: How safe is use Prophet Forecast with a business data? I mean, some business have a strong data policy and I'm not sure if using Prophet could break it. Is prophet in some way collecting some part of data?
With the Facebook's background about data management I think it's a better idea to ask before.
Oh. Interesting question. I think you can still keep everything local. It’s not like Facebook will see your data they just developed the code. But also check with your companies guidelines. I wouldn’t want you to get in trouble. Good luck!
I am facing challenges running the notebook on google colab especially on the Time series features section ,please help anyone
Hey, so i am getting yhat as negative since in my data the trend is that number of user decreased in last 4 months, what should i do?
Hello robert!
What about if you want to predict for multiple cities at the same time? (Having more than 1 entrie per time (for each city for example))
That gets tricky. You would need to train a model for each city, the model only can be trained on a single feature.
@@robmulla thanks!!
Hi Rob, thanks for sharing this. One question - what if i have many products to forecast sales, do i need to train/run model for each product? Or i could do this in one shot meaning ONE model to cover all products?
I guess you have longitudinal data and in this case you have a collection of time series, not one. In that case you can build a model per product, or use a global model. See M4 / M5 competitions. It is greatly harder to forecast from such data. If I misunderstood, ignore me.
You're right, it looks like panel data.
Just started with Prophet to model dispensing figures based on previous data but having problems, any help would be appreciated...Thanks Nick
Great Video! When I try to filter my Date Range for when you're doing "January 2015 Predicted vs Actual" I get an error like
Failed to convert value(s) to axis units: array([....]) -- Any idea what I could be doing wrong here?
Did you fix this Error? I have the same one
when I try to install fbprophet on my local machine, I get an error saying "ModuleNotFoundError: No module named Cyphon"
Hi Rob, great video I am wondering if you stopped making machine learning videos as this video was 1 year old
can you do with neural prohet?
I recorded a video but it wasn't good enough to release. Maybe at some point I'll re-try.
Hi Rob, I got a Ph.D in economics and just started Data Science courses. Could please help me to make choice as I am looking for the 5 BEST Machine Learning and Deep Learnin algorithms that I can master to be a Forecasting specialist.
Hey Gohou. Thanks for watching. I’d need to think that through. Certainly a component based model like prophet and tree based model like xgboost would be on the list. Deep learning models are still at the point where I haven’t seen them outperform the others. But things in data science change daily and I’ve heard some big companies use transformer architectures in forecasting.
when I ran the kaggle notebook Its showing dt is not defined which package is defined as dt? I searched on google it shows pandas series but exactly what is declared as dt?
Great question. dt allows you to apply datetime methods to a datetime type column or series. More info here: pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.dt.html Hope that helps!
@@robmulla sorry Rob I am a begginer I couldn't understand this I tired to import pandas.series.dt but it shows there is no module named pandas.series I am using Google Collab please help me tackle this problem
@@athithyabalasubramani1045 Not sure exactly what you are trying to do by importing it directly. These are just methods that you can apply to an existing series. You don't need ot import it directly.
Awesome video !! However, somehow I am getting the following error:
fig, ax = plt.subplots(figsize=(10, 5))
ax.scatter(datos_test.index, datos_test['Hs(m)'], color='r')
fig = model.plot(datos_test_forecast, ax=ax)
ax.set_xbound(lower='01-01-2019',
upper='02-01-2019')
ax.set_ylim(0, 60000)
plot = plt.suptitle('January 2019 Forecast vs Actuals')
ConversionError: Failed to convert value(s) to axis units: '01-01-2019'
Isn't prophet deprecated? I thought they switched it up to Neural Prophet which seems way better.
I’ve tried neural prophet but still think there is benefit to learning the OG version. Deep learning isn’t always better and can be less explainable. Read some articles where each can be better depending on the dataset.
towardsdatascience.com/prophet-vs-neuralprophet-fc717ab7a9d8
Is it?😅
kind of?
i dont know but when having this data is very predictable , tuning these models can be done, but the more real world application majority of these models fail... we need to keep feeding new data, new features and yet the model misses.
puting exampes everywhere like this, I think they are just bread crumbs to feed our imagination or desire or to get people excited but very quick you hit the wall and realize that this forcasting anything is a taugh task...
Actually i am getting error in installing fb prophet as it says building wheels for it was unsuccessful. Please solve this i have a very important project for my carrier.
Im using python version 3.9.13, im also unable to install fbprophet in my local machine, and in kaggle notebook too, fbprophet is not working.
@@arnabmukherjee3129 I'm also getting lots of error message installing pystan and fbprophet in jupyter notebook. There are 21261 lines of output! I don't know where to start and how to read any of the warnings and errors.
Ts forecasting never been accurate...if yes, everybody has been a billionaire by trading time series data like equity market, ficc etc...not much practical use
Wrong mindset. "All models are wrong, some are useful".
I was also going to respond with the "all models are wrong" quote, but Gerald beat me to it. Especially with forecasting you need to first ask yourself what the end goal is. I also think in most cases a human in the loop is still required, so an explainable model can be very helpful. But I totally agree that trying to predict something like stocks at large time horizons is a fool's errand.
@@robmulla so what value are we getting from ts models? Telling us something we already knew If no predictability? If it can't do a better job, why to hire expensive data scientist to do a job which a high school graduate can perform using Excel
Prophet is just 1 tool from the toolbox. It is not the whole toolbox. Knowing how to talk doesn't magically make you an orator. You need other skills
Excellent. But graphs are extremely awful! Can you improve them?
Music sucks.