Informer attention code - FROM SCRATCH!

In this video, we code the prob sparse attention and compare it to time series attention
ABOUT ME
⭕ Subscribe: www.youtube.co...
📚 Medium Blog: / dataemporium
💻 Github: github.com/ajh...
👔 LinkedIn: / ajay-halthor-477974bb
RESOURCES
[1] Main repo: github.com/zho...
[2] Code for the colab notebook: github.com/ajh...
PLAYLISTS FROM MY CHANNEL
⭕ Deep Learning 101: • Deep Learning 101
⭕ Natural Language Processing 101: • Natural Language Proce...
⭕ Reinforcement Learning 101: • Reinforcement Learning...
⭕ Transformers from Scratch: • Transformers from scratch
⭕ ChatGPT Playlist: • ChatGPT
MATH COURSES (7 day free trial)
📕 Mathematics for Machine Learning: imp.i384100.ne...
📕 Calculus: imp.i384100.ne...
📕 Statistics for Data Science: imp.i384100.ne...
📕 Bayesian Statistics: imp.i384100.ne...
📕 Linear Algebra: imp.i384100.ne...
📕 Probability: imp.i384100.ne...
OTHER RELATED COURSES (7 day free trial)
📕 ⭐ Deep Learning Specialization: imp.i384100.ne...
📕 Python for Everybody: imp.i384100.ne...
📕 MLOps Course: imp.i384100.ne...
📕 Natural Language Processing (NLP): imp.i384100.ne...
📕 Machine Learning in Production: imp.i384100.ne...
📕 Data Science Specialization: imp.i384100.ne...
📕 Tensorflow: imp.i384100.ne...

[@sudlow3860]

Thanks for a very informative explanation. This seems like a bit of a step up in complexity from earlier videos, so I suspect some viewers of earlier ones might not make it to the end of this one. I think the Quiz answers are A B B. Presumably this Probsparse approach is useful in other situations (image processing springs to mind) as well as time sequences.

[@scott7948]

In the final video are you going show an example when you feed data into the model and the interpret the output. It would be good to see any prepressing of the data to get it in the right format to feed into the model. I'm keen to use this model for a timeseries forecasting exercise 8 timesteps ahead.

[@lalatendusahu9941]

Hey I am not able to access the links you have provided under Math courses and other related courses

[@Akshaylive]

@4:38 are you sure d_q is the number of total time steps? I think it's supposed to be the dimension of the query & key.