AI Spectrum
AI Spectrum
  • 18
  • 435 957
How I get Tweet data for FREE in 2024 as a data scientist
Learn how to get tweet data for free! I’ll show you how to use Twikit, handle rate limit exceptions, save data in a CSV file, and compose complex search queries for Twitter scraping.
*UPDATE*:
To run the code, use *pip install “twikit==1.7.6”* to install the _twikit_ package. The latest update of _twikit_ deprecated the synchronous method which is used in the code.
Code on GitHub:
github.com/mehranshakarami/AI_Spectrum/tree/main/2024/Twikit
Resources:
- Twikit: github.com/d60/twikit
- Building queries for Search Posts: developer.x.com/en/docs/twitter-api/tweets/search/integrate/build-a-query
Who Am I?
Hi there! If you’re new to the channel, my name is Mehran Shakarami. I got my PhD in Applied Math, and I'm currently a data scientist in the aviation industry. In this channel I share videos about AI, coding and math.
Content:
00:00 What's up with Twitter?
00:32 Login to X.com
05:12 Search tweets
10:32 Save Tweets to CSV
12:38 Simulate human behaviour
15:06 Handle Rate Limit Exception
17:20 Complex search queries
zhlédnutí: 4 970

Video

Get Access to the FREE ChatGPT API with Python
zhlédnutí 21KPřed rokem
Learn how to start implementing ChatGPT in your Python code. In this video, I explain all the steps to set up a reversed engineered ChatGPT API (revChatGPT), and use it in terminal or in Python code. Resources: - revChatGPT: github.com/acheong08/ChatGPT - Projects made with revChatGPT: github.com/stars/acheong08/lists/awesome-chatgpt Who Am I? Hi there! If you’re new to the channel, my name is ...
ChatGPT wrote this video about itself.
zhlédnutí 350Před rokem
I asked ChatGPT to write a script about itself, and this is the result. My prompts to ChatGPT: 00:00 - I say hi to the viewer and explain that the script was entirely generated by ChatGPT, in conversation language 00:31 - explain what is natural language processing in conversation language 01:06 - explain how NLP has evolved over years in conversational language, and don't say "Keep watching .....
The evolution of CAPTCHA
zhlédnutí 3,2KPřed rokem
Since 2000, CAPTCHAs have been annoying us online. But are they able to keep robots away? Let’s find out... References: - I'm Not A Robot ✅: czcams.com/video/o1zNIm8GVPY/video.html - Why captchas are getting harder: czcams.com/video/lUTvB1O8eEg/video.html - I'm not a robot: czcams.com/video/jCr6rNaZ9EU/video.html - reCAPTCHA: Tough on Bots, Easy on Humans, Great on Mobile - Google I/O 2016: czc...
Run Python in the browser with PyScript
zhlédnutí 1,4KPřed 2 lety
Now you can use PyScript and directly run Python in HTML. GitHub page of PyScript: github.com/pyscript/pyscript/blob/main/docs/tutorials/getting-started.md List of 3rd party modules: github.com/pyodide/pyodide/tree/main/packages PyScript demos: pyscript.net/examples/ Content: 00:00 - What is PyScript? 00:30 - Hello World 02:30 - Writing into HTML elements 03:34 - Importing 3rd party packages 05...
How to Install Python on Windows in 2022
zhlédnutí 806Před 2 lety
Learn how to install Python 3.10.4 on Windows 11, and run your first Python script.
This AI sees math | Python project
zhlédnutí 945Před 2 lety
I wrote a program that sees handwritten math and plots it with the help of AI. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Math_plotter
Get UNLIMITED Tweets by Python Without Twitter API
zhlédnutí 104KPřed 2 lety
Learn how to get twitter data with no limits, no API and no authentication. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/snscrape
Twitter Sentiment Analysis by Python | best NLP model 2022
zhlédnutí 92KPřed 2 lety
Learn how to accurately do Natural Language Processing (NLP) on twitter data, and use roBERTa model with python for tweet sentiment analysis. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Sentiment_Analysis roBERTa on Huggingface: huggingface.co/cardiffnlp/twitter-roberta-base-sentiment
Amazon Web Scraping with Python for Beginners in 2022
zhlédnutí 4,5KPřed 2 lety
Learn how to do web scraping with Python and Beautiful Soup on Amazon.com, and save the data into a CSV file. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Web_Scraping Content: 00:00 - Intro 01:56 - Get HTML code of Amazon.com by requests 04:50 - Get all best sellers in books 07:25 - Extract ranks of books 09:13 - Extract titles of books 13:21 - Extract authors of books...
How to Get Real-time Tweets by Python
zhlédnutí 18KPřed 2 lety
A tutorial on real-time stream of Tweets by using TWEEPY. I show how to, in real-time, get full text of tweets by tracking keywords/hashtags or by following twitter accounts. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Twitter_API How to get TWEETS by Python | Twitter API 2022: czcams.com/video/Lu1nskBkPJU/video.html Get Tweets by User and Hashtag with Python | Twitter...
Get TWEETS by User and Hashtag with Python | Twitter API 2022
zhlédnutí 35KPřed 2 lety
Get TWEETS by User and Hashtag with Python | Twitter API 2022
How to get TWEETS by Python | Twitter API 2022
zhlédnutí 145KPřed 2 lety
How to get TWEETS by Python | Twitter API 2022
Is being SELFISH (NOT) effective?
zhlédnutí 278Před 2 lety
Is being SELFISH (NOT) effective?

Komentáře

  • @curious-z5r
    @curious-z5r Před 13 hodinami

    Thumns up🫡

  • @user-xp3co1bi5k
    @user-xp3co1bi5k Před 20 hodinami

    Bets video on the internet regarding this topic. You have no idea how much this helped me. 17:54 made me chuckle ahah

  • @JeremiahChalle
    @JeremiahChalle Před dnem

    @aispectrum kindly help as I'm facing this error: 2024-09-10 16:22:49.316101 - Getting tweets... Traceback (most recent call last): File "C:\Users\user\Desktop\Generative-ai\main.py", line 55, in <module> tweets = get_tweets(tweets) ^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\main.py", line 18, in get_tweets tweets = client.search_tweet(QUERY, product='Top') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 660, in search_tweet response = self._search(query, product, count, cursor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 602, in _search response, _ = self.get( ^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 169, in get return self.request('GET', url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 152, in request raise Forbidden(message, headers=response.headers) twikit.errors.Forbidden: status: 403, message: ""

  • @DullesAlan-w1r
    @DullesAlan-w1r Před 2 dny

    Garcia Amy Perez Robert Thomas David

  • @i.o4766
    @i.o4766 Před 2 dny

    1:24 can you explain why you imported all these things? I would like to learn this stuff but you're just saying what you're doing instead of explaining what it does or why you would want to do that.

  • @ebukachuqz
    @ebukachuqz Před 2 dny

    Thank you sir

  • @BitterSweet-x8m
    @BitterSweet-x8m Před 3 dny

    TypeError: AsyncClient.__init__() got an unexpected keyword argument 'proxy'. pls help me sir

  • @MikeHarris-p1b
    @MikeHarris-p1b Před 7 dny

    Hello, when I try to run everything up to and including the client.search_tweets command, I get the following error: RuntimeError: 'run_cell_async' needs a real async loop An initial search was not helpful. Do you have any insight on this error? Also, could you please tell me how to search for tweets within a certain time frame (date ranges)? Thank you!

  • @MikeHarris-p1b
    @MikeHarris-p1b Před 7 dny

    Hello, when I try to run everything up to and including the client.search_tweets command, I get the following error: RuntimeError: 'run_cell_async' needs a real async loop An initial search was not helpful. Do you have any insight on this error? Also, could you please tell me how to search for tweets within a certain time frame (date ranges)? Thank you!

  • @MikeHarris-p1b
    @MikeHarris-p1b Před 8 dny

    Hello, when I try to run everything up to and including the client.search_tweets command, I get the following error: RuntimeError: 'run_cell_async' needs a real async loop An initial search was not helpful. Do you have any insight on this error? Also, could you please tell me how to search for tweets within a certain time frame (date ranges)? Thank you!

    • @MikeHarris-p1b
      @MikeHarris-p1b Před 7 dny

      I am getting a different error after making some adjustments. RuntimeWarning: Enable tracemalloc to get the object allocation traceback Do you know how to resolve this issue? Thank you!

  • @zoheirkabuli
    @zoheirkabuli Před 9 dny

    You're a life saver

  • @user-jy9zv6gn7s
    @user-jy9zv6gn7s Před 9 dny

    can you make a script which will notify whenever the user posts a tweet ??

  • @OmerHarbi
    @OmerHarbi Před 11 dny

    how can I handle this error with twikit " BadRequest: status: 400, message: "{"errors":[{"code":366,"message":"flow name LoginFlow is currently not accessible"}]}"

  • @faqapour6166
    @faqapour6166 Před 20 dny

    I need at least 3,000 tweets on #diabetes. please help me🥺. I can't do it.

    • @LolaErogs
      @LolaErogs Před 15 dny

      Hello, have you been able to solve this?

    • @aispectrum
      @aispectrum Před 9 dny

      Have you been able to run the script? Is there a problem with data collection or are you getting some error?

  • @AndrianjafiniainaTojonirina

    I'm getting a "keyError : Open account" error when login. Could anyone help ?

  • @MidasEstanislao
    @MidasEstanislao Před 24 dny

    Me funciono. Gracias por tu tiempo, tienes nuevo suscriptor.

  • @abdullahqamar4317
    @abdullahqamar4317 Před 27 dny

    Thanks Buddy I first time visited you and this video helped me alot

  • @krishnakhandekar7165
    @krishnakhandekar7165 Před 29 dny

    not working packages\twikit\client\client.py", line 160, in request raise Forbidden(message, headers=response.headers) twikit.errors.Forbidden: status: 403, message: ""

  • @laidkahloul322
    @laidkahloul322 Před měsícem

    I am facing a problem with the code: for tweet in tweets: TypeError: 'coroutine' object is not iterable sys:1: RuntimeWarning: coroutine 'Client.search_tweet' was never awaited I am using Pycharm and Python 3.10 ??? any idea

    • @faqapour6166
      @faqapour6166 Před 20 dny

      I have this problem

    • @devanshnandwani4900
      @devanshnandwani4900 Před 3 dny

      same

    • @saikethanmadhadi1615
      @saikethanmadhadi1615 Před 2 dny

      They have upgraded the api from version 2.0.0 we need to call the api as async operation if we are calling with out async we will get coroutine issue check out there documentation for more details

    • @laidkahloul322
      @laidkahloul322 Před 2 dny

      @@saikethanmadhadi1615 Any code so? can you provide us with the required code please to get tweets? Thanks a lot !

    • @laidkahloul322
      @laidkahloul322 Před dnem

      @@saikethanmadhadi1615 Have you any code please? if you have it please send it or post it here! Thanks

  • @matejpatek9395
    @matejpatek9395 Před měsícem

    Keep it up man, appreciate the work

  • @kachiokeke2773
    @kachiokeke2773 Před měsícem

    is there a maximum number of tweets that can be fetched? I set a minimum of 1000 but it stops randomly between 90-600

  • @MehdiMahan
    @MehdiMahan Před měsícem

    واقعا عالی بودش ؛ ممنون بابت اطلاعات مفیدتون

  • @prateekrustagi537
    @prateekrustagi537 Před měsícem

    twikit.errors.Forbidden: status: 403, message: "" I am getting this error!

  • @ambaigbaig9459
    @ambaigbaig9459 Před měsícem

    Can somebody please help me on this.I copied and pasted the code word to word and yet there is still a Attribute error occurring can somebody point out the issue: Traceback (most recent call last): File "C:\Users\ambai\PycharmProjects\Twitter\tweets.py", line 1, in <module> import snscrape.modules.twitter as sns File "C:\Users\ambai\PycharmProjects\Twitter\.venv\Lib\site-packages\snscrape\modules\__init__.py", line 17, in <module> _import_modules() File "C:\Users\ambai\PycharmProjects\Twitter\.venv\Lib\site-packages\snscrape\modules\__init__.py", line 13, in _import_modules module = importer.find_module(moduleName).load_module(moduleName) ^^^^^^^^^^^^^^^^^^^^ AttributeError: 'FileFinder' object has no attribute 'find_module'

  • @KalpanaNarayan-z7k
    @KalpanaNarayan-z7k Před měsícem

    im getting error!!! help me out please

  • @aispectrum
    @aispectrum Před měsícem

    To run the code, use *pip install “twikit==1.7.6”* to install the _twikit_ package. The latest update of _twikit_ deprecated the synchronous method which is used in the code.

    • @laidkahloul322
      @laidkahloul322 Před měsícem

      thanks a lot for this nice video. I am facing a problem with this code: here is one of the first problems when doing it: for tweet in tweets: TypeError: 'coroutine' object is not iterable sys:1: RuntimeWarning: coroutine 'Client.search_tweet' was never awaited I am using Pycharm and Python 3.10 ??? any idea

    • @laidkahloul322
      @laidkahloul322 Před měsícem

      when I use the twikit==1.7.6, I got this error: raise BadRequest(message, headers=response.headers) twikit.errors.BadRequest: status: 400, message: "{"errors":[{"code":366,"message":"flow name LoginFlow is currently not accessible"}]}"

    • @carlota2001bbb
      @carlota2001bbb Před 24 dny

      @@laidkahloul322 Hey. I'm encountering the same error. Did you find a way of fixing it?

    • @MaxMinExtremum
      @MaxMinExtremum Před 22 dny

      Hi Mehran I have the same issue twikit.errors.BadRequest: status: 400, message: "{"errors":[{"code":366,"message":"flow name LoginFlow is currently not accessible"}]}" please help us Mehran jun

    • @AndrianjafiniainaTojonirina
      @AndrianjafiniainaTojonirina Před 21 dnem

      Hey ! I get a "key Error : Open Account" when attempting to log in. Could anyone help ? Pls

  • @busracelik1165
    @busracelik1165 Před měsícem

    I have to get the data of twitter nearly 1.000.000 tweets but these codes dont work it could you help me to get it?

    • @LolaErogs
      @LolaErogs Před 23 dny

      Hello, did you find a solution for this?

  • @codebyheart5067
    @codebyheart5067 Před měsícem

    this is very wonderful lecture keep it up bro you are doing wonderful job and this video is very helpful for me thanks for this

  • @Abi2win
    @Abi2win Před měsícem

    I tried to run this code to get the live stream tweets but now it shows an error ImportError: cannot import name 'OAuthHandler' from partially initialized module 'tweepy' (most likely due to a circular import) I tried a lot to solve this error but still I cannot find it. If you can help to solve this error, then it would be helpful.

  • @拔
    @拔 Před měsícem

    Synchronous version was deprecated from version 2.0.0. Therefore the code in this video will no longer work and needs to be updated to asynchronous. You can find more information in the documentation or on the release page.

    • @MonicaRotulo
      @MonicaRotulo Před měsícem

      it works for my by installing the appropriate version: pip install “twikit==1.7.6”

  • @sarahm8627
    @sarahm8627 Před měsícem

    I am getting this error when running the code- is there a way to fix it? TypeError Traceback (most recent call last) <ipython-input-18-6f730cd941a2> in <cell line: 59>() 72 break 73 ---> 74 for tweet in tweets: 75 tweet_count += 1 76 tweet_data = [tweet_count, tweet.user.name, tweet.text, tweet.created_at, tweet.retweet_count, tweet.favorite_count] TypeError: 'coroutine' object is not iterable

  • @Mouhzero
    @Mouhzero Před měsícem

    No longer working

  • @pensychan8053
    @pensychan8053 Před měsícem

    Not really for begginer. You just mentioned all the steps but without explaining what they do especially for begginers🙃

  • @tuvshuutuvshuu4447
    @tuvshuutuvshuu4447 Před měsícem

    i was using API before it was free. But it cant compare with API data. scraping data is way too different.

    • @aispectrum
      @aispectrum Před měsícem

      Agreed! Before it was way easier to get tweets 🥲

  • @stanleychukwuemeka8841
    @stanleychukwuemeka8841 Před měsícem

    Nice one man keep doing good 👍 Please will this work for streaming tweets I real-time for a specific region or will the rate limit affect it even after using the TooManyRequest class?

  • @ahmetefekoroglu1563
    @ahmetefekoroglu1563 Před měsícem

    it says RuntimeWarning: Enable tracemalloc to get the object allocation traceback how can i solve it

    • @stanislavchernyshov7916
      @stanislavchernyshov7916 Před měsícem

      Same. TypeError: coroutine object is not iterable. 'Client.search_tweet' was never awaited.

    • @aispectrum
      @aispectrum Před měsícem

      Apparently there is a new update that removed synchronous api. The easiest way to run the code is to install the package before that update. Try pip install “twikit==1.7.6”

    • @stanislavchernyshov7916
      @stanislavchernyshov7916 Před měsícem

      @@aispectrum works now, thank you. You should mention it in description, before you get the "does not work" sentiment in the comment section 😃

    • @antiwars9889
      @antiwars9889 Před měsícem

      @@aispectrum Code displays an error message when doing this: Forbidden: status: 403, message: ""

    • @migvie3404
      @migvie3404 Před měsícem

      @@antiwars9889 I also got the same... anyone figured out a fix?

  • @HitAndMissLab
    @HitAndMissLab Před měsícem

    you need to randomise it a bit more, with a decision tree, like doing some likes, re-tweets, maybe commenting with "Awesome" or "I agree" etc. That would look even more human. Can this new library handle proxies?

    • @aispectrum
      @aispectrum Před měsícem

      Nice suggestion! This can help to avoid the ban. And yes, you can pass proxy while initializing Client, like ‘Client(proxy=“0.0.0.0:0000”)’

  • @siddhubhai2508
    @siddhubhai2508 Před měsícem

    Finally bro got his account's password!!!!

  • @junbeompark3239
    @junbeompark3239 Před měsícem

    awesome

  • @stanleychukwuemeka8841
    @stanleychukwuemeka8841 Před 2 měsíci

    Is scraping millions of tweets still possible because i would like to build an AI model using the text data. Please if it's possible how can I go about it?

  • @deepakgoel1388
    @deepakgoel1388 Před 2 měsíci

    Still not working

  • @kaicyrus0
    @kaicyrus0 Před 2 měsíci

    GOOD JOB BUDDY

  • @phamanhtuan-10a39
    @phamanhtuan-10a39 Před 2 měsíci

    I am stuck on "failed to refresh session", please help!

  • @MohammadAli-bo3xl
    @MohammadAli-bo3xl Před 2 měsíci

    its ChatGPT, not GTP 😒

  • @droptimistic7419
    @droptimistic7419 Před 3 měsíci

    How to do that ?

  • @DailyVibz
    @DailyVibz Před 3 měsíci

    Ausome ! As an accountant, i am interested in getting into Python for various analysis . please which book to learn from from scratch and comprehensively.

  • @mahmoodmohamadinezhad4316
    @mahmoodmohamadinezhad4316 Před 3 měsíci

    Thanks bro, that was خفن !

  • @venusbhatia
    @venusbhatia Před 3 měsíci

    File "/Users/venusbhatia/twitter/tweets.py", line 1, in <module> import snscrape.modules.twitter as sntwitter File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/snscrape/modules/__init__.py", line 17, in <module> _import_modules() File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/snscrape/modules/__init__.py", line 13, in _import_modules module = importer.find_module(moduleName).load_module(moduleName) ^^^^^^^^^^^^^^^^^^^^ AttributeError: 'FileFinder' object has no attribute 'find_module'

  • @bishnudas8638
    @bishnudas8638 Před 4 měsíci

    This doesn't work anymore as I have tried doing it. Nevertheless, request you to post a video so that we can easily use Twitter data for Sentiment Analysis.

  • @lamourpaspourmoi
    @lamourpaspourmoi Před 4 měsíci

    there is no unofficial file inside the folder :(