- 18
- 435 957
AI Spectrum
Registrace 10. 10. 2021
How I get Tweet data for FREE in 2024 as a data scientist
Learn how to get tweet data for free! I’ll show you how to use Twikit, handle rate limit exceptions, save data in a CSV file, and compose complex search queries for Twitter scraping.
*UPDATE*:
To run the code, use *pip install “twikit==1.7.6”* to install the _twikit_ package. The latest update of _twikit_ deprecated the synchronous method which is used in the code.
Code on GitHub:
github.com/mehranshakarami/AI_Spectrum/tree/main/2024/Twikit
Resources:
- Twikit: github.com/d60/twikit
- Building queries for Search Posts: developer.x.com/en/docs/twitter-api/tweets/search/integrate/build-a-query
Who Am I?
Hi there! If you’re new to the channel, my name is Mehran Shakarami. I got my PhD in Applied Math, and I'm currently a data scientist in the aviation industry. In this channel I share videos about AI, coding and math.
Content:
00:00 What's up with Twitter?
00:32 Login to X.com
05:12 Search tweets
10:32 Save Tweets to CSV
12:38 Simulate human behaviour
15:06 Handle Rate Limit Exception
17:20 Complex search queries
*UPDATE*:
To run the code, use *pip install “twikit==1.7.6”* to install the _twikit_ package. The latest update of _twikit_ deprecated the synchronous method which is used in the code.
Code on GitHub:
github.com/mehranshakarami/AI_Spectrum/tree/main/2024/Twikit
Resources:
- Twikit: github.com/d60/twikit
- Building queries for Search Posts: developer.x.com/en/docs/twitter-api/tweets/search/integrate/build-a-query
Who Am I?
Hi there! If you’re new to the channel, my name is Mehran Shakarami. I got my PhD in Applied Math, and I'm currently a data scientist in the aviation industry. In this channel I share videos about AI, coding and math.
Content:
00:00 What's up with Twitter?
00:32 Login to X.com
05:12 Search tweets
10:32 Save Tweets to CSV
12:38 Simulate human behaviour
15:06 Handle Rate Limit Exception
17:20 Complex search queries
zhlédnutí: 4 970
Video
Get Access to the FREE ChatGPT API with Python
zhlédnutí 21KPřed rokem
Learn how to start implementing ChatGPT in your Python code. In this video, I explain all the steps to set up a reversed engineered ChatGPT API (revChatGPT), and use it in terminal or in Python code. Resources: - revChatGPT: github.com/acheong08/ChatGPT - Projects made with revChatGPT: github.com/stars/acheong08/lists/awesome-chatgpt Who Am I? Hi there! If you’re new to the channel, my name is ...
ChatGPT wrote this video about itself.
zhlédnutí 350Před rokem
I asked ChatGPT to write a script about itself, and this is the result. My prompts to ChatGPT: 00:00 - I say hi to the viewer and explain that the script was entirely generated by ChatGPT, in conversation language 00:31 - explain what is natural language processing in conversation language 01:06 - explain how NLP has evolved over years in conversational language, and don't say "Keep watching .....
The evolution of CAPTCHA
zhlédnutí 3,2KPřed rokem
Since 2000, CAPTCHAs have been annoying us online. But are they able to keep robots away? Let’s find out... References: - I'm Not A Robot ✅: czcams.com/video/o1zNIm8GVPY/video.html - Why captchas are getting harder: czcams.com/video/lUTvB1O8eEg/video.html - I'm not a robot: czcams.com/video/jCr6rNaZ9EU/video.html - reCAPTCHA: Tough on Bots, Easy on Humans, Great on Mobile - Google I/O 2016: czc...
Run Python in the browser with PyScript
zhlédnutí 1,4KPřed 2 lety
Now you can use PyScript and directly run Python in HTML. GitHub page of PyScript: github.com/pyscript/pyscript/blob/main/docs/tutorials/getting-started.md List of 3rd party modules: github.com/pyodide/pyodide/tree/main/packages PyScript demos: pyscript.net/examples/ Content: 00:00 - What is PyScript? 00:30 - Hello World 02:30 - Writing into HTML elements 03:34 - Importing 3rd party packages 05...
How to Install Python on Windows in 2022
zhlédnutí 806Před 2 lety
Learn how to install Python 3.10.4 on Windows 11, and run your first Python script.
This AI sees math | Python project
zhlédnutí 945Před 2 lety
I wrote a program that sees handwritten math and plots it with the help of AI. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Math_plotter
Get UNLIMITED Tweets by Python Without Twitter API
zhlédnutí 104KPřed 2 lety
Learn how to get twitter data with no limits, no API and no authentication. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/snscrape
Twitter Sentiment Analysis by Python | best NLP model 2022
zhlédnutí 92KPřed 2 lety
Learn how to accurately do Natural Language Processing (NLP) on twitter data, and use roBERTa model with python for tweet sentiment analysis. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Sentiment_Analysis roBERTa on Huggingface: huggingface.co/cardiffnlp/twitter-roberta-base-sentiment
Amazon Web Scraping with Python for Beginners in 2022
zhlédnutí 4,5KPřed 2 lety
Learn how to do web scraping with Python and Beautiful Soup on Amazon.com, and save the data into a CSV file. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Web_Scraping Content: 00:00 - Intro 01:56 - Get HTML code of Amazon.com by requests 04:50 - Get all best sellers in books 07:25 - Extract ranks of books 09:13 - Extract titles of books 13:21 - Extract authors of books...
How to Get Real-time Tweets by Python
zhlédnutí 18KPřed 2 lety
A tutorial on real-time stream of Tweets by using TWEEPY. I show how to, in real-time, get full text of tweets by tracking keywords/hashtags or by following twitter accounts. Code on GitHub: github.com/mehranshakarami/AI_Spectrum/tree/main/2022/Twitter_API How to get TWEETS by Python | Twitter API 2022: czcams.com/video/Lu1nskBkPJU/video.html Get Tweets by User and Hashtag with Python | Twitter...
Get TWEETS by User and Hashtag with Python | Twitter API 2022
zhlédnutí 35KPřed 2 lety
Get TWEETS by User and Hashtag with Python | Twitter API 2022
How to get TWEETS by Python | Twitter API 2022
zhlédnutí 145KPřed 2 lety
How to get TWEETS by Python | Twitter API 2022
Thumns up🫡
Bets video on the internet regarding this topic. You have no idea how much this helped me. 17:54 made me chuckle ahah
@aispectrum kindly help as I'm facing this error: 2024-09-10 16:22:49.316101 - Getting tweets... Traceback (most recent call last): File "C:\Users\user\Desktop\Generative-ai\main.py", line 55, in <module> tweets = get_tweets(tweets) ^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\main.py", line 18, in get_tweets tweets = client.search_tweet(QUERY, product='Top') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 660, in search_tweet response = self._search(query, product, count, cursor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 602, in _search response, _ = self.get( ^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 169, in get return self.request('GET', url, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\user\Desktop\Generative-ai\.venv\Lib\site-packages\twikit\client.py", line 152, in request raise Forbidden(message, headers=response.headers) twikit.errors.Forbidden: status: 403, message: ""
Garcia Amy Perez Robert Thomas David
1:24 can you explain why you imported all these things? I would like to learn this stuff but you're just saying what you're doing instead of explaining what it does or why you would want to do that.
Thank you sir
TypeError: AsyncClient.__init__() got an unexpected keyword argument 'proxy'. pls help me sir
Hello, when I try to run everything up to and including the client.search_tweets command, I get the following error: RuntimeError: 'run_cell_async' needs a real async loop An initial search was not helpful. Do you have any insight on this error? Also, could you please tell me how to search for tweets within a certain time frame (date ranges)? Thank you!
Hello, when I try to run everything up to and including the client.search_tweets command, I get the following error: RuntimeError: 'run_cell_async' needs a real async loop An initial search was not helpful. Do you have any insight on this error? Also, could you please tell me how to search for tweets within a certain time frame (date ranges)? Thank you!
Hello, when I try to run everything up to and including the client.search_tweets command, I get the following error: RuntimeError: 'run_cell_async' needs a real async loop An initial search was not helpful. Do you have any insight on this error? Also, could you please tell me how to search for tweets within a certain time frame (date ranges)? Thank you!
I am getting a different error after making some adjustments. RuntimeWarning: Enable tracemalloc to get the object allocation traceback Do you know how to resolve this issue? Thank you!
You're a life saver
can you make a script which will notify whenever the user posts a tweet ??
how can I handle this error with twikit " BadRequest: status: 400, message: "{"errors":[{"code":366,"message":"flow name LoginFlow is currently not accessible"}]}"
I need at least 3,000 tweets on #diabetes. please help me🥺. I can't do it.
Hello, have you been able to solve this?
Have you been able to run the script? Is there a problem with data collection or are you getting some error?
I'm getting a "keyError : Open account" error when login. Could anyone help ?
Me funciono. Gracias por tu tiempo, tienes nuevo suscriptor.
Thanks Buddy I first time visited you and this video helped me alot
not working packages\twikit\client\client.py", line 160, in request raise Forbidden(message, headers=response.headers) twikit.errors.Forbidden: status: 403, message: ""
did you find a solution to this?
I am facing a problem with the code: for tweet in tweets: TypeError: 'coroutine' object is not iterable sys:1: RuntimeWarning: coroutine 'Client.search_tweet' was never awaited I am using Pycharm and Python 3.10 ??? any idea
I have this problem
same
They have upgraded the api from version 2.0.0 we need to call the api as async operation if we are calling with out async we will get coroutine issue check out there documentation for more details
@@saikethanmadhadi1615 Any code so? can you provide us with the required code please to get tweets? Thanks a lot !
@@saikethanmadhadi1615 Have you any code please? if you have it please send it or post it here! Thanks
Keep it up man, appreciate the work
is there a maximum number of tweets that can be fetched? I set a minimum of 1000 but it stops randomly between 90-600
واقعا عالی بودش ؛ ممنون بابت اطلاعات مفیدتون
twikit.errors.Forbidden: status: 403, message: "" I am getting this error!
Can somebody please help me on this.I copied and pasted the code word to word and yet there is still a Attribute error occurring can somebody point out the issue: Traceback (most recent call last): File "C:\Users\ambai\PycharmProjects\Twitter\tweets.py", line 1, in <module> import snscrape.modules.twitter as sns File "C:\Users\ambai\PycharmProjects\Twitter\.venv\Lib\site-packages\snscrape\modules\__init__.py", line 17, in <module> _import_modules() File "C:\Users\ambai\PycharmProjects\Twitter\.venv\Lib\site-packages\snscrape\modules\__init__.py", line 13, in _import_modules module = importer.find_module(moduleName).load_module(moduleName) ^^^^^^^^^^^^^^^^^^^^ AttributeError: 'FileFinder' object has no attribute 'find_module'
im getting error!!! help me out please
To run the code, use *pip install “twikit==1.7.6”* to install the _twikit_ package. The latest update of _twikit_ deprecated the synchronous method which is used in the code.
thanks a lot for this nice video. I am facing a problem with this code: here is one of the first problems when doing it: for tweet in tweets: TypeError: 'coroutine' object is not iterable sys:1: RuntimeWarning: coroutine 'Client.search_tweet' was never awaited I am using Pycharm and Python 3.10 ??? any idea
when I use the twikit==1.7.6, I got this error: raise BadRequest(message, headers=response.headers) twikit.errors.BadRequest: status: 400, message: "{"errors":[{"code":366,"message":"flow name LoginFlow is currently not accessible"}]}"
@@laidkahloul322 Hey. I'm encountering the same error. Did you find a way of fixing it?
Hi Mehran I have the same issue twikit.errors.BadRequest: status: 400, message: "{"errors":[{"code":366,"message":"flow name LoginFlow is currently not accessible"}]}" please help us Mehran jun
Hey ! I get a "key Error : Open Account" when attempting to log in. Could anyone help ? Pls
I have to get the data of twitter nearly 1.000.000 tweets but these codes dont work it could you help me to get it?
Hello, did you find a solution for this?
this is very wonderful lecture keep it up bro you are doing wonderful job and this video is very helpful for me thanks for this
I tried to run this code to get the live stream tweets but now it shows an error ImportError: cannot import name 'OAuthHandler' from partially initialized module 'tweepy' (most likely due to a circular import) I tried a lot to solve this error but still I cannot find it. If you can help to solve this error, then it would be helpful.
Synchronous version was deprecated from version 2.0.0. Therefore the code in this video will no longer work and needs to be updated to asynchronous. You can find more information in the documentation or on the release page.
it works for my by installing the appropriate version: pip install “twikit==1.7.6”
I am getting this error when running the code- is there a way to fix it? TypeError Traceback (most recent call last) <ipython-input-18-6f730cd941a2> in <cell line: 59>() 72 break 73 ---> 74 for tweet in tweets: 75 tweet_count += 1 76 tweet_data = [tweet_count, tweet.user.name, tweet.text, tweet.created_at, tweet.retweet_count, tweet.favorite_count] TypeError: 'coroutine' object is not iterable
Me also!
No longer working
Not really for begginer. You just mentioned all the steps but without explaining what they do especially for begginers🙃
i was using API before it was free. But it cant compare with API data. scraping data is way too different.
Agreed! Before it was way easier to get tweets 🥲
Nice one man keep doing good 👍 Please will this work for streaming tweets I real-time for a specific region or will the rate limit affect it even after using the TooManyRequest class?
it says RuntimeWarning: Enable tracemalloc to get the object allocation traceback how can i solve it
Same. TypeError: coroutine object is not iterable. 'Client.search_tweet' was never awaited.
Apparently there is a new update that removed synchronous api. The easiest way to run the code is to install the package before that update. Try pip install “twikit==1.7.6”
@@aispectrum works now, thank you. You should mention it in description, before you get the "does not work" sentiment in the comment section 😃
@@aispectrum Code displays an error message when doing this: Forbidden: status: 403, message: ""
@@antiwars9889 I also got the same... anyone figured out a fix?
you need to randomise it a bit more, with a decision tree, like doing some likes, re-tweets, maybe commenting with "Awesome" or "I agree" etc. That would look even more human. Can this new library handle proxies?
Nice suggestion! This can help to avoid the ban. And yes, you can pass proxy while initializing Client, like ‘Client(proxy=“0.0.0.0:0000”)’
Finally bro got his account's password!!!!
awesome
Is scraping millions of tweets still possible because i would like to build an AI model using the text data. Please if it's possible how can I go about it?
Still not working
GOOD JOB BUDDY
I am stuck on "failed to refresh session", please help!
its ChatGPT, not GTP 😒
How to do that ?
Ausome ! As an accountant, i am interested in getting into Python for various analysis . please which book to learn from from scratch and comprehensively.
Thanks bro, that was خفن !
File "/Users/venusbhatia/twitter/tweets.py", line 1, in <module> import snscrape.modules.twitter as sntwitter File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/snscrape/modules/__init__.py", line 17, in <module> _import_modules() File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/snscrape/modules/__init__.py", line 13, in _import_modules module = importer.find_module(moduleName).load_module(moduleName) ^^^^^^^^^^^^^^^^^^^^ AttributeError: 'FileFinder' object has no attribute 'find_module'
This doesn't work anymore as I have tried doing it. Nevertheless, request you to post a video so that we can easily use Twitter data for Sentiment Analysis.
there is no unofficial file inside the folder :(
yah
hey install the 1.2.1 version it will be there