- 508
- 1 067 239
Python 360
United Kingdom
Registrace 25. 06. 2019
Regular Python videos, support my channel via Patreon:
www.patreon.com/join/3552586
► Automation - creation of documents (CSV, DOCX, PDF)
► EBAY API with Python
► Bitcoin
► ML and AI including chatGPT, LangChain and HuggingFace
► Google Cloud Platform Tutorials
► API integration + FastAPI
► Network Automation
► Web Scraping (Requests, Scrapy, BeautifulSoup)
www.patreon.com/join/3552586
► Automation - creation of documents (CSV, DOCX, PDF)
► EBAY API with Python
► Bitcoin
► ML and AI including chatGPT, LangChain and HuggingFace
► Google Cloud Platform Tutorials
► API integration + FastAPI
► Network Automation
► Web Scraping (Requests, Scrapy, BeautifulSoup)
Semantic Search for Real Estate - Vector Database (Qdrant)
Following on from the previous Rightmove web scraper video I have now added a semantic search capability using Qdrant vector database. (e.g Rightmove property listings often include semi detached houses even when you filter for detached).
Note : In my previous video I showed how to export JSONL format which means you need to use "lines = True" when you use pd.read_json
Web Scraping Rightmove - czcams.com/video/yfIFVVg_kvk/video.html
If you want a fast VPS server with Python installed check out :
🟩 webdock.io/en?maff=wdaff--170
Create a Simple Neural Search Service
---------------------------------------------------------------
qdrant.tech/documentation/tutorials/neural-search/
The workflow:
-----------------------
qdrant.tech/documentation/tutorials/neural-search/#workflow
Choose a pre-trained model :
-------------------------------
www.sbert.net/docs/sentence_transformer/pretrained_models.html#original-models
Run Qdrant in Docker
--------------------------------------
qdrant.tech/documentation/tutorials/neural-search/#run-qdrant-in-docker
Thumbs up yeah? (cos Algos..)
#webscraping #rightmove #pythonprogramming
Note : In my previous video I showed how to export JSONL format which means you need to use "lines = True" when you use pd.read_json
Web Scraping Rightmove - czcams.com/video/yfIFVVg_kvk/video.html
If you want a fast VPS server with Python installed check out :
🟩 webdock.io/en?maff=wdaff--170
Create a Simple Neural Search Service
---------------------------------------------------------------
qdrant.tech/documentation/tutorials/neural-search/
The workflow:
-----------------------
qdrant.tech/documentation/tutorials/neural-search/#workflow
Choose a pre-trained model :
-------------------------------
www.sbert.net/docs/sentence_transformer/pretrained_models.html#original-models
Run Qdrant in Docker
--------------------------------------
qdrant.tech/documentation/tutorials/neural-search/#run-qdrant-in-docker
Thumbs up yeah? (cos Algos..)
#webscraping #rightmove #pythonprogramming
zhlédnutí: 14
Video
Python Webscraper for Rightmove - 2024
zhlédnutí 57Před 2 hodinami
Rightmove listings often included semi detached houses even when you filter for detached. I've built a web scraper with filters to fix this.. # List of words to skip skip_words = ["semi - detached","semi detached","semi-detached", "shared ownership", "new home", "retirement","Auction", "Lodge", "terraced","terrace", "Semi - Detached"] # Check if the title or description contains any words to sk...
Build a Text Similarity Search App with Python & FAISS
zhlédnutí 106Před 7 hodinami
How to build a text similarity search web app using Flask, a pre-trained BERT model, and FAISS for efficient similarity search. A tutorial for mere mortals. No paid LLM API or 40GB LLM download required! Tutorial / Notes - redandgreen.co.uk/text-similarity-search-app-with-faiss/ai-ml/ If you want a fast VPS server with Python installed check out : webdock.io/en?maff=wdaff 170 Thumbs up yeah? (c...
How to Parse a List + Comparison v Rust
zhlédnutí 131Před dnem
How to extract just the floats from a list - a comparision using Python AND Rust code. If you want a fast VPS server with Python installed check out : webdock.io/en?maff=wdaff 170 Thumbs up yeah? (cos Algos..) #python #tutorial #pythonprogramming
Requests & Python | CERTIFICATE_VERIFY_FAILED
zhlédnutí 97Před 14 dny
How to avoid SSL "errors" when using Python and Requests on a server without an SSL cert. The certificate_verify_failed error occurs when the client cannot verify the server's SSL certificate. The error can be caused due to an expired or invalid SSL certificate, issues with the SSL certificate chain, or obsolete Python default certificates. import requests from requests.packages.urllib3.excepti...
5 Python tips | Python for beginners
zhlédnutí 52Před 21 dnem
5 must know Python tips including: list comprehension lambda unpacking and more.... If you want a fast VPS server with Python installed check out : webdock.io/en?maff=wdaff 170 Thumbs up yeah? (cos Algos..) #pythontutorial #tips #pythonprogramming
Single threaded v Multi threaded v Async Web Scraping
zhlédnutí 441Před měsícem
Single Threaded v Multi threaded v Asynchronous - comparison in Python - benchmarked using timeit urls = [ 'example.com', 'example.org', 'example.net', 'mockaroo.com', 'httpbin.org', 'books.toscrape.com', 'toscrape.com' ] The three types of multithreading models are many-to-one, one-to-one, and many-to-many. Threading can be easier to implement and understand, and it works well for I/O-bound ta...
Python Jobs - UK - June 2024
zhlédnutí 61Před měsícem
UK Python Developoer Jobs - Checking out job vacancies related to "Python" ~ June 2024 (UK) If you want a fast VPS server with Python installed check out : webdock.io/en?maff=wdaff 170 Thumbs up yeah? (cos Algos..) #uk #python #pythonjobs
My Programming Books - Top Books - Python, Rust, JavaScript, C
zhlédnutí 160Před 2 měsíci
Check out my thoughts and observations about my programming books that I've carefully chosen over the past few years. I buy one or 2 a year, so it's a big deal when I do buy one! Learn Python the Hard Way - Zed Shaw Python Crash Course - Eric Matthes Rust Crash Course - Abhishek Kumar and a few more! #pythonprogramming #programming #bestbooks
Using Pointers | C Programming
zhlédnutí 116Před 2 měsíci
A look at C programming - and pointers! Always good to have awareness of C no matter what your preferred programming language is! Buy Dr Pi a coffee (or Tea) ☕ www.buymeacoffee.com/DrPi #programming #tutorials #python
Troubleshoot Pytest VSCode "module not found" issues?
zhlédnutí 165Před 2 měsíci
Some set up tips and troubleshoot suggestions if you get "app module not found" when using VScode and Python. P.s Hold down shift when you drag and drop a file to get its path into where you want it, eg between the quotes! github.com/RGGH/mok Become a patron : 🌏 www.patreon.com/drpi Buy me a coffee (or Tea) ☕ www.buymeacoffee.com/DrPi If you want a fast VPS server with Python installed check ou...
Software Developer £££ Junior/Graduate Jobs | UK salaries 2024 | Can you afford to be one?
zhlédnutí 149Před 3 měsíci
Software developer jobs can be lowly paid at the junior level. Let's look at the requirements, the skills and qualifications required and consider alternative solutions. UK ONS Data for all jobs : - Average weekly earnings (AWE) were estimated at £677 for total earnings - £633 for regular earnings in February 2024. ~ source : www.ons.gov.uk/ ''' On a £35,000 salary, your take home pay will be £...
Python Jobs | April 2024
zhlédnutí 504Před 3 měsíci
Python Jobs | April 2024 (UK) - Jobs listed on Reed and Technojobs - a look at the skills required and the pay! Become a patron : 🌏 www.patreon.com/drpi Buy me a coffee (or Tea) ☕ www.buymeacoffee.com/DrPi If you want a fast VPS server with Python installed check out : webdock.io/en?maff=wdaff 170 Pytest with FastAPI course on "TestDriven.io": testdriven.io/courses/tdd-fastapi/? testdriven.io/c...
Recursion in Python - how to sum the difference between 2 lists
zhlédnutí 85Před 3 měsíci
Learn a simple technique for doing recursion in Python. I show the iterative and recursive solution in Jupyter Notebook and debug the recursion example in VS code. Check out the code and debug it here on PythonTutor : 🔗 bitly.ws/3huXW If you want a fast VPS server with Python installed check out : 🔗 webdock.io/en?maff=wdaff 170 code : 🔗 github.com/RGGH/recursion/tree/main/noteboobs Thumbs up ye...
Surrealdb & Python
zhlédnutí 305Před 4 měsíci
Learn how to use SurrealDB with Python using Jupyter Notebook and the SurrealDB Python SDK. The video shows also graph edges tables and how to use 'Let' to simplify SELECT queries. surrealdb.com/ Article on reaandgreen.co.uk: 🟢 redandgreen.co.uk/surrealdb-tutorial/surrealdb/ If you want a fast VPS server with Python installed check out : 🟢 webdock.io/en?maff=wdaff 170 Thumbs up yeah? (cos Algos...
Lazy Iterators v List Comprehensions in Python
zhlédnutí 237Před 4 měsíci
Lazy Iterators v List Comprehensions in Python
Stop blaming the language | RTFM and Chill out
zhlédnutí 78Před 4 měsíci
Stop blaming the language | RTFM and Chill out
Rye project and package management solution for Python
zhlédnutí 493Před 5 měsíci
Rye project and package management solution for Python
Image Similarity Search | Vision Transformer + Qdrant Vector Database
zhlédnutí 1,1KPřed 7 měsíci
Image Similarity Search | Vision Transformer Qdrant Vector Database
Learn LangChain - full tutorial | ChatGPT
zhlédnutí 1,1KPřed 11 měsíci
Learn LangChain - full tutorial | ChatGPT
Storing and reading prompts from database | LangChain OpenAI
zhlédnutí 921Před 11 měsíci
Storing and reading prompts from database | LangChain OpenAI
LangChain | Save tokens Caching & FakeLLM
zhlédnutí 458Před 11 měsíci
LangChain | Save tokens Caching & FakeLLM
OpenAI functions | control electronics
zhlédnutí 107Před 11 měsíci
OpenAI functions | control electronics
Save money with LangChain and ChatOpenAI
zhlédnutí 277Před 11 měsíci
Save money with LangChain and ChatOpenAI
Question Answering | LangChain and Vector Databases
zhlédnutí 1,4KPřed rokem
Question Answering | LangChain and Vector Databases
Let me know if you'd like to see a follow up video to this? - I can insert the scraped data into a vector database, so we can do semantic search, or I could do a more traditional insert into Sqlite? or PostGres?
Why does it display text out of the `sample_texts`?
That's just what I used to keep this video as simple as possible - to develop it properly you'd most likely use a vector database with "descriptions" and "full_article" - for instance if you were providing a semantic search on a collection of documents. I did one project before where it did a semantic search across youtube video transcripts if that's of interest too 👍
@@python360 But in your code did you have additional elements in the sample_texts list than what you show in the video or on your code page?
@@sammathew535 Yes, I added a few more whilst testing it, (apologies if I didn't make that clear)👍
@@python360 now it's clear. I would have tested it soon anyway. Thanks for the great work and sharing.
Thank You from the 'Distant Future' !
You are welcome!
Thank You. A useful review. Would you consider some vids on running everything locally via Ollama ?
Great suggestion, my laptop cooling fan kicks in and everything goes slow when I run locally. I may try on a Linux VPS again.Last time I tried that I had a message about CUDA and it wouldn't run. I could just do the vids on my laptop and edit at 4x or 8x speed! Thanks for the comment, will look into it!
kindly do an updated one of installation with regtest on mac
I don't have a MAC, but if you'd like to help test, we could sort something, I'd ask chatGPT to amend my current Linux script (for MAC) and go from there...
@@python360 This would be an honour sir...
@@python360 Also thank you for keeping your comment section lively and active
@ochiengodero2075 Thank you! I try to answer every comment, (I did miss some when I was rough with cv '19). I just thought, if you have a mac, you could run a VM on it with ubuntu for Bitxoin node btw. You'd just need to get the VM to use the network hardware for connection to internet.
"It's not easy is it?" -- You're telling me, I could have opted out!? Spent too many hours configuring the flask/ngrok backend
I now believe that it's only viable to jump through all the eBay hoops if you are a very motivated business owner coding it all for yourself, or you can spare a developer for a month to decipher their documentation. As a freelancer you couldn't make it pay!
Am on this video in 2024. I can just say amazing amazing
Ah thank you! - I am also using a mini think centre now too. (Running Linux Mint). I'll do an update in due course!
Thank you very much. That helped a lot.
Thank you for the kind comment!! 👍
Hello python360. It was a great video and i worked it. But i got a two problem. The first one is that, it extracts frames perfectly but the text extracted from frames are totally different. Like it extracting many symbols like (,''!;:"') . But these symbols are not present in the frame. It extracting the text but not accurate. Another problem is that, I downloaded Tamil and Hindi language file from github and paste it in exact directory. I also added Eng+tam+Hin in the code. But it's not working . It extracting only english language. What is the problem, can you please explain.
did the same but getting these errors: Traceback (most recent call last): File "c:\Users agha\OneDrive\Desktop\test\pycdemo.py", line 2, in <module> liObject = cdll.LoadLibrary('./cdemo.so') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users agha\AppData\Local\Programs\Python\Python312\Lib\ctypes\__init__.py", line 460, in LoadLibrary return self._dlltype(name) ^^^^^^^^^^^^^^^^^^^ File "C:\Users agha\AppData\Local\Programs\Python\Python312\Lib\ctypes\__init__.py", line 379, in __init__ self._handle = _dlopen(self._name, mode) ^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: [WinError 193] %1 is not a valid Win32 application
what does <b>python3 ytapithumbget.py<b> mean?
what does <b>python3 ytapithumbget.py<b> mean?
sir i did a code in jupyter notebook how do i transfer this code to html to be protrayed
If you use Jupiter 7 it's an option when you save now👍
Thank you for `python -m pytest` explanation.
Thank you!☺
shot thanks! perfectly simple video, let's see how it goes for me.
Thank you!!👍
Hi @python360, I was wondering if you knew how to pull shipping label cost information. I've been struggling with this. For some reason UPS labels show up in transactions but not USPS.
I don't I'm afraid, I'd maybe look to see if USPS have an API - might be simpler (asuming you know the exact weight/size ) ?
Hi, thank you for the great video. How can i select all the elements that i found with ctrl + f ?
Thank you !! Is this what you need ? ~ While on a search result, press Alt + N to add a cursor at the current occurrence? 👍
Nevermind, i found it , it is the setting: set hlsearch. This is set to off in default
Great instructions, thanks! To be clear, the email of developer account can be different then the email of the normal ebay account?
Thank you! Yes. 100% The developer account and normal account are totally separate, and so can email addresses be different (I know this as I use a dedicated email for developer) 👍
Прикольно, прикольно, но я бы подтянул синтаксис Python, а то функции вызывать забываешь))))
Ah, yeah, thank you - good point - thanks for pointing it out!
I have a question as I came across your video. First, thank you for your content as it is very helpful. Is there another method to extract the text from a video without taking the snapshot or picture every 4 second?
Hello! Thank you for your kind comment. You could maybe adjust the timing to say 3 seconds, or 2 seconds or whatever works, but using this method there will need to be some sort of capturing of the picture in order to do the text recognition. Perhaps there will be an AI product that can do it as the video plays, but using Tesseract and Python, this is the only way I know of. If someone else reads this and can advise then that'd be neat!
@@python360 awesome, thank you very much for your feedback. Greatly appreciate it.z have a good one!
@@hammadh2162 No problem, cheers Hammad, good luck with your project!😄
u r dressed like you are gonna fly a fighter jet in a minute
You're right. That's my day job. Python stuff is for lols 😎
where is the next part ?
I once made a video on pointers, no one has watched that)) But the topic is so basic and so important.
The CZcams algorithm seems to be very harsh on smaller channels - I made this last year and didn't dare make it public in case it lost viewers/caused unsubscribes. I'm past worrying about that now., "Que Sera Sera"
This is gold! Thank you so much for putting this together. Much appreciated. I would like to set up a basic application to pull my listings from ebay and push to another platform. Any suggestions on where to start? Easiest language to use, etc? Feel free to respond to me as a 5-year old LOL
Thank you for the good comment! Python will be the easier language to use. You can experiment getting data with the different API endpoints once you have your "developer key" - the authentication is quite complicated, and you'll need to get a new token, every 2 hours. The ebay developer forum is the best place for problems/tips.
@@python360 thanks again! Will definitely check out the forum and go the Python route to start. Thank you
It can be laborious, but if you follow their documentation eventually the mists clear! I've not used any API that uses X-auth as that's even more fiddly, certain API's need it, and some don't, so try find the ones that dont (the docs will state if x-auth is needed) while you're starting out 👍
since app is one directory up from tests/test_bcd.py, how can you import it such as from app.fetch_www import parse? This would need to go one directory up first?
. ├── app │ └── fetch_www.py └── tests └── test_bcd.py have you got the structure as above? (have you got tree installed, it's great for this kind of stuff)
Yes just like this.. even added __init__.py files in app/ and tests/ Been banging my head on this one. I'm working on a mac in vs code
I typed it all back in as I'd not kept the project, I tested it and it works ok : github.com/RGGH/mok One tip : specify your python version if you are doing it from command line : eg : $python3.10 -m pytest
@@python360 I saw this in the repo Add the project root directory to the PYTHONPATH # Replace '/home/rag/Documents/python/mok I didn't do this, that could be it no?
@@mytrustysteed8476 Yes, PATH can be a problem - if you use "python3.10 -m pytest" or whatever version of python you use then this should work. You can run "which python" and "which pytest" from terminal to check the paths.. See this : g.co/gemini/share/b76c08a4f516 PATH is a pain, but as it works on my set up then, check PATH, make sure you have __init__.py and also you can drag the file icon from VSCode file explorer, and hold down shift and drop it in between quotes when trying to get the right path into your python code.
Hi when running the tests I get error "modulenotfounderror no module named app" . I know there is something wrong with the relative imports? but I followed your structure exactly
ah ok, are you on windows or Linux/mac ?
try : python3.10 -m pytest see also : github.com/RGGH/mok clone it and see it it works?
What if you want to persist data? How can you set this up without marketplace notification exemption?
I'm not sure, other than if you maintain 2 database tables and check and store the differences?
10 years experience? Having a laugh
100% It makes a mockery of the whole thing! Better to avoid the recruitment circus if at all possible.
@@python360 absolutely. I'm in my first year of industry and feel absolutely blessed to not have to go through this hell! What a shit show.
Thank you for the video! Some of those salaries are VERY low considering living in the UK, how could someone lives there with that? Doing another video for graduates would be much appreciated, thank you again.
Yes, since the money printing/debasement during the pandemic the cost of living has increased way more than salaries. (Politicians now try and blame other things as if we don't remember). Thank you for the suggestion, I'll do some research and then do one on graduate/junior-mid level positions.👍
Hi thanks for the videos, I’m stuck on registration, I got the verification email and then was notified that my application was rejected… please advise? I’ve tried several different emails with and without vpn
Ah. I must have been lucky then. I had to wait approx 3 days but was approved eventually. Are you able to contact them and ask why? Fyi, I used a protonmail email address.
Very easy to follow and nice crisp content. This is what I was looking for. Thank you.
Ah brilliant. Thank you so much for letting me know. 👍
Is it possible to apply for jobs in UK as remote worker from other country?
Very good question! "it is mostly legal to work remotely from a different country from where your company is based. However, there is no one-size-fit-all approach" iasservices.org.uk/is-it-legal-to-work-remotely-from-another-country/
It would mostly depend on the tax system i.e. you pay income tax to the UK gov or your local gov? If you aren't working for a company that deals with international teams and has a local base it could be an issue, particularly with your local government proving you are paying your dues properly. So your first step is asking your government tax office to see how the rules are laid out.
Taxes, taxes, taxes...
@@scienc3tv (Thumbs up is me agreeing, not me in favour of taxes)👍
I need a code man.
A code?
@@python360 python code
@@python360 sorry, I need the python code which is used in this project. can you provide me?
I need help with getting 1xbet database for all live games. How can I do this
Have you tried ScraperAPI already?
Thank you so muchhhhh ! This was incredibly helpful
It's a pleasure. Really pleased it helped! 😊
הסרטון הכי מובן שראיתי בנושא הזה🥰
great video. Thanks
Thank you!
great tutorial, if your still interested in airport live stream there are 2 good channels to watch. they are Pomfus and Mark and SydSquad. Both are in Australia
Cool, thanks, I'll check them out!
Sounds interesting! Wonder if combining it with Proxy-Store's proxies would make the scraping even smoother. Thoughts?
You could, or scraperAPI offer smart proxy rotation also.
will check them out! thank you!@@python360
How can I create a hyperlink in a word document using python-docx?
I can't remember, (its v late in the UK) but the alternative would be to use 'python-docx-tpl' and do it that way. Use pip install and then add your own variable placeholders using jinja syntax.
Im trying to recreate from zero a complete template with python-docx like a resume filling its content, the body is ok but the problem is that i need to add in the header section an image like a background to cover all the header and text is front of this image. How can i achieve this?
You might be better using "python-docx-template" pip install docxtpl - so that you can start with a template file with an image already present that you put in once, manually and then reference that template and use the corresponding jinja syntax where you need to modify paragraphs etc
@@python360 thanks for the response. Im seeing your videos and i have this problem: as i see with python-docx-template i can replace names with curlys but in my case there are no curlies and also i have some bullet list as sample that i dont need It. My template IS a resume as you know there IS a section "skills" with these bullets sample, i need to eliminaré that list and populate with my own list of skills and also i have a work experience section list with position, dates, company and descripción (same with sample that i don't need It). Also the header section mentioned in the last comment. What can you recomend or It IS posible to work all these with python-docx-template instead DOCX?
@@henryvaldivia3355 Yes, so "Template" means "The word document you want to use with some variables" in this case? The word "template" in "docxtpl" is just a standard word/docx file, it's not a template file in the microsoft sense. This means you create your word doc as you would manually, and then add the jinja curly braces only where you need a variable? Forgive me if I not understood correctly, but I'd go with pip install docxtpl rather than pip install docx
Hello, just wanted to ask during the marketplace deletion/account closure notification process, where can I find the verification token and
This is the easiest and clearest explanation that i've seen so far!! It helped me immensely, thank you so much!!!
Thank you so much for your comment, this is really good to hear!!
Great tutorial, sir. But it would be great for me to hear about matadata-based text chunks filtering using langchain + qdrant! :) Liked and Subscribed : )
Noted! Thanks for the subscribe and comment sir. 👍
I'm having trouble installing tabula-py, an error pops up in JupyterLabs. Help?
Just what I needed, excellent video! Thank you.
Excellent. Thank you for the kind comment!
thanks for the video... if you want it to have a specific format do you just put the variable in that format like {{ variable }} has font courier, bold, size 14 while other items in whatever format you wish? also could you dynamically adjust a table like this too with a template variable...combining this concept and what you described in another video?
Thanks for the comment. Yes to your 1st question, andas for the table width, I don't know, I'll do some tests and maybe make a video if I can - maybe try this : stackoverflow.com/a/43053996/12670189
@@python360 thanks. I tried it out yesterday... it worked out just fine, but I did {{variable}} to avoid spacing issues. I also did it using pandas and used a ms excel file vs csv format file... that was good so I could have the input use drop down lists and such. Also I noticed the pdf conversion library needed office installed. As I do most of my development in wsl, I wasn't sure how to install it. I found another library that is convert2pdf. it required libreoffice installed, which seemed to be ok. it just had 2 snags 1) It is verbose about the conversion 2) my footer font size got adjusted. I emailed the developer to see if there is a workaround. if you know how to install ms office in a headless way in Linux, then I can go towards your referenced pdf conversion library instead.
you are an absolute madman
sir i have database in myworkbench , can i connect database with googlepalm model through sqldatabasechain? while trying to connect this error occurs :" cannot import name 'SQLDatabaseSequentialChain' from 'langchain.chains' (/usr/local/lib/python3.10/dist-packages/langchain/chains/__init__.py)"
Great topic, but your microphone sucks