Make Data Useful
Make Data Useful
  • 75
  • 946 268
Meta llama 3 unexpected results! 100 prompt test
Download ollama from ollama.com/
Learn more about metas llama models: huggingface.co/meta-llama
This video is sponsored by ME! consider subscribing.
In this tutorial I put Llama 3 to the test! Have you ever wondered how often AI models like Llama3 get the right answer? In this video, we're going to find out by asking Llama3 the same question 100 times and analysing its responses.
I'll guide you through the code used for this experiment step by step, so you can understand how it works and even try it out for yourself. By importing the ollama library and setting up a multiple-choice question about the location of a cake, we'll interact with Llama 3 repeatedly to see if it consistently selects the correct answer.
Throughout the tutorial, you'll learn:
How to set up and structure your code for interacting with Llama3.
Strategies for analysing and interpreting the responses given by AI models.
Tips for conducting your own experiments and evaluating AI performance.
Join me as we dive deeper into the world of AI testing and discover just how accurate Llama 3 can be!
Don't forget to like, share, and subscribe for more Python, web scraping, data and AI experiments and tutorials.
zhlédnutí: 2 265

Video

Shocked at the results Meta Llama 3 vs. Microsoft Phi-3 vs. OpenAI ChatGPT 3.5
zhlédnutí 4,9KPřed 2 měsíci
I gave the below question to all 2 local large language models (Meta Llama 3 & Microsoft Phi-3) and 1 hosted model (OpenAI ChatGPT) and was shocked at the results Question: There is a cake on a table in the dininig room, I walk over to the cake and place a plate on top of the cake, I then pick up the plate and take it into the kitchen. Which room is the cake currently in? The results surprised ...
How to install Microsoft Phi-3 on Windows Mac and Linux Local
zhlédnutí 2,7KPřed 2 měsíci
Learn how to download and install Microsoft Phi-3 on your computer with this quick and easy tutorial! Download ollama from ollama.com Whether you're using Windows, Mac, or Linux, I got you covered! 💪💪 I also test using Microsoft Phi-3 and Meta Llama 3 at the same time
How to run Llama 3 locally hands on easy to follow
zhlédnutí 2KPřed 2 měsíci
In this video I share how to get the most out of the llama 3 by installing a specific model variation 3:8b-instruct-q6_K and share the different variations available. I also test the model with jokes and maths and get some shockingly bad results.. as expected, poor prompting on my part. In the next video, lots of Python and data! Make sure you're subscribed.
How to install llama 3 on Windows Mac and Linux
zhlédnutí 17KPřed 2 měsíci
Learn how to download and install Llama 3 on your computer with this quick and easy tutorial! Download ollama from ollama.com Whether you're using Windows, Mac, or Linux, I got you covered! 💪💪s
THIS online Python tool makes testing scripts in 2024 too easy! Thanks Zancb
zhlédnutí 998Před 4 měsíci
In this photo I share my thoughts on learning Python in 2024 then in true Adam fashion we jump into a quick web scraping tutorial to get the price of a retail product from its JSON payload... What video would be complete without a quick tutorial! I am testing out a new mic setup so please let me know how the audio sounds.
Thank you
zhlédnutí 599Před 5 měsíci
Thank you
ChatGPT Vision API
zhlédnutí 5KPřed 7 měsíci
ChatGPT Vision API
GPT4 vision API Python Tutorial in to get you started
zhlédnutí 3,8KPřed 7 měsíci
GPT4 vision API Python Tutorial in to get you started
Learn Data Analysis with python ep 3
zhlédnutí 1,4KPřed 7 měsíci
Learn Data Analysis with python ep 3
Oops I accidentally exposed my ChatGPT API key in my last Python tutorial
zhlédnutí 757Před 7 měsíci
Oops I accidentally exposed my ChatGPT API key in my last Python tutorial
Learn Data Analysis with python chatGPT API ep 2
zhlédnutí 1,7KPřed 7 měsíci
Learn Data Analysis with python chatGPT API ep 2
Learn Data Analysis with python web scraping ep 1
zhlédnutí 26KPřed 7 měsíci
Learn Data Analysis with python web scraping ep 1
How to fix unbound local error in python
zhlédnutí 1,5KPřed 7 měsíci
How to fix unbound local error in python
Introduction to Plotly Dash with Adam from @CharmingData python data web app visualisation easy
zhlédnutí 1,2KPřed 10 měsíci
Introduction to Plotly Dash with Adam from @CharmingData python data web app visualisation easy
Python SQLite Tutorial I LOVE json_extract() 😍
zhlédnutí 2,3KPřed 10 měsíci
Python SQLite Tutorial I LOVE json_extract() 😍
ChatGPT API Python Tutorial, JSON Address Cleaner… Kind of
zhlédnutí 3,4KPřed 11 měsíci
ChatGPT API Python Tutorial, JSON Address Cleaner… Kind of
Escape the Confusion: Master JSON in Python
zhlédnutí 1,5KPřed 11 měsíci
Escape the Confusion: Master JSON in Python
Unlocking Hidden API Data: Man in the Middle Proxy - Introduction to MITMProxy's Practical Power
zhlédnutí 3,1KPřed 11 měsíci
Unlocking Hidden API Data: Man in the Middle Proxy - Introduction to MITMProxy's Practical Power
Your Opinion Matters: Help Shape the Future of this Channel
zhlédnutí 1,4KPřed rokem
Your Opinion Matters: Help Shape the Future of this Channel
Python Automation: Downloading 95 Excel Files in Seconds
zhlédnutí 8KPřed rokem
Python Automation: Downloading 95 Excel Files in Seconds
Data Binning Explained: A Python and Pandas Tutorial
zhlédnutí 7KPřed 2 lety
Data Binning Explained: A Python and Pandas Tutorial
When Excel Fails: Python to the Rescue
zhlédnutí 3KPřed 2 lety
When Excel Fails: Python to the Rescue
Python & MITMProxy: Web Scraping Secret iOS App Data
zhlédnutí 26KPřed 3 lety
Python & MITMProxy: Web Scraping Secret iOS App Data
Python Tutorial: Browser Automation & Web Scraping with Selenium - Part 2
zhlédnutí 11KPřed 3 lety
Python Tutorial: Browser Automation & Web Scraping with Selenium - Part 2
Mastering Browser Automation with Python and Selenium
zhlédnutí 29KPřed 3 lety
Mastering Browser Automation with Python and Selenium
How to scrape map data with Python | How to make money with Python Episode 5
zhlédnutí 39KPřed 3 lety
How to scrape map data with Python | How to make money with Python Episode 5
Python coding - How to access the Reddit API tutorial
zhlédnutí 9KPřed 3 lety
Python coding - How to access the Reddit API tutorial
Learn Python - Web scraping a private API - Questions from the comments episode 2
zhlédnutí 8KPřed 3 lety
Learn Python - Web scraping a private API - Questions from the comments episode 2
Python YouTube Scraper bonus - Easy step by step guide | How to make money with Python Episode 4
zhlédnutí 15KPřed 3 lety
Python CZcams Scraper bonus - Easy step by step guide | How to make money with Python Episode 4

Komentáře

  • @fatihahasus1069
    @fatihahasus1069 Před 8 dny

    its really hard for me to understand the documentation and function of binning thanks sir for this useful video!

  • @fatihahasus1069
    @fatihahasus1069 Před 8 dny

    is there any video about functions when we have to do functions

  • @ziyad.l252l
    @ziyad.l252l Před 10 dny

    Nice one

  • @FrenzyGangles
    @FrenzyGangles Před 12 dny

    I have been binge watching all the series. Where is episode 4? :( I really wanted to learn about the dashboards. Great work either way!!

  • @JustGiveUp.
    @JustGiveUp. Před 13 dny

    which browser are you using?

  • @MusaShaik1994
    @MusaShaik1994 Před 14 dny

    Even After 3 year ❤

  • @RidwanurRahmanextreme

    Hi, How did you add your device with your laptop please?

  • @BryanHawkshaw
    @BryanHawkshaw Před 18 dny

    Hey any news on when Episode 4 of Learn Data Analysis with Python is coming? The project is great and I’d love to finish it.

  • @thatoneguy1821
    @thatoneguy1821 Před 21 dnem

    Are you familiar with the DoorDash driver app? Is it possible to scrape data from that using MITM. Specifically, what I’m hoping for is a way to scrape all of my earnings from my time as a DoorDash driver (orders, stores, dates, base pay & tip). I’m not sure if you’ll even see this, but thanks in advance. I really appreciate these videos. Excellent content!

    • @MakeDataUseful
      @MakeDataUseful Před 20 dny

      Hey! Maybe!! There’s a couple of considerations 1) if you can see the data on screen you’re likely able to see it in an API call and 2) if they are NOT using ssl certificate pinning. Pinning is where the app doesn’t trust certificates on the device and uses its own certificates. There are ways around it but require some extra steps. Shoot me an email youtube@makedatauseful.com.au and we can explore a little more!

  • @arashhadadsoleymani3756

    Thank you so much for this incredibly helpful video on web scraping! Your explanations were clear and easy to follow, and the examples you provided made it much easier to understand. keep up!

  • @riccardoronco627
    @riccardoronco627 Před měsícem

    wonderful explanation - thank you!

  • @Nelnac
    @Nelnac Před měsícem

    I interpreted ollamas version as correct. It states a few things there. It says think about it. You placed the plate on top of the cake which means the cake is now on the plate, this is true if cake transferred to the plate when you placed the plate on top of the cake which would happen in real life. We're also assuming the cake has icing icing is sticky bits of cake stick to the icing that stick to the plate. It also very specifically stated in it's next paragraph, (on the plate) meaning the cake that's stuck to the plate is currently located.

  • @WaldoTheWombat
    @WaldoTheWombat Před měsícem

    does it have to be an online link or can send it an image from my own machine?

    • @MakeDataUseful
      @MakeDataUseful Před měsícem

      You can send an image from your local machine, checkout this part of the docs and if you want a tutorial let me know! platform.openai.com/docs/guides/vision/uploading-base-64-encoded-images

    • @WaldoTheWombat
      @WaldoTheWombat Před měsícem

      @@MakeDataUseful thanks

  • @TransmentalMe
    @TransmentalMe Před měsícem

    I get nothing but gibberish output when I install it and run ollama run llama3 on my M3 Max MBP and ask "Who are you?"

    • @MakeDataUseful
      @MakeDataUseful Před měsícem

      Weird, sounds like something's not right. I get a lot of gibberish when I crank up thinks like temperature and other params when defining a custom model file. Shouldn't be the case, what about llama2, same?

  • @lukuwazzow9553
    @lukuwazzow9553 Před měsícem

    After completing this course, are we truly able to make money with python?

    • @MakeDataUseful
      @MakeDataUseful Před měsícem

      In 2024? Make money solvving problems. Python can help

  • @dennissdigitaldump8619
    @dennissdigitaldump8619 Před měsícem

    For the "unsupported" machines LMstudio is the solution. Download the "model" and choose Llama3-whatever & go. I have a very high powered machine, but since it was a server it doesn't have AVX2, which most model loaders need.

  • @andyhelipilot3528
    @andyhelipilot3528 Před měsícem

    Binge watching your channel. I’m newish to Python. Looking primarily at what you’re able to achieve here. I use a browser but that is as far as I go. To enable me to understand more which paths shall I add to my studies, so that I can understand more regarding working with the inspect and developer tabs. Would adding to understand html, Python requests, pandas be sufficient. Essentially I’m if I was to find a course that covers how you understand websites and the inspect and developer tab views, what broad subject would one look for?

  • @RealEstate3D
    @RealEstate3D Před měsícem

    The name has nothing to do with the content at all. Where’s the installation process, which are the recommended hardware prerequisites, where’s Linux and where’s Windows. What was the initial sense of this video?

    • @MakeDataUseful
      @MakeDataUseful Před měsícem

      Hey RealEstate3D thanks for taking the time to leave some feedback. Let me know if you are still stuck, more than happy to go into a little more detail.

    • @RealEstate3D
      @RealEstate3D Před měsícem

      @@MakeDataUseful I am not stuck. I expected to find the information I already told you 2 hours ago. It might make sense to describe the content of your video honestly. Instead it seems you misnamed your video on purpose. That’s pretty questionable. As also your standard answer is kind of low effort.

    • @MakeDataUseful
      @MakeDataUseful Před měsícem

      @@RealEstate3D Okay, shoot me a video title and I'll update it

  • @0002nicejay
    @0002nicejay Před měsícem

    Thank you for being direct, clear, and straight to the point.

    • @MakeDataUseful
      @MakeDataUseful Před měsícem

      Hey you're welcome, I've been playing around with the API and Python package. More videos to come!

  • @dopplegangerdavid
    @dopplegangerdavid Před měsícem

    I didn't know Matt Walsh was a programmer.

  • @islamibrahim4382
    @islamibrahim4382 Před měsícem

    Hey Man you are one of the best, you always provide clear explanation to what's going on unfortunately there is not much video from you, I wish you can create a series on data analysis from start to end you are a very valuable resource to the community, keep going you are the best🔥

  • @zakaria20062
    @zakaria20062 Před měsícem

    Can remember faces ?

  • @thatoneguy1821
    @thatoneguy1821 Před měsícem

    THANK YOU!!! This is sooooo helpful

  • @abdullahAboelkhair
    @abdullahAboelkhair Před měsícem

    is this llma3 or llma3:instruct?

  • @rewinder2008
    @rewinder2008 Před měsícem

    funny when I tested it out, chatgpt got it wrong. I used llama 3 uncensored version dolphin and It got it right then on the explanation it hurts itself in its confusion lol Chatgpt: "The cake is in the kitchen. When you placed the plate on top of the cake in the dining room and then picked up the plate and took it into the kitchen, you effectively moved the cake into the kitchen along with the plate." >can you explain how did you solved this problem? "Certainly! Let's break it down step by step: You start in the dining room, where the cake is located on the table. You place a plate on top of the cake in the dining room. At this point, the cake is still in the dining room, but now it has a plate on top of it. Then, you pick up the plate with the cake on it and take it into the kitchen. Now, both the plate and the cake are in the kitchen. So, although the action of placing the plate on the cake initially occurred in the dining room, by moving the plate with the cake on it into the kitchen, you effectively moved the cake into the kitchen as well. Therefore, the cake is currently in the kitchen." Llama3dolphin: " The cake is currently in the dining room." >can you explain how did you solved this problem? "Sure! When you told me that there's a cake on a table in the dining room, and then later mentioned that you moved the plate (with the cake still underneath it) to the kitchen, I used deductive reasoning. Initially, the cake was in the dining room because you put a plate on top of it there. Then, when you moved the plate into the kitchen, the cake went with it, as it was still under the plate. Therefore, the cake is now in the kitchen."

  • @heritage1834
    @heritage1834 Před měsícem

    Great content 👍. It's very explanatory and easy to follow along

  • @haverts13
    @haverts13 Před měsícem

    There is a problem and when you close the terminal and when you open it again it gonna forget the last chat and information

    • @MakeDataUseful
      @MakeDataUseful Před měsícem

      Correct, there are ways to hold onto context.

  • @xlr555usa
    @xlr555usa Před měsícem

    Thanks for the video. Do you know if llama 3 has support for multiple gpus? Hoping it has support for Intel Arc cards as they are cheap and I already have three of them. Might have to get a bunch more if I get it working ;] I've seen Network Chuck here on snoozetube where he got multiple gpus working in llama, looks like it can sense multiple gpus. I get bombarded with advertisements for Nvidia H100 clusters and I'm seeing that just one H100 costs $43,000 on amazon. People are going to find another way besides Nvidia gpus and everyone should start looking at setting up AI locally in their homes. czcams.com/video/d8oVTKG39U8/video.html

  • @shawondas1689
    @shawondas1689 Před měsícem

    Can you please make a tutorial on how to download data from a local sales data software by setting the time period from the software? It'd be very very helpful. TIA!

  • @Bicyclesidewalk
    @Bicyclesidewalk Před 2 měsíci

    Just installed this on a Linux box - and wow, this one is great~

  • @dmitriyartemyev3329
    @dmitriyartemyev3329 Před 2 měsíci

    Ohh wow wow wow easy:))) thanks for video. Very interesting actually

  • @userou-ig1ze
    @userou-ig1ze Před 2 měsíci

    What about backtick

  • @ZacMagee
    @ZacMagee Před 2 měsíci

    Love your content man, keep it up❤

  • @simonsax9102
    @simonsax9102 Před 2 měsíci

    was this ever continued? I would be interested to build such a playground

  • @jindrichsirucek
    @jindrichsirucek Před 2 měsíci

    When you ask only for one letter response the model doesnt have time/space for thinking it through.. And also when you write the "please provide a one latter answer" AFTER then the last prompt is provide one letter a or b so small model will get confused and forget about the first part - maybe try to provide formating prompt at begining then pose question 🙂 keep digging

    • @MakeDataUseful
      @MakeDataUseful Před 2 měsíci

      Good insight, oh I've been digging! Next video is leaving two models to have a conversation with each other.. Things get WEIRD!

    • @jindrichsirucek
      @jindrichsirucek Před 2 měsíci

      @@MakeDataUseful Looking forward to that video.. if you will be willing to add url to the video to this comment, that would be great so I get notified :-)

  • @userou-ig1ze
    @userou-ig1ze Před 2 měsíci

    I thought this is what function calling is for?

  • @Koroistro
    @Koroistro Před 2 měsíci

    As a human I'd instinctivelly answer that the cake technically is in both rooms since in most cases some pieces of the cake would stick under the plate. Thought it's an overly technical point since some frosting/cream wouldn't constitute a cake. Lateral thinking is intersting in the context of the LLM. I'd be very curious on the path the prompt takes in the model latent space and what "node(s)" is/are at fault for the mistake.

  • @rodfer5406
    @rodfer5406 Před 2 měsíci

    I’ve seem historical errors in Llama3…

    • @MakeDataUseful
      @MakeDataUseful Před 2 měsíci

      It's interesting hey! In my latest video I tested it 100 times. It got it correct 98 times

  • @rodfer5406
    @rodfer5406 Před 2 měsíci

    Good

  • @sebkeccu4546
    @sebkeccu4546 Před 2 měsíci

    Try the rabbits in the garden exercise: There are two rabbits in the garden. In the garden is a whole. the first rabbit falls in the hole. How many rabbits are left in the garden? response should be still two because the hole is also in the garden. You can give the llm an second chance by explain the hole is also in the garden and that therfore both rabbits are in the garden. Now you tell it that the second bunny also falls in to that hole that is in the garden. It should responds , that both rabbits are in the garden.

    • @MakeDataUseful
      @MakeDataUseful Před 2 měsíci

      Great one! I love it. I was also going to try the “adding doubt” approach to any answer with a followup “are you sure?” to see if it flip flops.

  • @Rundik
    @Rundik Před 2 měsíci

    I also tried this model. It would often hallucinate and print random letters in a repeating pattern over and over again

    • @MakeDataUseful
      @MakeDataUseful Před 2 měsíci

      It's not without its quirks, multishot prompting may be the best way forward. I'm going to keep experimenting!

  • @kumarsalib722
    @kumarsalib722 Před 2 měsíci

    Why are they being compared to GPT 3.5 rather than 4?

    • @MakeDataUseful
      @MakeDataUseful Před 2 měsíci

      I knew 4 would knock it out of the water, I thought I would give it a chance against 3.5

    • @onlyms4693
      @onlyms4693 Před měsícem

      Because meta is a open source software and open source alwasy behind the closed source bwcause investor like the possibility to earn money from it.. So i love meta for it but we need to realise they will fell behind of other closed source llm