Is the new Raspberry Pi AI Kit better than Google Coral?
Vložit
- čas přidán 3. 06. 2024
- I've got my hands on the new Raspberry Pi AI Kit, an M.2 Hat with the Halio 8L module onboard. This brings advanced AI capabilities to the Raspberry Pi 5, and offloads this to the Neural Processing Unit.
Join this channel to get access to perks:
/ @kevinmcaleer28
💁♂️ For more information on SMARS Robots, tutorials and more visit: www.smarsfan.com
🎖To join the membership at 🥉bronze, 🥈silver or 🥇gold levels, head over to
www.smarsfan.com/membership
Join the list - action.smarsfan.com/join-the-...
☕️ Enjoy this video? Buy me a coffee! www.buymeacoffee.com/kevinmca...
📸 Follow me on Instagram - @kevinmcaleer / kevinmcaleer
🐦 Follow me on Twitter - @kevsmac / kevsmac
🙂📘 Join the Facebook group - Small Robots / smallrobots
👩💻 My Code on GitHub: www.github.com/kevinmcaleer
🎵 Music by Epidemic Sounds www.epidemicsound.com/referra...
#Pico #MicroPython #Robotics - Zábava
Hiiii Kev! 👋you could make your fortune telling robot EXTRA spookily accurate with this kit.
Cool! Yes, if I added a camera and ollama back end it could detect a person and then say something spooky about what it sees!
I'm so glad I found your channel so awesome and informative videos! keep it up
It great to have such a chip design with tightly integrated memory which is obvious the way to go for any inference tasks. But ive found it interesting how hailo manages to keep any information about the amount of that super useful memory 100% secret; and this product launch is no exception. Like, dont we all want to know what models we can actually run on there? Arnt we going to find out eventually what the memory limits are after we spend our money? Or is that somehow the point?
Thanks for the video. I got one of these kits and it is amazing when running the provided sample codes/apps. Pity to program something leveraging on this kit you need c++, the installation of the Hailo SDK and then it becomes very complicated for 99% of the people using a Pi. I would not recommend the AI Hailo kit for everyone. It is for developers with good knowledge of AI and c++. There is not a single video explains that, all video on CZcams just try to market it, I have not found any tutorial explains how to modify or make a new app. At present, if yogurt one of these kits you will be on your own.
Very interesting. It would be nice to hear more about supported frameworks in future. Realize this a hardware unboxing and quick demo.
Yes I’ll be following up with more info on that now that the full software stack is released
Thx for sharing! Apart from object recognition, any other use caes be benefit with edge compute power?
A great option for robot vision, IMO. I still wish there were at least a little NPU right on the Pi (like the RK3588)... but this is a worthy option in lieu of that.
Also regarding the TOPS/W rating, I had 13 TOPS at 1.5W-according to their specs, but I was wondering if you are using the full Pi + Hailo-8L power consumption for your rating of 3 TOPS/W?
I’ll look into that shortly Jeff - what equipment do you use to test the power draw?
Ah, I may have been reading the specs wrong-realized they are using "1.5W" on their site to indicate typical usage at 30-60 fps of a standard image processing model. It can use more than that when it goes all-out at 13 TOPS. Wish I could edit my video ha!
@@kevinmcaleer28 I use a ThirdReality Smart Outlet for measurements over time, it's connected to my Home Assistant. I have also used a few other power measurement devices, but the ThirdReality one's been the most reliable and seems pretty accurate!
Agreed. I was waiting for a promising tensor accelerator for the pi for a while. I just bought this to try it.
A possible advantage is that you can use the ram on the pi, and offload computation via the PCIe3.0 accelerator. This could allow to use larger models, a problem I have with e.g. the oak d lite is the 2GB ram limitation.
Something I want to try is to run a local LLM on the pi using halio acceleration. This could give a low latency LLM for things like e.g. Speech to Text to Speech. it's something that works poorly on the cloud due to the latency involved.
Got mine ordered.
I'm quite interested in seeing its performance running LLMs via Ollama. With a RPi5 alone, it takes vision model LlaVa about 5 whole minutes to interpret an image. I'm hoping the AI kit can improve that vastly! Have you given it a go?
Can I use this chip in a standard tensorflow model? If so how? Or is it automatic?
Very informative, I was looking for an accelerator for mobile robotics and found nothing useful. The Orin is powerful but has nvidia blessed binary blobs and is really expensive.
13TOPS, PCIE1x3.0 3.0TOPS/W are promising specs. On an 8GB RPi it might be enough to run a llama3-8B-4QKM model.
Is posible Rpi5 with SSD and Halo for run Asístant and Frigate? What are the correct hat configuration for this pourpose? Thanks
Does openvino work with this ?
(I'd like to use Audacity's openvino tools and Whisper for speech recognition).
When tried , the hailo causes issues in identification , and aside from the fps , accuracy is very low , is that accuracy lost during conversion ?, i would still use the coral for now
I'm interested in detecting ArUco markers and getting their pose estimates. Is that something this AI chip can do well? Would you still need to run something like OpenCV on it? I've never used AI specific hardware so not sure how it works.
I use the PCIe for an NVMe, so how would I add this AI kit, need a splitter?
Looking forward to a tutorial, inferencing custom trained Yolov8 object detection models using this....
Hi Kevin. Maybe you know a solution for connecting both an Ai Kit hat as well as an ssd on m.2 hat at the same time to a pi 5?
Wow, thank you very much for sharing! Always helping! I have a question, I think your website is down!
Thanks - its back up!
What about the cooling, I'm sure both the pi and ai chip will get boiling if you want to build some continuous running high-flops program
I can imagine nature watchers will want this to identify animals
+1 liked, for the late night dancing :-), seriously though, $70 for an entry point modest AI chip is pretty impressive. If you consider all the interesting things hobbits have done with PI's over the years, I'm looking forward to see what they come up with, when you add a NPU to the recipe
got a few pi's, but just wondering what uses would this have? o.0
I just got a milk-v meles and managed to get it to boot...(it won't power on using the raspberry pi 5 power but will from just a usb from my pc....) I think this RISC-V has potential but the OS support is lacking right now and RISC-V really needs some love from the open source community. I was hoping ubuntu had something already but they only have a broken version for Milk-V mars that the wifi won't even work... Yet... I think hobbiests are going to love this stuff!
now all that's remaining is: Frigate support!!
Can it run facial recognition model?
very cool !!
How well does it perform with LLMs against the base Pi 5?
This chip appears to be low on memory, and not fitting at all for LLMs. You want to look out for Hail-10H though (not released yet, no release date either)
Q:Hello..This is cool!... I was wondering if this is enough computer power to work with depth cameras with 3D object detection like the Realsesnse series?...
Same question… not sure if I should go for a Jetson Nano, Orin Nano or a PI5 + AI Kit. I want to run a RealSense + RPLIDAR for object tracking and SLAM.
That toilet detection was hilarious. You can see how it got matched: it looks oval and has a rim and that’s where the logic ends. No consideration of perspective or gravity. lol.
Hi Kevin. Is it posible to make a very small pc not much larger than an Apple TV box. It should be running Linux, and possible to connect, keyboard using USB/Bluetooth with a couple of additional usb ports 😃
Yep - Raspberry Pi 5 plus a tower case (SunFounder have launched one recently)
Thank You Kevin😀
Is it possible to use this technology to control a UAV so it can be autonomous without GPS? Maybe have it detect objects like buildings mountains lakes etc that it has seen a picture of prior? Where you can use identifiable objects as waypoints that you programmed in prior. I'm a total noob and wonder is there's a name for this idea so I can explore it further?
its called DSLAM, using pictures to map the environment in real time while you navigate it.
I have struggled SO HARD with the Coral and Raspberry Pi 5. If you are telling me I don't have to create Dockers and virtual environments and whatnot to run older versions of Python I'm IN!!!
Is it possible to install both the Acceleator and an SSD?
Yes - you'll need a Pimoroni NVMe Base Duo though
@@kevinmcaleer28 Thank you.
so no nvme ssd at the same time as the ai ?
Not with this Hat, but you can use the Pimoroni Duo to have both at the same time.
@@kevinmcaleer28 could we use the Pimoroni Duo and the Hailo accelerator with the same software, etc?
I got a coral and the coral hasn't helped at all. Looking forward to working it into my fabric project.
Nope it won't. If you're on a Pi 5 inferencing speed on the CPU is the same performance as the Coral TPU.
That's awesome can you still have an nvme drive
You'll need a Hat that can take 2 NVMe - such as the Pimoroni NVCe Duo
@@kevinmcaleer28 isn't the Raspberry Pi NVME hat stackable? so why cant we have two?
@@bern047 The Raspberry Pi M.2 Hat has one PCIe connector and one M.2 port. The GPIO pins are stackable, but you only get one PCIe connector. Pimoroni Duo has two M.2 ports. There are other Hats available that have multiple PCIe connectors so you can daisy chain them - check out PineBoards website for more info on those.
@@kevinmcaleer28 Thanks, Pimoroni have done it again for me, will order one thanks
Where’s the link for this module?
www.raspberrypi.com/news/raspberry-pi-ai-kit-available-now-at-70/
@@kevinmcaleer28 Do you know any reseller in the EU?
@@eMgotcha77 there’s links in that link
@@eMgotcha77 Pretty sure my friends at Pimoroni ship to the EU!
Thanks, I know. All shops, even Pimoroni, show either nothing or "pre-order, coming soon".
Well i have to exercise patience meanwhile 🙌
Can I use this AI kit to speed up my LLMs on the Raspberry Pi 5 with Ollama ?
how are you running llms on a rpi?
Bit of a misleading title as there was no real comparison between the Google Coral and Hailo 8L other than just a spec sheet side-to-side comparison. I guess it’s just a promo for the RPi AI Kit.
Is the 4gb ram raspberry pi 5 good enough for the A.I kit?
Sure is - in fact that’s the one I demoed it on
👍👍👍
Does not work with tflite?
According to the specs, yes!
@@kevinmcaleer28 Hi Kevin, Do you know of any good getting started guides for it? The demos are fine and the hailo site offers very little. I am interested in a guide to loading models and running inference in python so that I can make some particle use of it.
Can it run on a pi4?
Not without mods. It uses "PCIe Gen 3" connection on Pi5.
@@AerialWaviator Technically it could be made to run on a Compute Module 4-though I haven't tried yet.
@@JeffGeerling Was wondering. Perhaps a custom PCB where both plug-in. ;)
The RPI4 has a PCIe 1x2.0 connected to a USB3 host controller
The CM4 has the PCIe 1x2.0 lane free for the designer to use
Hardware wise, it should work, a compliant pcie 3.0 peripheral should communicate with a 2.0 pcie host at reduced speed. One has to see if the halio pcie silicon interface supports that.
The RPI4 has the BCM2711 SoC, the RPI5 has the BCM2712 SoC. One has to see if the provided Halio driver works on both SoC, or if it is specific for the RPI5 SoC.
I'd really like for the Halio M.2 accelerator to be compatible with CM4, there are lots of CM4 boards out there with a M.2 expansion slot already exposed. e.g. "EdgeBox-RPI-200 equipped a M.2 socket of M KEY type. ONLY 2242 size NVME SSD card is support, NOT mSATA."
😎🤖
❤❤❤❤❤🎉🎉
Unfortunately a lot of AI products look good just looking at the top level specs but then once you have a deeper look they are difficult to use and not all that useful. It will be interesting to see how this ends up working.
I have just bought a cheap jetson orin nano for a project of mine so I won't be buying this anytime soon but it will be interesting to see what people manage to use it for.
This is not like Rabbit or Humane, it actually does the work locally. I’ve made a lot of content about using object detection previously for various projects so this will take that workload off the main CPU freeing it to do other tasks.
your light does look like a toilet with the toilet cover down
But, any actual useful uses of this AI kit?
What actual uses would you find useful?
@kevinmcaleer28 anything besides imagine detection or recognition.
I guess you have a shitty light there!
Haha