Use ChatGPT-4o Vision on Images with Power Automate
Vložit
- čas přidán 25. 07. 2024
- This video is not just about Image Tagging but using an AI Model like ChatGPT-4o to pull back data based on your images. This includes:
1. Image Tagging
2. Generating Descriptions based on images
3. Transcribing Images
4. Translating Text in images
5. OCR
Using the ChatGPT API to analyze images involves uploading the image file to the API, which then processes the visual content to extract relevant information. The API can identify objects, read text, and provide descriptions or data points based on the visual analysis. This functionality is useful for tasks such as image classification, OCR (Optical Character Recognition), and generating detailed image descriptions for accessibility purposes.
Chapters:
0:00 Introduction
2:09 I like to use Leonardo.AI to Generate Images
3:10 Microsoft Syntex can do the same thing
4:26 Creating the Flow
6:36 Base64 Format for Images
7:40 Using ChatGPT API Documentation for API
10:03 Generating our API Key
13:00 Retrieving our Schema for Parse JSON
14:05 Fixing my error on application/json
16:27 Grabbing the first message Content
18:22 Testing our first Image Tag
22:00 Using the Actual Image Tag Column
22:50 Calculating Cost per Image
25:30 Transcribing Cursive Documents
29:19 Translating to Spanish
30:10 Conclusion - Věda a technologie
just discovered your videos, really fascinating. I've started using powerautomate a lot and chat GPT so these were really helpful. Do you think it would be possible using a similar method to use powerautomate and GPT vision to analyze an image and generate a PDF report (based on a template) and using a custom prompt. This could potentially save a massive amount of time at my workplace.
Yes totally possible. Worth checking out the Microsoft side of things too. With Power Automate you could customize the template just how you want it to too. 100% it's possible and the options are available for which AI tool you want to use: azure.microsoft.com/en-us/products/ai-services/ai-vision.
@@andrewhess123 yes!, thanks for this mate. it looks like Azure AI might be just GPT4 turbo and not GTP 4o? Cant quite find clear info.
Using powerautomate with gpt vision is definitely a game changer. So many practical applications.
@@mikeloopsthings I believe that is correct, currently it is just GPT4 Turbo and not GPT-4o
@@andrewhess123 thanks mate. i really love the idea of using powerautomate. Realistcally im just looking for a reliable automation to upload images to GPT and have it analyse them send back the data. I cant see any way to do this natively with GPT so powerautomate would be the most obvious way
Ok, it seems the ChatGPT + Power Automates videos are not hot enough currently 😀Will be switching topics next week. Back to Power Apps?
Hi @andrewhess123, you just come on my feed so maybe the youtube algo is taking a bit. I love this content and for my org is right up our ally! Would love to see how you would connect to GPT Assistants via PowerAutomate.
@@JulianTRamon You are right about the GPT assistants to Power Automate, that would be golden! 😀
Would like to throw my hat in the ring and give my appreciation for your informative videos. The gpt and power automate/apps integration is something that is seemingly invaluable at the moment. Would love to see some use cases and tutorials for gpt integration to dataverse as-well. Thanks for the great resource Andrew!
@@waleedkhan2500 Thanks I think the youtube algorithm may of hid my video a little bit 😀. Seems people are very interested in this stuff, just like I am.
Can you rename a Custom Connector?
I don't think its currently possible, you could completely remake it and give it a new name. Which I have had to remake these chatgpt custom connectors at least 20-30 times now 😅There is a vote to add that functionality: ideas.powerautomate.com/d365community/idea/556cfa11-0089-4467-80de-39ed7f3a9ceb
Can you make the same tutorial for resume attached in attachment (PDF) to extract the information regarding the PDF, like Years of experience, SKills?
Thanks Hammad! Let me see how well ChatGPT API does with this stuff, maybe it perfect, not sure yet. I know some options in SharePoint premium and Cognitive Services in Azure can do this though.
@@andrewhess123 Yeah but ChatGPT extract the details from the pdf. As we can upload any document to ChatGPT. Is it possible you can make tutorial on it?
@@andrewhess123 Which options can do this in sharepoint premium?
I just want to reduce cost thats why i am targeting chatGPT.
@@hammadyounas2688 I am not sure which cost is best yet. Many of these concepts are very new. There are "autofill" columns that you can set up: techcommunity.microsoft.com/t5/sharepoint-premium-blog/introducing-sharepoint-premium-the-future-of-ai-powered-content/ba-p/3981076.
Also using Microsoft Syntex you can do OCR also: www.linkedin.com/pulse/microsoft-premium-syntex-optical-character-recognition-eda-f0xme/
I am currently researching all of these options.
@@andrewhess123 You already pass the image to the chatgpt, so there is not much time to make a tutorial on pass pdf to chatGPT which produce the summary of the pdf.