You can now find the full application code in the video description. (or here: github.com/JohannesJolkkonen/llm-report-generation/ ) Remember to read the README, to understand what API-credentials you need for running the application.
The quiality of the videos rhar you post are just impressive, perfect videos, im just wondering if theres a way to build this app but entirely local to run in an ipad is it possible to build it ? Im trying to build an HIPPA prove app for a local doctor who wants the ia take the conversation and generate a report
@@Hector-zr2lq Thank you. I'm not very familiar with iOS/mobile development, but I'm pretty sure it would be possible, including local llms running on ipads or such. But it's not at all necessary (or worth it, in my opinion) to go down that route. It's perfectly possible to build HIPAA apps on the cloud, using platforms such as Microsoft Azure, where you can also run OpenAI's models. That's probably where you want to look, and perhaps get in touch with their sales for more specific requirements.
Thanks everyone for the positive feedback! As many are asking, will share the code within the next 1-2 days, just want to structure it a bit better and write a short README first
This is awesome Johannes - I think the approach you've taken (and crediting Jason Liu's work) is valid - the report itself has significant value. How can you 'protect' the data that is provided? The company information that is being queried in the LLM contains what might be sensitive company information - how do you know that it isn't being exposed for access by competitors or others?
Thank you! Regarding data protection, there's no particular risk of such exposure with LLMs. OpenAI and Anthropic both have it in their terms to not train models on API inputs or outputs, and also have robust security compliances (including SOC2 and HIPAA, which are 2 of the most common certifications that companies look for.) (links: trust.openai.com/, support.anthropic.com/en/collections/4078534-privacy-legal) There's also the option to use OpenAI's models via Azure, and Anthropic's via AWS. The guarantees are technically the same as above, but these major cloud platforms just have a longer track-record of trust and security behind them. So that's typically the go-to with more sensitive clients, like in my last 2 projects which were done for a bank and a major healthcare provider. But for most situations, using the APIs directly is perfectly fine. No more risky than any other SaaS, CRM or database the company is already using for their data. Might make a more detailed video about this at some point, as it is something that comes up quite often
@@artur50 Yup, all the LLMs used here (OpenAI, Anthropic, Groq for Llama3.1) follow the same OpenAI 'standard', so works with Ollama/LMstudio with very minimal adjustments
Not really, no. While you can run custom code on those platforms, that custom code can't use any external libraries (such as DocxTemplater, which is crucial here for the actual document creation). Then there's also stuff around handling the data structures, doing parallel API calls and validating the LLM-outputs, just to name a few, that I think would be impossible to do in no/low-code
You can now find the full application code in the video description. (or here: github.com/JohannesJolkkonen/llm-report-generation/ )
Remember to read the README, to understand what API-credentials you need for running the application.
The quiality of the videos rhar you post are just impressive, perfect videos, im just wondering if theres a way to build this app but entirely local to run in an ipad is it possible to build it ? Im trying to build an HIPPA prove app for a local doctor who wants the ia take the conversation and generate a report
@@Hector-zr2lq Thank you. I'm not very familiar with iOS/mobile development, but I'm pretty sure it would be possible, including local llms running on ipads or such.
But it's not at all necessary (or worth it, in my opinion) to go down that route. It's perfectly possible to build HIPAA apps on the cloud, using platforms such as Microsoft Azure, where you can also run OpenAI's models. That's probably where you want to look, and perhaps get in touch with their sales for more specific requirements.
Doing some research about NVIDIA NIM have you use it?
Thanks everyone for the positive feedback!
As many are asking, will share the code within the next 1-2 days, just want to structure it a bit better and write a short README first
Thank you very much Johannes for sharing! Looking forward!
Thanks Johannes!
Excellent work Johannes! Looking forward to play with the code when you are ready to share. Thanks !!
Thanks for showing a practical business use case. Very helpful.
Sharing code is invaluable. ♥️
It is very interesting ❤
Thanks so much
If there is a possibility to explain the project in detail, it will be appreciated
This is awesome Johannes - I think the approach you've taken (and crediting Jason Liu's work) is valid - the report itself has significant value.
How can you 'protect' the data that is provided? The company information that is being queried in the LLM contains what might be sensitive company information - how do you know that it isn't being exposed for access by competitors or others?
Thank you!
Regarding data protection, there's no particular risk of such exposure with LLMs. OpenAI and Anthropic both have it in their terms to not train models on API inputs or outputs, and also have robust security compliances (including SOC2 and HIPAA, which are 2 of the most common certifications that companies look for.)
(links: trust.openai.com/, support.anthropic.com/en/collections/4078534-privacy-legal)
There's also the option to use OpenAI's models via Azure, and Anthropic's via AWS. The guarantees are technically the same as above, but these major cloud platforms just have a longer track-record of trust and security behind them. So that's typically the go-to with more sensitive clients, like in my last 2 projects which were done for a bank and a major healthcare provider.
But for most situations, using the APIs directly is perfectly fine. No more risky than any other SaaS, CRM or database the company is already using for their data.
Might make a more detailed video about this at some point, as it is something that comes up quite often
@@johannesjolkkonenwill it be possible to run it with local stuff such as Ollama or LM studio ? OpenAI compatible API solution provided in the code?
@@artur50 Yup, all the LLMs used here (OpenAI, Anthropic, Groq for Llama3.1) follow the same OpenAI 'standard', so works with Ollama/LMstudio with very minimal adjustments
Thank you so much , excited to build AI applications and automations for our business.plz share repo link.
Can u share the GitHub link of the project?
Will you release the source? Would save me shit loads of timr
@@petersmith8153 Done, repo is now in the description and pinned comment (:
Excellent job 👏🏻
Amazing work🎉🎉
let's goooo
Hi, Is it possible to use programmes like "Make" or "Zapier" to automate this kind of process ? Thank you .
Not really, no.
While you can run custom code on those platforms, that custom code can't use any external libraries (such as DocxTemplater, which is crucial here for the actual document creation). Then there's also stuff around handling the data structures, doing parallel API calls and validating the LLM-outputs, just to name a few, that I think would be impossible to do in no/low-code