How can I do fine-tune using multiple books as dataset please? I have a many docx files (of those books) but I don’t understand how can we convert a story into many pairs of Q&A? Will doing that destroy the writing style of the author?
Also where do I download that notebook please? I have a Runpod account and I wanna try uploading the notebook into Runpod like you said in the video to start fine tuning.
Hello, you can generate a formatted and curated dataset from a book by using OCR (Optical Character Recognition) techniques. It converts the book's physical or digital text into machine-readable format. Tools like Azure Document Intelligence or Google Cloud Vision API can help with this by extracting text from images or scanned pages. Learn more about this: azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence
Amazing, could you share the notebook and dataset as well?
How can I do fine-tune using multiple books as dataset please? I have a many docx files (of those books) but I don’t understand how can we convert a story into many pairs of Q&A? Will doing that destroy the writing style of the author?
Also where do I download that notebook please? I have a Runpod account and I wanna try uploading the notebook into Runpod like you said in the video to start fine tuning.
any idea how to generate the dataset from a book, so it come already formatted and curated ? without human intervention , full automation style via ai
Hello, you can generate a formatted and curated dataset from a book by using OCR (Optical Character Recognition) techniques.
It converts the book's physical or digital text into machine-readable format. Tools like Azure Document Intelligence or Google Cloud Vision API can help with this by extracting text from images or scanned pages.
Learn more about this: azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence
@@Datasciencedojo perfect