How To Deploy Puppeteer Script in Vercel via CLI & GitHub in 2022
Vložit
- čas přidán 6. 08. 2024
- 🌐 NodeMaven Proxy Provider: go.nodemaven.com/scrape
💥 Special Bonus: Use "Michael" at checkout for an extra +2GB of bandwidth.
🤖 2captcha Captcha Solving Service: bit.ly/2captchapromo
This puppeteer tutorial is designed for beginners to learn how to use the node js puppeteer library to perform web scraping, testing, and creating website bots. Puppeteer is a Node library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol. Puppeteer runs headless by default but can be configured to run full (non-headless) Chrome or Chromium.
Playlist: • Nodejs Puppeteer Tutorial
Code: github.com/michaelkitas/Puppe...
Join our Discord: / discord
Puppeteer API: www.npmjs.com/package/puppeteer
Donate
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
PayPal: support@websidev.com
Bitcoin Wallet: bc1q05j8gcnq4mzvgj603cxdc8xxck4jgnu2ljsrt4
Ethereum Wallet: 0x5e7BD4f473f153d400b39D593A55D68Ce80F8a2e
Social
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Website: websidev.com
Linkedin: / michael-kitas-638aa4209
Instagram: / michael_kitas
Github: github.com/michaelkitas
Business Email: support@websidev.com
Intro (0:00)
Setup Puppeteer Script (0:31)
Vercel CLI Deployment (4:54)
GitHub Deployment (7:00)
Tags:
- Nodejs Tutorials
- Puppeteer Nodejs
- Nodejs Puppeteer Tutorial
- Puppeteer Tutorial for Beginners
#nodejs #puppeteer #webscraping
hi michael ! work perfectly !
Your tutorials are superb !!!!!
thank you much, you saved me a lot of time and headache of trying to make this work, liked and subscribed
Also asking if you could look into how to do this with Node 18 / newer versions of puppeteer and chromium on vercel. Thx!
Thanks. Never knew puppeteer would be this much of a pain in the a*ss to setup. Thanks for the video!
Thanks for this tutorial, I learned a lot of deployment, so many error. In the end vercel is not good to scrape, the basic tier have a request time of 5 seconds, that's impossible to scrape something in that time, after that the server goes to sleep mode and you get a time out error.
I love you man
A bit confusing but works fine, thanks!
You're welcome!
Sir please make video on playwright how to "deploy" playwright code on vercel or any ware so please make video on this topic sir
Thanks, but i believe vercel depreciated node14 from build therefore builds are failing. And this ONLY works kn node14. Correct me if im wrong? Any help appreciated.
Many requested this, I've added it to the to-do list, will try to make a video asap in later supported version
Hi, node 14 is not support now, only ^18 supported, but change to 18, It can't run well now
Make sure you also update nodejs locally to v18, and test/debug locally, before testing on production, you also need to make changes on json files for changing the version there as well
hi micheal ,
i am using whatsapp-web pkg in nodejs and this pkg uses pupetteer
in their documentaion they said they have given some command to run on linux server
how can i run that command in vercel because i want to deploy my nodejs on vercel but vercel blocks this pkg due to serverless function
Hi there! Vercel primarily supports serverless functions, which might not be compatible with packages like whatsapp-web that require persistent sessions and might use Puppeteer. Unfortunately, you can't directly run the Linux commands on Vercel as you would on a traditional server. You might need to consider an alternative hosting solution like a dedicated or virtual private server (VPS) where you have full control over the environment. Good luck!
Pretty nice tuto, but I've a message on Vercel that the function size is too big (52MB) and exceeds the 50 standard MB :/
Thank you! To fix the issue you need to either remove packages or use AWS Lambda (I have a tutorial on this)
Please respond from the channel owner. Is it possible to publish puppeteer on render because I could not publish it?
Going to make a video on that soon
@@MichaelKitas I'm waiting
I am facing an error code it is says server less function not found
Vercel, i have followed your tutorial exactly but I don’t why this issue is occurring please help
Can you rewatch the video? And try again maybe using my code this time to make sure?
@@MichaelKitas wow, what a fantastic video. great explanation and quality. it worked for me. came back after 3 months.
Can u make a video on the best free alternatives to heroku since heroku is removing their free plans for puppeteer
Great idea, I will 👍
cyclic
@@MichaelKitas because of heroku ending free plan I came😅😅😅😅 here.
@@rajeshchitikela5847 😄
@@MichaelKitas Bro I uploaded to vercel, but unable to get scraped items. And unable to understand the logs where in heroku it is easy to get logs
I followed your tutorial. It works amazing. But sometimes it times out the function. How can I display a error using express and not the vercel” branded error
You can catch the error and respond with a custom message
if possible, please upload the repo with updated code
Going to record one today, I've found a way better way to do this.
An error occurred with your deployment
FUNCTION_INVOCATION_TIMEOUT
This is because it took more than 60 seconds for the script to return output, Vercel which is server less are not designed for big tasks.
💪🔥🔥💜🧑💻🧑💻
Node 14 is now discontinued, so this one fails.
Thanks for pointing that out, will check and make a video on latest version of nodes soon, did you try changing version to 16?
Hello@@MichaelKitas, I forked your code in git, where I applied versions 16 and 18 of the node and both returned "504: GATEWAY_TIMEOUT Code: FUNCTION_INVOCATION_TIMEOUT" . I came to your video because I'm looking for a way to run puppeteer in production on Vercel, but so far I've only managed to do so in development mode.
very complicated config i guess there are others example more better than this !
Going to make an updated video today, stay tuned.
can you make a video but with nextjs as they have the api routes that we could use, i get an error for the aws lambda like this after downloading and need help (the problem shows after copying what you did here and it just can't seem to work)
let chrome = {};
let puppeteer;
let options = {};
if (process.env.AWS_LAMBDA_FUNCTION_VERSION) {
chrome = require("chrome-aws-lambda");
puppeteer = require("puppeteer-core");
} else {
puppeteer = require("puppeteer");
}
if (process.env.AWS_LAMBDA_FUNCTION_VERSION) {
options = {
args: [...chrome.args, "--hide-scrollbars", "--disable-web-security"],
defaultViewport: chrome.defaultViewport,
executablePath: await chrome.executablePath,
headless: true,
ignoreHTTPSErrors: true,
};
}
export const getBrowser = async () => {
console.log(AWS_LAMBDA_FUNCTION_VERSION)
return await puppeteer.launch(options);
}
- error ./node_modules/chrome-aws-lambda/build/puppeteer/lib/Browser.js.map
Module parse failed: Unexpected token (1:10)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See webpack.js.org/concepts#loaders
> {"version":3,"file":"Browser.js","sourceRoot":"","sources":["../../../source/puppeteer/lib/Browser.ts"],"names":[],"mappings":";;AAGA,IAAI,KAAK,GAAuB,IAAI,CAAC;AAErC,IAAI;IACF,KAAK,GAAG,OAAO,CAAC,4CAA4C,CAAC,CAAC,OAAO,CAAC;CACvE;AAAC,OAAO,KAAK,EAAE;IACd,KAAK,GAAG,OAAO,CAAC,iDAAiD,CAAC,CAAC,OAAO,CAAC;CAC5E;AAED,KAAK,CAAC,SAAS,CAAC,WAAW,GAAG,KAAK,WAAW,GAAG,KAAa;IAC5D,IAAI,IAAI,GAAS,IAAI,CAAC;IACtB,IAAI,KAAK,GAAW,MAAM,IAAI,CAAC,KAAK,EAAE,CAAC;IAEvC,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE;QACtB,KAAK,GAAG,CAAC,MAAM,IAAI,CAAC,OAAO,EAAE,CAAC,CAAC;KAChC;IAED,IAAI,GAAG,KAAK,CAAC,KAAK,EAAE,CAAC;IAErB,IAAI,KAAK,IAAI,IAAI,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,IAAI,EAAE;QAClD,KAAK,IAAI,IAAI,IAAI,KAAK,EAAE;YACtB,IAAI,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,CAAC;SACzB;KACF;IAED,OAAO,IAAI,CAAC;AACd,CAAC,CAAC;AAEF,IAAI,OAAO,GAAQ,KAAK,CAAC,SAAS,CAAC,OAAO,CAAC;AAE3C,KAAK,CAAC,SAAS,CAAC,OAAO,GAAG,KAAK,WAAW,GAAG,KAAa;IACxD,IAAI,IAAI,GAAS,MAAM,OAAO,CAAC,KAAK,CAAC,IAAI,EAAE,SAAS,CAAC,CAAC;IAEtD,IAAI,KAAK,IAAI,IAAI,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,KAAK,IAAI,EAAE;QAClD,KAAK,IAAI,IAAI,IAAI,KAAK,EAAE;YACtB,IAAI,GAAG,MAAM,IAAI,CAAC,IAAI,CAAC,CAAC;SACzB;KACF;IAED,OAAO,IAAI,CAAC;AACd,CAAC,CAAC"}
Okay, will check puppeteer with nextjs, and possibly make a series for that
@@MichaelKitas Hey Michael! Did you make a video on this? I'm also running on the same error with NextJS
Hi! I'm running into the same error. Did you manage to fix it?
@@csyago yeah no, i abandoned the project