Video

hI, I've been trying to do an exercise which consist on ingest data from an website (currencies), store them, and then show in a graphic the data collected, that's very simple to say but very difficult for me to do it, do you have any information I will really appreciate it. I have the API key from the source of data

Johnny the speed comes from partition by column we use while creating? Like if I used a different column insyead of date and and used the date related queries , will it still be faster or not?

Is this legit

Thanks! You helped me a lot! 😁

Thank you for this, and you have a most delightful accent.

Is the Gold folder redundant ? Seems like t is not needed. Or will it only be used if data in silver still requires further transformation ?

Sometimes you hide your owner account id, othrtimes you can't be bothered 😆 . Thank you very much for your tutorials. You are the best!

Hi all, when creating iceberg table in Athena , I get " Exception encountered when executing query, this query ran against ...... database, unless qualified by the query . please post the error message on our forum ....., anyone know the solution ?

just amazing 🥳

!!!

For a very large dataset (like around 15 billion rows overall) is it going to give good performance if we use iceberg to select/delete/update ?

Is there a way to overwrite the already present table? I cannot find this option anywhere at all.

line 3:5: mismatched input 'SYSTEM_TIME'. Expecting: 'TIMESTAMP', 'VERSION' I'm getting this error while running the timestamp querry. can you please tell me why?

thanks man

Can we create an iceberg table to S3 using multi region access point?

7:06

Today, Aurora is costlier and Aurora serverless is even costlier!!

Liked and subscribed 🤟

The lambda function is not accepting the python codes as they are of previous version of python. What should I do?

Hi Johnny, really appreciate your video. But when I created crawler in free trail access I am getting below error. Is there anything that you can help me on this? "One crawler failed to create The following crawler failed to create: "crawler_customer_csv" Here is the most recent error message: Account *************** is denied access."

Another good video from the Chiverse.. :)

amazing

You are amazing and a natural teacher !!

Is this guy Scottish or Jamaican? Never heard an accent like this before it’s wild

amazing work

You are amazing❤

best video

thanks for easy to follow video ...looking foreword to more such content on azure

You are Awesome. watching in 2024... ETL steps needs minor updating but I was still able to follow ! Keep up the great work !

thank you so much

Very good video, thank you! God bless - Matthew 11:28

Very thoroughly described. Thank you

Hey Jonny, there were only 2 rows in the bronze/ingest object which you pulled using Firehose, how come there are so many rows after the glue job to silver layer?

Amazing.

Hi thanks for your content. I got the following error while create CFN stack "Please check the role provided or validity of S3 location you provided. We are unable to get the specified fileKey: modules/599e7c685a254c2b892cdbf58a7b3b4f/v1/flink-sql-connector-elasticsearch7_2.11-1.13.2.jar in the specified bucket: ee-assets-prod-us-east-1" Do we need to download the .jar file and upload manually to S3 to make it work ?

This was an amazing tutorial. I understood every bit of it because of the way it was explained with hands-on. Loved hand typing of all commands which seemed very real world scenario. Thank you so much Johnny!

happy new year

Thanks @Johnny Chivers. This video unlocked a lot of confusions I had with RDS and Aurora. But doesn't Aurora global databases provide fault tolerance against Region outage?

Great tutorial - but a pro tip. You *totally* need a keyboard. Ideally one that really fits your finger size, pressing force and such. I mean it as an advice, not as a rude comment though. Give some a try. :)

I have not heard PIG in forever, really enjoyed that language.

I feel like now I am zero to a noob. It will take sometime to be a hero :)

Around 26 minutes after you queried the deleted data it said it scanned 5.76MB. That seems like a lot for just metadata!

thank you sir

@25:00 "I'll talk about connections quickly", LOL! That's what AWS Glue, Azure Data Factory, SSIS, Informatica, are all about: CONNECTIONS! You are moving data from a source to a target, and to do that, you need to be connected to both, the source and the target. Basically, you are an S3 guy, LOL!

Thanks for the video! What do you think about using ECS/EKS to run your python ETLs inside docker containers? so you can execute your tasks/pods from MWAA after. In case you don't need spark, could be an alternative to EMR and cheaper than Glue.

So clear! Thank you!

Awesome !!! Johnny...

Love your vids. Could you maybe do a vid on airflow hosting on fargate + simple pipeline? Something practical

Dude, your videos are so helpful, I got a Data Engineer job after practicing with your videos and they are still helpful.. More power to you man, I hope you get more success.

Thanks!