Anjan GCP Data Engineering
Anjan GCP Data Engineering
  • 87
  • 347 950

Video

Basics of Service Accounts and IAM Roles
zhlédnutí 1,4KPřed 9 měsíci
Basics of Service Accounts and IAM Roles
Big Query BigLake Tables
zhlédnutí 1,6KPřed 10 měsíci
Big Query BigLake Tables
Learn code free Data pipelines with Cloud Data Fusion | Packed with detailed Demos and explanations
zhlédnutí 7KPřed 11 měsíci
Learn code free Data pipelines with Cloud Data Fusion | Packed with detailed Demos and explanations
Serverless Change Data Capture (CDC) , Replication , Data Synchronisation with Cloud Datastream
zhlédnutí 1,8KPřed 11 měsíci
Serverless Change Data Capture (CDC) , Replication , Data Synchronisation with Cloud Datastream
ETL nested AVRO files into Cloud SQL, GCS using Dataflow Pipeline
zhlédnutí 1,1KPřed 11 měsíci
Code : github.com/anjangcp/GCP-Data-Engineering-Demo-Codes/blob/9b8b7b5c07129188237965b266f6d73f0350d5a0/Dataflow/batch_etl_avro_data_cloudsql.py Must watch below videos before you watch this : czcams.com/video/hWpNyGj5xVk/video.htmlsi=is1OHok0P0XMDRsS czcams.com/video/dKGB3d2XXiE/video.htmlsi=CvdRfpAeyl8NYaDd
Automate GCP Project’s IAM Roles Snapshot process using Cloud Functions, Bigquery, Cloud Scheduler
zhlédnutí 481Před rokem
Ref Code: github.com/anjangcp/GCP-Data-Engineering-Demo-Codes/blob/698552fb8352876303e3d41a2e3ea56ae552d103/Common_Realtime_Usecases/iam_snapshots.py
GCP Professional Data Engineer, re certified ,Exam experience and my suggestions to Aspirants
zhlédnutí 821Před rokem
GCP Professional Data Engineer, re certified ,Exam experience and my suggestions to Aspirants
Selecting GCP ETL/Data processing services | GCP Professional Data Engineer
zhlédnutí 876Před rokem
github.com/anjangcp/GCP-Data-Engineering-Demo-Codes/blob/3fe03fe8304dd34f3cfb3ae9ab43f47acf44b180/GCP_Data_Eng_concept_files/gcp_etl_services.png
GCP Professional Data Engineer | Storage and Database selection process, Test your knowledge
zhlédnutí 512Před rokem
GCP Professional Data Engineer | Storage and Database selection process, Test your knowledge
GCP Professional Data Engineer | Selecting suitable storage and database services based on use case
zhlédnutí 1KPřed rokem
GCP Professional Data Engineer | Selecting suitable storage and database services based on use case
Cloud Big Table - Introduction | Console , CBT - Basic Demo with Examples
zhlédnutí 1,7KPřed rokem
Cloud Big Table - Introduction | Console , CBT - Basic Demo with Examples
Import Bulk data from GCS into Spanner Table using Dataflow Jobs
zhlédnutí 890Před rokem
Import Bulk data from GCS into Spanner Table using Dataflow Jobs
Basic Cloud Spanner database operations using Python Client Libraries
zhlédnutí 580Před rokem
Basic Cloud Spanner database operations using Python Client Libraries
Cloud Spanner Introduction demo with examples
zhlédnutí 1,2KPřed rokem
Cloud Spanner Introduction demo with examples
Automate Dataproc workloads using Cloud Composer
zhlédnutí 3,6KPřed rokem
Automate Dataproc workloads using Cloud Composer
Composer - Airflow https operators, Extract data from web API using https operators with examples
zhlédnutí 3,8KPřed rokem
Composer - Airflow https operators, Extract data from web API using https operators with examples
Thanks for your support !!! and More to come
zhlédnutí 91Před rokem
Thanks for your support !!! and More to come
Schedule secured data load from Google sheets to Big query|Cloud Functions,Cloud Scheduler and more
zhlédnutí 2KPřed rokem
Schedule secured data load from Google sheets to Big query|Cloud Functions,Cloud Scheduler and more
Big Query Clustered Tables with Examples
zhlédnutí 4,5KPřed rokem
Big Query Clustered Tables with Examples
Big Query Table Partitions with Examples
zhlédnutí 7KPřed rokem
Big Query Table Partitions with Examples
Google Cloud Storage Object Versioning and Auto Class
zhlédnutí 1,2KPřed rokem
Google Cloud Storage Object Versioning and Auto Class
Schedule Dataproc Workflows using Cloud Scheduler
zhlédnutí 1,8KPřed rokem
Schedule Dataproc Workflows using Cloud Scheduler
Cloud Scheduler Introduction with simple examples
zhlédnutí 4,9KPřed rokem
Cloud Scheduler Introduction with simple examples
Export and Analyze GCP Billing data with BigQuery and Looker Studio
zhlédnutí 3,3KPřed rokem
Export and Analyze GCP Billing data with BigQuery and Looker Studio
Google Cloud Storage (GCS) Object Lifecycle Management with Examples
zhlédnutí 4,2KPřed rokem
Google Cloud Storage (GCS) Object Lifecycle Management with Examples
Dataproc Workflow Templates | Execute Jobs DAG on Managed Dataproc Cluster
zhlédnutí 3,2KPřed rokem
Dataproc Workflow Templates | Execute Jobs DAG on Managed Dataproc Cluster
Big Query Routines , Stored Procedures and Anonymous Blocks for repeated tasks
zhlédnutí 4,2KPřed rokem
Big Query Routines , Stored Procedures and Anonymous Blocks for repeated tasks
Big Query Federated queries | Connect and execute Cloud SQL queries from Big Query
zhlédnutí 1,9KPřed rokem
Big Query Federated queries | Connect and execute Cloud SQL queries from Big Query
Cloud SQL Python connector | Access Cloud SQL Instance Programatically
zhlédnutí 3,6KPřed rokem
Cloud SQL Python connector | Access Cloud SQL Instance Programatically

Komentáře

  • @bhargavpasupuleti1329

    Welcome back please upload more videos on GCP services and GCP certification related

  • @VenkatesanVenkat-fd4hg

    Your videos are great, kindly maintain frequent videos

  • @AnitaRaut-e8c
    @AnitaRaut-e8c Před 5 dny

    I would appreciate your response in advance. How can we identify the path of a specific resource? For example, in the video, you specified the path as: projects/{project_name}/datasets/.... Do we need to grant additional privileges to our default service account for executing the Eventarc trigger?

  • @coder_foo
    @coder_foo Před 14 dny

    This is very clear, thank you for this excellent overview.

  • @psylocko
    @psylocko Před měsícem

    Is it expensive to use Cloud Composer?

  • @kishorereddy3155
    @kishorereddy3155 Před měsícem

    how to contact you

  • @krishnanitturkar2793
    @krishnanitturkar2793 Před měsícem

    Thank you so much. It helped me a lot to understand difference between native and external table clearly. Thank you so much

  • @sunkarihari-x6w
    @sunkarihari-x6w Před měsícem

    Hi can you share sample xl data file this video

  • @narayanakp
    @narayanakp Před měsícem

    excellent Anjan, will you provide online training.

  • @analystakash
    @analystakash Před měsícem

    good

  • @ankitsoin5185
    @ankitsoin5185 Před měsícem

    Great explanation. I have 1 doubt that which flow is good Cloud scheduler scheduling dataproc workflows Or Scheduling dataproc jobs using Cloud composer? Can you pls help with use case sir

  • @harinireddy2875
    @harinireddy2875 Před měsícem

    Hi Anjan, I added test Cvs file in gcs bucket manually but I am unable to open in cloud storage default. Could you please explain why?

    • @anjangcpdataengineering5209
      @anjangcpdataengineering5209 Před měsícem

      @@harinireddy2875 are you not able to open the content of the CSV file ? Did you check the ACL of that bucket?

    • @harinireddy2875
      @harinireddy2875 Před měsícem

      Yes, access is present

    • @harinireddy2875
      @harinireddy2875 Před měsícem

      @@anjangcpdataengineering5209 when I am trying add the file which is separated with delimiter , it’s throwing an saying body_21 can not be null and with similar errors. I want add the files how is it and do transformations later. (I am not able on header toggle also .. if I do throwing an error with some error connection.

  • @ravipandey7488
    @ravipandey7488 Před měsícem

    Thank you for this awesome video, it explained the flow very well. Could you please also upload a similar video where we can loan on premise data in bigquery using composer. If the video is already there, please put the link in the comment.

  • @Maturshab2210
    @Maturshab2210 Před 2 měsíci

    How to schedule these sir

  • @AshishDukare-vr6xb
    @AshishDukare-vr6xb Před 2 měsíci

    Thanks for this video. Keep making more such videos

  • @AshishDukare-vr6xb
    @AshishDukare-vr6xb Před 2 měsíci

    what is this event and where it is coming from?

  • @TheSoftwareServices
    @TheSoftwareServices Před 2 měsíci

    Hope your health is good, great dedication and commitment Anjan

  • @TheSoftwareServices
    @TheSoftwareServices Před 2 měsíci

    great presentation, informative session tq Anjan

  • @daisyratto6647
    @daisyratto6647 Před 2 měsíci

    Thank you! This explanation help me a lot!!

  • @meenakshirandev4917
    @meenakshirandev4917 Před 2 měsíci

    Nice 👍

  • @TheSoftwareServices
    @TheSoftwareServices Před 2 měsíci

    informative session, excellent presentation

  • @lakhdariamine6794
    @lakhdariamine6794 Před 2 měsíci

    How to do this by terraform please

  • @santoshkumar-pk6xt
    @santoshkumar-pk6xt Před 2 měsíci

    Excellent usecase. Very helpful

  • @tammyajadi1574
    @tammyajadi1574 Před 2 měsíci

    Really good! Thank you so much

  • @TheSoftwareServices
    @TheSoftwareServices Před 2 měsíci

    its great and systematic presentation of content, highly appreciate your efforts, god bless you Anjan.

  • @meenakshirandev4917
    @meenakshirandev4917 Před 2 měsíci

    Nice 👍

  • @ReNewSelf
    @ReNewSelf Před 2 měsíci

    Super useful. Thank you for the post~

  • @Manuveerlapati
    @Manuveerlapati Před 2 měsíci

    Is there any way I can find path for csv file (greenhouse_dtls) file to download?

  • @pranaydhareshwar7755
    @pranaydhareshwar7755 Před 2 měsíci

    Thank you so much for your video. It has helped me a lot.

  • @digambarbhat8095
    @digambarbhat8095 Před 2 měsíci

    Your videos are very good and deserves good reach. Thanks for sharing all these videos.

  • @OmkarMusale-nd9wn
    @OmkarMusale-nd9wn Před 2 měsíci

    nicee

  • @suneelpaidi1311
    @suneelpaidi1311 Před 2 měsíci

    I was assigned to GCP project from JAVA development and was told to complete the course and do the certification. Your videos are helping me a lot. Also if you are active now, please make latest tools videos also and also IAM related videos etc. It will help a lot of IT guys like me. Thank you for your videos.

  • @AshishDukare-vr6xb
    @AshishDukare-vr6xb Před 2 měsíci

    bash_task = BashOperator( task_id='bash_task', bash_command='date:echo 'Hey I am BashOperator' ', ) In above code, I thing dag=dag line is missing Please suggest on this

  • @sharathkumar2506
    @sharathkumar2506 Před 2 měsíci

    Thank you

  • @JesusAlbertoGuerraPerez
    @JesusAlbertoGuerraPerez Před 2 měsíci

    hola, como podria subri un archivo desde local a Cloud storage usando cloud function?, Gracias

  • @endasil
    @endasil Před 2 měsíci

    I really enjoyed your first video in this series but please, try to keep the intro and the rest of the video the same volume. I had the volume turned up to max since the voice was soo low in the previous video, then i came to the next part and the introduction is EXTREMELY loud compared to the volume in last video. It was a very painful experience for my ears.

  • @ajaybhamare8237
    @ajaybhamare8237 Před 3 měsíci

    I need to connect to a third party API i want to do it via cloud composer in gcp basically airflow is it possible to do it and move the data to gcs how can i do ot

  • @DarkShadow-321
    @DarkShadow-321 Před 3 měsíci

    Awesome

  • @Rajesh-wr9im
    @Rajesh-wr9im Před 3 měsíci

    Nice and excellent content, and also very well explained the concepts with screen navigation steps. Are you giving any in person or online trainings? Can I get your email id?

  • @tanoojm1696
    @tanoojm1696 Před 3 měsíci

    Yeah the voice is so bad , couldn't hear most of the words properly

  • @abhijeet2218
    @abhijeet2218 Před 3 měsíci

    How to do the a ove for 100 Gbs of data that is going to come in the buckets

  • @AlejandroGarciaLagos
    @AlejandroGarciaLagos Před 3 měsíci

    Very useful, thanks

  • @abaghel007
    @abaghel007 Před 3 měsíci

    Sir how to reach out to you?

  • @abhisheknaidu8877
    @abhisheknaidu8877 Před 3 měsíci

    while creating the DAG i am getting error as SQLite cannot found , it is asking to change the configuration can you please suggest me to what could be done and suggest me the video which i can work on dependencies and fixing the error t

  • @sivaprasadanisetti7252
    @sivaprasadanisetti7252 Před 4 měsíci

    Is it possible to create Partition on already existed table in BigQuery?

  • @Ar001-hb6qn
    @Ar001-hb6qn Před 4 měsíci

    Hi @Anjan, I am unable to create a GCP Composer environment. After around 45 minutes it shows the error "Some of the GKE pods failed to become healthy. Please check the GKE logs for details, and retry the operation." I have configured the setting and given the necessary access. I am using composer-2.7.0-airflow-2.7.3. But it failed to create the environment. Can you please help with this? Thanks.

    • @Aadit_Shah
      @Aadit_Shah Před 3 měsíci

      Did you find any solution? I am facing the same problem.

    • @kaustubh747
      @kaustubh747 Před 2 měsíci

      Are you using a trial account? The composer would need more resources that the trial account does not provide

  • @swamyve
    @swamyve Před 4 měsíci

    Thank you for providing detailed video. Can you please able provide the dataset for testing ?

  • @Ilovefriendswebseries
    @Ilovefriendswebseries Před 4 měsíci

    Excellent Content and brief explanation

  • @aliel-azzaouy7007
    @aliel-azzaouy7007 Před 4 měsíci

    first thank you for this amazing coursers , seconde can you pleas sheare with us the presentation of all this playliste