Convert Parquet to Delta Format/Table | Managed & External Tables | Using Databricks | LearntoSpark

In this Video, we will learn to how to convert the parquet file format to Delta file format or delta table. We will also discuss on what is the difference between Managed and external table with one example.
Data used for this demo:
github.com/aza...
======================================================
Also check Other videos on delta:
Understanding Delta: • Delta Lake in Spark | ...
Understand concept of Vacuum and optimize: • Spark Delta Lake | Vac...
Understand Time-Travel: • Delta Lake in Spark | ...
Update and Delete in Delta : • Delta Lake in Spark | ...
Understand Schema Evaluation: • Delta Lake in Spark | ...
=======================================================
Blog link to learn more on Spark:
www.learntospark.com
Linkedin profile:
/ azarudeen-s-83652474
FB page:
/ learntospark-104523781...

[@matthewleon687]

youre a gem. Might've just saved my job

[@AzarudeenShahul]

Thanks for your support 🙂

[@Learn2Share786]

Great piece of info, thanks!

[@vijayashingote1148]

how to read c000.snappy.parquet file which is stored in dbfs location?
I am trying to read using spark.read.parquet(path) I got an error "incompatible file format" can you help me with this how to solve this error

[@vinayak6685]

❤️❤️ clarified a lot of my doubts

[@sivakumaresakkipillai642]

how to update records in particular partitions, can you suggest me how can I built in databricks.

[@joyyoung3288]

thanks very much for the clip, but the command: df_parq = spark.read.format("parquet"). load("dbfs:/pipelines/bob/tables/organic_farmers_market") returned the error as "
AnalysisException: Incompatible format detected." can anyone help? thanks

[@rajdeepsinghborana2409]

Please make video on that ,
Using structured APIs in Spark how to connect hive , Hbase , sqoop...

[@pavanp7242]

Nice explanation

[@gomangogalaxy6849]

We can directly convert to Delta right? Why using CSV->parquet->Delta format? Why not Dataframe->DeltaFormat?

[@AzarudeenShahul]

Yes, from df u can directly write it as delta. But in this scenario, i m trying to demo how to convert an existing parquet file to delta without changing the path.. hope u understood

[@aashish72it]

This is really great content

[@user-ez7he9iv5v]

Why we are converting the csv to parquet in the beginning, can't we direct change the csv to delta?

[@AzarudeenShahul]

As u know, there are still some project which uses target as parquet and wanting to convert to delta.. this is one of usecase that deal with conversion.. so to create a parquet, we use CSV in first.. hope u got it

[@mateen161]

Nice video. Thank you!

[@subhabasu8596]

Can the external table be converted to managed table

[@saurav0777]

Can we convert Hive table stored as ORC into the Delta table ?

[@AzarudeenShahul]

Yes we can achieve

[@prasannaboyapati3087]

Could u please upload all ur data sets used in demo into ur github link..?