Thank you for the explanation! Can please someone confirm that the files can only be read from FileStore and not blob storage by this approach (SQL)? Tried reading it from blob storage and cannot access using cloud_files() function. I guess it requires pyspark to access blob storage and create delta live tables.
Nice demo, but why does the comment for the "cleaned_station_status" table say "partitioned by station_id" when the code actually uses the last_updated_date column? You should update the comment in that notebook. :-)
He mentions the python scripts/notebooks that get the data. Probably using an api and saving results to DBFS. I’m sure you can find how to do that in other videos.
Great video, easy to understand, good overview for the Databricks beginners. Thanks!
This is great. Can you share notebook for querying the transaction log and presenting it in reDash?
Well explained. Thank you
Do you have a link to show how the queries work for monitoring the data quality. Thanks
GREAT VIDEO
Thank you for the explanation! Can please someone confirm that the files can only be read from FileStore and not blob storage by this approach (SQL)? Tried reading it from blob storage and cannot access using cloud_files() function. I guess it requires pyspark to access blob storage and create delta live tables.
Nice demo, but why does the comment for the "cleaned_station_status" table say "partitioned by station_id" when the code actually uses the last_updated_date column? You should update the comment in that notebook. :-)
Thank you for this video. I'm a little confused about what the "data.stations' refers to? Is it an array in the source json?
am I missing something or does the video really doesnt show how he got all those files in the data lake in the first place?
He mentions the python scripts/notebooks that get the data. Probably using an api and saving results to DBFS. I’m sure you can find how to do that in other videos.
Hes using autoloader to load the data probably from an s3 bucket, Azure cloud storage or volume
Anyone knows how to create a live dashboard like this in databricks?
Good video thank you. Quick question, is the DLT lineage also auotmatically avaialable and visisble in Unity Catalog?
Can an API hosted on an App service in anyway fetch Delta live tables data ? thanks
It’s been in gated preview for too long, when will it be made GA?
Today😃
@@adityaprakash6383 is this GA Aditya?
@@noorbashashaik2124 yes