![Tamarick Hill's Azure Data Training](/img/default-banner.jpg)
- 26
- 28 927
Tamarick Hill's Azure Data Training
United States
Registrace 16. 01. 2023
This channel focuses on Data related topics within the Microsoft Azure Cloud ecosystem, specifically SQL Server, Azure SQL, Azure Synapse, Azure Cosmos DB, and Azure Databricks. I am a Principal Cloud Solution Architect with more than 15 years of experience working with data solutions! My goal is to share knowledge about Azure Data Services and best practices around those services. Thanks for viewing and supporting my channel!
Meet Azure SQL Database Watcher
Azure SQL Database Watcher is a new fully managed monitoring solution for Azure SQL. In this video we will walk through the setup process for database watcher and review some of the dashboards that are generated to monitor your Azure SQL estate.
At the time of this recording, Database Watcher is in Public Preview and currently support Azure SQL Database (including elastic pools) and Azure SQL Managed Instance.
learn.microsoft.com/en-us/azure/azure-sql/database-watcher-overview?view=azuresql
At the time of this recording, Database Watcher is in Public Preview and currently support Azure SQL Database (including elastic pools) and Azure SQL Managed Instance.
learn.microsoft.com/en-us/azure/azure-sql/database-watcher-overview?view=azuresql
zhlédnutí: 919
Video
Azure SQL Managed Instance General Purpose Next Gen!!!
zhlédnutí 327Před 5 měsíci
Microsoft has recently announced the Next Gen General Purpose service tier of Azure SQL Managed Instance. In this video we will take a look at some of the new enhancements for this Next Gen offering. The blog post announcement shown in this video can be viewed at the link below: aka.ms/sqlmi-nextgp
How To Use a Pipeline to Load a Fabric Lakehouse from a REST API
zhlédnutí 1,1KPřed 6 měsíci
This video will provide you with a quick solution for loading a Microsoft Fabric Lakehouse from a REST API using a Data Pipeline.
Dynamic Data Masking in Microsoft Fabric
zhlédnutí 563Před 9 měsíci
This video will walk you through a quick demo of how to implement Dynamic Data Masking in a Microsoft Fabric Warehouse. We will walk through the different mask types that can be applied to your data and the resulting outcomes.
How To Implement Row Level Security Inside of a Microsoft Fabric Warehouse
zhlédnutí 1,5KPřed 10 měsíci
As part of the September 2023 Update for Microsoft Fabric, we now have the ability to implement Row Level Security inside of a Fabric Warehouse or SQL Endpoint. This video will provide a quick tutorial on how you can implement the feature.
What is Azure Data Share
zhlédnutí 646Před rokem
Azure Data Share is a service that allows you to securely share Azure Data sources such as Azure SQL Database and Azure Data Lake Gen2 with external vendors, partners, and customers. This video will provide a brief overview and demo of the service.
Azure SQL Managed Instance - General Purpose vs. Business Critical Tiers
zhlédnutí 1,5KPřed rokem
This video will provide you with of quick overview of the key differences between the General Purpose and Business Critical Tiers of Azure SQL Managed Instance. After watching you will be better equipped to decide on the correct tier for your workloads.
How To Access On-Prem Data Sources From Microsoft Fabric
zhlédnutí 6KPřed rokem
This video will walk you through how you can use On Prem Data Gateways and Microsoft Fabric to access On-Prem Data Sources. At the time of this recording, the gateway is only supported for DataFlow Gen2, so we will use that for the demo. learn.microsoft.com/en-us/data-integration/gateway/service-gateway-install
Multi-Cloud Analytics with Microsoft Fabric & Microsoft OneLake
zhlédnutí 265Před rokem
Microsoft Fabric is a new Unified SaaS based Analytical Platform offered by Microsoft. Its foundation is Microsoft OneLake which is a data store for all of your organization’s analytical data. Microsoft OneLake is the first multi-cloud datalake, and through the Shortcuts feature, we are able to easily and seamlessly perform analytics on data sources that may be spread across different clouds. T...
What Is Microsoft OneLake?
zhlédnutí 2,5KPřed rokem
This video will provide you with an overview of Microsoft OneLake that is provided as a part of Microsoft Fabric. learn.microsoft.com/en-us/fabric/get-started/microsoft-fabric-overview learn.microsoft.com/en-us/fabric/onelake/onelake-overview
How To Purchase Microsoft Fabric (Public Preview)
zhlédnutí 563Před rokem
This video will walk you through the options for purchasing Microsoft Fabric so that you understand the options available for your organization and can make the best decision. Buy a Microsoft Fabric Subscription: learn.microsoft.com/en-us/fabric/enterprise/buy-subscription Microsoft Fabric Licenses: learn.microsoft.com/en-us/fabric/enterprise/licenses Microsoft Fabric Capacities: blog.fabric.mi...
New Service Announcement - Microsoft Fabric (Public Preview)
zhlédnutí 478Před rokem
Microsoft Fabric was announced on 05/23/23 by Microsoft During the Microsoft Build Conference. Fabric is a new SaaS Based Data Analytics Platform. This video provides a quick overview with more deep dives coming in the upcoming weeks and months.
Getting to Know Azure Data Services Series: Azure Databricks
zhlédnutí 85Před rokem
Episode 7 of our Getting to Know Azure Data Services Series. In this episode we will provide a high level overview of Azure Databricks
Getting to Know Azure Data Services Series: Azure Data Factory
zhlédnutí 70Před rokem
Episode 6 of our Getting to Know Azure Data Services Series. In this episode we will provide a high level overview of Azure Data Factory.
Getting to Know Azure Data Services Series: Azure Stream Analytics
zhlédnutí 100Před rokem
Episode 5 of our Getting to Know Azure Data Services Series. In this episode we will provide a high level overview of Azure Stream Analytics.
Getting to Know Azure Data Services Series: Azure Data Lake
zhlédnutí 111Před rokem
Getting to Know Azure Data Services Series: Azure Data Lake
Getting to Know Azure Data Services Series - CosmosDB
zhlédnutí 74Před rokem
Getting to Know Azure Data Services Series - CosmosDB
Getting To Know Azure Data Services - Azure Database
zhlédnutí 112Před rokem
Getting To Know Azure Data Services - Azure Database
Getting to Know Azure Data Services - Azure SQL
zhlédnutí 180Před rokem
Getting to Know Azure Data Services - Azure SQL
How To Access Azure DataLake From Azure Databricks Using Service Principals
zhlédnutí 1,7KPřed rokem
How To Access Azure DataLake From Azure Databricks Using Service Principals
How To Restore TDE Protected Databases From AzureSQL Managed Instance To SQL Server 2022
zhlédnutí 1,2KPřed rokem
How To Restore TDE Protected Databases From AzureSQL Managed Instance To SQL Server 2022
How To Use Customer Managed Keys with Azure SQL Managed Instance
zhlédnutí 1,3KPřed rokem
How To Use Customer Managed Keys with Azure SQL Managed Instance
Manage Databases With Azure SQL Elastic Pools
zhlédnutí 3,7KPřed rokem
Manage Databases With Azure SQL Elastic Pools
3 Tips To Optimize & Save Money With Synapse Serverless SQL Pools
zhlédnutí 142Před rokem
3 Tips To Optimize & Save Money With Synapse Serverless SQL Pools
What Is Azure SQL Managed Instance Link?
zhlédnutí 584Před rokem
What Is Azure SQL Managed Instance Link?
Great video.
Thank you!!
do we need to perform the whole steps for restoring all the databases or is it a one time process?
Granting your SQL Server instance access to your key vault should be a one time process.
Hi. Quick question, @ 3:38 you tried to add users, how SSMS authorisesd them, were they already given access to the database, and from a active directory? Thanks in advance.
Yes these users are both from my Entra tenant and they were already provided the necessary permissions to access the database.
please provide the documentation reference page.
You can start out with this: learn.microsoft.com/en-us/azure/azure-sql/managed-instance/restore-database-to-sql-server?view=azuresql&tabs=managed-identity
@@DataBar for me the assymetric key is giving error. Key with name 'tdekey' does not exist in the provider or access is denied. Provider error code: 2058. (Provider Error - No explanation is available, consult EKM Provider for details)
@@DataBar Key with name 'tdekey' does not exist in the provider or access is denied. Provider error code: 2058. (Provider Error - No explanation is available, consult EKM Provider for details) I am not able to create the asymmetric key.
Hi, awesome vid. I need to connect my Fabric data with SAP that its in a VPN.... what can i use? thanks
Have you looked into using an on-prem data gateway with the SAP connectors?
Excellent..
Glad you enjoyed it!
How to do RLS in Lakehouse?
Have a look at this and it should help you. RLS is only supported on a Lakehouse through the SQL Analytics Endpoint. learn.microsoft.com/en-us/fabric/data-warehouse/tutorial-row-level-security
awesome video man!! Ive been looking for this solution the hole day!! since the Manage Roles in my PowerBi Desktop report is blocked, I guess because I connected my Report to the Warehouse semantic model.. this is the solution to RLS my diferent viewers, isnt it? My main table with sales dosnt have a column with the email, if I upload a table and relation it with the main sales table, is it enough¡? thanks bro!!
Thanks. If your semantic model is based on a star schema, you can filter your fact tables and slice based in your dimensions. So yes the relationships are key in the scenario you described.
It would helpful if you add video on how to load data from Oracle on prem to fabric using Notebook.Thanks
We recently launched a new Oracle connector. You can use this connector in addition to an on-premise gateway to access oracle on prem data. support.fabric.microsoft.com/en-ca/blog/announcing-new-data-factory-connectors-released-in-may-2024?ft=All
Great info and very informative 💪🏽…I’m currently a production SQL DBA on the road to becoming a Data Engineer so I enjoy your content. You gained a new subscriber 😊💪🏽
I appreciate it!!! 💪🏽
can we access SAP data in Microsoft fabric through On Prem Data Gateways ? I dont see any information on the internet that we are the source data that is accessible through MS fabric. If you know pls reply. thanks
at 5:31 in video , can you tell what is there in "more"
Yes SAP data can be accessed through an on-prem gateway. Check out this link for more details. blog.fabric.microsoft.com/en-us/blog/integrate-your-sap-data-into-microsoft-fabric/
Is the CMK works on auto failover group? if yes, then how key volt will be accessible by secondary database? Please help me to understand. Any documentation will be much appreciated
Just like we provided the primary instance with access to the key vault, we need to provide any secondary instances with that same access to the key vault.
Best and easiest way to explain the Elastic Pool.
Glad it helped!
Very useful and well paced video
Thank you! Please like and share!
Excellent video
Thank you!!
whats the latency of moving databases out of latency pool or increasing service tier of it. How to deal with noisy neighbour problem and how to mitigate the risk of kicking out the noisy neighbour from the pool without disruption or latency ?
Increasing a service tier can vary depending on if the current underlying VM has the resources to allocate to your instance, or if it needs to be moved to a different instance. However the downtime is minimal as some of this work involved with potentially moving to a new VM happens in the background while your current instance continues to run, and then we will do, in essence a DNS change, at the end. Moving databases into a new elastic pool is a relatively seamless experience. Within elastic pools, your can limit the amount a vCores that a database can consume from that pool helping to put guardrails around noisy neighbors.
@@DataBar my main concern is kicking out the noisy neighbour from the elastic pool to a dedicated azure SQL instance
Understood. That’s why it’s important to set up good monitoring and alerting procedures so you can identify things like that and take appropriate action!
So you either need to be using a Warehouse in Fabric or a SQL Endpoint. Therefore if I go into the Warehouse as my user I will only see my users information but if I go into a Lakehouse then this filtering will not be applied?
From the Lakehouse you are either using the SQL Endpoint to access data with T-SQL in which the row level security will apply, or you are accessing the data via Spark and hitting the underlying delta parquet files and RLS will not apply in that scenario.
nvarchar is not a supported data type?
That was at the time of this recording. nvarchar should be supported now as long as it’s not nvarchar(max) as LOB data over 1MB is unsupported.
just install 1 on prem data gateway in local PC to connect to multiple on prem databases? like oracle/SQL server etc
One gateway can be used to connect to multiple on-prem databases.
Great video Tamarick and at a pace I can follow and sufficient clarity through the process. Thanks
Thank you. Glad it was helpful!
Hey! Great video! Is it possible to use dynamic data masking for contributor users?
Any user with Admin, Contributor, or Member permissions on the workspace will be able to see unmasked data. If you need it to be masked for a user, either give the Viewer workspace role, or grant more granular SQL permissions using the GRANT statement.
can you do the other way around, accessing a databricks service (example search index) from an app on azure? What is the difference between Creating service principlal on azure and the one on databricks?
Could I access to SQL Server On premise with Notebook in Fabric?
I am not aware of a way to do this via notebooks. The ideal solution would be to use Fabric Mirroring to mirror your SQL Server database into OneLake. Currently the mirroring feature is in private preview and only supports Azure SQL DB, CosmosDB, and Snowflake, but sql server will be coming soon.
Please add more tutorials
Will do
What if more than 1 user need to access the on-prem data? Does every user need to install their own data gateway on the server?
No each user does not need to install their own gateway. The admin of the gateway, usually the person that installs it, can grant additional users permission to use the gateway. See the below doc for more info: learn.microsoft.com/en-us/data-integration/gateway/manage-security-roles
What if you want more than 1 user to access the on premise data? Do you have to make a gateway for each user?
No each user does not need to install their own gateway. The admin of the gateway, usually the person that installs it, can grant additional users permission to use the gateway. See the below doc for more info: learn.microsoft.com/en-us/data-integration/gateway/manage-security-roles
Really a great video, helped a lot.
Thanks! Glad it helped!!!
Easy to follow along. Thank you. Subscribed.
Thank you!
Very well explained
Glad it was helpful!
Hey Tamarick, do you have a reference how to configure the credentials as Datarbciks cluster configuration?
This video helped me a lot. I struggled for 4 days to mount Datalake on Databricks. This video explained exactly what I need. Thanks a lot for creating this video.
Thank you! Glad it was able to help!!
Would love to have more fabrics content, great job!
ok, but can i use on premise on data factory? not in dataflow gen 2? or KQL?
On prem gateways are used to access data on-prem and are currently only supported within Gen2 Dataflows.
That's amazing .... wanted someone to guide it in better way ... Luckily got this video .... Hope you will continue it n explore everything related to data science stuff which can empower data analysts with DS stuff.... 🎉
Glad this was able to help you!!!
i am getting this error after i publish error: GatewayCannotAccessSqlError Couldn't refresh the entity because of an internal error any idea how to fix it?
I would need to review it to really understand what’s going on. I recommend you open a support case for assistance.
Well done. Showing a lot of good detail. Thanks much.
Thank you! I appreciate you watching!!
Very good video. I"m learning. Still need hands on but that's coming.
I agree, I need hands on as well
Best of luck!
East Coast Sr - I plan tio complete all 7 videos in this series. Will have questions so I plan to reach out. Started with your video on Fabrics. Looking to take a deep intio Azure ecosystem. Tks for what you do
Thank you!!!
Just found your site today. Where have I been? 70 yrs old and retired. Now out of retirement to learn new technologies. Starting with Fabric. Will check out your vast collection of Microsoft related videos. Tks for sharing and being so passionate about what you love. It show! I will have questions and plan to reach out. 😅 East Coast Sr.
Thank you for watching sir! I appreciate it!!!
I will keep watching and learning. As I have questions I will reach out Keep doing what you do! I greatly appreciate it! 😊 ❤
Thank you! I appreciate you!
This video was very helpful to me as well! Thank you!
Thanks for watching!!
is Clustered columnstore index appropriate for real time replication? I saw blogs on why you shouldn't use indexes on columns that gets updated frequently. could you explain in brief please?
Indexes help with data retrieval, but they slow down data writes. In a real-time replication scenario you have lots of write activity taking place. Each time you write new data, any indexes you have will need to be updated, thus impacting performance.
Since the solution is centered around BC and zone redundancy could you use lb as opposed to traffic manage to reduce cost ?
Azure SQL only allows one replica to serve both read and write workloads so load balancing isn’t really a concept here. You can redirect read-only workloads to a secondary replica. Does this answer your question?
Great explanation to make decisions
Glad it helped!
very helpful information
Glad it helped!
The thing is, the replica is still in the same region right? So you're probably better off using GP and creating your own Auto-failover group with a second MI in a second region? That way you've got multi-region HA, which is better than what Business Crit offers.
It just depends on your use case. If your main goal is offloading read-only workloads then using the included replica of the BC tier meets that purpose. The replica is there for high availability purposes and to ensure Microsoft can maintain its SLA’s. But if you are looking for a solution for disaster recover, then, as you stated, auto-failover groups would be the best solution for that use case.
great explanation
Glad it was helpful!
Fantastic Video. Well explained and very detailed. Thanks
Glad it was helpful!
Great! But how did you do that: on 1:50 you have win10 inside win 11, that is inside macOS ? Have you 3 VMs ?
MacOS is my host OS. I then have an RDP connection to a Windows 11 VM. At the 1:50 mark I am running the SSMS app connected to a SQL Server 2022 instance hosted on Windows Server 2022.
Great video! when are they going to add support for more external sources like Google Cloud Storage?
We don’t have a date to announce publicly yet but other providers are in the works.
Thnx a lot. You really helped me!
Thanks for watching and glad I could help you!!