What's the real difference between a Power Automate flow and a Fabric Data Pipeline? (Great content BTW. We appreciate you covering all the new Fabric tech)
I would say Data pipelines are more data related. For example the copy data activity can use a compute power behind the scene to enable movement of trillions of rows.
HI Reza, I have a question related to copy between workspaces in Fabric. Can I copy reports, notebooks or pipelines from one workspace to the other? please advice.
For a first idea nice video👍 is there much data in the dataflow loaded because step did not finish till end of your video? Do you know which Fabric SKU is used in background to understand the performance levels better? Mana thanks 🤗
It did finish, if you check the photos in my article. in the video I did not wait for the execution to finish as it would have taken time. I am using Fabric's trial license.
Hi, although the below comments reference this, it's not entirely clear to me, are you saying that we cannot create a pipeline or gen2 dataflow that will populate a Fabric DW or Lakehouse with on-prem SQL server tables?
Hi. Not sure where I mentioned that. Perhaps the one you are talking about is the one asked if there is an on-premises version of Microsoft Fabric, which I said no. You CAN certainly create pipeline or dataflow gen 2 to load data into Warehouse or Lakehouse using on-prem sources or cloud sources.
The data in the report comes usually from a dataset. You can have a Dataflow get data from Dataset or even using XMLA endpoint and getting from AAS. However, if you want to get data from the report visual itself (to cover for measures etc), then the answer is no at this stage. You can do workarounds though, for example fetch the DAX code for the visual (using the performance analyzer), then run that using Dataflow and Data Pipeline on the XMLA endpoint. I might do a video on this sometime.
Thanks. It is very helpful I Still not Connected LakeHouse to my Local Computer what ever MS SQL Server or Excel file Since 2days This the Erroer "Connection could be created"
How many custom languages do they need? I can honestly think of at least four other ways of calling a today() function under Microsoft’s roof. Why the hell do they do this? These languages should at a bare minimum resemble each other. It pissed me off and makes me not want to invest in this.
I agree. Too many expression languages and each different. The problem is that each language comes from a different product and team, and now when they are consolidating all of these, it seems each is totally different. Let's hope Microsoft invest on bringing them all under a same layout at one stage. That said, in the world of tech, there are many language anywhere; R, Python, XAML, C#... That is part of the tech industry.
I don't understand why they keep making more complex stuff, more languages, and way more lot of buttons instead keeping just to azure. I'd rather stick to code in asembler than this.
Thanks. The firehouse of new info continues.
😉😊
What's the real difference between a Power Automate flow and a Fabric Data Pipeline?
(Great content BTW. We appreciate you covering all the new Fabric tech)
I would say Data pipelines are more data related. For example the copy data activity can use a compute power behind the scene to enable movement of trillions of rows.
Can you add actions to trigger a Power Automate flow (instead of Office 365 Outlook action)? That would be really powerful combination!
What a nightmare. I can't imagine a slower, unnecessarily rigid way of pipeline development.
HI Reza, I have a question related to copy between workspaces in Fabric. Can I copy reports, notebooks or pipelines from one workspace to the other? please advice.
For a first idea nice video👍 is there much data in the dataflow loaded because step did not finish till end of your video? Do you know which Fabric SKU is used in background to understand the performance levels better? Mana thanks 🤗
It did finish, if you check the photos in my article. in the video I did not wait for the execution to finish as it would have taken time. I am using Fabric's trial license.
What expression language is needed for the email?
How can i create Dataset which i used to create in ADF
Hi Reza, Can we have the transformed data exported to excel?
Hi, although the below comments reference this, it's not entirely clear to me, are you saying that we cannot create a pipeline or gen2 dataflow that will populate a Fabric DW or Lakehouse with on-prem SQL server tables?
Hi. Not sure where I mentioned that. Perhaps the one you are talking about is the one asked if there is an on-premises version of Microsoft Fabric, which I said no.
You CAN certainly create pipeline or dataflow gen 2 to load data into Warehouse or Lakehouse using on-prem sources or cloud sources.
Is it possible to use data pipeline in Fabric to pull data from Power BI reports? E.G. periodically pull data from a table in a report?
The data in the report comes usually from a dataset. You can have a Dataflow get data from Dataset or even using XMLA endpoint and getting from AAS. However, if you want to get data from the report visual itself (to cover for measures etc), then the answer is no at this stage. You can do workarounds though, for example fetch the DAX code for the visual (using the performance analyzer), then run that using Dataflow and Data Pipeline on the XMLA endpoint. I might do a video on this sometime.
Thanks !Great video. Is premium capacity needed?
You need Fabric license, which you can apply for trial
Thanks. It is very helpful
I Still not Connected LakeHouse to my Local Computer what ever MS SQL Server or Excel file Since 2days This the Erroer "Connection could be created"
Are you building it in a workspace that is not MY workspace?
@@RADACAD yes I create new workspace as you mention
There's no integration run time yet. No love for on premises pipelines in this preview. In this form, unusable for most big companies.
How many custom languages do they need? I can honestly think of at least four other ways of calling a today() function under Microsoft’s roof. Why the hell do they do this? These languages should at a bare minimum resemble each other. It pissed me off and makes me not want to invest in this.
I agree. Too many expression languages and each different. The problem is that each language comes from a different product and team, and now when they are consolidating all of these, it seems each is totally different. Let's hope Microsoft invest on bringing them all under a same layout at one stage. That said, in the world of tech, there are many language anywhere; R, Python, XAML, C#... That is part of the tech industry.
I don't understand why they keep making more complex stuff, more languages, and way more lot of buttons instead keeping just to azure. I'd rather stick to code in asembler than this.