This BIG Modeling Mistake Is Ruining Your Performance
Vložit
- čas přidán 21. 01. 2024
- Get the source code: / amantinband
My courses on DomeTrain: dometrain.com/author/amichai-...
In todays video we'll talk about one of the biggest performance mistakes your app is likely making today. I've encountered this mistake in almost all applications I came across.
In this video we'll see what the problem is, how to fix it, and how significant it is in terms of performance.
Connect with me on 'em socials:
Twitter: / amantinband
LinkedIn: / amantinband
GitHub: github.com/amantinband
Thanks for watching, don't forget to like & comment & subscribe! ❤️ 💻 - Věda a technologie
Most value from this video: "Create an object small enough that contains what's needed to make a business decision." I wrote that down! 😁
1) You started displaying an implementation which is actually a good implementation, then you asked about the problem with it. There isn't any problem with it actually, at least without a context. What you provided is just an performance optimization. So you solution is actually not needed if the app isn't suffered from performance issues.
2) What about DDD, if your User class is an aggregate root, and you want to apply action on the reminders via different flow, you still need it fully in the User class, if not consistency can't be accepted.
3) Appreciate your videos, as always 🙌
I am complete agree on both points.
If there is a performance problem then separate the write from the read. In general DDD and an ORM should only be used for writes. Use Dapper or full CQRS for reads.
Good point!
But Im wondering to hear your opinion. What if you'd take DDD approach to mutate only 1 AR per transaction and eventual consistency is not an option at a first glance, aka
1) validate if you can add a reminder in the first place
2) create Reminder and publish an event
3) User picks it up and increments that count
Would you lean towards idempotency to make it happen or something else?
The single aggregate that should change in your example is the user.
This is because the atomic decision is made by the user aggregate. Storing the reminder in the database can happen in a following transaction.
Check out my clean architecture template, it implements exactly this using single AR per transaction and eventual consistency.
@@amantinband Just watched it, thanks. Cool stuff, but it really feels off for a User to produce another AR event.
I was thinking more like User AR would validate and create Reminder AR using some internal static factory of Reminder class.
So basically my thoughts are:
1) Why would a user produce another AR event?
2) What topic/queue would that event be published? Users or Reminders? Should be reminders, right? But again feels off to me.
3) What event type would Reminder class store in domainEvents? Its own or the one user created?
4) Would that Reminder's event even be published or not?
5) And if we add CQRS/ES on top of that it will become even more messy. Because initial event that would have been stored along with User AR would contain data that he (user AR) doesnt need at ALL to be recreated, but is needed for Reminder.
Maybe because im not expereinced enough, thats why I asked.
Thanks for the good content tho
Thank you so much! I see this so much as well. In my experience, I see a lot of project that use an ORM system do this. That just get the relation/child entities (so full select and maybe other child entities) and do a count or whatever b.s. performace hit with it. Instead just a simple query to the db. I always encourage to use the database because in most scenario's the database is a lot faster, but don't over use it like that.
P.s. I am a query builder guy. I don't use EF core, i use sqlkata.
What if you need now to fetch the user with the Reminder object? Would you need to query it separately since your model does not have it together anymore? Or would you just use your DB models?
Hi Amichai love your vids and your style of teaching! Could you share what drawing tool you use for the vids? Its like powertoys but its looks so good! :)👍
Presentify (it's macOS only though)
Would it also be possible to have a reminderIdMap? I think this way, we'll be able to have a single source of truth for reminders on the user object.
Yeah definitely
Hey Amichai, what drawing tool do you use? It’s not zoom it, right?
Presentify (only available on MacOS though)
I like how you consistently write remidner 🙂
😂
Any particular reason why you don’t add a UserId to a reminder instead of the other way around?
User 'has' reminders not the other way around.
@@vivekkaushik9508 I get that. Another way of saying that, would be that a reminder belongs to a user. Represented by a user id.
@@Noceo I guess he just didn't in this example, but yes, that's required, otherwise how would you retrieve the reminders for the user from the database.
@@pazzuto Hmm, to follow his approach through the reminderIds list. Which has some drawbacks. To store this User Object in a DocumentDB will probably provide issues sizewise as most have document size limits(see explanation further down.). In a relational DB it could be moddeled as a reference table (reminderId, userId) which would result in an N:M relation, which would probably not be what he wanted. So yes in his example he probably should have added the UserId to the Reminders except he wants to make them shareable. I personally think that what he describes is not a good solution, and he also pointed this out by himself with another example, look at users over time. The RemindersMap and the _remindersIds is still growing indefinitely the longer the app runs for a specific user. So when Fetching the User Object, it not only can get large in size but also the computation which is needed for the aggregation grows aswell. Which obviously can be mitigated by looking only into the future. But the future itself is indefinitely large. So size wise, and this is for what he claims to optimise, reminders are no good candidate to get fetched preemptively with the UserObject. Considering the possible absolute size they should be fetched/calculated at will, which will probably not throw OutOfMemory exception in some SuperUser Cases and be more responsive on startup, but which might be slower on per Operation basis, although this can then be optimized in the Storage Engine through proper indexing (baiscally what he tried to do in the application itself)
How about lazy loading? All Reminders would load only when you ask for them. And when you count it would count objects in dB without loading them. What I'm missing here is how you are going to save both User and Reminder now.
Yeah our applications suffer terribly from this.
It's old and the exuse of "historically grown" is used a lot.
Data-Access is from hell and huge amounts of Data is joined and loaded many times.
None of the original devs is around anymore and nobody really dares to touch it, and if you do, the superiors won't accept the change out of fear it breaks the app. It's a legit fear though, but it really does feel like we're stuck with this.
Reason I wanna become an Architect is that I have seen terrible Applications in a whole line-up of companies I worked for, and since I can't trust others to make somewhat stable applications, I figured I'll do it myself.
Long way to go, and a cargoship load to learn, but I'll get there and just keep going. It benefits me as a Developer either way already.
Well, this makes sense for RDBMS. On the flip side, retrieval of data will require joins which can be slow. So, creation is fast, but retrieval is slow. 😊
In general, a computer spends 60% of it's time searching. So, retrieval should always be faster than creation.
If reminders are stored inside User this is still join. This only complicates saving.
I just noticed that this guy Vims. (I just started learning, it’s one of my 2024 goals to really build that Vim muscle memory)
Best investment you’ll ever make
When did he use it?
He is using VsVim which is an extension to Vscode that lets you use Vim movements and commands.
@@benatmartinez1365 he’s using some Vim plugin in vscode I believe
@@benatmartinez1365entire video. Vim plugin in VS code. There is Vim plugin for every IDE. I can't type anything without it.
Completely argee. That why I like to use my DbContext inside my Application layer instead of reusing the same repository and fetching way too much data. Using my DbContect directly in my application layer forces me to think about what data I really need for this exact business use case.
How do you handle the argument that "You're duplicating data by storing the counts of reminders, when it can be derived from the data by querying the actual list of reminders. You should look to tune your database indexes instead."
It all depends what you're optimizing for. Optimizing for database storage will lead to different decisions than optimizing for cost or performance
I think this is not very accurate. First, you violate liskov substition rule. AddReminder should accept only required fields, not the whole object. There is an atomic invariant between these objects, if you were to merge them into one document, then you wouldn't need to rely on a transaction primitive. Also, data access patterns should be noted.
I know that was just a trivial example but why have a public list of data on your record, that's also internal, and why not just use a check constraint so your application logic doesn't need to concern itself with transactional locking and concurrency? While the example was simple, I think it was too contrived to be realistically complex enough to illustrate a real scenario. I also wouldn't encourage using exceptions for control flow - it's not exceptional to consider the user has reached their daily threshold.
Check out my videos on the topics you touched and you'll see I more than agree. This was just a trivial example to get the underlying idea across
1st rule of programming - Never prematurely optimizer your code. 😅
I'll need another US for enhancement boss so that I can justify my man hours and charge you extra what could've been done in the first time only. 😂😂😂
That's exactly what it told to my self when he asked about the problem with the initial code