5 Tips To Achieve Low Coupling In Your Python Code
VloĆŸit
- Äas pĆidĂĄn 27. 07. 2024
- In this video I share 5 tips to help you write code that has low coupling. I'll show you several examples and also share a story of a technique I used several times in the past that has really help me reduce coupling and solve more complex software design problems.
The code I worked on in this video is available here: github.com/ArjanCodes/2022-co....
đ· Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
đ» ArjanCodes Blog: www.arjancodes.com/blog.
đ Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Designer Mindset Team Packages: www.arjancodes.com/sas
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
đ GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
đ If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
đŹ Discord: discord.arjan.codes
đŠTwitter: / arjancodes
đLinkedIn: / arjancodes
đ”Facebook: / arjancodes
đ Code reviewers:
- Yoriz
- Ryan Laursen
- James Dooley
- Dale Hagglund
đ„ Video edited by Mark Bacskai: / bacskaimark
đ Chapters:
0:00 Intro
1:32 What is coupling?
2:08 Tip 1: Avoid deep inheritance relationships
5:49 Tip 2: Separate creating resources from using them
8:01 Tip 3: Introduce abstractions
12:40 Tip 4: Avoid inappropriate intimacy
15:55 Tip 5: Introduce an intermediate data structure
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
đ· Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis.
Intermediate structures are the key idea of building data applications, analysis. Using data standards as an interface so that you can split the pipeline into decoupled parts. Create smaller data structures as objects when necessary to lessen the burden of larger classes. Thanks Arjan!â€
6:48 Using a helper function to create objects works great! Ever since I watched your other videos on decoupling, I was at a loss for a simple way to create objects that were previously in the class initializer.
10:58 Showing how a Protocol class is useful here especially with third-party libraries is really helpful. Again, after watching your previous videos on decoupling, I wasn't sure how to setup my class to work with both an ABC and the third-party library. Using a Protocol class will solve this.
Thanks for the great video!
Very good set of tips here. This is extremely useful for working with databases and schemas. It's way easier to test when you abstract the DB details away into a database adapter class and abstract schema. Need to change database? Now you only need to change the adapter instead of every piece of business code that interacts with the data.
Nice! I recently applied many techniques you have described in your videos to several projects at work. Protocol produced lots of awesomeness. We had this problem were we have several database sources that didn't share a common engine, and also different database destinations. Using protocols we just described the interface for the extraction class and the loader class and then, each time we had to add a new source or destination, we just added a new class from that protocol. A configuration file described which subclass to use. The course is so clean and concise now... Thank you again!
What really taught me to avoid strong coupling in my code is when I started working with rust.
The variable ownership concept of rust makes heavily coupled data structures (as I was used to to from working with strongly object oriented languages) very painful.
I really learned to think about software architecture differently from that :)
One thing he doesn't talk about is using concrete static types: they introduce coupling... You can avoid it by not declaring static types (if your a lone developer) or doing message: Message where Message is an alias of str instead of message: str.
We used intermediate structures a lot in the enterprise search I worked on. We had individual parsers for different file types like PDF or Word or HTML, etc, that all converted to our internal XML that kept the relevant information we needed like sentence and paragraph structure, but didnât care about any other formatting. So the actual search indexer didnât need to know anything about the original document formatting. We supported cross language search, as well, so we had intermediate objects to represent the âideaâ that the user searched for, which we could retrieve in any language, not just the one they used. (Mostly useful for multilingual support orgs where documentation is not always in sync across languages.)
Yup, this concept is something I teach to junior programmers since I've definitely painted myself into a corner when I wasn't aware of this
Interesting! I guess that using a formal like XML or JSON is preferred over a python object because it's language agnostic? I usually use dataclasses as an intermediate @datastructure and I wonder what the downsides are.
I was looking for this exact video in your channel over the last few days for a project where Iâm struggling with coupling. Amazing timing! This helps so much
Great video, Arjan. I agree, low coupling is generell a good thing to aim for. Moreover, when combined with high cohesion in each individual module, this is usually a sign of excellent code quality.
Thanks so much Robert.
wow, terrific video.
I have been watching your videos for for some past few months now and and you are really good at what your doing. if you could help us create a series of videos on a any of your side project showing us how you will apply all what you have been doing in your videos (from design, code refactoring etc), like combining everything in a single project. Thanks Sir @Arjan.
The "Inappropriate Intimacy" reminds me about a "code-smell" I look out for during reviews: Overuse of list-indices. I've come to realise that the more time I invested into code-maintainability, decoupling, separation of concerns and dependency injection/inversion the number of times I need to use list-indices has plummeted. They are still necessary at times. But more often than not, there are better/more appropriate datatypes than list or tuples. If I spot excessive index-usage I do tend to take a closer look to see if nothing can be done about it.
I love this stuff. I use it all the time, for as many third party libraries or utilities as I can. Configuration parameters, inputs and outputs, visualization data. A simple data structure goes a long way of making life so much easier when your dependencies change. And keep you sane while unit testing your code, since you can avoid any mocking or stubbing procedures over-engineered by the library developers.
I really like these kinds of videos, which are brief reminders of what your channel is really about. As a personal observation, intermediate data structures can be a challenge to design effectively in non-trivial cases. The sheer effort required to perform the raw analysis of a non-trivial problem domain and subsequent design of the data structure(s) to adequately cover the solution domain is only part of the challenge, the larger part of the challenge tends to be posed by the management style of the "average project manager" -- this translates into time pressure to "get things done". Then again, this could be said of any of the best practices that you cover on this channel.
This is a really great tips not just in python but almost all programming languages. Great stuff here Arjan!
Thank you!
I'm really amused by how good your content is. I've worked only 2 years as a software dev and can learn so much from your explanations. Thank you, Arjan! :)
Thanks so much Bogdan, glad the content is helpful!
You have no idea how long I've been looking for this kind of content. Thank you so much.
Glad to help!
This is amazing and so well explained with examples. The protocol way is much cleaner and makes more sense after watching this video.
Thank you so much!
One thing I have learnt the hard way, though, is that you sometimes have to be careful not to reduce coupling too much. Strong coupling is bad, but splitting your code into a million tiny pieces that barely interact with each other is also terrible.
I've now switched to writing coupled code first, and refactoring it once it becomes a problem. I suppose it is kind of like avoiding premature optimisation, if you abstract too much too fast, you might accidentally end up with Java.
Then again, especially some of the earlier (negative) examples in this video were... well. Gotta be honest, I wouldn't even have considered those approaches.
Note that Java has to abstract because it is statically type, it seems a lot of the time Python can do some Python dynamic thing with decorators or something else even when you practice YAGNI(your not gonna need it) since it is already has lots of levels of indirection/late binding. Also how come no one speaks about how giving concrete types to things causes code coupling, one way to reduce it is to either not declare types (single developer), or to alias types so that you can change them later.
Inappropriate intimacy is such a funny name
This is the perfect follow up to your ABC/Protocol and Uncle Bob's videos. Thanks so much for sharing. Your videos have helped me immensely. I now maintain several code bases that thanks to separation of concerns, are well tested And easily extensible. The last part is super important for the astrophysics research I do. Previously, I would write a new set of scripts, (20-100) for each research project just because they are all so different., and my attempts at creating libraries were never quite successful. But I realized that it was the degree of coupling which made it really hard to extend and re-use my codes. I am both more productive and a much better coder.
Thanks so much Emir, glad the content is helpful!
Fantastic video @Arjan, I love these best practice practical guide videos.
Thanks so much Concon, glad the content is helpful!
Really nice overview, thanks!
Thanks so much, glad the content is helpful!
Well-picked examples. Thanks!
Thanks so much Thijmen, glad you liked it!
To test project creation from Excel, I've introduced a excel dictionary data structure that was close to the original Excel but just a dictionary with column titles, a distilled version of a pandas DataFrame. To create the project I need to do a lot of transformations (e.g. Yes -> True, deleting some empty fields, adding ids, using unified field names, "10 mins" -> 600). So I was able to separate the IO step from the conversion, and to test the conversion without using BytesIO objects, just using that dictionary representation.
Sweet, simple.
Fully agree on the intermediate data structures.
Apart from separating own code, it also provides an important abstraction layer when having code that accesses external services. There, if you introduce intermediate data structures which you then return via a Facade, you ONLY couple the facade to the external system or library - not your entire project. If the external System ever changes - or you decide to swap it out for something else - you only have to adapt the Facade, not bits and pieces spread around everywhere.
Thank you!
Thank you Arjanđ
You're welcome David, glad you liked the video!
Thank you for the great insights
Glad it was useful for you, Stephen!
Again, great summarizarion of best practices đ
Thanks so much Mateusz, glad the content is helpful!
It is nice to see content with mid to advanced topics.
Thank you Ammar, glad you liked the video!
Another great video of good, solid coding reminders. Thanks for the effort you go to in creating these!
Iâm always interested in your grouping of functions, classes (or methods) and folder structures.
Thanks so much Andrew, glad the content is helpful!
Just when I needed it most! I've made services layer in my project and I was thinking about how to achieve low coupling between apps.
More tips: 1. no side effects on the module level 2. keep files small 3. keep imports between your modules minimal 4. use contextvars/contextmanagers for dependency injection 5. avoid inheritance 6. avoid, at all cost, multiple inheritance 7. especially in tests, consider putting import statements into functions, not at the module level 8. use existing abstractions and protocols as much as possible (iterators, sequences, callable, etc)
Thanks a lot for your video. It helps people like me understand a lot of things which I may clarify further for my juniors.
Thanks so much Pawan, glad the content is helpful!
Great video!
Thanks so much Nate, glad the content is helpful!
Suuupernice vid! I love these vids where you can take points and apply in other coding languages to :)
Thanks so much Carl, glad the content is helpful!
Hi Arjan! I found the last tip particularly useful! Another example of intermediate data structure would be OS drivers?
Great freaking video!
Thanks so much. :)
Thanks a ton đ
Most welcome Iyad! glad you liked the video!
Answering 5:10 in a big company: people that have not written code in years want to see high percentages of test coverage. So even though the function is silly, some developers have to write a test that executes that function.
One area I think intermediate descriptions can help are file importers, for a game engine or 3d software, this way as you said we can split the file parsing part from the recreation of the file in the engine or so
You could .... Draw your graph on the thumbnail with an automatic tool like graphiz. There's also networkx to help with a transitive reduction on your graph.
An automated process to keep a large project under control is a good idea.
On 10:55 well it is possible with abstract base classes. ABCs have a function called register and you just need to go ahead and register any third party or whatever class, as being an impementation of this ABC. Maybe not as elegant as the protocol, but possible.
Hi Arjen, great video again! A question about Tip1, arenât you worried about introducing too much functions, floating around in you codebase that are ârelatedâ to some objects in any way? I see this also applies to Tip 2.
Not sure if introducing a lot of functions is more readable over classes with methodsđ§
Hi Joris, thank you! And no, I'm not really worried :).
There are several ways to organize your code: in folders, files/modules, classes. I never noticed that I had a problem of having too many functions floating around or that I felt the need to introduce deeper inheritance hierarchies to be able to structure the code better.
I'm not against using classes with methods by the way, my point with the first tip is to be careful with deep inheritance hierarchies.
arjan is back
Thank you for the videos, they always make me want to look at my code for a few more seconds and ask if there is a better way of doing it. I have been able to identify code smells more easily in my own work and in others.
I have a request: Could you do a deep dive into code testing? I have a hard time understanding how you would write a test for code you have not written yet, and also how you would be able to come up with automated tests for every scenario. You mentioned something about using protocol classes to create mock objects and I would like to see that. Most of my testing right now is done by creating test objects and running the specific part of the project and checking to see if the output looks right. As time goes on, I might decide on a different approach to a problem or the requirements can change in between, how are pre-written tests going to work here? Also, if I was to create tests for something that requires a database, how would automated tests handle that in a CI/CD pipeline with external tools like github actions?
Two thumbs up for more on "protocol classes to create mock objects" - this sounds really interesting!
For #4, the signature of the generate_breadcrumbs could be further changed to accept postal_code, city, province to further avoid the intimacies
Good video.
Thank you Petr, glad you liked the video!
Adapter class FTW, version you need something that provides a specific Foo. Then you realize you can scrape Foo from a database, then you're reading in a Foo data from an XML dump, and then it makes sense to cache some of the Foo's stuff in a NoSQL database... All of these Foo's pretty much have the same behavior, but load from really different places.
Could we use the abstraction tip in the example you gave for Elastic Search and create a Protocol class for the same? Why did you move that code to methods?
Iâm not familiar with intermediate data structures. Could you please make a video about it? Iâve learned a lot from you! Thank you
Thanks for the suggestions, Giovanni, I've put it on the list.
So I get that the EmailServer Protocol removed the explicit dependency on SMTP but doesn't it implicitly still rely on it given that the protocol has to mirror the methods in that class? What if you wanted to add an alternative to SMTP but it had different properties/methods or even just different names?
1:44 i really was expecting u'd explain it in ur words...
7:51 When you pass that Message object into the send_message() function, is that an example of "dependency injection"? Since you're injecting the entire Message object into the send_message() function as an argument?
How are you switching between normal cursor and the Block Cursor? Is that Insert mode?
Hey, is there a link to the research paper he was talking about in the intermediate structures section?
Hey Arjan, your videos are really good. Wanted to point out one thing in the Protocol subclasses. Instead of using an ellipse "..." You can just write a docstring """my code comment""". It's more helpful and it is also valid sintax (since the ellipse doesn't do anything).
Do you happen to know if Sphinx recognizes and uses docstrings on the Protocol to generate docs if the implementation classes don't have docstrings?
@@rdean150 as far as I know you need to leave the docstrings yourself. However you don't need to specify the return. Since even if it's the same type it can follow another condition in the subclasses
I'm curious where did you do your intership at the time (25 year ago) ?
I've been using more Protocols lately but one minor detail I'm unsure about. I prefer to keep my protocols in a dedicated "protocols.py" module within my package, so that all of the various modules in my package can just import that rather than having circular imports or unnecessary cross-dependencies just to type-annotate parameters. Is this approach of defining protocols separately from the classes that implement them or use them considered good practice?
Sir, could you please help me to figure out that if I pass "msg" as an argument in send_message (method) So, where I should initialize it's class and then pass the object coz atleast somewhere I must initialize it. correct me please . Thank you
I'm not utilizing structural typing!! It finally clicked why I might like using Protocol instead of ABC.
i used protocol once then ABC several times, i still can not really understand the difference , i guess i am saying i wish it will click for me too,
Well, theyre pretty similar. But I think a protocol is just any object that has certain methods/members, whereas your class must actually inheret from the base abstract class to be considered a valid type. Hence you'd have to use multi-inheritance or some other fuckery to use it for static type/member checks where you want to have methods from a library or whatever enforced with the type check. Need to write some little mucking.py scripts to confirm.
11:00 So you are saying that such simple programming practices are not even standard?
Using interfaces/protocols/etc is well worth the effort since it makes other things so much simpler.
Along with what you are saying here and the corporate code I've been reading I'm starting to feel like an A+ developer even though I know I am shit.
i dont understand why tip2 reduce coupling.
but Great video, thank you!
If you separate creation from use, then you can decouple the code that uses the resource from anything thatâs needed to create it. And then you pave the way for introducing abstraction as well, which is the next tip đ.
Once you separate creation from use, you give yourself more freedom to change how the object is created. Hugely beneficial for automated testing
Where did you learn about all this?
Is that car crash project you worked on available ? Would love to show some folks interested in python in that type of capability to get them excited.
2nd is command-query separation from Martin Fowler, I guess
Hi Arjan!
One question about abstracting low level modules. If you abstracted database A module with a protocol/ABC that has to have a connect() method, how would you deal with replacing module A with module B if the methods are not named the same, but for example db_connect() ? Do they have to have same method names? Wouldn't that also then be a dependency in itself?
Suggest you watch Arjanâs video on the light switch/fan abstract example. I think that will clearly answer your question here.
You could use an adapter class that takes an instance of Database B and provides the correct interface to satisfy the Protocol / ABC by calling the appropriate methods on it, and then use a factory to choose and return the correct adapter instance.
Protocol classes are like interfaces right?
Want less coupling? Ditch OOP and go Functional. J/K
Use SOLID.
Can you please do a video on the inversion of the control container in python?
Something similar to Autofac in C#
Thanks for the suggestions, Sima, I've put it on the list.
I've noticed the more coupling I try to avoid, the more code I end up writing. As I have to write independent interfaces to completely isolate one module from another. Any for dependencies, I try to use buffer so each module can run asynchronously. But that ends up into more code. And this has served me well for years. But whenever I take a deep-breath and take a look at the code. It feels bloated. :/
Can coupling issue be avoided by using functional programming?
Not really, passing a struct as a first argument to a function isn't functionally different from calling a method on a class instance (in fact in python they would have the same signature). It does avoid a lot of headaches related to OOP but by itself doesn't remove the problem of a potentially mutable context object being a dumping ground for values. Another issue of pure functional approach at least in python is annotating functional types is a complete ass.
I do my c# and python in a solid way and I was told not to be an uncle bob.
âThe first big coupling problem that I had to solve myselfâŠâ - so awaited for you talking about your first wife and your marital problems⊠Next Time!
Haha, that's a topic for another video :).
Hi Arjan, you have a typo ("Enrol Now") on your code diagnosis sign up page, FYI.
Actually, the non US spelling is with a single L. But I've changed it now to use the US spelling since I also use that on the rest of my website.
czcams.com/video/qR4-PBLUZNw/video.html Where is the "Message" object defined, that is returned for the 'create_mime_multipart' function?
can't reach your website, any ideas why? đ
Not sure⊠it works fine here.
Well, it says "error 1020" every time I try to open it ÂŻ\_(ă)_/ÂŻ
Here is my Ray ID: 7487308bcd4d9d57
so now I figured why programmers are lonely. they avoid coupling
Could you put the link to your paper showed in the last section?
Here it is: ris.utwente.nl/ws/portalfiles/portal/28160157/00000051.pdf. It's really old though, so I'm not sure it's still of any use :).
I am too noob to understand all that code
Donc tu parles français ?
Oui :)
Link is dead lol
The link seems to work fine here?
I worked for me, too
why are the exaples so complex? I can follow until Elasticsearch and following tools i cannot remember. I have to leave your Video.
so i am out, and not able to understand the content. đ
I would like to have more examples on 'hello world' basis, to follow the content.
maybe i'm not experienced enough for your content.
If you could have a look into your analytics that many guys are breaking up at example time, i'm not the only one.
There are many other channels that can teach you the basics. Arjan's goal is not teaching you the basics, but the more advanced principles.
In the case of programming, I think it's more helpful to see how someone uses something practically rather than just the absolute techincal fundamentals of whatever's being discussed. Arjan's videos are generally like the former, but there are so many other channels you can watch that do the latter.
you are a good engineer but your courses are so fucking expensive. Please do a market research
He is very good at explaining things , good mix of theory and examples. The content is standard knowledge for object oriented devs so you can get that elsewhere if you look beyond python or study well known git hub projects. You need to decide if itâs worth the investment but your only as good as the information you take in or teams you work in.