Video není dostupné.
Omlouváme se.
Writing e2e tests isn’t hard, so do it
Vložit
- čas přidán 21. 08. 2024
- 📘 T3 Stack Tutorial: 1017897100294....
🤖 SaaS I'm Building: www.icongenera...
▶️ Generate Chapters: ytchaptersgene...
💬 Discord: / discord
🔔 Newsletter: newsletter.web...
📁 GitHub: github.com/web...
📺 Twitch: / webdevcody
🤖 Website: webdevcody.com
🐦 Twitter: / webdevcody
Playwright feels like it's taken the decade of e2e testing tools and perfected it
how so compared to cypress?
Love that you share on such diverse topic. I would also love to hear about how to setup a mock backend server. I always have trouble setting those up !
The only thing that normally gets skimmed over in e2e is booting you application into a testable state in CI. Yeah, clicking buttons and such is nice, but when you have to reproduce state, it gets tricky and you need to provision your backend, provision your database with dummy data, and all this takes more time to prepare and create than the e2e test itself. Probably why people somewhat hate e2e and frontend tests in general.
it depends on how complex your system is I guess, but honestly just doing a docker compose up and then run your cypress tests should be the only thing you need to do in CI/CD. Your compose file should load up your database and seed it, it should host your app, host any other local services you may need, and connect it all together. Then just run the tests
@@WebDevCody yeah, that's generally how you do it. The thing I was mentioning is the fact that you write a simple test for change a user profile or something and you need to provision everything. So, 4 lines of test code are equivalent of a bunch of lines of just having your application in a testable scenario.
@WebDevCody how do I this for a frontend that has multiple microservices backends
all microservices are behind api gateway.
How do you recommend i set up the backends?@@WebDevCody
I've been pushing for a system at work where we run a bunch of API calls first, then load a page, do an action, and check the result on the back end. The end goal is to actually mock all dependencies in TestContainers so you can just docker-compose up and run end to end tests as part of a pipeline without worrying about real infrastructure. As an automation guy I used to think this approach was crazy until I was forced into it. It's way harder to handle than just doing checks on the DOM though, so if you're solo, old school webdriver e2e is probably the way to go
@@Guergeiro We actually are facing similar problems in my startup, we had to orchestrate 3 codebases (~80k LOC) to be able to run an e2e test with Cypress. We ended up merging our codebases to streamline the CI/CD just to iterate faster on the tests.
Really great video! My problem with e2e is that for more complex use cases you need a functional backend if you want to test things really e2e. Maybe msw can simplify that mocking, but I think there are some problems with Nextjs and msw
at this point I'm convinced mocking causes more issues than not
This is soo valuable to people like me working on their own projects and trying to level up as a dev!
I had some problems to deploy a e2e tests using github actions and envs, did you manage to do a video about that? Should be fantastic!
your videos are just pure gold
ithink the biggest advantage that playwright has over cypress is multi lang support, and it's actually open source and not paid like cypress
Love this type of content!
Hire QA Automation (not you Cody, the collective dev workforce)
I mean if it’s your side project obviously that’s not a solution. But people still need to hear it even after all these years. I’ve seen project after project and entire companies fail because they tried to save a buck my putting all the responsibility on the Dev or Hiring manual testers, who let’s be real, just waste time. A well functioning framework executing hundreds of ui tests in a few minutes is a thing of beauty.
Oh and the testing in this vid will be fine for some quick smoke tests in a personal project, but will be flaky as heck and not a long term solution. For your company, get yourself a real framework
Would like to see you use playwright in a video. It also has a codegen tool for creating tests based on clicking around your UI.
Last I used Cypress it was not this easy. Love it. Thank you
The reason I like playwright is because they have really good methods for selecting elements by aria role, with no test ids required. I just still haven't figured where to srt the interface for mocking in e2e tests: http response vs data is db. It's hard to design e2e tests to get buy in from a large team to acrually maintain them!
Cypress has a plugin which gives those same features I believe
Ah cool, the reason I like that approach is that it makes the tests less flaky to change i.e. selecting by a partial text match on a button which is what the user actually sees.
You can make the partial text match selector look for say a value e.g. "$5.00" and then that test covers a lot of stuff - check data fetching works (mocked or not) and if currency formatting breaks.
It's sort of shifting the e2e tests away from control flow testing to trying to capture core user journeys and leaving control flow testing to integration tests (e.g. with testing-library)
Not fully developed my thinking around this so curious to hear other perspectives
The idea with not using data-testid, ids, classnames etc is that this "forces" the site/page to also be accessible or at least more accessibility friendly.
As you will need aria tags, labels etc to implement the tests which is what the user will interact with in the first place.
At least that is how I understand it to be. And from my understanding this is also what playwright recommends. Playwright is awesome btw.
Thanks for the content
You’re understanding is correct, but I’d need to get feedback from actual test engineers to get their opinion on test ids. I’m going to give cypress the benefit of the doubt and assume they know best practices. I’d rather not have tests fail because someone changes label text 🤷♂️
@@WebDevCody So true about text, we started doing e2e with text selectors and it was ok.
Now when we have i18n with 10 languages and it rapidly changes it just not an option. We rewriting our tests with test id, because it is more stable and maintaiting is way easier and faster.
Test engineers can add ids by themself. Altho some might argue that this approauch is not a "black box" testing.
At the end of the day e2e should bring your team value rather headache over selectors
Still not figured out great way to run it on CI before merging without creating entire deployment for full infrastucture (which is really expensive) so we doing it with feature-flags and manualy starting them from admin panel
@@WebDevCody I can see it from both sides tbh. At my place of employment they had set up a CD/CI to run all tests for that component/page/code base and then before a PR could be approved all tests had to pass.
And I knew when I changed text or something similair I should run the tests and fix the now brokens tests. So it just became a part of the proces of updating a label.
But I am not sure what is best. If in the company a11y is not necessary/needed for the custormers I think data-testid's would be the ideal solution. But if a11y is very important I think playwrights way of doing tests makes a lot of sense.
That is just my thoughts
@@WebDevCody But if someone inadvertently changes/removes the label text then is that not a valid failure? I get the usage of ids, but it isn't truly replicating the behaviour of a user.
@@danyates1 testing assessibilit in an e2e test feels wrong. There are accessibility tools you run on your ui which can also be used to block a deploy. e2e tests need to verify the core functionality of your app works. Don’t get me wrong, accessibility is important, but it isn’t core functionality. Many tools exist such as pa11y and axe to help verify accessibility, and one should manually verify screen readers work on the ui regardless before pushing.
Coincidentally I'm writing some new end-to-end tests for my web app right now
thanks, exactly what I needed
Good job bub ❤
It isn’t hard when the app has low complexity. Just with anything, as it scales it becomes harder and more expensive. Setting up testing infrastructure and resources, scaling the performance of large testing suite, writing sensible and pragmatic tests, setting realistic boundaries and patterns for testing, adding coverage on existing legacy codebase with low testability, working with volatile and ambiguous business requirements are some of the reasons why e2e testing is hard. So the aversion towards e2e tests is less about technical difficulties but more about these hidden costs people often don’t realize. I will say though that Playwright does make testing easier.
A lot of online demos even real apps with some so-called "complex" dashboard to showcase e2e are generally simple. It's so simple that even a well written typescript itself is enough to cover all of their tests. You don't even need an e2e for those examples. The true fix is for those cases are inside the codebase itself, fix your ts or whatever.
The hard part, as you say, it's something that starts to sit between general dashboard and toward the complex UI like Figma or Linear. Even writing e2e itself requires huge chunk of time and well concern, but you still have the same amount of time with a deadline.
At the end of the day, e2e or test in general is more useful for regression, it cannot predict and guard against the unknown bugs. You still need to manual test whatever you write even you have a 100% coverage.
@@doc8527 You are right. Also to clarify, when I was talking about complexity, I wasn't speaking exclusively of front end complexity, but actually about the e2e complexity. In the industry you can have a much more complex e2e complexity due to having more distributed / micro services architecture with various tech stacks and data stores, integrations with various external services etc. Because e2e is written from the front end people often think it only involves the front end but the whole idea of e2e is that it is "end to end" of the system and the larger and more complex and "end to end" becomes testing becomes harder, slower, complex and expensive to run and operate. But this is not to say e2e testing is not important. It is a valuable tool the business should invest in strategically. But thinking that it just easy and cheap, just because there's some library to help with it, I would say, is very misguided.
Liked but willl watch later.
Agreed 👍
I feel like e2e testing is still painful, i have written around 200-250 e2e tests for enterprise web apps in angular using playwright within the last few months. Managing complex states, validating request/response bodies, finding edge cases.. and so on is really boring and draining if you do it a lot
Is this with no mocking against a functional backend + seeded DB? Or are you intercepting the http requests
Try having to do it with selenium
@@cmyddxaa Some data is directly mocked into the testing database or by requesting the testing api. Because the entities and states are pretty complex multiple api/db calls are required to get the desired state. I mean we create factories/wrappers or other patterns which simplify writing new tests, but a good understanding of how database data effects the state of the application is still required, therefore it can be tedious.
There’s no silver bullet. If you’re not writing e2e you’re just testing a subset of functionality and probably writing tons of mocks and stubs, and by that point your tests have so much knowledge of implementation details that they are all brittle
E2E tests are brittle as fuck
Cypress is so powerful.
Haven't tried cypress so far, tho this is so similar to what we do with react-testing-library ...
Amazing
Hey Cody, do you like .NET Core and C#? Thanks for the videos.
@WebDevCody how do I this for a frontend that has multiple microservices backends
all microservices are behind api gateway.
How do you recommend i set up the backends?
You’d want to setup your ui to integrate with a live dev/test environment and then run your tests over your ui.
thanks @@WebDevCody