Tentacle Labs Blog

Testing is all about being confident that your changes to the software can be deployed to production without problems. While unit or integration tests help you to test details — like the results of a calculation, or whether two parts of your system are working together — you don’t know whether all parts of your system are working together as they should. Systems are getting ever more complex and we try to cope with that by splitting it up into parts so that we can handle them or distribute the complexity over multiple teams. Teams develop and deploy small, loosely coupled Microservices that reassemble a bigger system. How do you know that everything works together and that a user can use it? This is where End-to-end (e2e) testing comes into play. And Cypress can be a nice tool for that.

Cypress can be used for different kinds of tests of web applications: Unit tests, UI tests, or End-to-end tests. It is well known for being an easy to use, all-in-one solution, as it takes care of setting up the environment, of the asynchronous behavior of web applications, and has a nice UI. For sure, this comes with a bit of complexity, but for the most common use cases it is hidden away really well. I don’t like Cypress quiet technical functions for selecting elements for assertions as they make your tests very tightly coupled to your applications internals, like markup and classes. I prefer to see the application as a black box and test using user facing attributes like ARIA and semantics. This shifts accessibility left in your development process as every developer has to think about how the application can be discovered in that way. But @testing-library/cypress is a nice addition that fills this gap.

As I said, ”for the most common use cases”. Till now I had bad luck when using Cypress. I always had cases Cypress doesn’t support.

In the previous project I worked on an application with a complex SSO (single sign on) login flow based on OAuth2. Such a flow uses multiple origins as it redirects you to other parties to verify your identity. However, Cypress didn’t support multiple origins. Cypress therefor recommends to perform login programmatically (and because it is faster). But in our cases the SSO login is an important part of our user flow that we wanted to include in our end-to-end tests to be confident that everything works. We liked Cypress for the rest of the tests so much that we worked around it. Instead of performing the login in Cypress, we used puppeteer to automate that part and passed the login state into the Cypress tests. It wasn’t perfect, more complex than necessary, but gave use the confidence we need.

Cypress recently introduced two new features: multi-origin support and sessions. While I haven’t tested the multi-origin support yet, it is promising. It allows you to visit multiple origins in a single test like you need for a SSO flow. I already tested the session feature which allows you to cache your login state and reuse it in multiple tests. Previously you had to do this manually.

But the whole situation isn’t better in my current project. We are developing widgets for the Element messenger, which integrates with the Matrix ecosystem, and want to test whether everything works end-to-end. Element is a web application that injects micro-frontends (widgets) using iframes. Iframes are used for security and privacy reasons and isolate the widgets from Element. While Cypress does support iframes, it does not support cross-origin iframes. You can work around it by disabling web security, however, it is only supported by Google Chrome — once it’s disabled, your environment is a bit different when it comes to behaviors around security, which gives your less confidence in your tests. But besides that, iframes are still a second-class Citizen, as the really useful time travel feature for debugging tests is not available.

The new session feature is quite useful here, as logging in requires some more steps in Element due to the end-to-end encryption setup. The idea is to do it once, store the session, and reuse it between tests. But Element is using IndexedDB which isn’t stored and restored from a session at the moment 😦.

Our widgets are used to collaborate between users and we want to test such scenarios. As we treat our application as a black box where we only use the UI like a user would use it, it would be useful to open two browsers at the same time. This isn’t supported by Cypress, while other tools like Playwright support it. As an alternative we can fallback to sending the events of a second user programmatically, but that requires deeper knowledge into the system internal, diverging from the end-to-end testing concept.

So, is it just me, or does Cypress only support very easy or idealistic cases of testing? Why do I never work on such applications? Or do I just expect too much from it? Once the web application is a bit simpler, like this blog, Cypress can easily cover everything 😄. But as long as the developer experience stays this nice, I will probably still try to use it every time…