Selenium, Playwright, and Puppeteer all make use of Chrome DevTools Protocol (CDP) to access the browser, even Firefox. CDP is the current de facto standard to remotely access the browser. It actually starts an HTTP server in the browser that you send requests to. The W3C is working on actual standard for this space that will likely be more mature and durable.
My learning from playing CDP and the test automation applications mentioned is that this is way harder (and slower) than test automation should be and it’s a huge pain in the ass for authoring tests. The experience is so bad that it only seems to appeal to professional testers whose jobs depend upon it.
To solve for that I wrote my own test automation solution that does not require remote access to the browser. In my personal application where I use this form of testing I am able to execute about 30 user events per second in the browser. That performance speed is completely dependent upon other performance conditions of your application, hardware, and transmission handling. The test cases are simple data structures defined as TypeScript interfaces in a Node application communicating to the browser over WebSockets. The events are executed by creating a new Event object in the browser and executed with the event’s dispatchEvent property.
When your automation gets really fast (less than 30 seconds for a major test suite) everybody will make use of it including the developers, product owners, and even the business leaders. It becomes a faster way to setup and access various features of a browser application than doing it manually.
Good question. Same origin policy always applies so if you aren’t going to backdoor the browser as a bypass you need to in the same domain. The primary limitation then is that you must own the site you wish to automate, or run a replication of the site from either localhost or a domain you own.
I recommend running the site from an aliased subdomain. This will allow ownership of https certificates with a wildcard to your primary domain and that subdomain can point to the environment that contains your site database or services. You can also have a subdomain that uses production https certifications but resolves to a loopback IP like https://localhost.example.com pointing to 127.0.0.1 and/or a AAAA record pointing to ::1.
I cannot provide too many details without violating anonymity of this anonymous account. I can say that in my primary test suite I have about 85 tests each which comprise a variable number of user events and assertions totaling about 290 total assertions. When I was using HTTP that would take about 45 seconds to execute. When I switched to a home grown websockets solution the test time went down to about 6.5-8.5 seconds. Also, if you want things to be lightning fast try to shy away from using querySelectors.
85 tests and nearly 300 assertions in under a minute is still orders of magnitude faster than what I see elsewhere. There isn’t a single magic bullet performance killer. Extreme performance comes from absolutely everything: faster approach to testing, better hardware, super fast application to test, faster transmission, faster reporting and assertions, and everything else.
You can’t know what’s faster unless you are measuring things in isolation and making incremental improvements to a bunch of different bottlenecks. For example people love to tell me how fast their framework, their application, or whatever is but it’s clear that these are almost always anecdotal observations that are not measured or compared to anything.
When looking at performance it does not matter how fast something is, which makes anecdotal observations worthless. The only thing that matters is the difference in speed between things compared, the performance gap.
My learning from playing CDP and the test automation applications mentioned is that this is way harder (and slower) than test automation should be and it’s a huge pain in the ass for authoring tests. The experience is so bad that it only seems to appeal to professional testers whose jobs depend upon it.
To solve for that I wrote my own test automation solution that does not require remote access to the browser. In my personal application where I use this form of testing I am able to execute about 30 user events per second in the browser. That performance speed is completely dependent upon other performance conditions of your application, hardware, and transmission handling. The test cases are simple data structures defined as TypeScript interfaces in a Node application communicating to the browser over WebSockets. The events are executed by creating a new Event object in the browser and executed with the event’s dispatchEvent property.
When your automation gets really fast (less than 30 seconds for a major test suite) everybody will make use of it including the developers, product owners, and even the business leaders. It becomes a faster way to setup and access various features of a browser application than doing it manually.