Do you ever wonder what happens to your test data when you use external services / SaaS providers for browser and mobile testing?
SaaS solutions are popular options to enable cross browser and mobile testing at scale for software development and testing teams. They come with a rich feature set and a large variety of browser / operating system combinations, which is essential in today’s shift left driven organizations.
There is however one crucial and critical detail that is often overlooked by organizations that use these type of services— DATA PRIVACY.
When using SaaS solutions for browser and mobile testing, the typical setup is as follows:
While the customer’s build and test infrastructure is behind the corporate firewall, the browsers and mobile devices of the SaaS provider are outside of the corporate network, sitting in a public space.
In an automated or manual test, a request is sent to the SaaS provider, where a browser is started and controlled. In addition to the commands (e.g. open URL, click on button), the test data (e.g. name, birthday, financial information) is also sent to the browser (and therefore to the SaaS provider).
The transfer of the commands and data from the customer to the SaaS provider typically happens through encrypted tunnels which is considered secure. However, when the data arrives at the SaaS provider and is entered into the browser (e.g. via Selenium), all the information is visible in clear text to anyone who has access to the system where the browser or the mobile device is running.
This is not an issue when purely synthetic test data is used for the tests. But many organizations use “production like” or “production” data for test purposes.
When testing with production data, the moment that data leaves an organization’s network, data privacy regulations are potentially severely breached. The potential consequences of allowing private data to be accessed by an external organisation are numerous: legal action taken against the company; loss of client trust and business; negative effects on the reputation of the company; identity theft, fraud, etc.
In order to better understand the issue with data privacy, let’s look at what SaaS providers stipulate in their Terms of Service with regards to customer data and data privacy. In most SaaS providers’ Terms of Service you will find something like this:
One of the main reasons organizations are using SaaS providers is the simplicity and ease of running cross browser and mobile tests, without the struggle of building and maintaining their own Selenium Grid.
Element34’s Selenium Box — Enterprise Selenium Grid enables organizations to run their entire testing pipeline behind their firewall while at the same time enjoying all the rich feature sets and comforts of SaaS solutions.
Selenium Box is an enterprise level Selenium Grid that runs securely inside an organization’s network. No data leaves the corporate network and no external access is required.