Distributed testing, part 2
In this article, we continue discussing distributed testing and take a closer look at scopes and groups. We also introduce multi-agent testing, which is an example of using test scopes.
Note: If you haven’t read the first part of this series, I recommend looking at it here.
Introduction
To recap from the previous blog post, we introduce scopes and groups to capture that test automation workers are not the same. Like managing teams, some teams have different priorities than others, allowing for the grouping of automation test workers. Maybe your staging workers should not be dealing with other tests, so best to group them into a “staging” group. In another example, staging workers should not impede the work of the production workers.
Similarly, we introduce scopes to capture that different automation workers have different skillsets. In some cases, it’s the kind of data credentials they have, like login information. Here we use data-scopes to handle it.
We also introduce functional scopes to handle strict policy issues. Maybe some workers are part of a network where they have accessibility to systems that other workers don’t have, like billing systems in the figure above.
This might seem overly complex at first glance. Still, looking at the analogy of an Enterprise setting, where teams have different priorities, and different team members have different skillsets, it can begin to make sense.
In the end, it should allow you to set up a testing grid for an Enterprise company without any high setup cost. Boozang’s URL-based approach and Docker and Microservices allow us to do this in a simple manner. Instead of having a whole team working on DevOps, start some Docker containers with different URLs and let the magic happen.
Groups
Groups are a straightforward concept that allows you to cluster workers together. You can do this in any way you like. If you have production jobs that need to take priority, you might choose to create two groups: production and staging, where production workers take care of all CI jobs related to production deployment, while staging workers are used for lower-priority staging jobs.
Creating groups in Boozang
Go to Settings -> Integrations, scroll to “Cooperation Service,” and click the plus sign: New Group or Scope. Fill out the Group name, and your group will be created.
In this example, we create a ci group do deal with CI jobs.
Note: The role constraint can be ignored for now.
Starting a worker in a specific group
In the worker URL, add the “group” parameter
group=ci
You can now start a worker in a group by adding the group parameter, like
http://staging-be.boozang.com/extension?token=xxxxx&env=1&key=1&group=ci#5e3f275e64f84941a326d4d8/ci50
The worker will now be added to the group ci and will display under group “ci” in remote play as well.
Scopes
Scopes apply to one or more workers. There are two different types of scopes in Boozang
- Data scopes: Used to encapsulate a set of data, such as login information or role
- Function scopes: Used to categorize capabilities, such as admin worker, or billing worker
Simply put, data scopes are used to associate a set of data to a set of workers, while function scopes are used to express capability.
Data scope example
We have two sets of application roles: admin and user. We introduce two data scopes: admin and user, and we make sure to set different login credentials for them. We can now start a series of workers in the different scopes by specifying it in the URL
scope=admin scope=user
In order to start a worker dedicated to a scope, add the scope parameter like
http://localhost:8000/extension?token=xxxx&env=0&key=1&scope=admin#5f4927829535c45da9a140a2/master
We can tune the number of workers in the admin and user scope, respectively, or omit the scope from the URL altogether and let the dispatcher assign jobs freely.
Function scope example
We have server A and server B; server A has access to the billing system network, and server B has access to the application network. In this case, server A workers might be in the billing scope, while server B workers might be in the application scope.
Although similar, function scopes are stricter than data scopes. While data scopes can have generalized workers (meaning, workers who didn’t specify the URL scope), workers for function scopes are specialists. A data scope worker will never do any other work than work assigned to its scope.
Creating scopes in Boozang
To create scopes in Boozang, go to the IDE settings -> Integrations, scroll to “Cooperation Service,” and hit the plus sign: New Group or Scope.
You can now fill in Scope names in the box. If you want to create a function scope, check “As a designated function server”.
As you can see, in the below image, I have created two function scopes: Agent1 and Agent2.
Note: The * denotes that it’s a function scope.
Set the scope for test
To execute a test using a specific scope, you need to use the “plug test-case” feature in Boozang. Select the checkbox “Execute on remote server”.
Set worker to a specific function scope
To specify that a worker belongs to a specific functional scope, add it to the URL like (function scope “Agent1”)
http://ai.boozang.com/extension?token=xxx&env=0&key=1&scope=Agent1#5f4927829535c45da9a140a2/master
You can now select the specific scope to run the test.
Conclusion
Using groups and scopes can be daunting at first, but it can be very powerful.
To summarize
- Use groups to group workers, like “Jenkins-pipeline”, “staging,” or “IDE.”
- Use data scopes to pass data or support roles, like “admin,” “user,” or “trial-data.”
- Use a functional scope to denote functional access and policy, like “billing-scope,” “network-admin,” or “application.”
In the end, it’s up to the user to figure out the best uses of this experimental functionality. We believe that having granularity in which tests can be run with which agents can be powerful, especially in an Enterprise setting where companies want to set up their own test execution grid.
Let us how this is helping you and what can be improved.
