Blog

Read the latest news and stories about Boozang and test automation.

Tips & Tricks, New Features

Under the hood: Call flows

It can be helpful to look at the application’s call flows to understand the inner workings. For a casual user, this is, of course, not necessary. Nevertheless, I created this blog post for users who wants to know what goes on “under the hood” of Boozang. 

We will look at two different use-cases. The first case is when a user is working in the IDE, creating and modifying tests. The second one when a user is running tests using a CI server.

The project dashboard (in the Cloud)

The user must first be authorized to do any work in the IDE. The authorization happens via the Cloud instance login page.

  • Americas (Montreal): https://ai.boozang.com
  • Europe (Frankfurt): https://eu.boozang.com

After entering the login and password, the user will see the projects that he/she has access to and can create new ones. 

We are still in the Cloud in this view, and nothing is loaded on the client-side. 

Working in the IDE

As soon as the user wants to do any real work, he/she needs to launch a project. Typically, a user will only work on a single project at a time, and any existing open IDE project session will be terminated when creating a new one. 

After launching the project “The Cucumber Lab”, the user will be presented with a novel browser window: The Boozang IDE

Things have now changed: The user is now working locally in his browser, where the application IDE is loaded client-side. As soon as the user does a change, such as creating, updating, or deleting a test, the difference is synchronized via a web socket connection with the Cloud (either ai.boozang.com or eu.booang.com). As web sockets are a bi-directional (full-duplex) communication protocol, we can now synchronize this change across all other clients.

For example, if team member Anna and team member Adam is both working on Test A, they will see any changes made in real-time. This is often a great collaboration feature, similar to Google docs, but can also be undesired. In the cases where this is undesired, the team members should be working on different branches (see blog post on Branching and Merging: /enterprise-boozang-aligning-tests-with-code-branches/).

The call flow can be seen below

Running a test on the CI server.

After you have created your tests and organized them into test suites, you typically want to run them in a CI server. As Boozang is completely browser-based, there are many ways to do this. The easiest way is probably to use the Docker container boozang-runner, which is Open Source and can be found here

https://github.com/ljunggren/bz-docker-xvfb

The Boozang runner utilizes Puppeteer to control the Chrome browser and Xvfb to support headless runs. The Chrome browser’s built-in headless mode does not support running any Chrome extensions, so we have found that running Chrome in real mode with a virtual frame buffer is preferred. 

To run a test on a CI server such as Jenkins, you need to have Docker installed. To run a test you simply need to type in the test URL, like


BASE=http://ai.boozang.com
TOKEN=c3c3c3c3c3c3c3c33c3secret
ENV=0
PROJECT=60c94425994507096855aa53
BRANCH=master
SELF=0
TEST=m2/t1
GROUP=
SCOPE=
PARAMETER=
WORKERS=1

MASTER_URL="${BASE}/extension?parameter=${PARAMETER}&token=${TOKEN}${PROJECT}&group=${GROUP}&scope=${SCOPE}&env=${ENV}&key=1&self=${SELF}#${PROJECT}/${BRANCH}/${TEST}/run"
docker run --rm -v "$(pwd):/var/boozang/" --name=bzworker1 styrman/boozang-runner "${MASTER_URL}"

As you can see, there are many parameters to set, but most of them are to facilitate parallel test runs. You can generate all this config from the CI screen in the IDE interface (see below).

As you can see, each user has a personal token. It’s sometimes desirable to create a specific user CI user with its unique user token.

Below you can see the call flow for when a CI test is being run.

Conclusion

Going into depth like this around the inner working of an application is sometimes a little overkill. On the other hand, by going into details, it becomes more apparent to the end-user why a feature was built in a certain way and will help them troubleshoot the application easier. 

We also believe that the specific way Boozang was architectured is a great case for using it. By running a lightweight client that does all the testing instead of driving tests from the Cloud, we can avoid scalability issues. An additional benefit is also that there is no need to open any company firewalls, thus reducing the pain of adoption and potential security issues. 

Tips & Tricks, New Features

Element Selectors

Having a good element selection policy is key to good test automation. Why is this so hard? We believe it doesn’t have to be.

(more…)
Tips & Tricks, New Features

Getting more out of Cucumber logs

Using Cucumber to do Behaviour driven development (BDD) is great but sometimes a single bug can have impacts on several tests. This article tries to address this.

(more…)
News, New Features

New feature: Code Editor

We are super excited to introduce an enhanced code editor. This means it will be easier to find mistakes in Javascript, and easier to troubleshoot problems.

(more…)
Tips & Tricks, New Features

Logic and Loops using groups

This post explains the changes we made on groups to simplify handling logic and loops. We introduce group types that guide the user when solving more advanced test automation challenges. 

(more…)
Tips & Tricks

Under the hood: Architecture

In this series of blog posts, we will take a closer look at the Boozang technology. We will start with the architecture that explains how Boozang is built and how it interacts with the web browser.

(more…)
Tips & Tricks, New Features

Distributed testing, part 2

In this article, we continue discussing distributed testing and take a closer look at scopes and groups. We also introduce multi-agent testing, which is an example of using test scopes.

(more…)
News, New Features

Distributed testing, part 1

Introduction

We are working on a series of improvements to our platform. These consists of

  • Improving support for parallel test runs
  • Better job distribution
  • Support for multi-agent tests
  • Test handoff

Improving parallelism stems from the need to shorten execution times. In end-to-end tests, execution times are often very long. In a tool where test creation is fast (such as Boozang), long execution times dictate the upper limit of test coverage.

To shorten execution times, it’s also essential to have fair job distribution. Parallelism does little if one worker gets all the work. 

As we have several workers doing work in parallel, this opens up for multi-agent tests. Imagine where you have two active agents acting in a work-flow, like a two-sided marketplace. Usually, we solve this by writing a linear test that switches between the roles and often requires several logins and logouts. In a multi-agent test, you highlight which parts of the test are done by which agents and run the test with at least two active sessions simultaneously. 

In the new Boozang test architecture, we also support test handoff. In Boozang, being able to see the test executing in the browser greatly speeds up test creation. In some cases, the user wants to know the test outcome without seeing the test running in the browser window. Now the user can handoff the test to someone else and continue working un-impeded. 

Borrowing from Agile

To be able to do this well, we have used the analogy of Agile teams. In a team, different team members can have different skillsets. It’s ideal if the team volunteers for the tasks within their skillset. It’s also good for the team to work as an independent unit and handle assigned tasks without any centralized unit meddling. The team will report back to management when they have completed their assigned tasks and are ready to take on new ones.

Much in the same way, servers can have different areas of specialization. Some might live in a specific part of the network, giving them access to systems that others don’t. Some might run as users with a particular role, such as “admin” or “unauthorized.”

New terminology

To model this out on the application level, we introduce the following terminology.

  • Worker: An Boozang client instance that can do work. In Boozang, this is an instance created using the Docker container bz-docker-xvfb (or bz-docker-playwright).
  • Group: A worker belongs to a group, like a team member belongs to a team. In Boozang, a group is a set of client instances that are ready to accept work. 
  • Scope / Tag: Classification of a group. In our group analogy, the scope of a development team is “development.” In Boozang, the scope defines a type of worker. 

Now with this terminology at hand, we can understand the new Boozang view.  

Practical uses: Parallel runs

There are many practical uses for this functionality, but let’s start with the most important: setting up a test suite to run its tests parallel. If there is no specific need to group your workers, you don’t even have to define a group. 

  1. Go to Settings -> Integration -> Generate My account Cooperation Server Link
  2. Enter a unique server number (1, 2, 3 …)
  3. Enter password

The generated URL will look something like:

"http://ai.boozang.com/extension?token=xxx&env=0&number=2#5bea26946c43587a5950f410/0.0.1"

Now you have an executable URL for that worker. Note that the generated URL contains the server number, so it can be automatically assigned by Jenkins if needed. Use the boozang-runner Docker container (or boozang-playwright) to start clients using the URL, ensuring that each client has a unique server number. 

docker run --rm -v "$(pwd):/var/boozang/" styrman/boozang-runner "http://staging-be.boozang.com/extension?token=xxx&env=4&number=2#5e3f275e64f84941a326d4d8/ci44"
docker run --rm -v "$(pwd):/var/boozang/" styrman/boozang-runner "http://staging-be.boozang.com/extension?token=xxx&env=4&number=3#5e3f275e64f84941a326d4d8/ci44"

...
docker run --rm -v "$(pwd):/var/boozang/" styrman/boozang-runner "http://staging-be.boozang.com/extension?token=xxx&env=4&number=1#5e3f275e64f84941a326d4d8/ci44/m2/t1/run"

In the above test, we have created two “helper” workers first; then, by adding the test suite’s module and test suite in one of the clients (in this case, the master: number 1), the test is started.

"http://ai.boozang.com/extension?token=xxx&env=0&number=3#5bea26946c43587a5950f410/0.0.1/m5/t2/run"

The test clients will split up the test suite tests on a “first-come” – first-serve” basis and automatically stop upon completion.

Practical uses: Test handoff

To handoff tests in Boozang we created a new play mode in Boozang called “Remote play.” To test remote play, generate the URL as the previous example and use the Docker boozang-runner to start up the worker.

docker run --rm -v "$(pwd):/var/boozang/" styrman/boozang-runner "http://staging-be.boozang.com/extension?token=xxx&env=4&number=2#5e3f275e64f84941a326d4d8/ci44"
docker run --rm -v "$(pwd):/var/boozang/" styrman/boozang-runner "http://staging-be.boozang.com/extension?token=xxx&env=4&number=3#5e3f275e64f84941a326d4d8/ci44"

The workers will now idly wait for something to do. 

  1. Open the IDE
  2. Go to a test or test suite you want to run
  3. Use the Play dropdown to Click remote play
  4. The workers now show up as idling
  5. Select “Free combination of services”
  6. Click “Go”

You will now see the test being assigned to the first available worker. If it’s a test suite with several tests, the tests will be served up to the workers as in a “first-come”-“first-serve” basis.

As soon as the tests are finished, you should see the workers going back to “idle” status, available to take on more work.

Util functions

To create logic inside the tests that depend on this new code, we have added a set of util functions.

$util.getCoopKey
$util.getCoopScope
$util.getCoopGroup

It’s now possible to be able to do conditional logic inside the tests based on these parameters. For instance, to make sure all workers are using different user accounts, you can keep username and password in a matrix data file userAccounts defined as

_key	username	password
1	tom	secret1
2	anna	secret2

and simply retrieve the username and password using Javascript

let key = $util.getCoopKey;
$test.username  = userAccounts[key].username;
$test.password  = userAccounts[key].password;

Jenkins examples

You can find workable Jenkins examples in the Forum here. They are all based on bash and Docker and should work (almost) equally well for any CI server.

Read more

There is another blog post dealing with more complex parallel tests, where we introduce runner groups and scope. This can be found here.

New Features

New feature: Canvas testing & GoJS

Create tests on top of graphical elements in your application using canvas testing. Unlike other methods, Boozang allows us to record clicks and do validations inside the canvas, just like we do on elements in the DOM tree.

(more…)

Create your account for free!

Prefer a demo instead?