Different types of delay strategies
Delay strategies and stability patterns
Boozang tests are built to be able to run predictably and stable across different environments, even if these environments have vastly different response times. Often, the user simply has to record and replay tests, and Boozang will automatically adapt to run as stable as possible.
Let's look at some of the mechanisms Boozang uses, and how a user can calibrate these settings.
Different action-level settings
Timeouts
The main mechanism that is used is timeouts. Boozang has a default timeout set to 5 seconds, but will increase this if during recording a slow response is found. Under normal circumstances, this means that for any action, such as a click or a validation, Boozang tries to do it instantly. If unsuccessful, Boozang will re-try for 5 seconds until failing the action execution.
If a test action is expected to respond slower than 5 seconds in some cases, it's perfectly fine to increase the action timeout. As long as the element eventually renders, increasing the timeout will not increase the overall execution time of the test. This is the most common customization to do in Boozang.
Note: Timer shows as blue in the action list when timeout has been customized
Delays
Setting a delay is different from timeouts as it forces Boozang to wait the delay time before doing the re-try. The default delay in Boozang is 100 ms (the step delay time), and it's rarely recommended to change this setting as it can slow increase test execution times.
There are some exceptions to this rule when setting a delay is recommended
- Background process: In some cases, the UI update needs to wait for an asynchronous background process to complete. In this case, use a delay.
- Identical DOM elements: Sometime you'll have identical DOM elements that renders in different order. If these cannot be identified in any other way, it can make sense to use a delay to make sure both elements have rendered before executing the action.
Note: Timer shows as red in the action list when delay has been customized
Delays and Timeouts
There is nothing stopping you from using delays in combination with timeouts. If you know that an element will appear in minimum 5 seconds and maximum 30 seconds, you can set the action delay to 5 seconds and timeout to 30 seconds.
Note: Timer shows as red in the action list when both timeout and delay have been set
Timeout and fail element
A less known strategy to handle slow elements / response times is to combine timeouts with fail elements.
Imagine a very unstable page and slow page, where load times vary between 1 second and 2 minutes. In the case of a validation on this page, the user might need to set a timeout of 2 minutes to guarantee to avoid false negatives.
Now imagine you have two outcomes on this page, either the page renders "Yes" or "No", but it can still take up to 2 minutes. The normal way would be to look to Validate "Yes" and wait up to 2 minutes. But what if "No" shows up already after 10 seconds? Did we just wait for 1 minute and 50 seconds for no need?
In order to handle this case, we use "Fail element". Fail element will fail an action instantly if the element shows up. It's kind of an "anti-element".
Note: This might seem like a strange scenario, but is actually very common to greatly speed up tests checking if a user is logged in or not.
Conclusions
Usually, Boozang makes sure to wait for elements to render, and applies re-tries to make sure actions run as stable as possible. There are times when there is a need to "tweak" individual actions to make sure tests run stable, but still executes as fast as possible. Use these strategies to fine-tune tests for stability and performance.
