Quote from ljunggren on February 4, 2021, 12:13 am

Hi guys!

I've created a little script to do some simple data mining in the Cucumber logs. I feel that the Cucumber logs are really useful to express business impact, but not so good at assessing the technical work that needs to be done.

As we know have the concept of root-cause and known-issue, but not the summary logs that reflect this data, I created a simple bash script to extract some interesting facts from the run

#!/usr/bin/env bash

FILE_PATTERN="$@"

if [[ -z "$(ls -1 $FILE_PATTERN 2>/dev/null)" ]] ; then
    printf "\n"
    echo No report files exists. Exiting...
    printf "\n"
    exit
fi

if ! command -v jq &> /dev/null
then
    printf "\n"
    echo "JSON parser jq could not be found"
    echo "Please install it!"
    printf "\n"
    exit
fi

total_scenarios="$(grep Scenario $FILE_PATTERN | wc -l)"
failed_scenarios="$(grep status $FILE_PATTERN | grep failed | wc -l)"

printf "\n"
printf "#############################\n"
printf "### TEST SCENARIO SUMMARY ###\n"
printf "#############################\n"
printf "\n"

if [ -z "$total_scenarios" ]
then
    printf "No scenarios found\n"
else
   printf "Total: %s\nFailed: %s\n" $total_scenarios $failed_scenarios
fi

total="$(grep status $FILE_PATTERN | wc -l)"
failed="$(grep status $FILE_PATTERN | grep failed | wc -l)"
skipped="$(grep status $FILE_PATTERN | grep skipped | wc -l)"
pending="$(grep status $FILE_PATTERN | grep pending | wc -l)"

printf "\n"
printf "#########################\n"
printf "### TEST STEP SUMMARY ###\n"
printf "#########################\n"
printf "\n"

if [ -z "$total" ]
then
    printf "No test steps found\n"
else
    printf "Total: %s\nFailed: %s\nSkipped: %s\nPending: %s\n" $total $failed $skipped $pending
fi

all_issues="$(cat $FILE_PATTERN | jq -c '.[] |.elements'[0].extraData |jq -c 'select(.rootCase != null)' |jq -c '"\(.rootCase.errHash)#\(.rootCase.desc)#\(.rootCase.type)#\(.rootCase.scope)#\(.id) "' | sed 's/^.//;s/.$//')"
unique_issues="$(cat $FILE_PATTERN | jq -c '.[] |.elements'[0].extraData |jq -c 'select(.rootCase != null)' |jq -c '"\(.rootCase.errHash)#\(.rootCase.desc)#\(.rootCase.type)#\(.rootCase.scope)"' | sed 's/^.//;s/.$//'| sort | uniq -c | sort -r)"

total_issues="$(echo "$unique_issues" | grep "#" | wc -l)"
tbd_issues="$(echo "$unique_issues" | grep null | wc -l)"
application_issues="$(echo "$unique_issues" | grep app | wc -l)"
automation_issues="$(echo "$unique_issues" | grep auto | wc -l)"
unknown_issues="$(echo "$unique_issues" | grep unknow | wc -l)"

printf "\n"
printf "###########################\n"
printf "### ROOT CAUSE ANALYSIS ###\n"
printf "###########################\n"
printf "\n"

if [ -z "$unique_issues" ]
then
    printf "No issues found\n"
else
    printf "Total issues: %s\nTo be defined: %s\nApplication: %s\nAutomation: %s\nUnknown: %s\n" $total_issues $tbd_issues $application_issues $automation_issues $unknown_issues
    printf "\n"
    printf "Most impactful issues\n"
    printf "\n"
    echo "${unique_issues}"|sed 's/#/ /g' 
fi

slow_steps="$(cat $FILE_PATTERN | jq -c '.[] |.elements' | jq -c '.[] |.steps'[] | jq '"\(.result.duration/1000000)ms_\(.name)|"' |sort -Vr | head -10 |sed 's/^.//;s/.$//')" 
#times="$(cat $FILE_PATTERN | jq -c '.[] |.elements' | jq -c '.[] |.steps'[]| jq '"\(.result.duration) \(.name)"' |sort -Vr | head -10 | sed 's/^.//;s/.$//' | awk '{system("date -d@"$1/1000000000" -u +%H:%M:%S")}')"
printf "\n"
printf "##########################\n"
printf "### SLOWEST TEST STEPS ###\n"
printf "##########################\n"
printf "\n"

if [ -z "$slow_steps" ]
then
    printf "No test steps found\n"
else
    echo $slow_steps|sed 's/_/: /g' |sed 's/|/\n/g'|sed -e 's/^[ \t]*//'
fi

worker_list="$(ls -rt $FILE_PATTERN | xargs grep -ho "\[m.*t.*\].*" | sed 's/..$//')"

printf "\n"
printf "####################################################\n"
printf "### JOB/WORKER LIST SORTED BY TIME OF COMPLETION ###\n"
printf "####################################################\n"
printf "\n"

if [ -z "$worker_list" ]
then
    printf "No jobs found\n"
else
    echo "$worker_list"
fi

printf "\n"
printf "### END REPORT ####\n"

In order to use this script, simply run the following commands in your bash shell

curl -o generate_summary.sh https://raw.githubusercontent.com/ljunggren/bz-utils/main/scripts/generate_summary.sh

chmod +x generate_summary.sh

./generate_summary.sh *.json

I hope this simple script can inspire you to create your own utility scripts, and maybe I can share them in the bz-utils repo

https://github.com/ljunggren/bz-utils