Cucumber BDD testing using TestNG framework with Selenium webdriver and Cucumber reports

This new age requires all development to be test driven and Cucumber is a great tool to support behavioral driven development. It also creates documentation automatically and easily shareable. Cucumber bridges the gap between business and technical users. Users can write test cases in plain English and the implementation can follow. It also serves the purpose of end-to-end testing framework.

TestNG is a testing framework like Junit. The advantages TestNG provides are:

1. It can run tests in parallel
2. Supports advance annotations like BeforeSuite/AfterSuite etc.
3. Grouping of tests are allowed

and so on.

So it makes sense to create a test suite which has a combination of both. My post here provides the details of the test suite which combines both TestNG and cucumber and uses selenium webdriver to execute test cases on a couple of websites. There are a bunch of features I implemented which can be utilized to create enterprise grade test suite. In the end, I’ll also highlight how to automate the test suite using Gitlab. So let’s jump in.

This app is a simple gradle app and gradle is configured to execute TestNG framework.

So first thing first are the dependencies —

dependencies {
    implementation 'org.seleniumhq.selenium:selenium-java:4.1.3'
    implementation 'io.github.bonigarcia:webdrivermanager:5.1.0'
    implementation 'org.testng:testng:7.5'
    implementation 'io.cucumber:cucumber-java:7.2.3'
    implementation 'io.cucumber:cucumber-testng:7.2.3'
    implementation 'org.slf4j:slf4j-simple:1.7.35'
}

Below is snippet of build.gradle that can be used to configure gradle to execute the TestNG suite —

test {
    useTestNG() {
        suites "testng.xml"
        testLogging.showStandardStreams = true
        //report generation delegated to TestNG library:
        useDefaultListeners = true
        outputDirectory = file("$project.buildDir//testngOutput")
    }
    //turn off Gradle's HTML report to avoid replacing the reports generated by TestNG library:
    reports.html.enabled(false)

    systemProperty 'platform', System.getProperty('platform')
}

While here on the build.gradle, I also configured the gradle cucumber reporting so as to generate cucumber reports automatically —

plugins {
    id 'java'
    id "com.github.spacialcircumstances.gradle-cucumber-reporting" version "0.1.24"
}
cucumberReports {
    outputDir = file('build/reports/')
    buildId = '0'
    reports = files('build/cucumber-reports/cucumber.json')
}

So pretty much when the suite is executed we’ll see below outputs:

1. TestNG report under build/testngOutput
2. Cucumber reports under build/reports/
3. Screen shots of test cases under build/screenshots (To be covered later in the post)

Let’s dive-in on the TestRunner. It is a JAVA class implementing the TestNG suite which is used to execute cucumber features. Cucumber features are configured as —

@CucumberOptions(
      features = {"src/test/resources/featureFiles"},
      glue = {"com.company.step.definitions"},
      monochrome = true,
      dryRun = false,
      plugin = {
            "json:build/cucumber-reports/cucumber.json",
            "rerun:build/cucumber-reports/rerun.txt"
      })

glue option here tells which package has the implementation of features.

To execute the TestNG suite, I have implemented these functions —

@Test(groups = "cucumber", description = "Runs Cucumber Scenarios", dataProvider = "scenarios")
public void runScenario(PickleWrapper pickleWrapper, FeatureWrapper featureWrapper) {
   testNGCucumberRunner.runScenario(pickleWrapper.getPickle());
}

@DataProvider
public Object[][] scenarios() {
   return testNGCucumberRunner.provideScenarios();
}

DataProvider here provides all the scenarios and using TestNGCucumberRunner all those scenarios are run.

This TestRunner class is configured to run using a testng.xml file (which was configured using gradle) —

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Test Suite">
       <parameter name="platform" value="${platform}" />
        
         <test name="TestSuite" >
          <classes>
              <class name="com.company.runner.TestRunner"/>
          </classes>
        </test>
</suite>

Let’s jump to Selenium webdriver initialization. Here I have downloaded the chrome drivers and hardcoded them in the project. Selenium also automatically downloads the webdriver based on the browser installed (also configured in the code) and execute test cases.

In my case I have downloaded, but if you want selenium to use latest drivers, just delete the drivers and configure code accordingly.

Selenium supports different browsers and also remote browsers. The project doesn’t touch those points, but a simple functionality can always be written to execute all test cases in all the browsers.

Let’s initialize the webdriver —

public class TestRunner {
   private TestNGCucumberRunner testNGCucumberRunner;
   public static WebDriver webdriver;

   @BeforeSuite(alwaysRun = true)
   @Parameters({"platform"})
   public void setUpCucumber(String platform) throws Exception {
      testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());

      if (platform.equalsIgnoreCase("Windows")) {
         System.setProperty("webdriver.chrome.driver",
               System.getProperty("user.dir") + "/src/main/resources/Drivers/chromedriver_windows_100.0.4896.60.exe");
         ChromeOptions options = new ChromeOptions();
         //options.setHeadless(true);
         options.addArguments("--disable-notifications");
         webdriver = new ChromeDriver(options);

      } else if (platform.equalsIgnoreCase("Linux")) {
         System.setProperty("webdriver.chrome.driver",
               System.getProperty("user.dir") + "/src/main/resources/Drivers/chromedriver_linux64_100.0.4896.60");
         ChromeOptions options = new ChromeOptions();
         options.setHeadless(true);
         options.addArguments("--disable-notifications");
         webdriver = new ChromeDriver(options);
      }

      log.info("Chrome Driver Initialized.");
   }

A few things are happening here —

1. @BeforeSuite annotation is used. This will ensure that webdriver is initialized before the test suite is started
2. Hardcoded drivers based on platform which is either Windows or Linux
3. @Parameters annotation is used. This ensures that any parameter that is passed to test suite is automatically captured. In our case, we configured gradle to pass the parameter as property parameter like this —

systemProperty 'platform', System.getProperty('platform')

You can check build.gradle section when we configured the test stage of gradle.

Similarly, I have also implemented the @AfterSuite annotation to close webdriver and quit TestNG —

@AfterSuite(alwaysRun = true)
public void tearDownClass() throws Exception {
   webdriver.quit();
   testNGCucumberRunner.finish();
}

So now our Runner is ready. Let’s implement the test case cucumber scenarios. For scenarios, first thing is feature files. I have two feature files for sample testing here:

• Tests the reset button functionality of guru99 website

Feature: Reset Functionality Scenarios

  Scenario Outline: Reset Functionality
    Given User navigates to "<website>"
    When User enter the Username "<username>" and Password "<password>"
    Then Reset the credential

    Examples:
      | website                        | username | password   |
      | https://www.demo.guru99.com/v4 | test1    | test1      |
      | https://www.demo.guru99.com/v4 | test2    | test2      |

• Test to check the Home button functionality of javatpoint website

Feature: Home Button Functionality Scenarios

  Scenario Outline: Home Button Functionality
    Given User navigates to "<website>"
    When User clicks on Home Button
    Then Home website "<website2>" is reached

    Examples:
      | website                                | website2                    |
      | https://www.javatpoint.com/gui-testing | https://www.javatpoint.com/ |
      | https://www.javatpoint.com/gui-testing | https://www.javatpoint.com/ |

Plain English and this is the advantage Cucumber provides. Business users and write test cases and implementation can follow later

Before we jump into implementation, you need a @Before function to provide the scenario —

@Before
   public void before(Scenario scenarioVal) {
       this.scenario = scenarioVal;
       log.info("Scenario: " + scenario.getName());
   }

This needs to be there in all the test classes extending TestRunner.

Let’s see the implementation now for test1 —

@Given("^User navigates to \"(.*)\"$")
 public void user_navigates_to_website(String website) throws Exception {
  log.info("User navigates to website: "+ website);
  webdriver.get(website);
  webdriver.manage().window().maximize();
  Thread.sleep(2000);
 }
@When("^User enter the Username \"(.*)\" and Password \"(.*)\"$")
 public void enter_the_Username_and_Password(String username,String password) throws Throwable
 {
  webdriver.findElement(By.xpath("//input[@name='uid']")).sendKeys(username);
  webdriver.findElement(By.xpath("//input[@name='password']")).sendKeys(password);
 }
@Then("^Reset the credential$")
 public void reset_the_credential() throws InterruptedException {
  webdriver.findElement(By.xpath("//input[@name='btnReset']")).click();
 }

Here we load the website into webdriver and then use XPATH to locate the website components. I am using relative path and found the most easiest way to do this.

I installed a cromeplugin from — https://selectorshub.com/

Once you have it, its easy to locate xpath by right clicking on the component and copy relative path. The advantage relative xpath provides is if there are any changes on the website, it has a high chances of being immune to those changes.

Please note the way data is propagated from feature file to implementation.

For test2, here is the implementation —

@When("^User clicks on Home Button$")
public void user_clicks_on_home_button() throws Throwable
{
   webdriver.findElement(By.xpath("//a[normalize-space()='Home']")).click();
}

@Then("^Home website \"(.*)\" is reached$")
public void home_website_is_reached(String expectedUrl) throws InterruptedException {
   String url = webdriver.getCurrentUrl();
   Assert.assertEquals(expectedUrl, url);
}

I am also asserting the value of URL here, test case will fail if the values won’t match.

You can also get the screenshots of each test case using below code —

@After
public void screenCapture(Scenario scenario) throws Exception {
    String filename = System.getProperty("user.dir") + "/build/screenshots/"
          + scenario.getName().replace(" ", "_") + "_" + scenario.getLine() + ".png";
    log.info("Taking Screenshot --> " + filename );
    byte screenshot[] = ((TakesScreenshot) webdriver).getScreenshotAs(OutputType.BYTES);
    File f = new File(filename);
    f.getParentFile().mkdirs();
    f.createNewFile();
    FileOutputStream fos = new FileOutputStream(filename);
    fos.write(screenshot);
}

This will ensure that screenshots are available at the end of each scenario and can be verified later.

Let’s talk about execution now. Execution is simple, just issue below gradle command —

gradle test -Dplatform="Windows"

Let’s talk about outputs and reporting.

• TestNG reports looks like this

• Cucumber reports looks like this

• And screenshots also got captured

Pretty neat, right!

Now lastly, as I said I’ll try to touch base on how to automate all this using Gitlab pipeline. This is not a part of the repo, but you can always implement using script feature of gitlab. What you need is implement .gitlab_ci.yaml like this:

image: busybox:latest

test:
  stage: test
  services:
    - name: selenium/standalone-chrome:99.0
      alias: chrome-browser
  image: gradle:latest
  before_script:
    - export GRADLE_USER_HOME=`pwd`/.gradle
    - chmod +x gradlew
  script:
    - ./gradlew test -Dplatform="Linux"
  artifacts:
    paths:
      - build/
    expire_in: 1 week
    when: always

There are a few things happening here —

1. Implementation of additional services before the stage starts. This will ensure chrome-browser is ready served by a docker image at a URL — http://chrome-browser:4444/wd/hub
2. Reports and screenshots are uploaded as artifacts which can browsed using gitlab
3. All artifacts expire after sometime

There is one thing to notice during webdriver initialization here. As the browser is now available as separate service, so instead of using ChromeDriver, you need to use RemoteWebDriver, like this —

webdriver = new RemoteWebDriver(new URL("http://chrome-browser:4444/wd/hub"), options);

That’s all folks. Please leave your comments and I’ll improve this post as much as I can. Gitlab repo is available in below section.

References:

Github: https://github.com/paras301/testng-with-cucumber

Cucumber selenium post: https://www.guru99.com/using-cucumber-selenium.html

Selenium webdriver: https://www.selenium.dev/documentation/webdriver/getting_started/first_script/