Test Parameterization on Robot Framework

Introduction

In this article, you can learn the basics of parameterizing tests on Robot Framework.

It is assumed that you know the material of the article «Templates in Robot Framework»

The official recommendations on the project structure can be studied here

Analogue of this article for PyTest can be studied here

Solution to the problem with visibility of modules - here

Three-digit numbers

Let's consider an example of test parameterization using templates . The project structure has the following original form.

filter_three/ ├── three_digit.py └── tests └── robot └── test_app.py

The filter_three() function and the app.py module return True if the number is three-digit.

def filter_three(x: int) -> bool: return x in range(100, 1000)

*** Settings *** Library ../../three_digit.py Test Template Validate Number *** Test Cases *** number expected_result Digit 8 ${False} Two Digit Number 99 ${False} Lower Border 100 ${True} Three Digit Number 101 ${True} Upper Border 999 ${True} Four Digit Number 1000 ${False} *** Keywords *** Validate Number [Arguments] ${number} ${expected} ${res}= Filter Three ${number} Should Be Equal ${res} ${expected}

python -m robot .\tests\robot\test_filter_three.robot

============================================================================== Test Filter Three ============================================================================== Digit | PASS | ------------------------------------------------------------------------------ Two Digit Number | PASS | ------------------------------------------------------------------------------ Lower Border | PASS | ------------------------------------------------------------------------------ Three Digit Number | PASS | ------------------------------------------------------------------------------ Upper Border | PASS | ------------------------------------------------------------------------------ Four Digit Number | PASS | ------------------------------------------------------------------------------ Test Filter Three | PASS | 6 tests, 6 passed, 0 failed ==============================================================================

Multiplication of two numbers

Let's consider the parameterization of simple multiplication of two numbers

Project structure:

parametrize/ ├── prod │ ├── prod.py │ └── tests │ └── robot │ └── test_prod.robot └── venv

# prod.py def prod(a, b): return a * b

We will run tests from the prod directory using the command

python -m robot .\tests\robot\test_prod.robot

*** Settings *** Library ../../prod.py Test Template Validate Prod *** Test Cases *** arg1 arg2 expected_result Zero Zero ${0} ${0} ${0} Positive Negative ${7} ${-8} ${-56} Float Positive Positive ${13.0} ${14} ${182} *** Keywords *** Validate Prod [Arguments] ${arg1} ${arg2} ${expected_result} ${product}= Prod ${arg1} ${arg2} Should Be Equal ${product} ${expected_result}

============================================================================== Test Prod ============================================================================== Zero Zero | PASS | ------------------------------------------------------------------------------ Positive Negative | PASS | ------------------------------------------------------------------------------ Float Positive Positive | PASS | ------------------------------------------------------------------------------ Test Prod | PASS | 3 tests, 3 passed, 0 failed ==============================================================================

Quadratic equation

Project structure

quad/ ├── quadratic_lists.py └── tests └── robot └── test_quadratic_lists.robot

Let's take a script from here that solves a quadratic equation.

Change the return type from a tuple to a list

# quadratic_lists.py from math import sqrt def quadratic_solve(a, b, c) -> list: if not all( map( lambda p: isinstance(p, (int, float)), (a, b, c) ) ): raise TypeError(TYPE_ERROR_TEXT) print("Types are OK") if a == 0: if b == 0: # a и b 0: решения нет return [None, None] return [-c / b, None] d = b ** 2 - 4 * a * c if d < 0: return [None, None] d_root = sqrt(d) divider = 2 * a x1 = (-b + d_root) / divider x2 = (-b - d_root) / divider if d == 0: x2 = None elif x2 > x1: x1, x2 = x2, x1 return [x1, x2]

Parameterization will be done using templates

# test_quadratic_lists.robot *** Settings *** Library ../../quadratic_lists.py Test Template Verify Solution *** Variables *** @{roots1}= ${4.0} ${-1.0} @{roots2}= ${1.0} ${None} @{roots3}= ${None} ${None} *** Test Cases *** a b c expected_result Two Roots ${1} ${-3} ${-4} ${roots1} Single Root ${1} ${-2} ${1} ${roots2} No Roots ${0} ${0} ${0} ${roots3} *** Keywords *** Verify Solution [Arguments] ${a} ${b} ${c} ${expected} ${res}= Quadratic Solve ${a} ${b} ${c} Should Be Equal ${res} ${expected}

python -m robot .\tests\robot\test_quadratic_lists.robot

============================================================================== Test Quadratic Lists ============================================================================== Two Roots | PASS | ------------------------------------------------------------------------------ Single Root | PASS | ------------------------------------------------------------------------------ No Roots | PASS | ------------------------------------------------------------------------------ Test Quadratic Lists | PASS | 3 tests, 3 passed, 0 failed ==============================================================================

Now let's return the original return type - a tuple in the script with solving a quadratic equation

quad/ ├── quadratic.py └── tests └── robot └── test_quadratic.robot

Now the robot needs to adapt to tuples

*** Settings *** Library ../../quadratic.py Test Template Verify Solution Suite Setup Prepare Variables *** Variables *** @{roots1}= ${4.0} ${-1.0} @{roots2}= ${1.0} ${None} @{roots3}= ${None} ${None} *** Test Cases *** a b c expected_result Two Roots ${1} ${-3} ${-4} ${ex_res1} Single Root ${1} ${-2} ${1} ${ex_res2} No Roots ${0} ${0} ${0} ${ex_res3} *** Keywords *** Verify Solution [Arguments] ${a} ${b} ${c} ${expected} ${res}= Quadratic Solve ${a} ${b} ${c} Should Be Equal ${res} ${expected} Convert List To Tuple [Arguments] ${list} ${tuple}= Evaluate tuple(${list}) RETURN ${tuple} Prepare Variables ${tup1}= Convert List To Tuple ${roots1} ${tup2}= Convert List To Tuple ${roots2} ${tup3}= Convert List To Tuple ${roots3} Set Suite Variable ${ex_res1} ${tup1} Set Suite Variable ${ex_res2} ${tup2} Set Suite Variable ${ex_res3} ${tup3}

Checking a list of unknown length

The get_digits() function from the list_of_digits.py module must return an arbitrary set of digits.

Repetitions are allowed.

The digits are 0, 1, 2… 9.

The function is written with an error, but it is not easy to catch with a single test.

This is the most important example of this article. Before this, we simply used Template with a limited number of tests.

Now we parameterize the check of a list of previously unknown length. That is, we do not know how many tests there will be. It depends on the input data, which we do not control.

Project structure that I would like to have

list_of_digits/ ├── list_of_digits.py └── tests └── robot └── test_list_of_digits.py

# list_of_digits.py import random def get_digits() -> list: n = random.randint(1, 10) result = [] for i in range(0, n): result.append(random.randint(4, 10)) return result

It is not yet clear how to adequately solve this problem on the robot. Below I propose a solution using DataDriver .

The point is that the resulting list is saved as a .csv file and then, using DataDriver, individual tests are run for each line.

It must be installed separately, since it is not included in the standard robot library.

python -m pip install robotframework-datadriver

The project structure now looks like this original image. After running the test, a file will be created next to list_of_digits.py UserData.csv

list_of_digits/ ├── list_of_digits.py └── tests └── robot ├── libraries │ └── save_to_csv.py └── test_list_of_digits.py

# save_to_csv.py def save_list_to_csv(list_of_digits): with open("UserData.csv", "w") as f: f.write("*** Test Cases ***;${var}") for digit in list_of_digits: with open("UserData.csv", "a") as f: f.writelines("\n") f.writelines(";") f.writelines(str(digit))

# test_list_of_digits.py *** Settings *** Library DataDriver ../../UserData.csv Library ../../list_of_digits.py Library libraries/save_to_csv.py Suite Setup Prepare Variables Test Template Variable Is Digit *** Test Cases *** Test ${var} *** Keywords *** Prepare Variables ${list_of_digits}= Get Digits Save List To Csv ${list_of_digits} Variable Is Digit [Arguments] ${var} ${num}= Convert To Integer ${var} Should Be True ${num} >= 0 Should Be True ${num} < 10

python -m robot .\tests\robot\test_list_of_digits.robot

============================================================================== Test List Of Digits ============================================================================== Test 4 | PASS | ------------------------------------------------------------------------------ Test 8 | PASS | ------------------------------------------------------------------------------ Test 1 | PASS | ------------------------------------------------------------------------------ Test 5 | PASS | ------------------------------------------------------------------------------ [ WARN ] Multiple tests with name 'Test 4' executed in suite 'Test List Of Digits'. Test 4 | PASS | ------------------------------------------------------------------------------ Test List Of Digits | PASS | 5 tests, 5 passed, 0 failed ==============================================================================

============================================================================== Test List Of Digits ============================================================================== Test 2 | PASS | ------------------------------------------------------------------------------ Test 10 | FAIL | '10 < 10' should be true. ------------------------------------------------------------------------------ [ WARN ] Multiple tests with name 'Test 2' executed in suite 'Test List Of Digits'. Test 2 | PASS | ------------------------------------------------------------------------------ Test 9 | PASS | ------------------------------------------------------------------------------ Test List Of Digits | FAIL | 3 tests, 2 passed, 1 failed ==============================================================================

The first test run consisted of five tests and found no errors. The second test run, although it consisted of only three tests, was luckier and caught the error already on the second test.

Unique test case names

Warnings of the following type periodically appear in the logs

[ WARN ] Multiple tests with name 'Test X' executed in suite 'Test List Of Digits'.

The name of the test case consists only of the word Test and the variable ${var}, which can be repeated.

*** Test Cases *** Test ${var}

Therefore, if there are two twos in the list, we will get two test cases Test 2

This can be fixed by changing the code save_to_csv.py

# save_to_csv.py def save_list_to_csv(list_of_digits): with open("UserData.csv", "w") as f: f.write("*** Test Cases ***;${var}") i = 0 for digit in list_of_digits: i += 1 with open("UserData.csv", "a") as f: f.writelines("\n") f.writelines("Test-" + str(i) + ". Digit: " + str(digit)) f.writelines(";") f.writelines(str(digit))

python -m robot .\tests\robot\test_list_of_digits.robot

============================================================================== Test List Of Digits ============================================================================== Test-1. Digit: 5 | PASS | ------------------------------------------------------------------------------ Test-2. Digit: 9 | PASS | ------------------------------------------------------------------------------ Test-3. Digit: 4 | PASS | ------------------------------------------------------------------------------ Test-4. Digit: 5 | PASS | ------------------------------------------------------------------------------ Test-5. Digit: 9 | PASS | ------------------------------------------------------------------------------ Test List Of Digits | PASS | 5 tests, 5 passed, 0 failed ==============================================================================

This test run had a repeat of the five and nine, but since test case names now have an index, repeats are impossible

If you want to deepen your knowledge about preparing test data, read the article about DataDriver

Site check

A practical example of using parameterization to check site page responses.

The task is to check whether all pages of the site heihei.ru respond with HTTP code 200.

Project structure:

from_site_map/ └── tests └── robot ├── libraries │ └── save_to_csv.py └── test_site_map.robot

My hosting Beget.com creates a sitemap for free. I download it to the parent directory of tests

from_site_map/ ├── sitemap.xml └── tests └── robot ├── libraries │ └── save_to_csv.py └── test_site_map.robot

The site map looks like this original image:

<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> <loc>https://heihei.ru/Spain/cities/malaga/</loc> <priority>0.8</priority> <changefreq>daily</changefreq> </url> <url> <loc>https://heihei.ru/Finland/destinations/</loc> <priority>0.6</priority> <changefreq>daily</changefreq> </url> </urlset>

Only there are many more pages there.

For parsing the site map we will use the library XML and for accessing the site - the library RequestsLibrary

Let's prepare the test environment

python -m venv venv
venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install robotframework robotframework-requests robotframework-datadriver

*** Settings *** Library XML Library RequestsLibrary Library Collections Library DataDriver ../../UserData.csv Library libraries/save_to_csv.py Suite Setup Prepare Variables Test Template Status Is OK *** Variables *** ${XML}= sitemap.xml @{urls} *** Test Cases *** Test ${url} *** Keywords *** Prepare Variables ${urls}= Get Urls Save List To Csv ${urls} Status Is OK [Arguments] ${url} ${response}= GET ${url} expected_status=200 allow_redirects=${False} Get Urls @{children} = Get Elements ${XML} */loc FOR ${child} IN @{children} Append To List ${urls} ${child.text} END RETURN ${urls}

# save_to_csv.py def save_list_to_csv(list_of_urls, file_name="Urls.csv"): with open(file_name, "w") as f: f.write("*** Test Cases ***;${url}") i = 0 for url in list_of_urls: i += 1 with open(file_name, "a") as f: f.writelines("\n") f.writelines("Test-" + str(i) + ". URL: " + str(url)) f.writelines(";") f.writelines(str(url))

python -m robot .\tests\robot\test_site_map.robot

============================================================================== Test Site Map ============================================================================== ------------------------------------------------------------------------------ Test-1. URL: https://heihei.ru/Spain/cities/cordoba/ | PASS | ------------------------------------------------------------------------------ Test-2. URL: https://heihei.ru/Finland/holidays/easter.php | PASS | …

Troubleshooting

If Python does not find the module with the code, you can try to go to the directory parent to tests from_site_map and add it to the system path.

In PowerShell команда будет выглядеть так:

$Env:Path += ";$pwd"

In Bash it's a little different.

export PATH=$PATH:$(pwd)