Python test frameworks and testing

satya - 3/15/2024, 8:46:49 AM

Roughly I was told by popularity

  1. pytest
  2. unittest
  3. doctest
  4. nose2
  5. Robot Framework
  6. hypothesis
  7. behave
  8. tox

satya - 3/15/2024, 8:47:39 AM

A bigger list

  1. pytest
  2. unittest
  3. nose2
  4. nose
  5. doctest
  6. tox
  7. hypothesis
  8. Robot Framework
  9. behave
  10. lettuce
  11. unittest-xml-reporting
  12. green

satya - 3/15/2024, 1:23:39 PM

Example with unittest module, setup, tear down


import unittest

class MyTest(unittest.TestCase):
    def setUp(self):
        # Setup code here
        print("Setting up before each test")

    def tearDown(self):
        # Teardown code here
        print("Tearing down after each test")

    def test_hello(self):
        print("hello")

    def test_ha(self):
        print("ha")

if __name__ == '__main__':
    unittest.main()

satya - 3/15/2024, 1:24:40 PM

notes

  1. uses unittest built-in module
  2. setup and teardown are called before and after each test
  3. This is different from setup and tear down "once" for "ALL" tests

satya - 3/15/2024, 1:25:06 PM

Setup for all tests example


import unittest

class MyTest(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        # Setup code that runs once before all tests
        print("Setting up before all tests")

    @classmethod
    def tearDownClass(cls):
        # Teardown code that runs once after all tests
        print("Tearing down after all tests")

    def test_hello(self):
        print("hello")

    def test_ha(self):
        print("ha")

if __name__ == '__main__':
    unittest.main()

satya - 3/16/2024, 12:18:18 PM

A larger example


"""
*************************************************
* Test Configuration
*************************************************
data files used:
/data/testconfig.toml

Code:
1. write me a unit test class TestConfig
2. Provide methods to setup and tear down at the class level
3. Create 1 test to start with called "test1_simpleConfig()"

"""

import unittest
from baselib import baselog as log
from appwall.appinitializer import AppInitializer
from appwall.appobjectsinterface import AppObjects
from baselib import fileutils as fileutils

def _getConfigFilename() -> str:
    curdir = fileutils.getCurrentFileRoot(__file__)
    tomlConfigFilename = fileutils.pathjoin_segments(curdir, "data", "testconfig1.toml")
    return tomlConfigFilename


class TestConfig(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        log.ph("Setting up the test environment", "start")
        configfilename = _getConfigFilename()
        AppInitializer.initializeApplication(configfilename)
        log.ph1("Test setup complete. Real tests start.")

    @classmethod
    def tearDownClass(cls):
        log.ph("Stopping the tests", "done")

    def test1_getValueExisting(self):
        cfg = AppObjects.getConfig()

        # Test direct key at root level
        log.info("Testing direct key at root")
        value = cfg.getValue("key1")
        self.assertEqual(value, "key1")

        # Test a sub key under a category
        log.info("Testing a sub key: topic1.key1")
        value = cfg.getValue("topic1.key1")
        self.assertEqual(value, "topic1 key1")


    def test1_getValueNonExisting(self):
        cfg = AppObjects.getConfig()
        log.info("Testing a non existent key for exception")
        with self.assertRaises(Exception):
            cfg.getValue("Non-existent-key")

def test():
    unittest.main()
    #log.ph("Configuration filename", _getConfigFilename())

def localTest():
    log.ph1("Starting local test")
    test()
    log.ph1("End local test")

if __name__ == '__main__':
    localTest()

satya - 3/16/2024, 12:21:33 PM

A list of asserts

  1. assertEqual(a, b)
  2. assertNotEqual(a, b)
  3. assertTrue(x)
  4. assertFalse(x)
  5. assertIs(a, b)
  6. assertIsNot(a, b)
  7. assertIsNone(x)
  8. assertIsNotNone(x)
  9. assertIn(a, b)
  10. assertNotIn(a, b)
  11. assertIsInstance(a, b)
  12. assertNotIsInstance(a, b)
  13. assertRaises(Exception, func, *args, **kwargs)
  14. assertRaisesRegex(Exception, regex, func, *args, **kwargs)
  15. assertAlmostEqual(a, b)
  16. assertNotAlmostEqual(a, b)
  17. assertGreater(a, b)
  18. assertGreaterEqual(a, b)
  19. assertLess(a, b)
  20. assertLessEqual(a, b)
  21. assertRegex(text, regex)
  22. assertNotRegex(text, regex)

satya - 3/16/2024, 6:17:45 PM

Return values from test functions

  1. They don't return any values.
  2. Instead assert functions are expected through exceptions to detect a success or failure.