Changeset 6ffc0b for src/documentation/tests

Timestamp:

Dec 14, 2012, 1:03:12 PM (12 years ago)

Author:

Frederik Heber <heber@…>

Branches:

Action_Thermostats, Add_AtomRandomPerturbation, Add_FitFragmentPartialChargesAction, Add_RotateAroundBondAction, Add_SelectAtomByNameAction, Added_ParseSaveFragmentResults, AddingActions_SaveParseParticleParameters, Adding_Graph_to_ChangeBondActions, Adding_MD_integration_tests, Adding_ParticleName_to_Atom, Adding_StructOpt_integration_tests, AtomFragments, Automaking_mpqc_open, AutomationFragmentation_failures, Candidate_v1.5.4, Candidate_v1.6.0, Candidate_v1.6.1, ChangeBugEmailaddress, ChangingTestPorts, ChemicalSpaceEvaluator, CombiningParticlePotentialParsing, Combining_Subpackages, Debian_Package_split, Debian_package_split_molecuildergui_only, Disabling_MemDebug, Docu_Python_wait, EmpiricalPotential_contain_HomologyGraph, EmpiricalPotential_contain_HomologyGraph_documentation, Enable_parallel_make_install, Enhance_userguide, Enhanced_StructuralOptimization, Enhanced_StructuralOptimization_continued, Example_ManyWaysToTranslateAtom, Exclude_Hydrogens_annealWithBondGraph, FitPartialCharges_GlobalError, Fix_BoundInBox_CenterInBox_MoleculeActions, Fix_ChargeSampling_PBC, Fix_ChronosMutex, Fix_FitPartialCharges, Fix_FitPotential_needs_atomicnumbers, Fix_ForceAnnealing, Fix_IndependentFragmentGrids, Fix_ParseParticles, Fix_ParseParticles_split_forward_backward_Actions, Fix_PopActions, Fix_QtFragmentList_sorted_selection, Fix_Restrictedkeyset_FragmentMolecule, Fix_StatusMsg, Fix_StepWorldTime_single_argument, Fix_Verbose_Codepatterns, Fix_fitting_potentials, Fixes, ForceAnnealing_goodresults, ForceAnnealing_oldresults, ForceAnnealing_tocheck, ForceAnnealing_with_BondGraph, ForceAnnealing_with_BondGraph_continued, ForceAnnealing_with_BondGraph_continued_betteresults, ForceAnnealing_with_BondGraph_contraction-expansion, FragmentAction_writes_AtomFragments, FragmentMolecule_checks_bonddegrees, GeometryObjects, Gui_Fixes, Gui_displays_atomic_force_velocity, ImplicitCharges, IndependentFragmentGrids, IndependentFragmentGrids_IndividualZeroInstances, IndependentFragmentGrids_IntegrationTest, IndependentFragmentGrids_Sole_NN_Calculation, JobMarket_RobustOnKillsSegFaults, JobMarket_StableWorkerPool, JobMarket_unresolvable_hostname_fix, MoreRobust_FragmentAutomation, ODR_violation_mpqc_open, PartialCharges_OrthogonalSummation, PdbParser_setsAtomName, PythonUI_with_named_parameters, QtGui_reactivate_TimeChanged_changes, Recreated_GuiChecks, Rewrite_FitPartialCharges, RotateToPrincipalAxisSystem_UndoRedo, SaturateAtoms_findBestMatching, SaturateAtoms_singleDegree, StoppableMakroAction, Subpackage_CodePatterns, Subpackage_JobMarket, Subpackage_LinearAlgebra, Subpackage_levmar, Subpackage_mpqc_open, Subpackage_vmg, Switchable_LogView, ThirdParty_MPQC_rebuilt_buildsystem, TrajectoryDependenant_MaxOrder, TremoloParser_IncreasedPrecision, TremoloParser_MultipleTimesteps, TremoloParser_setsAtomName, Ubuntu_1604_changes, stable

Children:

7ef5b9a

Parents:

3b75c2

git-author:

Frederik Heber <heber@…> (04/12/12 16:01:24)

git-committer:

Frederik Heber <heber@…> (12/14/12 13:03:12)

Message:

DOCU: Enhanced documentation of tests with information from TRAC.

Location:

src/documentation/tests

Files:

• regression-tests.dox (modified) (2 diffs)
• unit-tests.dox (modified) (2 diffs)

src/documentation/tests/regression-tests.dox

@@ -85 +85 @@
  * \section regressiontest-add Adding a new test
  *
+ *  The basic working is: have a input file, do something with it and compare
+ *  to output file against a stored one.
+ *
  *  \attention Name convention of files and directories, e.g.
  *  \b tests/regression/Parser/Pdb with \b testsuite-parser-pdb-save.at
@@ -110 +113 @@
  *  testsuite is automatically re-compiled when one of the test files has changed.
  *
+ *  On how to write a testsuite test, we refer you to one of these .at files to get a
+ *  notion and  here to have a reference of the possible autotest commands at hand.
+ *  The scheme is always the same basically:
+ *  \code
+ * AT_SETUP([General test part - description of this testsuite section])
+ * AT_KEYWORDS([<some keywords>,<actionname>,[undo/redo]])
+ * ...
+ * AT_CHECK([this], <return value>, [ignore], [ignore])
+ * AT_CHECK([that], <return value>, [stdout], [stderr])
+ * AT_CHECk([fgrep "test fine" stdout], 0, [ignore], ignore])
+ * ...
+ * AT_CLEANUP #remove all temporary files
+ * \endcode
+ * where <return value> is some number the code returns to indicate everything
+ * worked fine. The global theme is specified only once per .at file, file
+ * AT_SETUP and AT_CLEANUP sort of embrace every specific test that you want to do.
+ * Note that it is required to list the action name under AT_KEYWORDS and also
+ * give undo oder redo as an additional keyword if the undo or redo of the action
+ * is tested. It is advised to give further keywords, e.g. the directory name
+ * giving the general theme of the tests (selection, analysis, ...). Also note
+ * that all keywords are always lower-case!
+ *
+ * Note that testing the undo/redo-functionality of an Action is always placed
+ * into the same test file along with the normal functionality but in different
+ * tests (i.e. undo and redo each have their own AT_SETUP .. AT_CLEANUP wrapping).
+ *
  *
  * \date 2011-10-31

src/documentation/tests/unit-tests.dox

@@ -75 +75 @@
  *  -# Add the test to the \b Makefile.am contained in \b unittests.
  *
+ * Writing a new unit test then consists of writing the source and the header
+ * file, where you should guide yourself by the already present files. Naming
+ * convention is usually to suffix the name with UnitTest, e.g.
+ * \b SingletonUnitTest.cpp.
+ *
+ * The test class should be named in a similar manner and has the following
+ * look:
+ * \code
+ * class SingletonTest : public CppUnit::TestFixture
+ * {
+ *   CPPUNIT_TEST_SUITE( SingletonTest );
+ *   CPPUNIT_TEST ( blaTest );
+ *   CPPUNIT_TEST ( blablaTest );
+ *   CPPUNIT_TEST_SUITE_END();
+ *
+ * public:
+ *   void setUp();
+ *   void tearDown();
+ *
+ *   void blaTest();
+ *   void blablaTest();
+ * };
+ * \endcode
+ * It inherits CppUnit::TestFixture and defines it as a TEST_SUITE via the
+ * initial macros which also give the test functions to call. Below the test
+ * functions are defined along with setUp() and tearDown() which embrace the
+ * call of each test function, i.e. which create a common test environment for
+ * each test function, by allocating some memory, setting some stuff and
+ * cleaning up in the end again.
+ *
  *  If there is not yet a \b unittests folder:
  *  -# Create a new \b Makefile.am, check out one of the present ones to get
@@ -98 +128 @@
  *    -# ...LDADD gathers all libs that are required for compilation.
  *
+ * \section unittest-failures Marking tests as expected to fail
  *
- * \date 2011-10-31
+ * If a unit tests fails for a given commit but this is expected because the
+ * failure is repaired after some other implementation lateron, it is useful
+ * and required to mark the test as expected to fail.
+ *
+ * To do this, add a variable \b XFAIL_TESTS if not present to the Makefile.am
+ * in which the failing unit test is contained as follows
+ * \code
+ *   XFAIL_TESTS = SingletonUnitTest
+ * \endcode
+ * This will mark \b SingletonUnitTest as expected to fail and continue testing
+ * even though the test is not passed.
+ *
+ *
+ * \date 2012-04-12
  *
  */