How effective is YAML Testing in OpenERP?
What is Yaml?
YAML is a human-readable data serialization format that takes concepts from programming languages such as C, Perl, and Python, and ideas from XML and the data format of electronic mail. YAML stands for YAML Ain’t Markup Language (yes, that’s a recursive acronym). YAML is available as a format for OpenERP data as of OpenERP 6.0, featuring the following advantages:
- User friendly format as an alternative to our current XML data format.
- Same system to load data or tests, integrated in modules.
- Built in OpenERP so that you can develop complex Python tests.
- Simpler for non developers to write functional tests.
How to write a Yaml Test?
This piece of information is written on the basis of the recent experience I had in writing YAML test for OpenERP. This mode of testing may be the best available approach to do automated tests in OpenERP which is really a major milestone during any enterprise implementation of this great application.
Writing a Yaml Test is a two phase process. In the first phase the person with a strong domain knowledge (need not be a developer) writes the required functional tests in plain English. In the second phase a developer converts these tests to codes.
The Test scenario is described for testing a wizard to create a Budget for a particular activity. It can be tried in any custom modules or functionalities. Some tests are already written in the existing OpenERP code and documentations are also available.
A sample test case can be written by a non developer as follows:
-
To test the budget creation wizard's working I create department, program, and activity and perform budget creation using these.
-
I create a department named Test Department 1
-
I create a program named Test Program 1 under Test Department 1
-
I create an activity named Test Activity 1 under Test Program 1
-
I select Test Department 1, Test Program 1, Test Activity 1 and 1991-1995 budget period
-
I create a budget using the previously selected values.
-
I test that the state of the budget is Draft.
Next a developer adds the actual code to perform these tests as follows:
-
To test the budget creation wizard's working I create department, program, and activity and perform budget creation using these.
-
I create a department named Test Department 1
-
!record {model: zb.department, id: department_id_0}:name: "Test Department 1"
-
I create a program named Test Program 1 under Test Department 1
-
!record {model: zb.program, id: program_id_0_0}:name: "Test Program 1"
department_id: department_id_0
-
I create an activity named Test Activity 1 under Test Program 1
-
!record {model: zb.activity, id: activity_id_0_0_0}:name: "Test Activity 1"
program_id: program_id_0_0
-
I select Test Department 1, Test Program 1, Test Activity 1 and 1991-1995 budget period
-
!record {model: zb.budget.target, id: budget_wizard_id}:department_id: department_id_0
program_id: program_id_0_0
activity_id: activity_id_0_0_0
budget_period_id: budget_period_id_0
-
I create a budget using the previously selected values.
-
!python {model: zb.budget.target}: |result = self.create_budget(cr, uid, [ref('budget_wizard_id')], {})assert result, 'The budget was not created.'
-
I test that the state of the budget is Draft.
-
!python {model: zb.budget}: |department_id = ref("department_id_0")
program_id = ref("program_id_0_0")
activity_id = ref("activity_id_0_0_0")
budget_period_id = ref("budget_period_id_0")
ids = self.search(cr, uid, [('department_id', '=', department_id), ('program_id', '=', program_id), ('activity_id', '=', activity_id), ('budget_period_id', '=', budget_period_id)])assert len(ids) == 1, 'There should only be one budget for a combination of a particular department, program, activity and budget period.'
budget_obj = self.browse(cr, uid, ids, {})[0]assert budget_obj.budget_state == 'draft', 'The newly created budget must be in Draft.'
Save this file with an extension .yml for example budget_creation_wizard.yml.
How to run the test?
In the __openerp__.py file add the name of the test file including its relative path in the test tag. That is,
'test' : ['test/budget_creation_wizard.yml']
Start the openerp server with the following parameter --log-level=test. This will log the details of the test being done. By looking at the log we can find which all tests were performed and which all tests passed and the failed tests.
./openerp-server.py --log-level=test
In the log file we can see lines like the following:
TEST:tests.budget_performance:To test the budget creation wizard's working I create department, program, and activity and perform budget creation using these.
TEST:tests.budget_performance:I create a department named Test Department 1.
TEST:tests.budget_performance:I create a program named Test Program 1 under Test Department 1.
TEST:tests.budget_performance:I create an activity named Test Activity 1 under Test Program 1.
TEST:tests.budget_performance:I select Test Department 1, Test Program 1, Test Activity 1 and 1991-1995 budget period.
TEST:tests.budget_performance:I create a budget using the previously selected values.
TEST:tests.budget_performance:I test that the state of the budget is Draft.
Limitations of YAML Testing
Since YAML testing is a server side testing, there is a limitation in testing the view of the model. Eventhough we cannot test the view completely some features of the view that we can test includes:
- When we change a particular field's value another field(s) depending on this field changes. This scenario can be simulated by writing a test which calls the onchange function in the corresponding model.
- Check the view's rendering. Even though this test does not guarantee that the view will be rendered correctly but at least we can make sure that the view is not having any major error. This can be accomplished by writing a test that calls the fields_view_get function of the corresponding model.
- Button click. The button click can be simulated by writing tests which calls the underlying function to which the button calls.
Conclusion
Yaml testing makes regression testing easy. With a fair amount of good tests we can be assured about the quality of the code and any modification of the existing features or addition of the new features does not introduce any new bug.
Some useful links
1. http://doc.openerp.com/v6.0/developer/5_16_data_serialization/
2. http://doc.openerp.com/v6.0/contribute/15_guidelines/coding_guidelines_testing.html
3. http://fptiny.blogspot.com/2010/09/new-automated-test-framework-for.html
4. http://fptiny.blogspot.com/2010/09/improved-automated-tests-for-openerp.html