What happened last week,
It's been a while since I posted part 2 of the series and for that I apologise, we have been busy working on the testing framework and also integrating unit tests into the application which I will cover in tutorial sometime.
When I wrote the first two posts we were just starting to use the framework to test our app, Anu our test automation engineer most certainly hates me now as she has spent quite a while writing functional tests but I hope now we have everything written and the tests are running when the application builds on the CI server the pain she must suffer with manual testing each release is about over.
Our application as show below was built for mobile and desktop environments,
Because it was intended to be displayed on multiple devices testing could be a real pain, automation testing allows us to test automatically on multiple display resolutions and check the outcome is correct.
Our application as show below was built for mobile and desktop environments,
As I discussed previously we decided we were going to use a Behavioural Based method for our functional tests. Normally these would be written as part of the development process but we were bad developers and did not do this so it was an arduous task to write them after development had completed. The tool we use for most of our BDD is Cucumber (http://cukes.info/), we use this for testing native iOS, Android, APIs and anything else we can find. The benefits of this tool is the language Gherkin (https://github.com/cucumber/cucumber/wiki/Gherkin). This is plain english and should tie pretty close to your agile features.
Thankfully we are only going to be testing the starling demo project however this is more than complicated to get your feet wet.
Lets look at how we can test this using BDD first we are going to write a feature that describes an element of the functionality.
Consider the following example...
Thankfully we are only going to be testing the starling demo project however this is more than complicated to get your feet wet.
Lets look at how we can test this using BDD first we are going to write a feature that describes an element of the functionality.
Consider the following example...
Feature:
When the user clicks on the benchmarks button they should see a screen which allows them to test the performance of the framework.
Scenario:
Given I am on the main screen
And I can see the "Benchmarks" button
When I click the "Benchmarks" button
Then I expect to see a button with the text "Start benchmark"
The above should be understandable by the guys in the suits, in fact we actively encourage them to write these features. It should also be understandable by the guys in the T-shirts as we are the people who will be making this a reality.
Using Cucumber and Melomel allow us to automate our starling application, we can turn the above text into clicks and swipes. This way we can write our tests before our code give ourselves a repeatable method for testing our feature.
Ok boring stuff over tech stuff time.
I am assuming you have followed the previous post part 2 which describes the build process and what is needed to get going. If not head on over there now (http://ttmmhe.blogspot.co.uk/2014/01/functional-testing-starling_15.html)Go to the folder where you cloned the example project and update the repository, it has been a couple of months since I wrote the last post so I will most definitely have updated some stuff. You can do this by executing the command.
git pull
You will see a directory structure which should look like the below:
Also update the gems in the project by executing the ruby command "bundle" from the command line.
bundle
Enable Melomel in Starling
To enable melomel we must reference the melomel.swc in our project then all we need to do is to call the below methods passing a reference to our starling stage.
Melomel.stage = new StarlingStage(stage); Melomel.connect();
Cucumber Structure
Cucumber requires several folders and files in order to run, there is the main "features" folder this contains all your feature files and code to run the tests. The "step_definitions" which contains the code matching your plain english feature steps and a "support" folder which contains well support code and files.
Feature Files
Feature files are plain text files which describe the stuff your application should do just like we looked at earlier. Cucumber parses these files and attempts to match them to ruby functions stored in the step_definitions files.
Step Definition Files
The step definition files contain code that matches the feature step, cucumber uses regex to parse the ruby functions and when it finds a match it will execute the code. This way you can write multiple features that re-use step definitions.
Consider the below step:
"Given I am on the main screen"
This would match the below ruby function:
Given /^I am on the main screen$/ do
menuScene = Melomel.wait_until_visible("MainMenu", nil, nil)
menuScene.should_not be_nil
end
If a visible instance of "MainMenu" is not present in the application an error is thrown. We can also use rspec to assert that the object is not nil.
Lets look at another example, this time we are going to find a button which has a particular text. In cucumber we can write a generic step that will match multiple feature steps.
"And I can see the "Benchmark" button"
This would match the ruby function:
Given /^I can see the "([^"]*)" button$/ do |arg1|
button = Melomel.find!("starling.display.Button", nil, :text => arg1)
button.should_not be_nil
end
This time we are using the standard Melomel.find method, this does not wait for an object to be displayed it assumes that it is already present. We also are using regular expression instead of the direct text reference. This way we can re-use the step for multiple buttons.
Lastly here is an example to show how we can touch a button
Lastly here is an example to show how we can touch a button
When /^I touch the "([^"]*)" button$/ do |arg1|
button = Melomel.find!("starling.display.Button", nil, :text => arg1)
button.should_not be_nil
Melomel.touch(button)
end
Thats really all there is to it, we can build up our features before we start creating our code and in a nice test first manner watch the tests fail then slowly start to pass.
Support Files
Cucumber has a nice feature that allows us to execute an application before we run our steps, inside the 01_launch.rb file you will se the following:
Before do |scenario|
#start the app
@pid = Process.fork do
flex = ENV["FLEX_HOME"]
launchon_mobile = "#{flex}/bin/adl -profile extendedMobileDevice -screensize iPhone -XversionPlatform IOS #{Dir.pwd}/bin/Demo_Mobile-app.xml"
exec launchon_mobile
end
Melomel.connect()
end
After do |scenario|
puts "AFTER TO SCENARIO KILL #{@pid}"
Process.kill 9, @pid
end
at_exit do
end
This code starts our application in the adl and then closes it once our steps have executed.
Running the Example
So all we need to do to run our example is build it and run cucumber
rake build cucumber
If everything has worked you should see output like the following:
Conclusion
To retro fit this testing into our existing application was a bit of a pain however we did have to modify the Melomel framework first to work with Starling. However it has been worth it as now updates are release with much greater confidence and much less pain as the automation testing takes all the load away from testing the application. In a future post I will explain how we have taken this application one step further by re-architecting all the existing code removing PureMVC and implementing RobotLegs so we have full unit test coverage in addition to our functional tests.
You can find all the source for Melomel on the Marks and Spencer public Github repository here:
At present there is no gem uploaded to rubygems for the starling version of melomel, we plan to upload this in the next week.
Have fun.
Great post & clear explanation on Functional testing
ReplyDelete