Tuesday, July 12, 2011

Test Automation Process Overview

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions.[1] Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.

Overview

Although manual tests may find many defects in a software application, it is a laborious and time consuming process. In addition, it may not be effective in finding certain classes of defects. Test automation is a process of writing a computer program to do testing that would otherwise need to be done manually. Once tests have been automated, they can be run quickly and repeatedly. This is often the most cost effective method for software products that have a long maintenance life, because even minor patches over the lifetime of the application can cause features to break which were working at an earlier point in time.

There are two general approaches to test automation:

* Code-driven testing. The public (usually) interfaces to classes, modules, or libraries are tested with a variety of input arguments to validate that the results that are returned are correct.
* Graphical user interface testing. A testing framework generates user interface events such as keystrokes and mouse clicks, and observes the changes that result in the user interface, to validate that the observable behavior of the program is correct.

Test automation tools can be expensive, and it is usually employed in combination with manual testing. It can be made cost-effective in the longer term, especially when used repeatedly in regression testing.

One way to generate test cases automatically is model-based testing through use of a model of the system for test case generation but research continues into a variety of alternative methodologies for doing so.[citation needed]

What to automate, when to automate, or even whether one really needs automation are crucial decisions which the testing (or development) team must make. Selecting the correct features of the product for automation largely determines the success of the automation. Automating unstable features or features that are undergoing changes should be avoided.

Code-driven testing

A growing trend in software development is the use of testing frameworks such as the xUnit frameworks (for example, JUnit and NUnit) that allow the execution of unit tests to determine whether various sections of the code are acting as expected under various circumstances. Test cases describe tests that need to be run on the program to verify that the program runs as expected.

Code driven test automation is a key feature of Agile software development, where it is known as Test-driven development (TDD). Unit tests are written to define the functionality before the code is written. Only when all tests pass is the code considered complete. Proponents argue that it produces software that is both more reliable and less costly than code that is tested by manual exploration. It is considered more reliable because the code coverage is better, and because it is run constantly during development rather than once at the end of a waterfall development cycle. The developer discovers defects immediately upon making a change, when it is least expensive to fix. Finally, code refactoring is safer; transforming the code into a simpler form with less code duplication, but equivalent behavior, is much less likely to introduce new defects.
[edit] Graphical User Interface (GUI) testing

Many test automation tools provide record and playback features that allow users to interactively record user actions and replay them back any number of times, comparing actual results to those expected. The advantage of this approach is that it requires little or no software development. This approach can be applied to any application that has a graphical user interface. However, reliance on these features poses major reliability and maintainability problems. Relabelling a button or moving it to another part of the window may require the test to be re-recorded. Record and playback also often adds irrelevant activities or incorrectly records some activities.

A variation on this type of tool is for testing of web sites. Here, the "interface" is the web page. This type of tool also requires little or no software development.[citation needed] However, such a framework utilizes entirely different techniques because it is reading HTML instead of observing window events.[citation needed]

Another variation is scriptless test automation that does not use record and playback, but instead builds a model of the application under test and then enables the tester to create test cases by simply editing in test parameters and conditions. This requires no scripting skills, but has all the power and flexibility of a scripted approach. Test-case maintenance is easy, as there is no code to maintain and as the application under test changes the software objects can simply be re-learned or added. It can be applied to any GUI-based software application.
[edit] What to test

Testing tools can help automate tasks such as product installation, test data creation, GUI interaction, problem detection (consider parsing or polling agents equipped with oracles), defect logging, etc., without necessarily automating tests in an end-to-end fashion.

One must keep satisfying popular requirements when thinking of test automation:

* Platform and OS independence
* Data driven capability (Input Data, Output Data, Metadata)
* Customizable Reporting (DB Access, crystal reports)
* Easy debugging and logging
* Version control friendly – minimal binary files
* Extensible & Customizable (Open APIs to be able to integrate with other tools)
* Common Driver (For example, in the Java development ecosystem, that means Ant or Maven and the popular IDEs). This enables tests to integrate with the developers' workflows.
* Support unattended test runs for integration with build processes and batch runs. Continuous Integration servers require this.
* Email Notifications (automated notification on failure or threshold levels). This may be the test runner or tooling[clarification needed] that executes it.
* Support distributed execution environment (distributed test bed)
* Distributed application support (distributed SUT)

Framework approach in automation

A framework is an integrated system that sets the rules of Automation of a specific product. This system integrates the function libraries, test data sources, object details and various reusable modules. These components act as small building blocks which need to be assembled to represent a business process. The framework provides the basis of test automation and simplifies the automation effort.
Defining boundaries between automation framework and a testing tool

Tools are specifically designed to target some particular test environment. Such as: Windows automation tool, web automation tool etc. It serves as driving agent for an automation process. However, automation framework is not a tool to perform some specific task, but is an infrastructure that provides the solution where different tools can plug itself and do their job in a unified manner. Hence providing a common platform to the automation engineer doing their job.

There are various types of frameworks. They are categorized on the basis of the automation component they leverage. These are:

1. Data-driven testing
2. Modularity-driven testing
3. Keyword-driven testing
4. Hybrid testing
5. Model-based testing

Notable test automation tools
Tool name Produced by Latest version
HP QuickTest Professional HP 11.0
IBM Rational Functional Tester IBM Rational 8.2.0.2
Parasoft SOAtest Parasoft 9.0
QF-Test Quality First Software GmbH 3.4.0
Ranorex Ranorex GmbH 3.0
Rational robot IBM Rational 2003
Selenium Open source 1.0.10
SilkTest Micro Focus 2010 R2 WS2
TestArchitect LogiGear 6.0
TestComplete SmartBear Software 8.5
Testing Anywhere Automation Anywhere 7.0
TestPartner Micro Focus 6.3
TOSCA Testsuite TRICENTIS Technology & Consulting 7.2.2[3]
Visual Studio Test Professional Microsoft 2010
WATIR Open source 1.6.5
WebUI Test Studio Telerik, Inc. 2011.1

Sunday, July 3, 2011

Difference between says and said in Direct speech

I would say that you would use says when you're talking about something which has been said very recently and is still true. For example:

You and Mike are in the living room. He says "I'm hungry". You go into the kitchen and say to another friend "Mike says he's hungry, so shall we make some lunch now?"

D.S. : Kamala said, 'I meet (simple present) my friend often.'.
I.S. : Kamala said that she met (simple past) often.
A reported verb in the simple past in direct speech remains as it is even in the indirect speech, only if the reporting verb (verb outside the quotations) is in the present/ the future tense
D.S.: Kamala says, 'I went there yesterday'.
Here the reporting verb 'says' is in the present tense. So the tense of the reported
verb, 'went' (simple past) remains as it is.
I.S.: Kamala says that she went out yesterday.
If the reporting verb is in the past tense the tense of the reported verb
must be changed and the simple past must be changed into past perfect,
whether it is an immediate past action or distant past action. ('long past' as
you say)
e.g. : The constable said to the inspector, 'We arrested him ten minutes
ago' (Ten minutes ago- not distant/ long
past). Even then the I.S. is:The constable told the inspector that they
had arrested him ten minutes before
It is not question of near/ immediate past or distant past/ long past. If is depends on whether the reporting verb is in the present or the past tense.
Q. (i) I read in a book (DGP publications) that long past actions changed from D.S to I.D.S
simple past - Past perfect tense.
e.g.: He said, 'Rama Killed Ravana'
He said that Rama had killed Ravana.
(ii) He said, 'I lived many years in England.'
He said that he lived many years in England.
Please explain above example.
A. (i) D.S. : The teacher is saying, 'Rama killed Ravana'.
I.S. : The teacher is saying (Reporting verb - present tense) that Rama killed
Ravana. (Correct). Here, we don't say,
Rama had killed Ravana.
(ii) Similarly: D.S. : He says, 'I lived many years in England'.
I.S.: He says that he lived many years in England.
D.S.: He said, 'I lived many years' in England.
I.S.: He said that he had lived... in England.
Please understand the use of the past perfect tense correctly. The past perfect is used for the earlier of two past actions.
He told me that he had seen the movie. There are two past actions here - 1) His
telling me, and 2) his seeing the movie. Both are past actions. However, his seeing
the movie is the earlier, and his telling me about it is the later past action. So, his seeing the movie - must be in the past perfect tense.
Also look at the following:
He slowly got up from bed. He remembered what had happened the night before. Two
strangers had entered his room about 6 in the evening yesterday. They had hit him
hard on the head. That was all that was able to recollected. He now looked around. His wrist watch and rings he had kept on the table were no longer there. He understood that the strangers had taken them away.
Observe that his getting up slowly is a past action. all the other events had taken place 'yesterday', that is, they are all earlier past actions. So they are in the past perfect tense. So the use of past perfect has no connection with time of happening.