Wednesday, November 16, 2005
Tales of a hobbyist software architect
I am working on some automated testing at work. The product I test is very complicated. I was doing some procedural testing on one property window which has 300 children across 15 tree items (plus three more trees and a few modal dialogs of additional options). It occurred to me that OO was the way to go here, but the implementation was an issue.
The implementation I've hashed out so far is as follows. A class representing the entire window is created (the facade). Objects for each type of child in the property window are created via a factory which also populates them with data needed to find them (the factory creates four types of objects so far representing 150 different choices in the window). These objects are passed to the facade in an array. The facade then opens the properties window and iterates through each class. Because all four classes have a common interface, the facade doesn't care what sort of class they really are. The method it calls is always the same.
There's a few problems with this approach. First is VBScript. TestComplete is the automation tool we are using, and it was decided that VBScript is the language we should use. There are some pros to this choice since VBScript makes things like WMI very simple. The cons relate to things like objects. There's no inheritance! These four classes have a considerable amount of common code.
The second is TestComplete itself. TC has a neat Object Driven Testing (ODT) UI. Unfortunately it has limitations. First, the UI itself is cumbersome - keyboard navigation is kind of a pain, and copy/paste doesn't seem to work. Second is portability - how do I share my ODT tests with other testers? The third seems a little more abstract but I'll try to explain it. The model I architected represents each option in the properties window (but feel free to abstract this idea to the entire UI). This model can be incorporated into tests, where you are using the model to control the UI programatically. This has a few benefits over simple recorded scripts which I won't get into too much (unless you want me to). The thing is that AutomatedQA's idea of ODT is to use the object model to represent the tests rather than the UI. Here's a comparison:
My model - The user instantiates the option factory and creates option objects representing the settings she needs to test in the properties window. She then uses the facade to open the window and set the settings. This may generate immediate results or further steps may be required to validate results. One thing is certain - most test units will involve multiple trips to this properties dialog. This is supported by simply using the factory to create different options and passing them to the facade. It's a repeatable process.
TestComplete's model - Each test you run is represented by a class. A facade class would have subclasses that perform tests or components of tests. A factory class is not supported. Representing the UI, therefore, is extremely difficult and it's even more difficult to say "first, do this in the UI, then exit that and do this, finally go back to square 1 and do some more". At least, that would be impossible to do with the architecture I have chosen.
So... It's an impasse of sorts. If this was "the TestComplete way" to do ODT, that would provide some guidance. If I knew other testers would use this model, that would give some guidance. I feel like it's pretty easy to maintain and the interfaces are pretty easy to use, but it will require some time investment to get it right.
Ultimately, it's a selfish decision. Each big project like this seems to take something out of me. I'm trying to evaluate whether the emotional return I get when people really use my code will equal the emotional toll creating the code will take. Creating the code actually isn't that bad, but "selling" it to peers, documenting it, supporting it... This is daunting.
The implementation I've hashed out so far is as follows. A class representing the entire window is created (the facade). Objects for each type of child in the property window are created via a factory which also populates them with data needed to find them (the factory creates four types of objects so far representing 150 different choices in the window). These objects are passed to the facade in an array. The facade then opens the properties window and iterates through each class. Because all four classes have a common interface, the facade doesn't care what sort of class they really are. The method it calls is always the same.
There's a few problems with this approach. First is VBScript. TestComplete is the automation tool we are using, and it was decided that VBScript is the language we should use. There are some pros to this choice since VBScript makes things like WMI very simple. The cons relate to things like objects. There's no inheritance! These four classes have a considerable amount of common code.
The second is TestComplete itself. TC has a neat Object Driven Testing (ODT) UI. Unfortunately it has limitations. First, the UI itself is cumbersome - keyboard navigation is kind of a pain, and copy/paste doesn't seem to work. Second is portability - how do I share my ODT tests with other testers? The third seems a little more abstract but I'll try to explain it. The model I architected represents each option in the properties window (but feel free to abstract this idea to the entire UI). This model can be incorporated into tests, where you are using the model to control the UI programatically. This has a few benefits over simple recorded scripts which I won't get into too much (unless you want me to). The thing is that AutomatedQA's idea of ODT is to use the object model to represent the tests rather than the UI. Here's a comparison:
My model - The user instantiates the option factory and creates option objects representing the settings she needs to test in the properties window. She then uses the facade to open the window and set the settings. This may generate immediate results or further steps may be required to validate results. One thing is certain - most test units will involve multiple trips to this properties dialog. This is supported by simply using the factory to create different options and passing them to the facade. It's a repeatable process.
TestComplete's model - Each test you run is represented by a class. A facade class would have subclasses that perform tests or components of tests. A factory class is not supported. Representing the UI, therefore, is extremely difficult and it's even more difficult to say "first, do this in the UI, then exit that and do this, finally go back to square 1 and do some more". At least, that would be impossible to do with the architecture I have chosen.
So... It's an impasse of sorts. If this was "the TestComplete way" to do ODT, that would provide some guidance. If I knew other testers would use this model, that would give some guidance. I feel like it's pretty easy to maintain and the interfaces are pretty easy to use, but it will require some time investment to get it right.
Ultimately, it's a selfish decision. Each big project like this seems to take something out of me. I'm trying to evaluate whether the emotional return I get when people really use my code will equal the emotional toll creating the code will take. Creating the code actually isn't that bad, but "selling" it to peers, documenting it, supporting it... This is daunting.
# posted by James Tatum @ 6:30 PM 
