Selecting a Tool. When to start evaluating: Current method is not effective Can solve an immediate problem/challenge Can increase productivity and deliverables Budget is available. 1. Selecting a Tool. Management’s Priorities: Total Cost Return on Investment
How to work with Vendors
(A view behind enemy lines)
Minimizing the time of the evaluation:
1. What are our primary objectives for a tool?
2. Describe the organizations development and testing process / methodology.
3. What type(s) of testing are we doing currently?
4. What tools do we currently own / maintain? (Purchased and Homegrown)
5. What language(s) is our application developed in?
6. Is the application web or browser based? If so, which browsers and versions do we test on.
7. What operating systems do we test on?
8. Are there any third party controls or grids?
9. Are there any Active X controls?
10. Do we need to test server side com objects or other parts of our application not accessed by a GUI?
11. Describe the application architecture?
12. What is the communication protocol between the different tiers of the application?
13. What databases does our application work with?
14. How do we create our test data?
15. Are we trying to validate data in the GUI or the back end database or both?
16. How often does our group test new builds of applications?
17. Do we have a dedicated Test Lab or would testers be using their own desktops?
18. What amount of money are we budgeting for this project / tool?
Questions that few customers ask……….
But they would be much happier if the had!!!!!
1. Given our environment, how will your tools help support it?
2. Will your tool work natively in our environment (i.e. with our own JVM) or is there an add on?
3. Will there be any additional modules if we change our environment?
4. Are those an additional cost?
5. What integration’s exist between you tools as well as the tools we already own?
6. How easy is it to maintain scripts?
7. What functions exist to help in error detection and error recovery.
8. What are the different components of your tool (i.e. Requirements Mgmt, Test Mgmt, Data Mgmt, Record and play back, Defect tracking etc.)?
9. How is your tool licensed? (Floating or per seat)
10. What is the single user list price of your software?
11. What is the maintenance cost of your software?
12. What does maintenance include?
13. What are the hours of Technical support?
14. Is training available?
15. Is training and technical support provided by your company or a third party vendor?
16. Where is your development done?
17. Are they available for feature requests and more advanced technical support?
Compuware -- QA Center