EXPERT RESPONSE
I think we need to keep the following challenges in mind:
1) Unlike traditional apps (web development, rich client apps), there is no direct user interface to the services you are building and testing, so you need bring that up into an interface that allows you to interact with these services easily (i.e., provide data, construct messages, etc.)
2) Similar to traditional apps, there are functional requirements that your services need to adhere to, i.e., implement the requirements correctly and executes your use case scenarios correctly, but there are also non-functional quality aspects as well:
- compliance with standards (W3C, OASIS, WS-I, etc.) , so that you can reuse and access these services successfully today and tomorrow while maintaining vendor independence and avoiding proprietary lock-ins.
- adherence to policies, these include standards as well as design and runtime policies and which insure consistency in services, enable the reuse and reinforce trust so that as your SOA grows, you are truly capable of reusing services and have full visibility into their quality status (like a credit report)
3) Functional and performance considerations in SOA require somewhat more attention and a different approach from traditional apps, because once you build business processes around the services you have in your SOA, your services may be accessed and consumed in ways that are not always easy to predict, because the scope of their use is no longer controlled by the specific processes and scenarios that are bound by, say, a specific web application, so you need to pay additional attention to negative testing and ensure that the services function reliably under unexpected conditions (e.g. bad data, missing data, etc.)
4) Security. Never forget about security considerations: both from inside and outside even if your services are internal only. There are certain vulnerabilities that are specific to XML that need to be avoided, and even traditional security considerations like SQL injection vulnerabilities need to be prevented.
From a process perspective, a continuous testing process needs to be established where you can isolate parts of the system (sometimes you need stubbing or emulation to achieve that), and constantly (automatically) run regression tests against it. Some key tips here are:
1) Put your resources into building the tests, not running tests. The execution should be automated. Whenever you find yourself spending time running tests (sending XML messages around and analyzing them) then you are doing something wrong.
2) Maintain your regression suites and test that you created, these are very valuable assets, because once you need to make a new version of a service, these test assets become valuable so you can run and ensure that nothing breaks from one release to another.
3) Having an infrastructure to share these tests assets is critical as well, so that different teams, QA and devs, etc. can access, leverage and reuse the tests that others have created. This is a result of the nature of Web services being reused and consumed by different groups.
|
