In my previous post titled “Dont Just Detect Defects Early“, I focused on how crowd-sourced testing is a fundamental shift from the traditional testing approach and why, in this age of customer empowerment, the power of the crowd cannot be ignored. Without doubt, crowd-sourced testing is helping businesses reach market faster, with products that adequately and accurately address requirements of a large, geographically disparate customer base with minimal infrastructure and staffing overheads. While these benefits paint a promising picture, businesses must also be aware of the risks, challenges and pitfalls to avoid.
A major concern and the gravest risk that crowd-sourced testing poses, is that of security and confidentiality.
- Many companies release beta versions of their products, with the intention of tapping feedback from early adopter customers. This early adopter release is actually crowd testing at work. The pre-release (beta version) product runs the risk of getting exposed to competitors. Plus, if the beta version is ridden with defects, theres also the risk of negatively influencing mindset of the early adopter user base.
- Crowd testers or beta version adopters are not keen to share personal data together with test issue logs. This results in inadequate data for analysis, leading to compromised testing.
One approach can be urging testers to accept and sign a non-disclosure agreement. This restricts distribution and disclosure of business and customer sensitive information. In a geographically diverse deployment scenario, the NDA can be served electronically, as part of overall terms and conditions, while on-boarding crowd-testers through an online test management portal. Additionally, an amalgamated team comprising a mix of internal, permanent, and external crowd testers can ensure superior testing quality, without compromising on security and confidentiality. Business and customer sensitive testing tasks are retained in-house, while non-critical testing are outsourced to the crowd.
Another remedy is to disseminate test data in the log through a test data management tool that disguises sensitive data before it is acted on for further analysis.
Crowd-testing is based on the premise of a diversified team, managing a crowd of geographically, culturally and linguistically heterogeneous testers. This also poses a project management challenge. Effective and real-time feedback and communication are absolutely essential tools for crowd-driven testing projects. Adopting agile practices with a team that works in well-organized sprints is also a great way to manage crowd-sourced testing.
Another significant challenge is when the crowd veers to the wrong direction, resulting in poor quality testing output. Well-defined testing objectives, communicated at the outset, through a formal on-boarding process, help address this challenge. Before embarking on a journey to ensure product quality and performance, the crowd must have a clear definition of the purpose, with processes and mechanisms to clearly measure testing outcome.
The key to succeed with the crowd, is to first understand the fitment criteria. Then, using it prudently and diligently, in line with tactical and strategic business needs. In the right situation, with the proper checks and balances, crowd-testing is a robust assurance platform for ensuring product quality.
For your business too, crowd-testing can yield un-explored and un-imagined answers. Answers which are a surefire way to create competitive edge.