Crowd-source testing within an organisation !!

Tags

Bug Bash, Collaboration, CrowdSourcing, Testing

Most commonly known as Bug Bash, Break the App session, but they are all really just few names of this practice where we crowdsource* the testing to the project teams.

It’s been a while since I have been following this practice of crowd-sourcing the testing to the project team, occasionally outside project team as well and found that it is the most easiest and cost effective way to implement exploratory and usability testing prior to the release every iteration.

Team who have built a feature would always like someone else to have a look at it prior release, it will get them honest, unbiased feedback which is very valuable before the release, it also saves lot of time and money for the project teams. Throughout the iteration teams test the stories and try to capture maximum defects and prevent them to go ahead in the final product but just like other team members (developers, business analysts, ux) they must have been 100% focused on the feature and there is a possibility that they know something too well and overlook the obvious.

What is needed are fresh pairs of eyes provided with a mission to play with the feature developed and see what they think. An end user, someone who is playing with an application for the first time and asked What do they think about it? This also gives the feedback as we consider first impression lasts, something that either isn’t intuitive, doesn’t work as expected or just looks crap which is not going to live it’s life in production.

On the other side if the same task is given outside organisation to the team of experts onshore/offshore or usability experts, it will definitely get the job done but it’s an additional cost to the company versus in this case if you choose to crowdsource it internally, it’s faster cheaper and quicker solution.

How does it work

It is a time boxed activity where teams are given approximately 30 – 40 minutes to play with another team’s new feature and provide feedback within that timeframe, related team member then review the feedback and act on them, few things needs to be taken care are:

• The exercise should ideally take 30 mins but should not exceed more than an hour, based on when is it being performed as time is precious.
• Inform the teams involved prior to this exercise of what they’re going to look at and precisely guide them to provide focus but also allow people the freedom to do as they wish within that guidance.
• Everyone available can join in.
• All testing is performed on the same test environment.
• Freeze the build.
• If it’s a web app decide browsers and devices to be used and divide them accordingly.
• No discussion is allowed between team members.
• Feedback can consist of bugs found, questions, comments, general feedback, whatever they like.
• The remit is to “play with the feature and see what you find/think”.
• After 30 minutes, everyone returns to their normal work and send their feedback to the task owner.

If company has existing tool to collect feedback then it can be used. I have used Mingle, Jira and spreadsheet on various occasions, constraint is that it may waste time if people don’t already know these tools, and here our main objective is to collect feedback so even if someone likes to write on post-it’s then allow them and collect it at the end, so that time blocked can be utilised completely for testing.

Benefits of this practice:

• It’s a short and time-boxed activity where technical people provide focused information on a new feature and its integration with all others in the system.
• It also acts as an unofficial demonstration for each team and as a learning exercise for all.
• This activity mostly uncover all glitches in functionality, recommendations for future work and overall opinions.
• Users performing this activity will also build knowledge about the features being built in the organisation.

One thing to be noted is the timing of this activity during the iteration, it should not be done near to a release, else you will not have time to incorporate the feedback from this activity. If something really bad is found then it can be scheduled for resolution in the same iteration or next iteration and importantly it has been prevented from going live.

*crowdsourcing: It is a distributed problem-solving and production model. In the classic use of the term, problems are broadcast to an unknown group of solvers in the form of an open call for solutions. Users—also known as the crowd—submit solutions. Solutions are then owned by the entity that broadcast the problem in the first place—the crowdsourcer.

Attributes of shared understanding !!

Attributes of shared understanding

• Clear business requirement
• Simple
• Concise & Precise
• Well defined features
• Well defines scenarios
• Clear business goals

Qualities of good acceptance test

• Closely coupled with business requirement
• Shared understanding
• Should restrict to scenarios
• Completeness
• Compact
• Use of examples
• Data should not be part of the tests
• Readable
• Understandable

Living Documentation

• Concise
• Should be inline with business requirement
• Single source of truth
• Avoid technical jargons
• Consistent terminology should be used
• Readable
• Understandable
• Easy to access

Dev Box Testing – Step to faster feedback

Tags

Agile, Collaboration, Dev Box Testing, Testing

Dev Box testing, name comes from the practice where initial round of testing is done on developer’s(Dev) machine(Box) before the developer checks in the code to the source repository and mark the task (story or defect) as development complete.

I feel this is one of the crucial step towards getting faster feedback, every task done by the developers gets tested on their machine by BA’s & QA’s and feedback is provided on the spot, it sounds very simple and yes it is simple if followed properly else it could turn out to be chaotic.

How it should be done ??

There is no formal process of how it should be done so it’s best left to the team, few ways I have done it so far.

• The developer demonstrates the functionality implemented and run through the changes that has been made and impact on other functional areas to BA & QA who took the responsibility of that defect or story.
• Developer handover his/her machine to QA or BA to run through scenarios, mainly happy path scenarios and make sure goal of the task is accomplished.
• Time spent for this should range between 10 to 15 minutes based on the complexity of the story or defect implemented.

Where does it fall within iteration ?

Why teams should follow:

• Faster feedback, it reduces the significant time to find defect and get it fixed, in usual iteration model where this process is not followed feedback will come very late and if defect found it will get into priority queue and developer available at that point in time will fix it, turn around time in this case is quite high.
• BA gets an opportunity to ensure that implementation meets business requirements.
• Developers & QA’s can share knowledge and context around how QA’s test complex stories beyond UI e.g. with service calls, databases, etc and developers can share better ways of doing the same thing.
• Most importantly, if issues are found developer can quickly fix them and get it tested faster.
• Time utilisation, developers can utilise the time when they run the build locally before check-in.

Dev box testing encourages collaboration within team, specially BA, QA & Developer and aim towards reducing bugs that are committed into  source repository and collaborate to work towards the same goal which is to deliver business value by building a quality product.

Test early, fail fast !!

Tags

Agile, Collaboration, Testing

Innovation and failure go hand in hand, fearing failure stifles creativity and progress. If you’re not failing, you’re not going to innovate.

Introduction:

We have always read about testing should start early in the life cycle of development, here is some analysis why.

Irrespective of Waterfall Model or Agile, we usually fall in trap of bringing in Testing in our last phase, benefit in Agile methodology is you failing faster then waterfall projects but still not faster as you can.

Usual software development life cycle:

• Planning: requirements are expressed, relevant people are contacted, few meetings takes place. Then the decision is made that we are going to do this project
• Analysis & Design: BA does Analysis and designs are prepared.
• Code/Build:Now developers write the code and hand it over to QA, in waterfall model after months and in Agile mostly at end of iteration.
• Testing: Now it’s your turn: you can start testing.

But when it’s planned this way, mostly time to test gets shrunk from what it was initially planned, reason business do not want to move deadlines as they have already committed the date, which results in testing team investing more unplanned hours getting the build tested and get discovered defects fixed.

Cost Effectiveness

The earlier you find a bug, the cheaper it is to fix it.

If you are able to find the bug in the requirements gathering phase, fix is going to be of nearly of no cost versus if found in testing phase or post release, fixing them is 100 times expensive.

Conclusion: Start testing early & Collaborate

This is what you should do:

• Make testing part of each phase.
• Start test planning the moment the project starts.
• Start finding the bug the moment the requirements are defined.
• Keep on doing that during analysis and design phase.
• Make sure testing becomes part of the development process.
• And make sure all test preparation is done before you start final testing.

“I have not failed, I’ve just found 10,000 ways that won’t work” - Thomas Edison

Agile distributed teams

Tags

Agile, Distributed Teams

I have been working in distributed teams for close to 5 years now, here are few things about distributed teams.

Distributed Team

A distributed team (also known as a geographically dispersed team) is a group of individuals who work across time, space and organizational boundaries.

Working with distributed teams gives companies access to talent that they may have otherwise not have access locally. Additionally, companies gains experience in working with different global markets. Moreover, a project can be completed in a faster manner if people in different time zones are continuously working on a particular project. That’s not all, companies can obtain significant cost savings if they work in distributed team environment.

Advantages of Distributed teams

• Talent
• Productivity
• Diversity
• Minimal infrastructure
• Cost Savings
• Ecological
• Work – life balance
• Individual control

Forming the Fully Distributed Team

• Shared ownership from the start
• Decide architecture together
• Get to know the client and domain
• Form personal relationship
• Communication is the key

Distributed Team Meetings

• Video Conferencing is must for all the meetings like Daily standups, Planning and Retrospective
• Same rule applies for all teams
• Planning poker over video or digital tool
• Digital wall, it should be kept updated all the time.
• Get extended day when you work in distributed teams, project can be completed in a faster manner if people in different time zones are continuously working on a particular project

How to start with Distributed Teams

• Start with one location and bring few people onshore and then those people go back ad set up a distributed team.
• Onshore team can measure velocity across few iterations and then almost same velocity should continue.
• Quality is still the same as you still write Unit test, Acceptance tests and get stories tested within iteration.

Understand that not everything can be distributed

• Enterprise architecture often does not.
• Software architecture distributes easily enough.

Difficulties Faced 

• Initial reluctance to comunicate extra
• Culture makes it hard to get aligned, misunderstandings about priority and value
• Local team taking aggressive ownership
• Not enough context information offsite
• Both sides need to adjust
• Trust

When to start with distribution

• Get your local organisation capable of running Agile projects
• Get quality up with XP practices
• Stop thrashing, focus people (Stop trying to do too much)
• Think of scaling distributed team

Summary

• Working successfully in a distributed way is all about handling the ‘distance’ between people
• Classical approach is with more detailed instructions and control, not suited for knowledge workers
• Agile can tie people together across distances
• Agile benefits (Time to market, performance, quality ) mixed with offshoring benefits is a killer combo
• Introducing Agile and distribution at the same time is often too much to take in

Conclusion

Fully distributed teams has more value then localised agile teams.

Tools used

• Skype for continuous video conferencing
• Swapping people onshore & offshore – share context
• Mingle for digital wall
• Go for Continuous Integration
• Showcases on join.me

What to automate and what not to automate !!

Tags

Automation, Testing

This question is often raised by testing teams, what should we automate, or should we automate everything.

I would recommend teams to find answer themselves by asking the question what value will all the automated tests provide vs cost & time to be invested to build the automation suite. We automate to get faster feedback and shortened the testing time, which leads to a huge saving in terms of time and money for the team.

The automation requirements define what needs to be automated looking into various aspects. The specific requirements can vary based on product, time and situation, but still I am trying to sum-up few generic tips.

Test cases to be automated

• Tests that need to be run with every build of the application (sanity check, regression)
• Tests that use multiple data values for the same actions (data driven tests)
• Complex and time consuming tests
• Tests requiring a great deal of precision
• Tests involving many simple, repetitive steps
• Testing needed on multiple combinations of OS, DBMS & Browsers
• Creation of Data & Test Beds

Test cases not to be automated

• Usability testing – “How easy is the application to use?”
• One-time testing
• “ASAP” testing – “We need to test NOW!”
• Ad hoc/random testing – based on intuition and knowledge of application
• Device Interface testing
• Back-end testing

Prioritize those tests that are to be automated, give them weightage on risk and ease of automation, a simple matrix can then be drawn up to show which tests will take the most time to automate and their relative risk/importance to automate. You can then place them into priority order and plan accordingly.

Running Cucumber features across browsers

Tags

Capybara, Continuous Integration, Cucumber

I use Cucumber and Capybara for testing my Rails application, it’s easier to write tests using Cucumber tool, like an example mentioned below:

Most of the time on CI (Continuous Integration) server we execute our tests in headless mode, but recently I had to run my tests across many browsers like Firefox, Chrome, Internet Explorer 8 and 9 and that’s when the need to run these test across multiple browser arise.

You can also configure your capybara to run on remote hosts, that means from your machine you can connect to browser installed on some other machine which is on same network, It’s really simple to configure on Rails application.

 

Install Cucumber and Capybara

To configure Cucumber and Capybara in your Rails application, add these dependencies to your Gemfile and then run bundle install:

If you don’t define the version, it will pick the most recent one when you run “bundle update”, Once these gems are installed then run the following command to generate your cucumber config:

Now, lets use the above mentioned example of Cucumber feature and get it working (assuming you will write step definition, use it for reference). By default  test run’s in Capybara’s headless browser, so before we test on real browser, lets get this test working on default one.

 

Run in default headless mode

To run the test, type this command from your web folder.

Now, when feature is passing in headless mode, lets try to do some configuration changes to run tests in various browsers.

 

Run tests on remote Internet Explorer (Windows Machine)

Use any windows machine on your network to set up.

Selenium requires Java so you should have it on your machine, you can download it from here. Open new command prompt (Start..Run…cmd) and run the java command to see it has installed successfully, if not then set the environment variable to get it working.

Now from Selenium page get the latest selenium-server-standalone.jar and from your command prompt you should be able to navigate to where the file is downloaded and run:

It should show some logs on command prompt and let you know its running.

It’s time to hook Capybara to Windows

To get your features running on that box go back to the Rails application, open up features/support/env.rb and add this snippet after capybara.default_selector line

Make sure you change the ip address in the snippet to the correct one and now when you type:

You should see IE browser launched on your Window’s box and test should pass.

 

Run tests on Firefox browser

To get your features running on your local Firefox instance go back to the Rails application, open up features/support/env.rb and change the default driver to be selenium instead of capybara.

Now type the same command and your tests should run on firefox browser.

 

Run tests on Chrome browser

To get your features running on your local Chrome browser go back to the Rails application, open up features/support/env.rb and change the default driver to be selenium_chrome instead of capybara and add following snippet.

Also on your machine you will have to install chromedriver before you run the test against chrome browser

Now type the same command and your tests should run on chrome browser.

I am yet to try it against Safari and will keep this post updated, let me know if you had any issues.

 

References and examples picked from:

• https://github.com/cucumber/
• https://github.com/jnicklas/capybara

“

My View of Specification by Example

Tags

Specification by example, Testing

It’s been few weeks I attended a 3 days Advanced Agile Acceptance Testing workshop by Gojko Adzic at ThoughtWorks, Sydney office. I am trying to put down some points which I feel might be helpful for my friends.

What is Specification by Example

Specification by example is just the concept underlying our test driven development, in theory each test should indicate how a system will be used to build the software. Many times we think tool is important for making our life easier, but the fact is best of the tools,will also not help us unless we define specifications collaboratively correct and clear requirements with examples.

Most of the time teams blindly start implementing solution on the requirements which are given by the business, but in order to build the right software, we should define scope by derving it from goals and business should tell us the value they are looking for, delivery teams should use their knowledge and suggest a easier, faster and cheaper solution to the business.

Specifying Collaboratively

Delivery teams should collaborate with business users in specifying the solution, each team members have their own capability and it can be leveraged, it also creates a collective ownership of specifications, making everyone involved and more engaged in the delivery.

Team starts writing these specifications using Natural languages so that it’s easier for other team members to understand, but  they can’t be concrete in first attempt, which leaves a gap in understanding of the consumers mind because each individual perceives it differently, so in order to make it concrete, team should be using examples. Teams identify the key examples to fulfil a specification and later they can be expanded to cover other edge cases which testers find, in this way everyone in the team will share the common understanding of what needs to be delivered, which will avoid any kind of rework and since key examples are easy to understand and communicate, we can replace the requirements with these key examples.

Refine the specification

Now these key examples can be refined by cleaning them, by removing all additional information, they should only tell us what the software is suposed to do, not how it does. Once the examples are refined they can be used as an acceptance criteria, so when team is doing development they have to make sure that software works correctly for for all these examples. These key examples identified plays a various roles as acceptance test, functional regression test and specification of work.

Now in order to get the quicker feedback we automate these key examples and this way we reduce the cost of fixing the failing examples earlier. An automated specification with examples is written in a human readable format and should be easily accessible to all members of the team. We can use these executable specifications as a target for development and easily check if the system does what we agreed and the same document can also be used by the business when needed to change the specification.

Living Documentation                                                                                                                                                                                                   

Now since we have these specifications documented we should ensure that they are well organised, easy to find and consistent, when business rules changes these specifications should also be updated so that gradually they become a living documentation for the team.

Living documentation is a reliable source of information which tells us how the system functions currently, It is much easier to read and understand by any user, Business can use it as a reference for analysis of new requirement, Developers in the team can use it for development and we testers can use for the testing purpose.

Collaborative analysis reduces testing efforts !!

Tags

Analysis, Testing

Preventing defects is much more important than finding and fixing them, we can do that by doing a better analysis of requirements. It takes lot of effort and a higher cost to get the bugs/defects fixed at later stages in a software lifecycle  as compared to finding/fixing them much earlier in the cycle.

We all know that defects get injected into the software right from the requirement gathering stage, so if the requirements are properly analysed and cleared than it leads to developing the high quality software, whereas if it’s other way round then it leads to a bad code which may or may not work. Common example of poor requirement gathering is teams develop and test some piece of code, show it to business and then business realizes that the solution developed does not solves the problem completely, which generates some new requirements and the cycle continues, all this time we spend time in testing, re-testing the same piece of code. Getting the requirements right in the first place will greatly reduce the effort which in turn leads to less retesting.

Defects in requirements can be very costly for the organisation as it involves large amount of rework or may be project failure, so before the requirements get converted into design or get into implementation phase, it must be ensured that they are correct, complete, still valid, current, testable and relevant.

Since we work in collaborative environment, I would not leave the requirement gathering only on business analyst (BA) but to the entire team, specially quality analysts. I have tried pairing with BA’s on the team and found that BA-QA pairing helps designing the better requirements, In fact QA’s can think of the requirements to be testable or not which also helps in analysis.

Morale at Workplace

Tags

Management, Morale, Workplace

I do not have much experience as I started my carrier just 3 years back. There are various factors which motivated me to write this blog. I’ve fallen victim to it’s lows and have experienced the occasional highs.How strange that the one thing that drives a company the most is the one thing most companies don’t drive…”Morale”. Morale is often related to “Job Satisfaction”, But I feel, ‘Job Satisfaction’ is an individual phenomenon; while Morale indicates a group phenomenon. In better words, Morale indicates the happiness of the employees with the organizational environment.
Morale is also often believed to be all to do with ‘reward’ and it’s no wonder company directors are so dismissive of improving it. Better morale suggests better reward, and reward suggests ‘money’.What I have found by personal experience and reading various articles is that when poor morale finds a crevice to breed, it festers, grows and consumes all others. It has such a grip that it begins to cross teams, departments… entire infrastructures.

Do you want to know what morale is really about? It’s not money. It’s ‘Recognition’. Recognizing that someone is valuable. Sure, you could do that with money. But how many people do you know that say that they never earn enough. We live in a society that encourages us to live beyond our means. We will never earn enough. It’s not money. It’s recognition.

Morale is normally high when everything is going ‘just right’. When people come in to work, enjoy their job, go home feeling like they accomplished something, and are welcome home. Everything is just right.

What set’s off morale on a downward slope?

Something inevitably goes wrong. Whilst morale is high, we tend to deal with it by happily going about fixing whatever it was that went wrong. If it’s a short fix. A small problem. We deal with it effortlessly with ease.

So it’s not when things go wrong? Or is it? No. It’s when we stop communicating. It’s when something goes wrong (or generally, when it’s about to (we’re all pretty intuitive to what upsets the balance of our happiness)), and we hold back communication.

Even in best of times, morale is delicate, unpredictable thing, Bad morale is insidious. It skulks, lurks, and it simmers just beneath the conversation at the water cooler. But if you keep your eyes and ears open, you’ll know when it’s there. Morale is a business issue, it also causes decline in productivity and quality. Morale is not a software program so can’t be fixed by service packs or patches. It won’t be solved by pizza parties, free gifts and functions.

Reason of Bad Morale

Lack of communication and bad management, or lack of confidence in management, are the two biggest cause of low morale. It doesn’t matter what the economy is like.
In tough times such as these, the people are looking for support, leadership and reassurance, If they are ignored or underestimated there is a morale problem on hands.
* First step to fix it is acknowledging that the problem exists.
* Second would be, realizing that it’s my responsibility to make it better.

Some steps should be taken for recognizing and rehabilitating low morale

a). Look, Listen, Take Nothing for Granted.
The biggest mistake you can make is to ignore the existence of a problem or rationalize it away. People shouldn’t be constantly uncommunicative. You should try to know them, what they feel, let them know about yourself. You might have open culture but people learn from their previous jobs to keep their mouth shut. Employees try to hide their feelings because they are afraid of a departmentative backlash if they say anything. The best way to counteract that kind of wariness is to talk to your staff in a consistent honest manner.

b). Honesty Is the Best Policy
The biggest mistake is to withdraw information and keep it yourself and don’t share information with the employees. It’s insulting to your employees’ intelligence, and it’s very destructive in terms of morale. As a leader, even if your own morale is down, you have to keep your head up even if it’s just by keeping a smile on your face.
Building morale through communication is more than smiling and keeping the doors open, however. It means developing the trust between you and your team and that kind of connection is made by opening up about yourself, both personally and professionally. Let them see you as you are. Hold a series of meeting with them to improve morale, update them on what’ being done and gather feedback on the process as it moves forward. By involving them, it will make them feel powerful and pat of solution rather then problem. By soliciting their advice, you demonstrate a level of respect and trust that they need to see.

c). Don’t take away Their Trainings
This is another effective way for boosting morale and solidifying a sense of commitment between you and your staff is to shore up your professional development and training program.

d). No cheap tricks.
You have to do something every day that will make them want to come to work. Employees are smart; if you treat them badly specially during tough times, they will remember. If you treat them well, they’ll be loyal and stick around. If not, they’ll leave for the better place as soon as they can.
Appreciate your staff as much and often as possible, treat them like they are the best. Do whatever you can, even in small ways, through emails, meetings, though one-on-ones, let them know you value them and the work they do. For such efforts to be effective, they have to be genuine.
People don’t stay for the cheap tricks, they stay if the environment is supportive and challenging, if business is functional and there are positive working relationships.

Improving Employee Morale

Improving employee morale benefits everyone involved in a workplace. Boosting morale means they will feel more pride in their work, call in sick less often and be more productive. Happy employee means happier employer. Improving employee morale can be accomplished fairly
easily.

Most people thrive on feeling appreciated, so you can improve employee morale by showing you appreciation in simple ways like saying “job well done”, “ good work”, etc. Another way to show appreciation is by being friendly and interested in your employees. A warm smile and sincere query about how one is doing will in turn motivate employee. Encouraging social interaction is another way of doing it, yet another way is by rewarding your employees.

A very important factor in improving employee morale is the work environment. Research shows that atmosphere greatly and directly affects the motivation level and feeling of well being of the employees in a workplace.