[Day 11/12] Why Testing Matters

caroline-ambros · ‎12-18-2017

It can be easy to dismiss software testing as an unnecessary investment of time and money. One or two little bugs couldn’t possibly hurt anyone, right?

But for many organizations, one little bug spiraled out of control! The good news is, you don’t have to learn how vital software testing is to your business the hard way. We want to hear your stories about #WhyTestingMatters. Comment on this post and tell us about your favorite software fails.

Over the past few weeks, we’ve asked our social media followers and some of our favorite experts to share their opinion on #WhyTestingMatters, too. Check out what we’ve learned so far.

We started by doing our research.

Turns out, lots of big brands have missed bugs in their testing. Here are some of our favorites:

Apple’s recent iOS 11.1 update came with a host of glitches. Users report that the new system won’t type the letter “I.” They’ve since solved the issue, only to reveal a new bug which turns “it” into “I.T.” Tweet this.

Apple wasn’t the only company to fail to deliver on a cell phone’s most basic function this year. One of it’s competitors, the Android OnePlus 5, disconnects when users try to call 911. Tweet this.

Have you ever wondered who removes the more unsavory content from your newsfeed so that you don’t have to see it? This summer, a software bugat Facebook helped hate groups answer that question by revealing the identities of 1000+ of their moderators. Tweet this.

Read the rest of our favorite software fails here.

Next, we asked the experts.

We asked Jess Ingrassellino, T.J. Maher, Mike Lyles, Lino Tadros, Trevor Atkins and Rosie Sherry why they think testing matters, and we got some great advice.

T.J. Maher, Senior QA Engineer and Ministry of Testing Boston Organizer, shared a software fail story with us:

“The team developing a new European Union shopping cart found out quickly that the VAT tax test server was flaky. But was that the cause of the one-cent discrepancy between the estimated and actual price that randomly appeared? Everyone on the team thought so. I wasn’t sure.

Taking my Selenium script that placed an EU order, I looped it twenty times, collecting data all the while. It had a failure rate of 40 – 60%. After showing the metrics gathered to the new developer assigned to the EU cart, he checked his code. It turned out he had made a rounding error calculating the estimated and actual tax differently.

Does something not feel right? Don’t just go with a hunch or a gut feeling. Make sure to do the legwork required.” (Tweet this)

-T.J. Maher, Senior QA Engineer and Ministry of Testing Boston Organizer, Adventures in Automation and on Twitter as @tjmaher1

We got advice from Trevor Atkins, Principal Consultant at ThingTesting.com, about how testing can affect your brand’s reputation (both positively and negatively):

“There are so many factors that contribute to software being ‘of quality‘ that typically you have only the resources to make a few stand out. Those that are emphasized become part of your brand. You wouldn’t want your brand to become infamous for a faulty decision that could have been prevented by smarter testing, would you? (Tweet this)

Testing can be a trusted advisor to help the business make informed decisions around quality – like when to release. In other words: Testing matters because quality keeps customers coming back for more.”

-Trevor Atkins, Principal Consultant at ThinkTesting.com and on Twitter @thinktesting

Read the rest of the article, including what Mike, Lino, Jess and Rosie had to say here.

Why do you think testing matters?

We want to hear from you! Share your favorite software fail in the comments below and on Twitter or LinkedIn!

Resources:

https://blog.smartbear.com/software-testing/why-testing-matters/

https://blog.smartbear.com/testing/experts-insight-why-testing-matters/

https://twitter.com/search?f=tweets&vertical=default&q=%23WhyTestingMatters&src=tyah

Now, it’s your turn to participate!

Why do you think testing matters?
What is your favorite software fail? Share an example from your work or the news!

Want to do even more to spread the word? Share your software fail on Twitter or LinkedIn using #WhyTestingMatters. Add yours!

The article is posted within the 12 Days of Software Quality event. Here are simple rules of the event:

- Each day, our experts publish interesting industry articles

- You share your thoughts on a topic by commenting an article and vote for the others' comments you like

- Daily, we select a winner based on the number of Kudos his/her comment got.

12 Days of Software Quality Schedule

kaftw · ‎12-18-2017

Testing matters because a strong, left-shifted quality process enables the organization to move faster and deliver faster with confidence. It allows for stability in planning and for pivoting when necessary because you know changes are safe to make. If you can't reliably change and release quality software, you're dead.

sweta_desai · ‎12-18-2017

1. Why testing matters?

In industry where now and then there is a release/patch applied to an existing system. It become necessary to test the
existing working plus the new feature that is added to the release. Testing has direct impact on industry as well as customer.
Customer satisfaction is the key to every business platform. A wrong release/feature defect in any product in any industry:
may it be IT or any, can lead to the large lose of customer.

To check for the all the combination of tests [Test cases: in terms of testing] gives the whole picture of the correct
working part of the system.

I have worked in semiconducting as well as telecom industry. A failure over in determining route path/finding IP/recognizing the packets leads to breaking of the whole chip functionality [semiconductor] while the facilities like can not able to reach call/not able to use data on mobile/wrong billing calculation on invoice/wrong taxation calculation based on location can lead to the lose of customer satisfaction [telecom].
In both the industries, it directly affects the product and the customer. To make sure, considering all the situation and combination, testing becomes the key part.

2. What is your favorite software fail? Share an example from your work or the news!

The one of the example that has currently been in to the news related to software fail over:

Macy's US is the top shopping destination. One of the best season to earn and make profit for Macy's is Black Friday.
This black Friday, there was an issue with working of Credit cards. Customers have lined up for hours and then if the
system doesn't respond, the frustration level lines up too!!!

According to the news report:

1. Macy's revenue dropped 6.1% in its most recent quarter, which marked the 11th straight quarterly decline.
2. Macy's is struggling as consumers opt to shop on Amazon and other websites.

This situation was the result of lot of user request at a time.

This would have been avoided, if the Web Stress test, Performance Test is being performed keeping all the load in mind
during the peak season.

This has an impact on the revenue as well as customer satisfaction.

Thank you,

Sweta Desai

baxatob · ‎12-18-2017

I. Testing absolutely matters! Even if you have not QA department or no one tester working with you, you perform testing during software development. Even if you are the single developer on your project and your application has only one button... probably you want to know does this button work or not 🙂 So you do testing anyway. And I believe that this approach is not only about software, because it refers to everything surround us.

II. My favorite bug was in one authorization test. If application user has 3 unsuccessful login try (wrong password) - he should be blocked. But the bug blocks ALL of users! We spent a lot of time to understand what happens, because each time we deployed the fresh system image and database dump, then made steps to reproduce the issue -- finally we blocked the system again and again.

sanj · ‎12-18-2017

Why does testing matter?

You can catch bugs in different stages

dev

qa

preprod and

prod

Cost increases as bugs are caught at each stage

There are also other consequences for eg when recalls, retrying to define your brand.

Look at when Noote7 would catch fire!

The issue is with integration tests/

We tend to do component level test really well but forget about or integration tests lacks test coverage hence amazing bugs are found when products make to their customers for eg the classic dreamliner fire issue.

How would anyone catch that.

Apple and Microsoft have tried to alleviate this problem by releasing beta version of their OS so more "free" qa testers can test and validate their OS

I think that is a good is trying to reduce number of buds found in prod hence reducing cost of fixing these bugs.

My memory of most spectacular bug is windows NT crashing and saying this program has performed an illegal operation.

sounds like a memory violation issue which required restarts!

Favorite software fail

blackberry demise as apple pushed iphones which was easier to use than awkward blackvery phones!

as tech makes advancement it leaves some biting the dust while others who have leveraged new technologies soar!

kodak is another classic case which lost to digital cams

Marsha_R · ‎12-18-2017

Testing is needed because it makes everyone happy! 😉

Testing that starts at the design level and continues through the development cycle produces a better product. This gives the team a better reputation internally and externally, and keeps the customers happy.

If the external customers are happy, then they aren't complaining to Support or Sales (and really they are your customers too) and nothing escalates so you don't suddenly have management standing over you wanting to know what is going on!

No one wants to fix and retest bugs in a hurry at the last minute either. Deploying code that is already known to be stable keeps the team happy.

A team that is known for consistently good deployments can be given some leeway if something unforeseen happens. Testing will support this as well.

Fewer complaints means less time putting out fires and letting the team work on new development and isn't that the fun part? Do the testing even if it seems boring and it will end up giving you more time to do what you really want to do.

msiadak · ‎12-18-2017

Why do you think testing matters? Testing matters because it keeps me with a paycheck. 😉 All kidding aside, it matters because no one person, be it a developer or a designer or an architect, can imagine every scenario or prepare the code for it. All said and done, it's a last line of defense before passing things to the end user. Any problems that can be found early in the development cycle simply cost less to fix: less time, less money, less resources, etc. Testing matters because it helps to ensure a functional product being delivered to the end user. The more that can be found ensures a smooth experience for a delivery and for maintaining the software at a later date.
What is your favorite software fail? Share an example from your work or the news! Favorite piece of software fail is more of an example I have seen too often rather than a specific case. This has to be when a user somehow manages to uncover such a specific scenario that no one could have ever planned for it, and yet it pops up during the first use with a customer after/during a delivery. This happens a lot! There's always that one corner-case that just could not be accounted for no matter what. It just goes to show there's always room for improvement. 😉

tristaanogre · ‎12-18-2017

1) Why do you think testing matters?

In this day and age, almost EVERYTHING runs on some sort of software, from medical care to transportation to financial systems. And all of these situations, if there is a problem in the software, can have a catestrophic effect on many people that may even risk lives. If a software application designed to schedule dosages of medications applied automatically by a computerized IV system in an intensive care ward malfunctions and gives incorrect dosages (or none at all), it could kill the patient. Software testing is THAT serious... lives MAY be at stake... literally.

One of my co-workers used to work for a railroad company where the software that controls the automatic breaking of the engines was being tested. He used a rule of thumb in that environment that, when it came to release date, they would all climb aboard the train with the new software running and aim the train at a wall, expecting the software to kick in and stop the train. If they could not confidently do that, then the software was not ready for release.

Not every software application is life or death... but honestly, that's a mindset that should be used. If you are developer creating software and you are not confident in it where, if it controlled life or death for you, you would put your life at stake... don't release it. And the only way to be sure of that is to test the h*** out of it. Whether you are a professional tester, a developer, an automation engineer, a buisness analyst or whatever... test the software. Who knows what might happen if you don't?

2) Favorite software fail - We had a "Murphy's Law for Software Testing" that we employed at another company. "If the software will fail catastrophically if a user does something completely unexpected and out of scope of the software requirements, then the user will do something completely unexpected and out of scope of the software requirements, usually within the first day of receiving the new version." We've seen this happen. "Oh, that will never happen in the field. Not an important bug..." And wouldn't you know it... This happened with a credit card payment protocol implemented on a web ecommerce site. Without going into details, the effect was that the credit card holders ended up getting overcharged by thousands of dollars because the website let them input information improperly... and the bug was deemed "unimportant" when found in the testing phase because, well... "No one would EVER do something THAT stupid..."

TanyaYatskovska · ‎12-19-2017

Awesome comments, guys!

I want to leave my 2 cents, although I do not take part in the competition 🙂

I hate working with software with a bad design! In my opinion, software companies should pay much attention to usability tests in addition to traditional functional ones. I cannot forget one unpleasant situation when, because of the lack of usability of one mobile app, I sent one (kind of personal) message to a person who I didn't know. Bad memories...

I'm happy that, in SmartBear, we do a lot of usability tests, especially with new employees.

NOTE: to help us select a winner of a Day, please give Kudos to comments you like the most in addition to posting your own comments. I'll announce the Day 11 Winner very soon. Hurry up to vote!

P.S. there is no need to cheat. I see everything 😉

mandhale · ‎12-19-2017

Testing is very important phase of any product (software)life cycle.
If the software is not well tested then it is having very bad quality and loopholes.
Testing improves the software quality and its user-friendliness which is the very important part today.
Testing helps to remove unwanted behaviour in software.
It helps to improve quality so it will automatically improve the popularity of the software.

durgesh · ‎12-19-2017

Testing is required for an effective performance of software application or product. It's important to ensure that the application should not result into any failures because it can be very expensive in the future or in the later stages of the development. It's required to stay in the business.

It matters when it comes to Quality of the product.

gijoshi · ‎12-19-2017

Why do you think testing matters?
Yes, Testing an application matters the most.
If we think professionally, Customer pays for quality of products, that should be easy to Use, safe & secure.
We have to think what if there is no Testing team (Dev as one-man show), then End user comes into picture, and a person who knows nothing about software/technology have to face issues with application.
Ultimately, customer may loose trust on the application which may violate the security of Data/Info.
I would rather say, there should be team of many people in testing team who can see a single application/problem in different way.
Team should be combination of male and females, the way they think is different.
Not only in Software but also in hardware we need good testing that should not cost an end-user's Life.

What is your favorite software fail? Share an example from your work or the news!
- Android OS on Micromax, I don't remember the android version, if someone send '=0' (kind of) text message on Micromax phone then it gets restarted and user don't know why?
That might got fixed now.
- FoodPanda to order a food, if user waits for 5 minutes approximately on Transaction page, then it comes with message as payment done and order is placed.
- Amazon prime user subscription (Don't know if it is true) have to choose Net Banking and give HDFC account number as 123456789, then it shows message as Subscribe as Prime user.
- Mi A1 Android facing issues with HeadPhones having mic, other user not able to hear voice.

Olga_T · ‎12-19-2017

My favorite software fail is in a time management tool which counts the time I spent working on this or that task every day. And, from time to time, this tool removes the list of the tasks, so I have to recreate it from scratch once or twice a week. That’s what I have done this morning actually . I still hope that the developers will fix this, but I think that this product has already lost a lot of users because of this drawback - you just don't want to waste your time anymore. So, guys, test more and gain more!

TanyaYatskovska · ‎12-19-2017

Great comments, guys! I want to announce that today's winner is @gijoshi. Congrats!

We've already published the last article of this program - today we will talk about the importance of testing of microservices. Join the conversation here:

https://community.smartbear.com/t5/Community-Matters-Blog/Day-12-12-Microservices-Should-We-Test-The...