Now your product went live. All going good until you found a bug. Now after fixing, as you doing have a testing environment, you push to the common environment. You made syntax error and whole system goes down. You cannot afford to have that and we are left with no other choice - a testing environment.
You have found interest from others on your product and the traffic has gone up. You income started to grow and 2-3 developers cannot maintain everything. Its when a QA joins you. The same issue comes again, tester wants to test a place where everything is stable. So developers do some coding and after doing a sanity, will push to testing environment with the consent of tester.
There can be other environments as well like
Integration Environment : A place for developers to test how different modules interacts before sending for testing
developer environment
testing environment
Staging Environment : A pre-release stage, where the release is done once QA's signs-off for release. It will be closest to production
The main purpose for having different environments is to provide stable and independent environments for all. There are companies which might have even 10 or more such environments with specific purpose for each like - functional testing, load testing etc.
Production data must be readily available for frequent and efficient access and is stored persistently. It is therefore often stored on high-performing Tier-1 storage that can be easily managed.
Should you use production data or generate test data for testing?
Posted on March 7, 2013
This post is part of the Pride & Paradev series.
Should you use production data or generate test data for testing?
You should generate test data for testing
You should use production data for testing
You should generate test data for testing
Generating test data is the only reliable way to accurately run tests repeatedly and consistently knowing that the input test data hasn’t changed.
Some applications rely upon specific data which is either hard to find, or hard to fake. For example, the web application I am working on displays different promotions based upon which day of the week you are using the system, and also changes prices depending on the day of week and time of day.
If you were using production data for testing, you would either have to run tests at specific dates/times to test different promotions/prices, or you would have to change the server date/time to test these. Changing the date/time on the server will effect anyone else using that server, so should be avoided. It also means that as you run your automated tests continuously against new check-ins, if you don’t use a known set of generated test data, you will get different results depending on time of day.
When developing an entirely new feature, there won’t be production data that you can use for testing, so you will need to generate some in this case.
Generating specific test data will often take longer sourcing production data, but will retrieve results over time as tests are run very consistently against a known data set.
You should use production data for testing
When you’re testing a web application, you’re as much testing the data as testing the application behavior. Using production data will ensure that what you are testing will be as close as possible to the actual behavior once the feature is released to production users.
If you generate test data and use it to test, who is to say that this test data is actually valid. If you generate test data through lower level means (such as SQL insert scripts), you may introduce test data that isn’t representative of that in production that may either introduce errors in functionality when actually running against production data, or errors in test that won’t actually exist in production. As your database schema updates and evolves, you will need to also keep your data generation scripts up to date so they are reflective of production at all times.
If you do use production data, you need to be clever about how to source data. Querying the database using SQL scripts is an effective approach as it will enable you to quickly find real data that you can use to verify a story has been implemented correctly.
It will also allow you to identify outliers and edge cases that can be tested using real production data against the system in development.
If there any concerns about using production data for testing, these can be mitigated by obfuscating the data so it is indistinguishable.
Staging Environment : A pre-release stage, where the release is done once QA's signs-off for release. It will be closest to production
The main purpose for having different environments is to provide stable and independent environments for all. There are companies which might have even 10 or more such environments with specific purpose for each like - functional testing, load testing etc.
Production data must be readily available for frequent and efficient access and is stored persistently. It is therefore often stored on high-performing Tier-1 storage that can be easily managed.
Should you use production data or generate test data for testing?
Posted on March 7, 2013
This post is part of the Pride & Paradev series.
Should you use production data or generate test data for testing?
You should generate test data for testing
You should use production data for testing
You should generate test data for testing
Generating test data is the only reliable way to accurately run tests repeatedly and consistently knowing that the input test data hasn’t changed.
Some applications rely upon specific data which is either hard to find, or hard to fake. For example, the web application I am working on displays different promotions based upon which day of the week you are using the system, and also changes prices depending on the day of week and time of day.
If you were using production data for testing, you would either have to run tests at specific dates/times to test different promotions/prices, or you would have to change the server date/time to test these. Changing the date/time on the server will effect anyone else using that server, so should be avoided. It also means that as you run your automated tests continuously against new check-ins, if you don’t use a known set of generated test data, you will get different results depending on time of day.
When developing an entirely new feature, there won’t be production data that you can use for testing, so you will need to generate some in this case.
Generating specific test data will often take longer sourcing production data, but will retrieve results over time as tests are run very consistently against a known data set.
You should use production data for testing
When you’re testing a web application, you’re as much testing the data as testing the application behavior. Using production data will ensure that what you are testing will be as close as possible to the actual behavior once the feature is released to production users.
If you generate test data and use it to test, who is to say that this test data is actually valid. If you generate test data through lower level means (such as SQL insert scripts), you may introduce test data that isn’t representative of that in production that may either introduce errors in functionality when actually running against production data, or errors in test that won’t actually exist in production. As your database schema updates and evolves, you will need to also keep your data generation scripts up to date so they are reflective of production at all times.
If you do use production data, you need to be clever about how to source data. Querying the database using SQL scripts is an effective approach as it will enable you to quickly find real data that you can use to verify a story has been implemented correctly.
It will also allow you to identify outliers and edge cases that can be tested using real production data against the system in development.
If there any concerns about using production data for testing, these can be mitigated by obfuscating the data so it is indistinguishable.
Posts
