Live testing, A/B testing, and segmentation
Live tests are no replacement for other types of tests conducted in lower environments. Each type of live testing serves a unique purpose in that it can only be executed in a live environment.
Post-release testing
Some systems integrate with external systems that do not provide a lower environment for testing. Engineers would normally mitigate this risk by having a simulator running in lower environments. The simulator is a fake component that runs simplified logic just to act like the target external system. Engineers rely on documentation or information from the third-party company to implement the simulator.
This approach is not ideal, but it is better than having nothing to detect defects in lower environments. Several risks come with this approach:
- The simulator logic needs to closely follow the steps of external system changes. Otherwise, it creates a time gap of discrepancies.
- The external system may release its...
