A scenario which is in sunny day for a while but turns to rainy day in another while or another day. This scenario is also called blinking story/test. A more formal term, occassional failure. They all refer to this kind of issue.
It is quite important to resolve blinking scenarios because the issue is destorying the trust in automation. Without a full trust on automation, CI and further migration is still a dream.
Here are several hints to resolve flickering/blinking/Occassional-failing stories:
- Environmental reasons, perticuplarly the shared elements;
- Leakage of resource, fd, memory, defragmented buddy pool, combined ID space, et;
- Race condition, it is more difficult to find and usually lead to a hidden problem in products;
Yes, we are talking about the three dimensions again. Logic(scope/concept/recognition), Space (leak/id/dirty/init/clean/oversize) and Time (racing/history/cache/lifetime/frequency/multithreading) are still the three friends for software engineers to think about in typical tough moment.
Good luck, testers.
[EOF]