156
votes
If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?
No.
The simplest example I've seen is: branch A cleans unused imports in a file. Branch B adds code that actually uses some of the unused imports. Git merges automatically since the lines that were ...
- 5,178
114
votes
If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?
No. As a counter example, consider branch A adds a unit test that uses reflection to check for a misspelling in an enum. And branch B adds a misspelling. Both pass because a misspelling doesn’t fail a ...
- 110k
81
votes
Is the usage of random values in unit testing a good practice?
Yes, I agree that randomness shouldn't be part of a testing suite. What you want is to mock any real randomness, to create deterministic tests.
Even if you genuinely need bulk random data, more than ...
- 59.5k
68
votes
Accepted
How do we avoid CI-driven development...?
CI-driven development is fine! This is a lot better than not running tests and including broken code! However, there are a couple of things to make this easier on everyone involved:
Set expectations: ...
- 136k
56
votes
Is the usage of random values in unit testing a good practice?
I've worked on projects which use anywhere from no to extensive randomness in tests, and I'm generally in favour of it.
The most important thing to remember is that the randomness must be repeatable. ...
- 11.6k
47
votes
Unexpected Code Coverage Reduction
The problem I see here is that you have made the code coverage a trigger for build failure. I do believe that code coverage should be something that is routinely reviewed, but as you have experienced,...
- 46.5k
37
votes
Unexpected Code Coverage Reduction
Have you considered not using code coverage metrics?
I'm not going to argue that code coverage isn't something that you should look at. It absolutely is. It's good to keep track of what was covered ...
- 85.9k
36
votes
If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?
Here is an example which neither does require changes to the existing tests itself, nor reflection, nor a failing build, for not giving the wrong impression such cases can only happen under artificial ...
- 221k
34
votes
How do we avoid CI-driven development...?
Building a sustainable plugin model requires that your core framework expose a stable interface that plugins can rely on. The golden rule is that you can introduce new interfaces over time but you can ...
- 5,004
33
votes
Accepted
Debugging a performance issue, do I commit the timing code?
Yes, you should commit this code, for the reasons you already mentioned. It is a form of test code, and similar to other test code which checks just correctness, it has some value and will probably be ...
- 221k
27
votes
Which comes first: CD/Trunk-based development or microservices?
Consensus is you want CI/CD first and anyway, independent of your application's language, design or architecture. Whether you deliver trunk-based or use feature branching is also independent of CI/CD.
...
- 18.6k
24
votes
Accepted
continuous integration for scientific software
Testing scientific software is difficult, both because of the complex subject matter and because of typical scientific development processes (aka. hack it until it works, which doesn't usually result ...
- 136k
18
votes
Accepted
Unexpected Code Coverage Reduction
You can mitigate the effect to some degree by allowing the relative code coverage to reduce when the total number of uncovered lines also reduces, or when the total number of lines reduces, since this ...
- 221k
17
votes
What does "continuous" mean in "continuous deployment", "continuous delivery" and "continuous integration"?
Traditionally, before Continuous Integration, Continuous Build, Continuous Deployment, Continuous Testing, etc., the system was split into modules and those modules were developed by independent teams ...
- 105k
16
votes
When do you have enough automatic testing to be confident in your continuous integration pipeline?
In General
When do you have enough automatic testing to be confident in your continuous integration pipeline?
The answer probably becomes clear if you think about what you want to be confident about....
- 2,598
16
votes
Accepted
Is CI/CD a myth
In my opinion, testing can be done locally on development computer;
Of course it can. And it should be done. CI/CD is not "instead" of it, but rather "in addition" to it.
testing ...
- 3,085
15
votes
Unexpected Code Coverage Reduction
This is called Simpson’s paradox, and is a well known statistical issue with your approach.
You could even construct cases where you refactor and afterwards every single method has a higher coverage, ...
- 1,473
12
votes
If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?
Approaching this from a different angle, there's a simple process to ensure that the tests continue passing after merging both branches: a branch must pass CI after being applied to the current target ...
- 11.6k
12
votes
Is the usage of random values in unit testing a good practice?
No. Random values in unit tests cause them to be not repeatable. As soon as one test will pass and another will fail without any change, people lose confidence in them, undermining their value. ...
- 110k
12
votes
Which comes first: CD/Trunk-based development or microservices?
Trunk based development with continuous develivery and microservices are mostly orthogonal concepts. However, for a huge monolith, the delivery cycles which usually require a certain amount of testing ...
- 221k
11
votes
Microservices shared end-to-end testing: Which version(s) of other microservices to use?
Your question boils down to "What is the goal of your tests?"
If you want to pre-test your next deployment before it goes into production tomorrow, test the combination of versions you ...
- 221k
9
votes
Accepted
Why the devops pipeline reads "Code, Build, Integrate, Test"..why build is second?
I write some code, then I build it (compile it) then I run the tests. This happens on my local machine. That's the "code & build" phase in that diagram.
Then I check my code in to a central ...
- 39.6k
9
votes
If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?
If two individual branches pass unit tests, once they're merged, is the result also guaranteed to pass unit tests?
Taking the question at face value, it's very simple to create an example where one ...
- 59.5k
9
votes
How to automate version bumping, when version bumping involves changes in source code and a git tag?
Am I trying to implement a bad strategy here?
If the strategy works for you, then it is not a bad strategy.
On the other hand, changing the same line of __init__.py in every feature branch is a ...
8
votes
continuous integration for scientific software
It feels kind of strange to me to start writing unit tests for
hundreds of functions which are already implemented.
You'll want to (typically) write the tests as you change said functions. You don't ...
- 820
8
votes
Accepted
Are CCQ (Continuous Code Quality) tools like SonarQube expected to deny version control changes?
Quality gates – checks that must pass before some changes can be merged – are a useful way to detect quality problems early. These gates can include any kinds of quality checks, including running test ...
- 136k
8
votes
How do we avoid CI-driven development...?
To be honest, I don't think you can handle this in a better way - if changes result in breaking maintained parts of your project the CI should fail.
Does your project have a contributing.md or ...
- 660
8
votes
How do we avoid CI-driven development...?
so when they propose refactoring changes in the core (which these days happens on a quite regular basis), they checked that the code compiles on their machine before making a pull request on github.
...
- 15.3k
8
votes
What's the proper order of stages in a CI build?
There is no proper order.
What is good for one repository is bad for another.
The proper order is the one that makes the most sense for your code base. This might mean:
several pipelines
keeping on ...
- 16.6k
8
votes
Accepted
How to make the tests run faster?
Your tests are doing way too much.
Your unit tests take on average nearly a second. A typical unit test's running time is less than 10 milliseconds.
It's more common for integration tests to take ...
- 51.6k
Only top scored, non community-wiki answers of a minimum length are eligible