When To Unit, E2E, And Integration Test

Never in my 2 jobs have we had failing tests across several projects, or any tests whatsoever 😂

What is the difference between unit tests and integration tests?

Grug seems experienced :)

your description of seperating data sources and the actual code under test (passing the file content) did really help me write better code and have more fun writing tests.

mocking sucks, but you have little options with 3rd party code if it doesnt implement dependency injection (which can also become tedious if its overdone, but it sure beats mocking)

"if you have to mock you've done something wrong" and "dont like flaky tests".. well how do you do an integration test decoupled from services you don't control? Surprise, you mock them.

Its so true, the word 'integration test' causes so many sour faces.

"I'm a tool and I build tools ..." hahahaha

Why the hell would you make a function for a function call?

I love mocking.

Omg this video is a massage for my ears and brain on this topic.

I absolutely highlighted the same problem of having way too many unit tests early in the implementation of a component on a new service at AWS. It was a nightmare and the engineers thought they were doing good stuff because they had a Jacoco coverage report number that said 99%/100% but not even the "happy path" for the overall functionality was close to being there; nor did any integration test exist for it as they made changes. I kept my mouth shut until I had to deal with it -- and it was exactly the problem of not being able to change code that needed to be changed quickly and freely. I politely brought it up as a challenge to adding new functionality, and the other two engineers scoffed and essentially essentially implied "git good." I remained calm because I know the primary component(s) I was responsible for would continue to develop and grow smoothly, and soon it would be obvious they were in over their heads and despite rosy standup status reports, nothing meaningful actually worked. Needless to say, it wasn't the only signal that either of those engineers had some severe shortcomings to succeed in a sufficiently complex project. One of those engineers left the company, the other switched teams. When me and another engineer took over that component, we used basically zero of the code they left behind.

I also dealt with a ton of flaky tests across many teams. Baffling how many "good"(bad) engineers don't understand criticality of disabling them. If a test is rarely flaky and a re-run usually makes it pass, fine. But a set of tests resulting in frequent re-runs or ignoring the result? Disable it -- plain and simple. Ignoring a signal is no signal at all. I wish more dev managers vocally supported this, but it's hard to "cover your ass" if a problem happens later and you approved of disabling the specific test that "covered it." Strong engineers would still understand even in hindsight -- that test never really worked.

Mocks -- there is a use case for it. I agree with you though that 98% of devs use it inappropriately and you gave a perfect example of how to breakdown functionality to actually test a thing without needing a mock. They are needed when the thing you want to test absolutely depends on a hard to simulate interaction with the environment -- like "is this code failing fast on 400 exceptions?" and "is my code and retrying 500 exceptions?" You don't want to have to stand up a test DB server that can simulate error conditions that are not normal, but need to be handled correctly.

Not sure what the aversion to mocking is about. Even in the fs example you provided, your file read function would still need try-catches or other branching logic to validate the file was successfully read. And then at that point you need a way to emulate what happens if the file is not found or some other error occurs (hence the need for a mock). Mocking is just a natural extension of dependency injection.

I like TDD for Go w/ Testify, personally. Often I'm using "golden" tests as my starting point though because I can define my expected inputs and outputs. Then, I write more verbose tests on bits that I'm not as confident in. I find that I always end up wasting time tracking down bugs when I get lazy and skip unit tests to try to get it done

That comment about not mocking was gold to me, as right before you said that I was thinking, "But how can you not mock? I don't want to use the real thing in tests." And yet there was a third way: factor out the dependency so that to write a unit test for that function amounts to testing a third party tool, so you wouldn't, and then you have the complex function that just needs a simpler input that's easy to construct in a test.

His definition of "inbetween tests” is still a unit test. Use these as your unit tests, and TDD is easy. More micro level tests are pointless, as these higher level tests will fail. It took me 15 years to begin understanding this.

The place I work at now as a paid part-time intern, the first real task given to me when switching to the backend was to write mock tests to increase branch coverage for the whole repo to 80%. The only tests we have now are some sort of integration/system test based on a few hundred megabyte large files. Soo, two paid interns are now tasked with writing tests with mocking for the whole codebase. It's the most painful experience of my life.

Congrats on 100K!

He remembered!

I miss the notifications

What is your github?

Grug is a smart grug. Use club to smash like.

Stop unit testing things that are not units.

Im on the build team. Devs hate me, managers love me because i used the iron rod of god where if your PR fails any unit tests, it doesnt get to be in master. Simple as that. Now our unit tests work and no one ignores failing tests

Property testing is not used nearly enough. I think a lot of less experienced devs might not even know that quickcheck libraries are a thing that exists.

TDD actually works really really well but it's like religion, trying to convert anyone by force into it is not going to work. If people don't see it and get it and immediately fall in love with it, you're not going to force them to fall in love with it. Just impress the importance of testing and pray that some day in the future they will be graced to see the light of how much easier their lives could be.

TDD for me is not about any kind of testing ideology, it is just about embracing how powerful it can be and how much easier your life can be if you have code constantly running as you work. Manually clicking around and exploring things also has its place, but any time you can plop out a line of driver code that will run and do it for you instead of having to do it manually, it can just make life way way easier. And no, it's not like this kind of code ever has to be checked into the code base. It's not about code coverage or test philosophy or safety, it's just about going fast and getting rapid feedback as you prototype.

When I am guided by nothing other than the principle that I want to go as fast as possible and I want to get instantaneous feedback, I end up settling into a programming loop that looks more or less a lot like TDD. At the very least, I absolutely always have some sort of code running in an automatic watcher loop that re-runs every time I make an edit for that instantaneous feedback.

Congratz!!!!

Not a TDDer here, but in my understanding, unit tests ARE supposed to be deleted if your API or functionality changes. At that point you are not offering that service, so why test it?

People should rather learn that most things don't need unit tests. You only test a total of 3 things:

1. Any incoming query - assert it returns what it should.

2. Any incoming command - assert it changes what it should, given it's somehow publicly accessible.

3. Any outgoing command - mock to see if it gets sent with the correct arguments.

That's all. You don't go unit testing implementation, cuz grug club you in face.

Unit testing ought to use for testing the behavior not implementation.

Talking about unit test? It's not Dave Farley, since he hates unit tests basically. :)

Grug speaks how prime reads normal english. I'm happy that Prime found a writer that he can read well 😁 (all jokes I love Prime).

People are missing out on the true power of TDD and BDD. You are supposed to spend time thinking about and writing the tests. This guides how your code is written in a much cleaner way than simply writing the code. Of course, write the tests from the view point of the business value being added and not from the viewpoint of your state design pattern.

I have never seen prime read an article with as little reading error, I think he really mentally vibes with this kind of writing

There's a certain point where I start to naturally "test" my code to make sure it does what I think it does. You write a little code to make sure it works, and that's that. I usually just throw that code away when I'm done with it. But I think this is the precise moment when unit tests start to make sense. I'm literally already writing them, might as well keep them around.

The whole writing tests before you write code thing never really made sense to me though. Unless you have a crystal clear specification for how something should perform, writing tests beforehand seems quite counterproductive.

dependency injection.

Never test until you know what the program output should be. If you know what the output should be, you can write a test. You should write that test as soon as you know the output, no sooner, no later. Don't add more than one test at a time. If a test you didn't just add is failing, it's a regression. Keep your testing simple or it will summon the shadow demon.

We have so called time bombs. These tests would run normal all the time, but then out of a sudden they fail because of some datetime things.
One test used to fail every day between 0-1pm 🤷‍♂️

Using mostly e2e and never have problems with Grugs kind of mystery breaking tests.

Many years ago the team were into unit testing but we found those rarely saved us. The problems were often in steps between the units.

Worked with TDD fans for a while, it can work without too much messing around if you just assert html containing text. But stopped doing it, I could never really feel at ease with thinking a failing test was fine and normal.

i usually make a test whenever i notice a spot where a certain level of complexity passes through. it's great intuition that saved me a lot of time. the more serious tests come later when the code feels more solid like mentioned in the video.

I have only used mocking when working with a flaky 3rd party API that required consumers to build controllers, payloads, and issue requests. There were so many points of failure in implementing their code that mocking had to be the only right solution.

I have a sneaky suspicion that Primeagen is Grug, Grug is Primeagen... heh!

Is truth. Is good.

Doing the whole "never integrate with your environment thing" is really really hard though. Very often you need little pieces of data that you do not know at the start of the function so you cannot really extract it out entirely. Then you can either mock part of your data access layer or run a local db or similar for the test. I sometimes prefer the latter, but the downside is that your tests that could have taken milliseconds now take seconds to complete.

My experience with test first has changed how I write code. My code is more reliable and my tests are easy to change when business requirements change. The secret? I focus on business value and not on implementation. Also, I use dockerized databases for my testing so I never have to mock repos. It isn’t difficult but it seems that way to those who haven’t done it before.

Edit: fixed a typo

Added note, I’m not saying that I’m more talented than others. My team and I have figured out how to use BDD to guide our development. We believe in BDD and TLD. Everything is just a tool to help you write code. Use what works for you.

Mocks are proof that OO is bollocks

grug is a genius