xUnit or NUnit? Picking the Right Testing Library

Ok looks like I need to make a video on MSTest

I mostly use fluentassertions so I don't care too much about the testing framework.
The setup of complex dependencies is a one time thing

Which is the best way approach on sending a list of test case objects (usually values and expected values, but with the flexibility to design the TestCase class with what ever properties you like) in xUnit? I have used this in NUnit with [TestCaseSource(typeof(TestCases), nameof(TestCases.TestCasesCollection<TestCase>))] and found it to be nice to make a clear representation of what a TestCase is in the given context as well as separating the test data from the actual test and its logic. Great video/content as always and keep up the good work!

For simple unit tests I do not care. But NUnit is my option for intergration tests. For exeample when you want to isolate tests with transaction scope to keep database state pristine for other test scenarios, in xunit ClassFixture is initialized on a separate async context thus that scope does not carry over to facts (no, async option on transaction scope is not a solution here). also tests ordering or some global inits are harder in xuinit, require custom code. With NUnit all those issues do not exist.

no. you don't need TestFixture unless you have some setup to put in it. TestCase working fine themselves. The one time init showcase is not representative at all, as we have SetUp for it. NUnit is sequential execution by default, so SetUp works pretty solid for, you guessed it, test run setup. And NUnit being translation of JUnit - we can say it was not designed from scratch in the first place.
On the contrary, xUnit was created from scratch when guys realized testing in C# missing a lot with NUnit (say Java) way and it's worth to give C# it's own clean test way. And we don't want to mess with long time living NUnit and make some non-compatible changes - so they decided to create a new testing lib. And they did it the right way so each run is from scratch and test class also from scratch - so your setup not is a constructor - we're good to run all of these in parallel in the most cases.
Not sure it's fair to day NUnit tests are flaky. You need to know tools you're working with. For NUnit you know it's not going to create new test class for every test run. For xUnit - you know it will.

I use MSTest V2

Nick, yes, please, make a video about the MSTest.

I use XUnit because Nick uses XUnit

It would be cool if you went over playwrights cogen tool. How you can set up the code it generates in a way you can just copy and paste each new test case as an individual file.

A Side note is that not only is the test case operated on a single class in NUnit, it's operated on that class in PARALLEL by default, which is a recipe for disaster.
I would add for the people who don't have the option of switching to XUnit you can achieve the separate class per test functionality in NUnit with some annotations on the class with tests in it.

[TestFixture]
[Parallelizable(ParallelScope.All)]
[FixtureLifeCycle(LifeCycle.InstancePerTestCase)]

If you want to have separate instances of a class in NUnit, would you not simply create the object inside the test method, instead of a class variable?

Class variables are supposed to be reused among all methods.

the arguments are pretty "random". As said in the intro: a matter of taste. Both are available, both have are large community. Essentially you can't go wrong. It is more important to provide good documentation how to solve common testing issues, e.g. how to genrate read-world data to test DB performance read AND write operations. also: how to add failure to in certain layers (with a random chance of occurance) etc. Most tests, acutally test only that a) the compiler works, b) that the specification - which changes all the time - is being fulfilled.

I use NUnit mainly because thats what we use at work. However i feel like its actually the best option from this description aswell. Having the classes be singletons makes a lot more sense to me. That way its also cleaner to read what variables are useable for that specific test and what variables are not available. Also if you have some sort of global test setup logic like creating a file, downloading a file etc. Im not sure how you would do that using xunit without having to do said logic multiple times over.

First of all, love your videos! :)

I disagree with your "instantiating the test class for every test is better" argument, cause I would expect a property in my test class to be the same when instantiating it inline and not changing it afterwards. Also you can mark it as readonly (as you do) if its not supposed to be changed by your tests (generally I would avoid global variables to be changed in methods - so for me it allows for a bad pattern, and NUnit will force you to do it right).

Also, I like the [Setup], [Teardown], and also [OneTimeSetUp] and [OneTimeTearDown] attributes that exists in NUnit, as to me, it's more explicit than implementing IDisposeable, and using IClassFixture.

I clicked on the video to kind of get the "this is why you should pick XUnit in this scenario or this is why you should pick NUnit in that scenario" - instead I got "They can do pretty much the same things in different ways, but XUnit is more popular".

I value your points and video though :D I'll keep searching for someone to change my mind about NUnit versus XUnit :) But it also seems there's a good reason both exists, which annoys me a bit :D

I have a problem with xUnit, it is missing some features that exist in NUnit. It doesn't support generic TestFixture. Also, it doesn't have retry/replay attributes (but some nugets add this functionality).

Nick, in the “complex” example you showed some shared context and seems like you are using ICollectionFixture for it. We are doing exactly the same but we faced a terrible consequence we don't have a workaround for: tests in the same collection can't be executed in parallel. Did you manage to fix it somehow? I think about switching to NUnit because of it.

May I ask your thoughts on using var versus explicit data types? Example: "Guid id = ..." vs "var id = ..." I recently made the switch to using Rider from VS, and I find it odd how Rider always throws warning when I try and use the explicit version, and it recommends I change them all to var.

I just use MSTest so far. What you showed with xUnit I achieve by writing a nested class that I instantiate at the start of every test. it's usually disposable and it will have the SUT and any mock dependencies created with options for setting known values and so on. what you did with xUnit is slightly cleaner by basically using the test class itself as the SUT wrapper.

MSTest is so much better. It creates an instance per test, just like XUnit. It can easily do method level parallelization, which is something XUnit can't do. A lot of the async support is more straightforward.

It's been a while since I'd moved to xUnit, as that's what the new team had + I wanted to check the hype around it, but I still find nUnit to be more CONSISTENT and CLEAR. You can change from per test to per run setup by simply replacing an annotation and when reading existing code it's very clear what it's going to do without looking into docs. Also nUnit works better with multiple sets of data as you can specify what RETURN RESULT you expect, instead of introducing additional 'expectedValue' input -> much cleaner. Plus no nonsense [fact]/[theory] distinction.

Also if you want to use original Asserts I find nUnit ones better and more obvious, plus they support '.That' pattern which is nice. On top of that we've experienced multiple deadlock problems with xUnit, that were terrible to debug, and the author's response to issues in the framework that were pointed out in github wasn't exactly building confidence.

Nunit allows you to change default behavior of using one class instance for all tests

Do not create sut outside of test and do any setups there. After some time you have setups which are hard to understand and do to much for some tests. Always use Auto fixture with automocking. I'm talking about unit tests.

MSTest is fine.

Btw. NUnit also had a similar interface for tests as XUnit with something like Assert.AreEqual(), but there is this newer fluent API using Assert.That(). I started using FluentAssertions for that. Technically, it is fully redundant, but you can use it with any of these frameworks and you get the same assertions.

I started using NUnit very long time ago even before MSTest was available. A couple of years ago I switched to XUnit.
I have never used MSTest regularly, but it looks like polished NUnit for me.
From my point of view there are 2 bad things about XUnit: missing documentation and that it is relatively slow (in comparison with others). On the other hand, you can easily run tests in parallel using XUnit, so it is better scalable.

Ja Ja ja ja G U NET


G-UNIT

MSTest2 + Fluent Assertions it's being sufficient to me.

I share similar feelings about NUnit and xUnit, about Assert.Pass(): I don't think there is anything wrong to just say Assert.Equal(true, true), but I go for FluentAssertions usually anyway.

Once I tried xUnit 5 years ago, I couldn't return to NUnit. It's like when you buy a new and much better car

NUnit. Tried XUnit and when straight back. NUnit is an easier way to quickly test the app realistically. Stuff that is setup on startup and a bunch of methods that are ran right after. Initiating everything from start for each test isn't how an application operates in 2023 and trying to set it up that way in XUnit just wastes a ton of time where in NUnit you're already getting test results because that's its "default" logic.

I only use xUnit nowadays, basically because of the reasons they wrote in the "Why Did we Build xUnit 1.0?" documentation article.
But let me name the most important things: better isolation, by default parallel, more natural constructs (as you mentioned too, C# constructs instead of attributes only).

And yeah, maybe you need to make a video on why MSTest is/was garbage.

Good reminder video of why I used xunit for that many years.

Together with Fluent Assertions for asserting, NSubstitute for mocking and AutoData for injecting mocks and data I’m happy writing tests.

Thank you for this video. Everything was new to me.
I'm used to using NUnit and am familar with the syntax. However xUnit is also very straight forward.

I like the default of xUnit as well. But NUnit has a FixtureLifeCycle attribute. With LifeCycle.InstancePerTestCase you can ensure your classes will be instantiated the same way as in xUnit, with the constructor and IDisposable. You can also mark your assembly to use this for all the classes. Very handy when running test in full parallel, because SetUp is not 100% safe.

1 second in and I am like ok cool using XUnit for the rest of my life.

I use xUnit for exactly that it creates one instance of the test class for each test execution (even for theories). If a system gets a large set of devs banging out stuff it less likely to break due a lack of understanding of instance state. Otherwise all the three main test frameworks are much of a much-ness, but this one behavior of xUnit is the "killer app" that makes it the choice.

Just one general questions,
will the courses on dometrain also be updated?

Glad to see this video, at least the reason is not following what internal dotnet team uses blindly. I have been using xUnit 2 years, previously I used NUnit more than 6 years. The only reason I uses xUnit is because my team/tech lead wants it. I never use xUnit without FluentAssertion. xUnit's assertion is poor compared to NUnit. One of missing feature in xUnit from NUnit is test session fixture setup. xUnit only allows per test/class level setup. I used it before in NUnit and don't see feature equivalent in xUnit. One of the best feature from FluentAssertion is object graph comparison. I use it a lot in my test projects, it's very flexible and powerful.
So my best combination will be NUnit+FluentAssertion. NUnit covers most of the case in my project (assertion helper, setup). I like the new style which is more BDD. It's very flexible, either for single object testing or collection testing. I will use FluentAssertion for object graph testing only. Previously, I have to write my own implementation just to implement this in NUnit.

In xUnit I'm really fond of TheoryAttribute, and using InlineData/ClassData

I use MSTest.

NUnit using the same instance for all Tests in the class hit me hard when working with Unity3D... Also the version used was so old, it didn't support the "instance per test" behavior at all.

xUnit is indeed a better alternative to nUnit. But in all honesty, ever since MSTest added exception-based asserts a few years back (which was its biggest downside) there has been no reason anymore to use anything else other than Microsoft own library. And I also prefer the overall semantics of MSTest. I must say xUnit way of naming things always made me itch.

Hi Nick. In NUnit you usually would initialize SUT in SetUp method. Then you will get a separate instance per test and TestContext.Out.WriteLine will print different GUIDs.

We’re using MSTest and it’s been perfectly good. The DataRow attribute saves a lot of repetition.

MSTest! Originally mostly because it was developed by MS. Then I quickly started enjoying its simplicity and how easy it is to have up and running (dotnet new mstest && dotnet test, that’s it!) (I guess other frameworks are easier to run nowadays with the dotnet CLI too). And still using it because… there isn’t a day where I feel I’m missing something. No frictions. It’s better than good enough: it just never gets in my way, and does what it says it does. A tool that weighs like a feather. Never makes me think about testing frameworks, and so I never think about switching.

I came to the video looking for why you'd be using NUnit not xUnit 😁
A lot of people use xUnit because it's getting Microsoft's 1st class support (in docs and examples, etc.) since the creator joined the company.

Lol I literally program around the class initialization problem in nunit all the time. I just add a initialization method on the test initialization attribute, it’s fine.