Test-Driven Development In Python // The Power of Red-Green-Refactor

Who's using TDD regularly in Python? If you have any tips to share, please post them here!

"TDD IS THA SHIZNIT" indeed. :)

If TDD is basically setting requirements in advance, is it then a waterfall process in disguise?

Actually, it is much better to only add a single test per red-green-refactor loop. This enable step-wise and incremental implementation of the production code. It also makes it really easy to see if you are in green since all tests pass, you don't have to remember which tests are "supposed" to fail at this step.

thank you!

Hi Arjan, great videos as always, wish I had watched this a year ago. After watching this video, I have done a little more research my own on TDD, then I come across another method/framework called BDD (behaviour driven development). Have you had any experience on it? Just curious if you have any preference over either one and why, thanks!

Anyone else prefer to have the expected value before the function call: assert expected == func()?

I've seen someone use this approach and it was absolutely miserable to watch them. The problem is that people tend to not think about the problem at hand enough, and only focus on making tests pass. This can lead you to an over-complicated implementation, and possibly to a dead end.
My approach is to take time to think how the problem can be approached, then settle on the best option and implement it first. Then, using that initial implementation, I would use the red-green-refactor method.

Thank you very much for this!

Given the overhead, does TDD's benefits carry over to small personal projects, especially when it may be a single person working on said project, meaning the same person writing all the code and tests? If not, at what project size should TDD be seriously considered?

I've learned about TDD previously in a programming course. It was a completely new workflow for me and I found it really cool, but it seems like overkill to me for small projects.

Would you recommend testing with data in files? I mean if you're just looking an unt output, you can hardcore this in the unit test, but if you generate a large dataset should it be numpy or pandas, my approach is to save known good data results to a file in the test folder and have the unit test compare to that.

I find your tutorials so easy to understand. Could you do something in continuous integration and deployment?

Why didn't you select pytest instead of unittest...pytest requires less typing.

I write embedded C, but have started writing TDD for the Python tools I create. I much prefer the fine-grained approach. One test at a time lets the test/spec guide the design better and puts fewer details in the air (avoiding confusion). Arjan seemed to already have a solution in mind for his example; maybe that was a side-effect of preparing to make the video.

You said write test first, then write code, then how come you have all the methods and attributes and instance name of your actual object and funtionalites, everything handy ? or atleast you had it's structure crafted...before writing the test.

I've got two questions. I would appreciate if you assist me there.
1. In TDD, we write tests first. What if we implement a feature that is logically implementable and then we write tests and then improve the functionality?
2. Is it ok to create a new object in the `setUp` and then check the equality of each field of the created object with a valid sample object? Like firstly, I declare a variable. Then I create a new object with that variable's value then I check the equality of the created object's value with the declared one.

Great video, but can you tell why you are using unittest instead of Pytest? Is there any reason to switch?

How can I get some of the benefits of test driven development when writing code that's experimental in nature, like is often the case in data science?

I would like to see you performing code roasts, or refactoring videos using TDD.

I love how we are both using python but your code hardly looks python especially the strict typing. Do you think this is necessary, should I try to change my habits. I get it in big projects but do you always do this

This channel has quickly become my favorite in all things Python.

Loved this thank you

If you want to use TDD, but you don't know the requirements exactly before starting the project or you don't know exactly what you want to use to structure and implement those requirements, you could take the following approach:

Step 1: try to make the requirements less ambiguous and research technologies that could help achieve those requirements

Step 2a: make a proof of concept (no tests, rapid dev cycle, experiment with the technologies to see what works best, iterate to find the best way to structure the program)
Step 2b: demo the PoC (verify if the requirements are met, find things that need more attention or need to be faster. Go back to step 2 if necessary)

Step 3a: create a NEW repo, DON'T copy over the code from the poc, lay down the decided upon framework for the program, without actually adding code. Use the PoC only for reference/"lessons learned" from now on. (You don't want untested and possibly buggy PoC code in production)
Step 3b: lay down your high level tests (itest/e2e) based on the entry/input points of your program.
Step 3c: starting at the entry point to the application (REST/SOAP/JMS/gRPC/event/CLI call/etc.), lay down the interface/code skeleton for that feature and create your tests, TTD style! (Red phase)
Step 3d: implement the tests. (Green phase). if the implementation defines more methods, don't implement those yet, unless they are simple and the existing test fully covers these methods. Mock where necessary.
Step 3e: refactor your code. Keeping your framework in mind when doing this and ask yourself if the framework still suits your needs the way it is now.
Step 3f: recursively go back to step 6 for all unimplemented methods.
Step 3g: verify if all methods are implemented and if the itests / e2e tests are now succesful.

Step 4: deploy your program

Feel free to correct me if I missed something or give your opinion if you disagree with this approach.

Noob question: how did you refactor your code on the fly with a shortcut? Is that a custom shortcut running black?

I still don't know how to properly write tests for code and sometimes code is so complex, e.g. use requests to AWS or require some messaging in RabbitMQ to deal with response later, that I always asking team lead with helping to write tests for my case. There is some mock magic, that I particulary not understand right.

I’ve worked on teams that do this. Development is extremely slow and whenever a stakeholder changes requirements it leads to huge amounts of tests that are broken and now you’re fixing the tests and the code.

Which is all fine, but your stakeholders better understand why you’re doing it up front or it’s just going to look like you’re moving slow.

I will be exposed to TDD a bit, I will consider all those items you mentioned thanks.

The best thing is yes when you apply test first then code approach you really don't make functions complicated and you write better testable code..

Nice video. My biggest shock with TDD was when I saw Uncle Bob in his clean code video. There he would have had 5 of those cycles for 5 unit tests. I'm curious about your opinion, when would you commit in your cycle?

I know this isn't the topic at hand but I didn't know you could explicitly declare types in Python until watching this video. After reading up a bit, I see Python doesn't enforce types but some IDE's will call out type differences if types are annotated in your code the way Arjan does here. Since he's using VS Code, is that the benefit he gets in annotating types in his code here?
Edit: I guess he's doing it because he's using dataclasses?

For generating types of test data, pytest + hypothesis are a good combo. You can even cover None, nan and other outlier cases easily

thanks for the video Arjan! I teach testing and TDD, your video is extremely clear and easy to understand, I'll recommend it to my students. Thanks!

Very nice Arjan, as always.
Just my two cents here, maybe it was only for the sake of didactics but I believe the best approach to TDD is baby steps. So it's better to write a single test, make it pass and, refactor and then go to the next.
Also, when you talked about having someone who writes tests and someone who writes the code, well, in a steady environment, developers should write tests, specially when we are talking about TDD, but we still should have QAs who would:
1. Pair with devs to think about test cases and test strategies
2. Validate those tests after they're done.

Nonetheless, I've been referring your videos to my friends. You're great!!!

I'm interested in how to design complexe setups for tests. My main method is to go with pytest fixtures but there are handle dynamically and I hate how IDE can't identify them (for example impossible to ctrl click definition to find the fixture definition in vscode)

Cool video

Thank again Arjan. Looking forward to Mocks and Patch videos ... I particularly find them complicated.

I was the 610th like on the video. Fibonacci would be pleased. Thank you for these videos.

Which domain do you use?

Testing manually is boring and takes forever. Let's automate it. -Kent Beck, probably.

Great introduction to testing video. Useful to have the pros and cons

It would be great to have a good explanation about unit testing for Models and database interactions, it's a topic in unit test that I never see anyone tackle in a simple intuitive way.

I really struggle with this TBO... I don't know how to find the time to do the tests first... not the mention get 100% coverage on the code base. TDD is a great methodology and would love to learn TDD first approach... but it is hard, and convincing colleagues that this is the way to go is hard... justifying time spent on this... is hard...

I would love a video on pytest! I have been using it recently but I feel like I would get a better idea with your input. Thanks for the video!

did anyone else notice the “TDD IS THA SHIZNIT”

I'd appreciate a video about mocking, an extremely useful technique which really takes your testing capabilities to the next level

Great video Arjan! Didn’t know about TDD before this video, might start to implement it at work.

Why unitest instead of pytest?