Comment

We join the comments at https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam already in progress. I promise, it won’t make you cry.

https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam#comment-3966797910

You say it yourself, that “unit tests should be a […] developer tool”. Exactly right: they are, and that’s how I use them. I use them specifically to drive the design and help me identify design risks. They help me build components that I can confidently and safely arrange in a way to solve business problems. — J. B. Rainsberger, replying to https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam#comment-2706552631

For me, I think this might be getting closer to my trouble with trying to use unit tests in some circumstances. Sometimes when I write a component that I intend to be reused and have a nice API and needs to behave in a particular way and is naturally isolatable, the first thing I do is write a bunch of unit tests because I need to consume that component from other components and have confidence that it will behave correctly. For example, something which converts string representations within my application without talking to external systems.

However, when I am writing glue code which builds an Entity Framework LINQ query from user inputs, I can’t figure out how to make that a unit test properly isolated from a database because I find it to be uncoupleable from the database. I feel like if I wrote a unit test in that situation, my tests would be something like “compare this LINQ Expression tree to the expected one”—when I actually care more about whether or not the expression tree is capable of getting me the right rows from the database than whether or not it looks like what I think will give me the right rows from the database. For this system, if I were to try to decouple components and were to write unit tests for the part that created my SQL/LINQ query, the unit test would not be useful because all it would be verifying is that my method is implemented a certain way. I.e., the unit tests would basically be like using jest’s snapshot feature and running jest --updateSnapshot without taking the time to read the changes made to the snapshot.

Discussion

I’m going to show my age here. I remember thinking about this exact problem when I started working with container-managed persistence Enterprise Java entity Beans (“CMP entity beans”). The year was 2000, I worked at IBM, I’d worked in Java for nearly two years full time, and WebSphere Commerce (as it was becoming) used CMP entity beans as the standard way to read and write data from the database. They were similar to Hibernate entities. Nowadays, I think the cool kids use JPA for this. Or something better, I hope.

I mention this because I felt exactly the same hesitance. I thought about how I would write solitary tests for these CMP entity beans, isolating them from the database. I saw no clear way to inject a DataSource interface into them. Even if I did, I had previously ruled out stubbing and mocking the Java Database Connectivity (JDBC) interfaces, because they certainly violated the Interface Segregation Principle as though they earned royalties for doing it. After putting a lot of thought into it, I concluded that writing solitary tests for CMP entity beans would probably not return much on the investment, so I gave the idea up. I wrote this advice in JUnit Recipes: here I’ve found one place where writing solitary tests just doesn’t make sense. Don’t bother.

Mock Roles, Not Objects

I later understood a theoretical basis for defending this decision. I found the now-famous “Don’t mock types that you don’t own” advice, which I later saw repeated in a paper at OOPSLA. Instead of writing solitary tests for the CMP entity beans, I hid the entire family of classes behind Repository interfaces, which allowed me to write solitary tests for the clients of the Repositories. Most of the work in creating CMP entity beans lay in metadata to configure the entity bean container, anyway. (Choose this transaction isolation level. Choose that way of mapping data types.) I wrote almost no code beyond duplicating SQL database row schema information in Java bean classes. I imagine that someone wrote a generator for that years ago, but I never used one. Looking at that code, I had nothing really to test unless I ran it with the database. In this one case, I tried very hard it isolate my code from the expensive external resource—the database—and I couldn’t justify the investment, so I didn’t bother.

Of course, if I found myself adding logic to my CMP entity beans that could run entirely in memory, then I moved it out so that I could run it entirely in memory. The CMP entity bean would pass itself (or part of itself) into that logic so that the logic would neither know nor care where the data came from, nor where it would go. Standard Dependency Inversion Principle stuff.

I feel confident that if you confronted me with this decision again, even knowing what I know now, I would decide the same way.

Just Try It

I find it difficult to describe a rule that an inexperienced me could have followed to lead me to this judgment. I can say that I generally recommend trying very hard to isolate your code from the database, even going as far as writing several tests, ruthlessly removing duplication, and then seeing where you end up. After a few hours’ investment, you might judge better whether to keep going or to give it up. You might need to go to far and risk wasting a few hours in order to develop the judgment that you need for future work. Don’t worry: good judgment comes from experience; experience comes from bad judgment. Let yourself give in to some bad judgment every so often. Try something that looks like it will never work, just in case it does. But remember to set a timer, then stop when it sounds, and then evaluate where you stand.

In a similar situation, if you can extract an interface from the annoying thing that you need to connect to, then you might find that helpful. You could then write solitary tests for the clients of that new interface. If you can’t extract one, then you can’t. That might work just fine too. It matters to me that you think carefully about it and even try it, rather than give up at the first minor sign of difficulty.

I used to think of Rails as a hopelessly coupled mess (I still do), but one day Pat Maddox showed us how to turn hopelessly-coupled Rails into nicely-decoupled Model/View/Controller components, by taking advantage of the fact that in Ruby everything is an interface all the time. Sadly, his article has disappeared from the web. We can’t do this so easily in Java nor C#, but it continually surprises me what we can do when we apply the principle and let our imagination run wild.

Remove Duplication, Even Without Tests

If you decide not to write tests, then at least remove duplication in your glue code. I find this very valuable in finding missing abstractions along the integration boundary between My Stuff and Their Stuff. To remove duplication without tests, commit annoyingly frequently. If I don’t have tests, then I want the world’s best undo button as a backup. Remove duplication along the integration boundary and then see what happens. Quite often, when I do this, I find an abstraction that I can put between My Stuff and Their Stuff that plays the same role as the Repositories do in my CMP entity bean adventure. Most programmers ignore this opportunity at their peril. I encourage you to try it.

References

Steve Freeman, Nat Pryce, Tim Mackinnon, Joe Walnes. “Mock Roles, Not Objects”. The first paper to codify how test doubles (mock objects) supports programming to interfaces. Functional programming enthusiasts would laugh that we poor object-oriented programmers would even need to independently discover this principle.