Friday, July 29, 2011

Why I created Fluent Assertions in the first place

A few weeks ago I read The value of open-source is the vision not the source code and that made me think about my own reasons for starting Fluent Assertions, now more than a year ago. In the light of that article, lets briefly go over my own goals for Fluent Assertions.
The intention of a unit test must be clear from the code
An intention revealing unit test has the following characteristics
  1. The arrange, act and assertion parts are clearly distinguishable
  2. The name represents the functional scenario the test verifies
  3. The assertion provides sufficient context to understand why a specific value is expected
For instance, consider this small test.

        [TestMethod]
       
public void When_assigning_the_endorsing_authorities_it_should_update_the_property
()
        {
           
//-----------------------------------------------------------------------------------------------------------

            // Arrange
            //-----------------------------------------------------------------------------------------------------------
            Permit permit = new PermitBuilder().Build();
 
           
//-----------------------------------------------------------------------------------------------------------

            // Act
            //-----------------------------------------------------------------------------------------------------------
            permit.AssignEndorsingAuthorities(new[] { "john", "jane"});
 
           
//-----------------------------------------------------------------------------------------------------------

            // Assert
            //-----------------------------------------------------------------------------------------------------------
            permit.EndorsingAuthorities.Should().BeEquivalentTo(new[] { "john", "jane"
});
        }
Yes, I agree that it's a very simple test and that this scenario should be tested as part of a larger scope. Nonetheless, it clearly illustrates my three points. Now consider this test .

        [TestMethod]
       
public void When_a_substance_is_specified_that_is_not_required_it_should_throw
()
        {
           
//-----------------------------------------------------------------------------------------------------------

            // Arrange
            //-----------------------------------------------------------------------------------------------------------
            var permit = new PermitBuilder().Build();           
           
var someUser = new UserBuilder().Build
();
 
           
var dataMapper = new InMemoryDataMapper
(permit, someUser);
           
var service = new CommandServiceBuilder().Using(dataMapper).Build
();
 
           
//-----------------------------------------------------------------------------------------------------------

            // Act
            //-----------------------------------------------------------------------------------------------------------
            try
            {
                service.
Execute(new AddMeasurementsCommand
                {
                   
Id = permit.Id,
                   
Version = permit.Version
,
                   
Username = someUser.Username
,
                   
Measurements = new
[]
                    {
                       
new MeasurementData("Oxygen", 1.1d, new DateTime
(2011, 4, 13, 16, 30, 0))
                    },
                });

               
Assert.Fail("The expected exception was not thrown"
);
            }

           
//-----------------------------------------------------------------------------------------------------------

            // Assert
            //-----------------------------------------------------------------------------------------------------------
            catch (InvalidOperationException exc)
            {
               
Assert.IsTrue(exc.Message.Contains("not required"
));
            }
        }
It is testing a particular business action against one of its business rules. It's quite easy to understand, but there's still a lot of noise caused by the try...catch construction and the additional Assert.Fail() to assert that an exception was thrown at all. Less important, but still a bit obscure is the usage of the DateTime constructor. With Fluent Assertions, we can do better.
            //-----------------------------------------------------------------------------------------------------------
            // Act
            //-----------------------------------------------------------------------------------------------------------
            Action action = () => service.Execute(new AddMeasurementsCommand
            {
               
Id = permit.Id,
               
Version = permit.Version
,
               
Username = user.Username
,
               
Measurements = new
[]
                {
                   
new MeasurementData("Oxygen", 1.1d, 13.April(2011).At
(16, 30)),
                },
            });
 
           
//-----------------------------------------------------------------------------------------------------------

            // Assert
            //-----------------------------------------------------------------------------------------------------------
            action
                .
ShouldThrow<InvalidOperationException
>()
                .
WithMessage("not required", ComparisonMode.Substring
);

Something similar is also possible for verifying that events have been raised, with special support for the INotifyPropertyChanged interface so common in Silverlight and WPF projects. I blogged about that earlier this year.
A unit test that fails should explain as best as possible what went wrong
Nothing is more annoying then a unit test that fails without clearly explaining why. More than often, you need to set a breakpoint and start up the debugger to be able to figure out what went wrong. Jeremy D. Miller once gave the advice to "keep out of the debugger hell" and I can only agree with that.
For instance, only test a single condition per test case. If you don't, and the first condition fails, the test engine will not even try to test the other conditions. But if any of the others fail, you'll be on your own to figure out which one. I often run into this problem when developers try to combine multiple related tests that test a member using different parameters into one test case. If you really need to do that, consider using a parameterized test that is being called by several clearly named test cases.
Obviously I designed Fluent Assertions to help you in this area. Not only by using clearly named assertion methods, but also by making sure the failure message provides as much information as possible. Consider this example:
"1234567890".Should().Be("0987654321");

This will be reported as:

clip_image001[6]

The fact that both strings are displayed on a separate line is on purpose and happens if any of them is longer than 8 characters. However, if that's not enough, all assertion methods take an optional formatted reason with placeholders, similarly to String.Format, that you can use to enrich the failure message. For instance, the assertion

new[] { 1, 2, 3 }.Should().Contain(item => item > 3, "at least {0} item should be larger than 3", 1);
will fail with:

clip_image002[6]
The code itself should be a great example of what high-quality code should look like
This goal should be rather obvious. I don't only want to deliver a great framework, I also want people to learn from it. And isn't that the single biggest reason why people join open-source projects? However, the article I mentioned before states that the vision behind an open-source project should be much more important than the quality of the source code. As you might expect, I don't agree with that. In fact, one of the challenges I ran into was my desire to control all code contributions so that they complied with Clean Code, my own coding guidelines and my ideas about unit testing.
At first the article made me doubt about the approach to take, but then I decided that the quality of the code was just as important. I now respond to all contribution requests with some information on the way I'd like FA to see evolve. Additionally, I've set-up a dedicated contribution branch that they can use. Depending on the quality of their contribution, merging them into the main branch requires a corresponding amount of work at my side. I know that this might keep some contributors away, but up to now, most of of them agreed with my approach and were willing to deliver high-quality code.

Tuesday, July 05, 2011

Fluent Assertions 1.5 is done! Now it's time for that summer.

In the last couple of months, me and colleague Martin Opdam have spent a considerable amount of time on both improving the reporting capabilities of Fluent Assertions as well as fixing and incorporating various community contributions. Because of the many changes required, a very busy client project, and my endeavors around moving to a new house, it took us over four months to complete the next version. So let's quickly highlight the changes.

clip_image001[8]

So what's new?

A contributor working under the CodePlex account MatFiz added the missing counterpart of string.Should().Contain(), consistently named string.Should().NotContain(). And while he was doing that, he also included the case-insensitive versions string.Should().ContainEquivalentOf() and string.Should().NotContainEquivalentOf(). He also included fluent equivalents of string.IsNullOrWhiteSpace() through string.Should().BeBlank() and string.Should().NotBeBlank().

Asserting that a string matches a particular wildcard pattern can now be done using string.Should().Match() and the case-insensitive MatchEquivalentOf(), both supporting the familiar * and ? Characters. You might wonder why we choose for ordinary wildcards rather than Regular Expressions, and the answer has everything to do with keeping your tests intention revealing. I haven't met a lot of developers that know all the regex escape codes by heart. But I do know a few that are capable of writing with some very obscure (but technically correct) expressions. And that isn't going to help the majority of the developers.

Another request involved asserting that an object can be serialized and then deserialized using the binary or XML formatters. I never felt the need for something like that, but I decided to include the object.Should().BeBinarySerializable() and object.Should().BeXmlSerializable() anyhow. Both of them will serialize an object to a MemoryStream using the corresponding serializer and then use FA's property comparison assertions to compare the properties of the deserialized object with the original object. Notice that this works only for objects that support full round-trip serialization.

While Martin introduced the possibility to execute various assertions on IDictionary<T>, including checking for specific keys, values or a combination of both, at roughly the same time I added support for IComparable<T> through methods such as comparable.Should().BeLessThan(), BeGreaterOrEqual(), BeNull() or BeInRange().

One of the things I've always have been annoyed with is the fact that when you assert that an exception was thrown with a particular exception message, you had to specify the entire message, including punctuation and whitespace. But usually I don't really care about the specifics, and only need to ensure myself that parts of the message match. But since we've introduced wildcard-based string matching in this release anyway, it didn't require a lot of work to support this:

  action.ShouldThrow<ArgumentOutOfRangeException>().WithMessage(

       "code InvalidCode does not match a known", ComparisonMode.Substring);

Or this:

  action.ShouldThrow<AssertFailedException>().WithMessage(
       "Expected object*World*to be less than*City*because a city is smaller than the world."
,

       ComparisonMode.Wildcard);

Another little neat new feature is a fluent API for specifying dates and times. For example, consider this.

  var period = new Period(
        new
DateTime(2011, 2, 1, 8, 0, 0), new DateTime(2011, 2, 1, 18, 00));  
           

If you're into fluent interfaces like me (if not, why would you be here anyway :-)), this is a much better read, don't you think?

  var period = new Period(1.February(2011).At(08, 00), 1.February(2011).At(18, 00));

What property?

Because I don't like spending time in my debugger, the majority of this release was spent on the reporting side. I've tried to make sure that FA reports failures as clear and intention revealing as possible.

For instance, object.ShouldHave().AllProperties().EqualTo(object) now internally uses the equality assertions appropriate for the type of property. This significantly improves the details reported for a string or IEnumerable<T> property.

clip_image002[6]

The property assertion also throws with a clearer explanation when the types of equally named properties are not convertible.

Where's the difference?

Another area that has been improved is the way string differences are reported. This applies both to exception message assertions as well as direct string comparisons. The exception assertions will always display the expected and the actual messages on separate lines so that it is easier to spot the difference.

clip_image003[6]

String assertions will do the same, but only if one of the involved strings contains a newline or is longer than 6 characters. As you've probably already noticed, line breaks and other special characters are escaped now.

An object is not an object

Something I already benefited from during dogfooding an intermediate release in a client project is that FA will now display the public properties of the involved objects whenever an assertion failure occurs.

clip_image004[6]

Before this change, the only thing you got was a message along the line of "Expected object classname to be equal to classname, but they weren't". This will surely help tracking down the problem in your production code. And notice, FA will only do this if the object involved doesn't override ToString(). And since the collection assertions now use the same formatting infrastructure as the rest of FA, collections of objects will also be displayed using the actual structure of the object. In fact, it even support nested collections.

Did I break anything?

Upon specific request from a current user, I did change the behavior of the collection.Be(Not)SubsetOf() method so that an empty set is now treated as a subset of any set. Something similar happened to the collection.Be(Not)EquivalentTo() method; an empty set is now treated as equivalent to another empty set. So if you find that some of your tests start to fail after the upgrade, make sure they don't rely on this behavior.

Also, if you have been extending FA using the Execute class, pay particular attention to the release notes. We've changed the way you need to refer to the because of an assertion and marked the old API as obsolete.

How do I get started?

Just install NuGet and download the latest version of Fluent Assertions from its corresponding NuGet page. If don't want to use NuGet, then download it from CodePlex directly.