Friday, June 26, 2015

Why you should abandon TFS and adopt Git

Almost every time I do some kind of talk somewhere, people ask me for advice on how to convince their management that they should drop Microsoft Team Foundation Server's source control system and move over to Git. In this post, I'll be talking about the source control system only. Both Visual Studio Online (Microsoft's TFS-in-the-cloud), GitHub and BitBucket support this heavily. We use GitHub ourselves, but I've noticed that Microsoft made some considerable improvements since we moved away from TFS.



Anyway, in January 2014, after three years with TFS, we finally moved to Git. With 1.5 years of experience, I think we're entitled to having an opinion about how they compare. To help me compile this post, I asked my colleagues about things they can do now they couldn't do when we were still using TFS's centralized source control system. One word of disclaimer though.

  • You can create and merge branches in seconds rather than minutes (or hours if your project is anything but trivial). Switching between branches happens in milliseconds rather than the typical 15-30 minutes in our projects. This completely changes the way you work. It allows a level of isolation that is inpossible to achieve with a centralized source control system.
     
  • Pull Requests (PR), a feature supported by all online Git platforms is a crucial tool for teams to break down complex software systems in components owned by teams without blocking anybody. It allows developers to make a kind of copy of projects they don't own, create branches, work on improvements and new features, and then send a request to the owning team to pull in their changes. Pull Requests are the single best way to scale a software development organization into multiple autonomous teams.
     
  • How often did you encounter useless comments like "Oops, I broke the build" or "Some review rework". You can clean up the history of a feature branch before you merge it to its final location by using an interactive rebase to amend existing commits (the equivalent of a TFS changeset), squashs a couple of commits together, or to fix up those commit comments.
     
  • Developers can work in parallel on the same branch by merging with other team members, either from the central repository or on forks owned by individual team members. If they want, they can use PRs to control what gets in the shared branch.
     
  • So you've worked on that feature for a week or so, but in the mean time, the main development branch has been changed considerably. Wouldn't it be cool if you could apply your changes on that newer branch as if you started from that version in the first place? Well, that's what a Git rebase can do for you. It keeps the history clean and avoids all those unreadable merges.
     
  • Compared to our time with TFS, we have much less merge conflicts now are on Git. The main reason is that Git does a three-way merge because it knows exactly where two branches started to diverge. TFS typically does a pretty dumb text-based merge. You also have a lot of control how merging should be handled per file type in that particular project.
     
  • You never have to check out anything explicitly. Just edit it the file in whatever tool you prefer and Git will pick that up. The project's .gitignore file ensure it will only monitor specific files. If the net result of your changes is that you didn't make changes at all, Git will not see that file as changed anymore. Say goodbye to all those TFS changesets with unmodified files. We did try TFS's Local Workspaces feature, but that collapsed with large repositories.

  • Everything you can do online you can do off-line. Last year, I was traveling through Switserland for a couple of days. The simple fact you can branch, merge and rebase, all from your local laptop is something you really start to value if you can do that without an internet connection. This has become instrumental when the internet connection of my client's project went down. You can even merge with another colleague's repository over a local network share or USB drive.

  • Since Git treats branches as a pointer at a linked list of commits, the entire source code, including all branches and tags can be downloaded in minutes, if not seconds.

  • And finally, even Microsoft made the jump. Git is now the primary source control system for the on-premise version of TFS as well as Visual Studio Online's. And Microsoft is maintaining the entire .NET framework on Github.

If you still can't convince your boss with these arguments, tell your boss how much more productive you can be with Git. Or you can reverse that argument by telling him or her how much money he's throwing away every day by sticking with a centralized source control system.

So what do you think? Are you moving over soon? Did I miss any good arguments I should include? Did I exaggerate with any of this? Let me know by commenting below or tweeting me at @ddoomen.

Wednesday, June 24, 2015

False positives and semantic versioning

As part of stabilizing an upcoming release, I always dog food a beta package against the 12000 unit tests in one of our bigger projects. In the early days, that would surface all kinds of edge cases I never thought of. In every single case, the first thing I would do is to add a new unit test to Fluent Assertions to make sure that edge case would be covered from that point on. But during the last couple of releases, finding a failing unit test would be pretty unique. What an unpleasant surprise it was when I encountered about 150 failing unit tests when running against a beta of v3.4.

The cause of this was FA's most powerful extension method ShouldBeEquivalentTo. It performs a recursive comparison of two object graphs. To determine which properties have to be included in the comparison, by default it's supposed to use the compile-time type (a.k.a. the declared type) of the objects in the graph. However, in early releases this didn't behave entirely consistent. Both me and top contributor Adam Voss have worked on the internals many times thereby slowly improving the consistency with every release. In v3.4 we both applied some more improvements, but those unfortunately surfaced a false-positive from an earlier release.

In our particular case, we were comparing a collection of events, declared as Event, with another collection. Because of that existing bug (I'm not sure when it was introduced), it would still use the run-time type of the particular event during the comparison. The call to ShouldBeEquivalentTo excluded the properties of the Event base-class. And since v3.4 will now include the properties defined by Event class only, the net result is the following InvalidOperationException:

No members were found for comparison. Please specify some members to include in the comparison or choose a more meaningful assertion.

This particular exception can be solved by using the IncludingAllRuntimeProperties option. But be prepared for unit tests that suddenly fail on some nested property's value that didn't match the expectation. In all cases where this happened in our code base, the unit test failed correctly. In other words, we had some pretty serious false positives.

So what does this have to do with semantic versioning? Well, Fluent Assertions' release strategy is based on semantic versioning. This requires me to carefully think about the version number increment and how that affects the changes I make in a particular release. To elaborate on that a bit, assume the current version is v3.4.0. If I change some internal logic without affecting the public API and/or fix a bug in a backwards compatible fashion, I'm supposed to increment the version to v3.4.1. If instead, I'm adding new extension methods, additional overloads or marking APIs obsolete, the version should get bumped to v3.5. And finally, if I would drop those obsolete APIs or a particular .NET framework variant, I must change the version to 4.0. In short, the versioning strategy is not based on a marketing decision, but purely derived from the changes you made. By strictly adhering to this, people using v3.3 should be able to upgrade to any other v3.x version with confidence.

So looking back at this false-positive, the question remains whether this release demands a major, minor or patch increment. Since this release also includes new backwards-compatible API changes, it's going to be at least a minor increment. Treating it as a patch release is out of the question. However, when somebody updates their FA version from v3.3 to v3.4, chances are that it'll break some unit tests. You could say that is a breaking change they want to postpone to a later point in time. On the other hand, we're talking about a false positive here. I guess people would like to know that their production code contains a problem not previously covered by Fluent Assertions. Having said that, I've decided to treat this fix as a normal bug fix.

Considering this story, I can imagine more gray area exist in the semantic versioning realm. I couldn't find a nice spot to discuss those topics, not on Gitter nor on Jabber. So until somebody else comes up with a better place, I've created a Gitter room. Please join me here if you have questions about semantic versioning.

Sunday, June 21, 2015

How we document stuff

A recurring topic in every software project I've been involved with is what to document, when to do that, and where to store it. So it wasn't a big surprise that at a recent event, somebody asked me how we track and communicate design decisions. I initially pointed him to an article I wrote in February, but then I realized that that one only covers documentation in the context of an evolving architecture. So let me elaborate on documentation a bit more.

In short, we make a distinction between structured and unstructured documentation. We had lots of discussions about that, in particular because some people would really like to introduce a single tool for both of these categories. But I don't believe in silver bullets and prefer to use the best tool for the right job. Neither Microsoft's SharePoint nor Atlassian's Confluence does both well.

Let's start with unstructured documentation. This category covers notes you make during development, usually tightly couple to a user story in Atlassian's JIRA, our tool of choice for agile work item tracking. For those kinds of notes we still use Microsoft OneNote. A great example where OneNote shines is a detailed check list with things that still need to happen within a user story.

clip_image001

We generally start off with a couple of items, but this gets expanded heavily during development. Since we practice Test Driven Development, we track the refactorings, the edge cases and any leftovers here, including the blog post we need to write about some important change we introduced. In my team, having unchecked checkboxes means we still have work to do. So anything that pops up during discussions or sprint demos ends up here. Even if we decide to not do something, I expect the team to track it here. I really feel uncomfortable when open ends are discussed that are not in OneNote. And yes, I still frown upon people that use Notepad to track a couple of notes…

Discussions are also treated as unstructured documentation. For this we use Flowdock, a team-oriented threaded discussion platform, about which I blogged before. Obviously we prefer face-to-face discussions, but with an increasing group of professionals that are less vocal or working remotely, Flowdock has become invaluable to us. Even if we had a face-to-face discussion, we try to add a summary of that to the respective discussion flow, just to allow those that are out-of-office to catch-up the next day. Flowdock is particularly strong because it can serve as an efficient replacement for those long email threads where anybody who has access can read along or decide to participate. You'd be surprised if you see some very valuable contributions from people that would do so during a live conversation.

Related to these discussions we have documentation with semi-structured characteristics; blog posts. Whenever a major architectural, product-wise or process-related change is introduced, the team involved is supposed to write a blog post about that on a SharePoint Blog. SharePoint's WYSIWYG editor really has a lot to wish for and doesn’t support MarkDown. As soon as we find a way to migrate all those posts to Confluence, we'll do. Note that most of the people I know who really treat those posts seriously, write those posts in OneNote so that the team can contribute and provide feedback. We then use Microsoft Word's blog editor to post the final result to SharePoint.

clip_image002

Structured documentation ends up somewhere else. But within that category, we treat product-specific documentation different from project and team documentation. The former is tightly coupled to the version or variant of the product and should follow its development lifecycle. Examples of those are installation guides, (web service and REST) API documentation and release notes. Because of that, we really want people to be able to update that documentation as part of the product or architecture changes happening in the appropriate source control branch. So for that category we use Markdown as well, especially because it's a text-based mark-up, where merging concurrent changes is pretty painless. For editing we either use GitHub's built-in Markdown support or MarkdownPad 2.

clip_image003

Documentation not directly related to stories, products or systems ends up as Confluence pages. Although Confluence can be a bit sluggish sometimes (I suspect Atlassian is still catching up with its popularity), its editing and collaboration features are marvelous. I love getting those emails with recent changes and such. It really allows me to see what's going on within the organization. Architectural or high-level PowerPoint presentations also end up here, but need a special document library to track them. I have the feeling Confluence's integration with Office is not as strong as SharePoint's, but it's acceptable for now. By the way, did I mention that we often create carbon-board posters from those slides? We use them during discussions or as part of the introduction of new people, and also put them on the walls for everyone to see. Those really spark off interesting discussions with new candidates, prospects or existing clients…

So how do you approach software documentation? Let me know by commenting below or tweeting me at @ddoomen.

Friday, June 12, 2015

It's not about what you know right now, it's what you'll know in the near future

Unless you're looking for some really specific skills, hiring somebody purely based on somebody's current skills is the worst thing you can do. In my current position, I regularly get to do job interviews, both for my current client as well as for my employer Aviva Solutions. And while evaluating potential new colleagues, I tend to mostly ignore his or her current knowledge and focus on those characteristics and skills that somebody needs to solve real-world problems in real-world projects.


Mentality and passion have a big part in this. I've met a lot of highly technical engineers with a passion for building elegant and abstract technical solutions. Passion is good because it makes you care about what you’re building and gives you the energy to continuously improve your solutions. But what I've often missed is the understanding that they have certain responsibilities towards their team; building software maintainable by the other team members, keeping track of the work that still needs to be done, or simply the sharing of information with the others. You need to have the mentality to not just make things work, but to make it work for others too.

Another very important skill is being able to quickly find the information you need to solve problems, either online or through your professional network. I would never judge somebody for not knowing certain things needed in a project, as long as they do what it takes to acquire that information as soon as possible. For instance, if you don't have any experience with Git, I would give you a link to a tutorial, article or eBook and expect you to first study it yourself. I've been surprised how much difficulties some people have finding a solution on Google where a couple of carefully chosen keywords would have giving them immediate results. Consequently, having access to the internet, your Twitter network, Github and StackOverflow are a primary need for me. Without it, a client can't expect me to bring the value they pay for. In fact, I'd probably refuse the assignment in the first place.

So what about the culture of our organization? How do you know if somebody will be a good fit with the other colleagues? Each potential candidate has at least two interviews, each with a different pair of people. The first interview is mostly about learning to know eachother. E.g. who is this person? What ambitions and drivers does he (or she) have? Any passions, hobbies or other things that tell more about a person? How well does he express his thoughts and ideas. That will tell a lot about that person. The second interview is there to determine the things I mentioned before, such as skills, structure, mentality and ways of learning. But does that say anything about whether or not that person is a fit for our culture? No, not really. But joining our annual company-wide trip to a Southern European city for a weekend of fun, drinks and reflection most definitely will.

I purposely didn't mention experience yet. But don't get me wrong, experience is extremely valuable. But it isn't the experience with a particular technology or framework that I care about. What I care about is the experience of solving complex problems, the problems you face when introducing new technologies or frameworks, and the experience with the dynamics of people working in groups. So even if you've barely left school, and you have hardly any experience, it's the things I mentioned earlier that we will value. In my opinion, those are the ones that will determine how fast you'll be building up valuable experience.

So it's great to have intimate knowledge on for example OWIN, HTTP and REST APIs, but if you miss the skills to (learn to) work in teams, have trouble communicating your solutions to both technical and business people, or don't really have what it takes to understand the challenges of commercial projects, you're not going to be of much use for us. And the same applies to our clients. Yes, they initially hire us to help them solve specific problems or build great software with the experience we have right now, in the end it's the other skills that will make the project a success. Technologies change, processes change, tools change, and projects change. Your success depends on how easily you adapt and embrace those changes.

What do you think? Does it make sense? Love to hear from you. Just comment below or tweet me at @ddoomen.

Thursday, May 28, 2015

Dude, you can't solve all the problems by yourself

I think that the gist of this post should be pretty clear. Unfortunately I've fallen in that same trap myself many times. So often, I had to refrain myself from pulling somebody's keyboard from under their hands, just because I thought I could fix the problem at hand myself much faster. But with the risk of sounding arrogant, quite often I can. And sometimes, when the circumstances demand, I do. But in almost every occasion I did, a bit of patience would have increased the chance that the involved colleague could have solved the problems themselves or even propose a better solution. And isn't that a much better approach for scaling development projects? So assuming you recognize this kind of (mis-)behavior, here are a few more good reasons for keeping your patience.

  • People might second guess your solution, potentially identifying a flawed design more quickly. Similarly, by clearly explaining your solution or approach, you might surface new insights yourself (a.k.a. the cardboard programmer). At the same time, you might get more buy-in from your team.

  • It increases trust and respect the people have for you. They won't only see you as the fix-it-all-guy, but also as the go-to-guy for advice. In a way, it makes you approachable. Especially if you're filling a high-profile role, being approachable by new people or people without strong communication skills is essential for an open and efficient work environment.

  • Seeing your colleagues solving the problems with a bit of help can increase the trust you have for them. And if you trust the people you're working with, you’ll also more easily delegate responsibility to them. I'm pretty sure that will make your live much easier.

  • Gives people more autonomy and allows them to learn from their mistakes, which will significantly increase the capacity of those people. In retrospect, the single biggest mistake I made in my career is to try to keep people from making mistakes. It has cost me a lot of energy, and never gave them a chance to learn and experiment. 

  • If you teach somebody a new shortcut key, a debugging trick or a convenient command-line tip as part of a solution, chances are that that person will cascade that knowledge on to other colleagues, much faster than you do alone.

  • If people feel the solution is theirs, they usually also feel more responsible for it, automagically increasing the commitment they'll give to it. At the same time, successfully solving a problem will increase their security level and increase the energy they will take up the next challenge.

  • Being the one with all that knowledge and skills may put you in a powerful situation for a while, but at some point, you simply won't be able to handle all that work anymore. Being able to distribute the work to others so that you can take a couple of days off to spend some time with your family or attend that awesome conference will quickly become a difficult or impossible thing to do.

If those arguments are not enough, what do you think this will do to the strength and autonomy of the team as a whole? So the next time you tend to make that mistake, think of the old phrase "The whole is more than the sum of its parts".