Thursday, April 23, 2015

Software versioning without thinking about it

Solving the versioning problem
If you're building libraries, products or any other software system, versioning is usually a pretty big deal. It's the only way to determine what version of that library, product or system you're looking at. Before an organization settles on a versioning strategy, many discussions have been held on what constitutes a major release versus a minor release, how to version the component from a marketing perspective, and how to deal with bug fixes. In addition to that, if that software system involves a library or framework, or just component, then you'd be pretty interested to know when an update to that component involves breaking changes.

Fortunately, the open-source community has solved both of these problems for us. First, we have semantic versioning, which unambiguously defines how to update your version when you do hot fixes and patches, minor back-wards compatible improvements or breaking changes. They even define how you should post-fix your version numbers to denote pre-releases and build numbers. Assuming all decent software project are using Git these days, then the other problem is solved by following the GitFlow branching strategy, an initiative by fellow countryman Vincent Driessen. GitFlow describes in detail on what branch you do your main development work, how you stabilize an upcoming release and how you track production releases.

So I'll assume for a minute you are going  to embrace the semantic versioning scheme as well as follow the branching strategy prescribed by GitFlow. Wouldn't it be cool if some kind of tool existed that used the names of the branches and the tags on master (representing the production releases) to generate the proper version numbers for you? Well, once again, the open-source community comes to the rescue. Jake Ginnivan and the guys from ParticularLabs, the company behind NServiceBus, have build just that and named it GitVersion. So let's see how that works.

Showing of the beauty of automatic versioning
First, we need to install GitVersion, which is pretty easy through Chocolatey.

> choco install gitversion.portable

This allows you to run it from the command-line. You can also hook it into your build system by copying the single executable into source control or adding it as a NuGet package. It will detect build engines like TeamCity and AppVeyor and adopt its output. Now let's assume you have a new Git project with a first commit on the master branch:

> mkdir GitDemo
> git init
Initialized empty Git repository in D:/Tmp/GitDemo/.git/

> echo "hello" demo.txt
> git add .
> git commit -m "My first commit"
[master (root-commit) 2a34238] First commit
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 demo.txt

Now run the GitVersion command-line tool

> gitversion

This will result in the following JSON output:


So, by default your version numbering starts with 0.1.0. Notice the many variables targeted to specific uses. For instance, we store the InformationalVersion that includes the hash of the commit in the AssemblyInformationalVersion attribute of the assembly. Similarly, we use the NuGetVersion for our NuGet packages. Finally, our build numbers are mapped to the FullSemVer variable. That last part is pretty important, because traditional build numbers don't say much about the actual changes. With Git versioning, rebuilding a particular commit renders the exact same number. This creates a whole lot more tracability.

Let's follow Gitflow and continue development on the develop branch and run GitVersion.

> git checkout -b develop
> gitversion

Ignoring the remainder of the variables for now, this is the result:


You'll notice two things. First, the minor version number is automatically updated. Second, the version number is post-fixed to make it pretty clear your working a development branch. Now let's add a couple of commits and see what happens.

> echo "test" > demo2.txt
> git add .
> git commit -m "another commit"
> gitversion

This will result in the last digit representing the number of commits since we branched of from master.


So let's work on a little feature through a feature branch.

> git checkout -b feature1
> echo "test2" > demo2.txt
> git add .
> git commit -m "another commit"

Running Gitversion again will give you:


And again it's crystal clear what version you're working on. To simulate working on some feature, I'll assume a couple of more commits without repeating myself here. Running GitVersion after those changes results in:


Now it's time to integrate those changes back into the develop branch.

> git checkout develop
Switched to branch 'develop'

> git merge feature1
Updating eb79902..8c374d1
 demo.txt  | Bin 0 -> 16 bytes
 demo3.txt | Bin 0 -> 14 bytes
 demo4.txt | Bin 0 -> 14 bytes
 3 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 demo3.txt
 create mode 100644 demo4.txt

> gitversion

So, whatever you do, the version is always in sync with the changes without any manual tasks.

Now suppose principal development has completed and it's time to stabilize the code base. Let's make that explicit by starting a release branch.

> git checkout -b release-1.0.0
Switched to a new branch 'release-1.0.0'

> gitversion

So it seems release branches are not for shipping release versions. Instead, they are used to ship beta versions of your system. Again, the +0 is used to denote the number of commits since the last time you shipped a beta package. Consequently, if you do ship such a package, you're supposed to tag that commit with something like 1.0.0-beta.1. When you do, and you add any additional commits to that branch, the following happens.

> git tag 1.0.0-beta.1

> ..make some changes

> git commit -m "some more changes"
[release-1.0.0 76682ac] some more changes
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 demo5.txt

> gitversion

As long as you're stabalizing a system or shipping intermediate versions for acceptance testing or similar situations, stay on the release branch. By doing so, you can support both the upcoming version as well as the current production version. Now, when you're ready to go into production it's time to merge to master.

> git checkout master
Switched to branch 'master'

> git merge release-1.0.0
Updating 2a34238..76682ac
 demo.txt  | Bin 0 -> 16 bytes
 demo2.txt | Bin 0 -> 14 bytes
 demo3.txt | Bin 0 -> 14 bytes
 demo4.txt | Bin 0 -> 14 bytes
 demo5.txt | Bin 0 -> 14 bytes
 5 files changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 demo2.txt
 create mode 100644 demo3.txt
 create mode 100644 demo4.txt
 create mode 100644 demo5.txt

What's important to remember is that you should tag the commit you ship, so in this case we're going to tag the merge result with the final release number. Let's see what GitVersion will do with that.

> git tag 1.0.0
> gitversion

This marks the end of the development cycle. Any hot-fixes should end up on master as well, and will have their last digit incremented, e.g. 1.0.1, 1.0.2. Any successive development should continue on develop again, so as a last step, let's switch back to that branch and see what happens.

> git checkout develop
Switched to branch 'develop'

> gitversion

Nice isn't it? GitVersion understands the concept of that tag on master and will assume you'll continue with the next minor version on the develop branch. If you really want to continue with a different version, there are ways to make that happen. And I've been just showing you the most common flow. I highly recommend checking out the examples in the GitVersion wiki.

In summary
In short, you just need to remember a couple of things:

  • Development happens on the develop branch
  • Stabilizing upcoming releases and shipping beta packages happens from release- branches
  • The master branch tracks production releases and hot-fixes.
  • Anytime you ship something, you must tag that commit. No need for tracking releases anywhere else.

So what happens if you don't have regular releases and your project needs to deliver continuously? Well, the guys at Github had the same questions and came up with an alternative to GitFlow called GitHubFlow. Fortunately, GitVersion supports this flow out-of-box as well.

So what branching strategy do you use? And how do you track your releases? Let me know by commenting below or tweeting me at @ddoomen.

Sunday, April 19, 2015

Prioritizing projects and tasks with minimum loss of time and money

Last week, I was so fortunate to attend a marvelous talk by Donald Reinertsen on a amazingly effective approach for prioritizing projects within an agile organization (and which is part of SAFe). Most managers will tend to prioritize their projects based on non-quantifiable attributes such as the strategic importance of a project or the risk of loosing a contract. But if you have a multitude of those projects, all equally important, how do you make a decision on what project should be done first? In his opinion, prioritization should happen on a principle named Weighted Shortest Job First. In this principle two figures are of upmost important:

  1. The duration of the project
  2. The cost of delaying that project

By dividing the former by the latter, you get the weighted value on which you prioritize. Consider for example three projects, A, B and C, all with the same cost of delay but with varying lengths. If you would put them on a graph like the picture below shows, it becomes pretty obvious which project should be done first. So although my first instinct would be to try to get that big project out of the way first, that's actually the least economical thing to do. And it doesn't matter whether you define your units in weeks, months, story points or cows, as long as you don't compare apples and oranges.


Now how do you deal with those strategic projects I started this post with? You still need a way to quantify those. One way is to try to estimate for how much money completing such a strategic project will enable other projects. Another possibility is to ask the involves stakeholder on how much money he or she is willing to pay for the advantage of starting this project first.

And what about projects you really can't predict the length of? Well, in that case, Donald recommends allocating fixed chunks of capacity to a project. When that chunk of capacity expires, you simply put that project back in your queue and reprioritize. This might work particularly well with technical innovation projects where you need a couple of those chunks to be able to reliably estimate the remainder of the work. And guess what you need to do when you have that information…

So, one particular question I tried to get an answer on is how this all applies to user story prioritization. Over the years, I have had many discussions on stories needed to bring down technical debt or architectural changes needed to handle non-functional requirements. Quite often, and despite of my efforts to convince the product owner of their value, I get overruled and other features get priority. By trying to estimate the cost of delay for those technical stories as well as their length and comparing those to the weighted values of the functional stories, you might make a better chance of convincing the product owner. You could even use this within the team to help determining whether taking a shortcut is worth the trouble to meet a deadline early. I haven't tried this myself, so I don't know if this would really work, but I'm sure going to try that soon. Anyway, if you want to learn more about the scientific aspects of this technique, I recommend reading the original article on the SAFe website.

So how do you do prioritize projects, tasks and technical debt? Let me know by commenting below or tweeting me at @ddoomen.

Monday, April 06, 2015

How to do a sprint demo without utterly boring the audience

Over the last year or two I attended numerous sprint demos and product demos hosted by development teams. Although I can definitely improve as a speaker myself, standing at the side-line and observing other speakers have made me pretty opinionated. I also get bored pretty easily and have a very short attention span, so the things I'll be saying might not be applicable to everybody. So with this disclaimer out of the way, let's see what you can do to prevent people from getting bored to dead….

Adapt to your audience

Most experienced presenters know that they have to think about the intended audience when they create the slides and presentation structure. But I barely see people adopt their presentation style, their vocabulary and slides they really show to the actual audience. Sprint demos tend to attract a variety of people, so be prepared to change what you were planning to demo based on that. Don't bore that product owner or COO with technical terms or the number of story points you've finished. Just skip those slides if you feel nobody cares about them.

Don't read from your slides

The best thing you can do to move away any attention from you is to put lots of text on your slides and read from it. But if you want people to listen to you, keep them as empty as possible. Ideally, you don't have slides at all (if you've ever attended a Dan North session, you'll know how powerful that can be). I need my slides to keep some structure in my talks, so I prefer to use just keywords and tell my story around them.

Don't try to impress the audience with loads of little things

Pick a couple of features that you think will interest the actual audience or are interesting for the people represented by the audience. Don't demo or name all those little features nobody really cares about. Just invite them to join you after the meeting or visit you at your work place instead.

Cut of any discussions not relevant for the rest of the audience

In every demo, you'll find somebody that likes to profile himself by asking lots of questions and making statements about what you could or should have done instead. Unless you really believe the rest of audience is interested as well, cut of that guy gracefully by proposing to discuss his point after the demo. The reverse is also pretty common; the guy doing the demo taking the opportunity of a question to show off how much he knows or how difficult it has been been to complete a certain feature. Although some of that might include a couple of valuable lessons, don't forget that the audience is there to learn something, not to see how great you are.

Prepare your demo environment

If you need to demo an app on a mobile device or a tablet install some kind of screen sharing functionality, so you don't have to ask hordes of people to look at a little device. Likewise, if you need to use a remote desktop session to demo something, make sure you create shortcuts that automatically connect and logon using the correct credentials. Don't let people wait until you've found the right machine and have them see you filling in the credentials wrong a couple of times. If you demo something using an IDE such as Visual Studio, hide all unnecessary toolbars, tool windows, etc. to allow people to focus on one thing. Oh, and please hide your start menu as well.

Don't touch that freaking mouse

If I had to name the single most annoying thing I see people doing is moving the mouse using the laptop's track pad. No, scratch that. The most annoying thing is people moving the mouse in an erratic pattern, revealing the presenters lack of preparation or their nervousness. So, unless you really need to show the audience what you're doing, remember keyboard shortcuts and ditch that mouse. But….if you really need to use that mouse pointer, bring a real external mouse. And never ever use a trackpad…..ever….

Avoid the sound of silence

If I've ever experienced a cringe-making moment while attending a talk, it must be that moment where the speaker ran into an issue with his demo and the audience of 70 had to go through 5 minutes of total silence. I felt so ashamed for the guy, I had to restrain myself from breaking the silence with some (bad) jokes. Having a technical problem or forgetting what you were going to tell now is something that happens to everybody. Just don't make it too big of a deal and avoid the silence. Explain what you are doing or make a joke about it. Humor is in my opinion the best you can do at that point, especially if it's self-effacing.

Size does matter

Sure, that tool or IDE looks cool on a HD or HD-ready projector, but don't expect people to be able to read those tiny lines of code. So please crank up the font size to 13 or 14 if you want people to be able to read your code. The same goes for your slide deck. If you need a small font size to get all that content on your slide, you must have violated tip 2. Oh, and don't forget to be prepared to change your color scheme. Although your slides may shine on your expensive display monitor, they may get unreadable on a projector. I've made this mistake myself a couple of times, especially when the company I was visiting thought projecting on a wall was a good idea…

Don't bother people with your personal life

No, I'm talking about those cute pictures of your kids… That might actually work to connect to your audience. No, what I am referring to is all those notifications that appear while you do your talk. Do you really want to let the audience see that email your wife just send about her 'plans' for tonight's date while the kids are out of the house? Or worse (but less likely in a sprint demo), having somebody you follow on Twitter tweeting about the worst sprint demo ever. So enable Presentation Mode or whatever it is called on your platform to prevent any notification for an hour or so. As a nice side-effect it will also prevent the screen from going into your screensaver.

Delegate note taking

The most unprofessional thing to do is ask people for feedback during a sprint demo or discover problems in whatever you're showing and have those same issues or questions appear in the next demo. So make sure you ask somebody from your team to actively take notes about product issues, unanswered questions or anything that needs improving the next time you do your demo.

Now let's hope I don't violate any of these myself the next time I do a demo or talk….

So what do you do to make your sprint demos shine? Let me know by commenting below or tweeting me at @ddoomen.

Wednesday, April 01, 2015

Getting your teams to communicate effectively

Lets be honest here. For a very long time, if I had to choose between an averagely skilled but well-spoken developer and a very skilled and experienced introvert, I would probably choose the first. And yes, I do realize that these are two extremes and that most people have characteristics of both. So please don’t flame me for saying this. Nonetheless, I (still) believe that multi-disciplinary agile teams need to foster open communication. In such a setting, you just can't work in isolation that long. XP practices like pair programming and swarming on stories is the common thing people do.

Consequently, having somebody in the team that has difficulty expressing him or herself in Dutch or English can be considered as a impediment. But before I'm going to talk in solutions, let me share some of the symptoms I've observed first hand.

Communication Impediments

I think many experienced scrum masters will recognize this, but when I was still part of a Scrum team, we regularly had retrospective and sprint demos. Somehow, the same people were either always speaking or always silent. Even tricks like brain storming using post-its and dot voting didn’t seem to give the result we were hoping for. Some of the more quieter persons simply didn’t participate. If you would be really lucky, those guys would send you an email with their feedback afterwards.

Another issue I've seen happening in multiple teams is developers shutting themselves off by using headphones. I don't really mind headphones and I understand that some people simply don't switch their attention that easily. But in my experience, the people that do tend to wear headphones, are also the people that are least involved in team discussions. I've noticed several situations in which some team members were discussing a certain topic in an ad-hoc fashion with the guy wearing headphones completely oblivious of this. Sure, the guys in the discussion could have asked the headphones guy to get involved, but they might not think he or she would have anything to add or they simply don't want to disturb him or her. Either way, unless the headphones guy is very perceptive of what's happening around him, I would recommend against headphones at all.

I myself am very sensitive to people that don't share their progress pro-actively. It might be my tendency for over-understanding what's going on, but somehow I tend to have more trust issues with people working from home if they don't actively share what they are doing. In my opinion, it should not matter whether you're working from home or at the office. The difference is that your team can't tap you on your shoulder if they need you, so you need to compensate for that.

A final impediment that I've observed first-hand is that developers that have difficulty expressing themselves are often overruled by product owners, business people or more senior developers, even though their point is very valid. For the bigger part of my career I would treat people like that as irrelevant and kind of walked over them. I've even been reprimanded for that at some point. However, over time I've learned that giving them more room to make their point can result in much better solutions in the long run. And not only that, it will most definitely improve your professional relationship. However, not everybody is aware of this or has the patience to give the other guy some slack. Hey, even with this knowledge, I still have days I simply can't make myself patient enough. So in a way, I still see this as an impediment for effective collaboration.

Based on these experiences, you might be tempted to take the easy road and only select developers with great communication skills. But over the last year or so I've learned that some colleagues just need a different means of communication. It might require a bit of patience, but if do give them room enough, they might surprise you with great solutions, ideas and feedback. They sure managed to surprise me on more than one occasion. In retrospective, I was just plain wrong in my assumptions. In fact, my wrongness was the primary reason for writing this post in the first place.

Tips that can help

Now that I've cleared the air, let's see what you can do to improve on this.

  • Use a chat tool like Flowdock, Hipchat or Slack to have discussions about certain topics. You'll be surprised of how people change when they have time to put their thoughts together without the pressure of a face-fo-face discussion.
  • Invite people for certain tasks by email or Flowdock. Where they might not respond to a public invite, e.g. during a meeting or stand-up, they might turn out to be very eager if you approach them 'digitally'. Happened to us many times.
  • Try alternative techniques for retrospectives. Just doing a round table inquiry about the good and the bad isn’t very effective. I particularly like Sandi Mamoli’s Deep Tissue Massage Retro and the 5-Why Root Cause Analysis Retro.
  • Have one-to-one meetings before or after that team retrospective to make sure his or her feedback is listened too. Since I can be pretty direct sometimes, meeting up with individual colleagues in a private meeting helps create a better environment for a respectful conversion.
  • Set-up a pairing station outside the vicinity of the team. This allows two developers to work together and have discussions and ask each other questions without the whole team overhearing them. That might give an insecure developer just enough confidence to ask the right questions.
  • Try to create a couple of tasks for those developers that just don't work well with other developers so that you can still benefit from his or her unique skill set. I know that doesn't align well with multi-disciplinary teams swarming on stories, but sometimes thinking outside the box is worth the trouble.
  • Speak in a common language, even if the people involved in the discussion were not planning to include anybody else and speak the same language. They might not even realize the other guys have something valuable to add to the discussion. We have mixed Dutch and non-Dutch teams and we don't force people to learn Dutch. So in our case, I recommend teams to speak English at all times. If you don't, you'll never give the other guys a chance to join in on the topic.
  • When working from home, actively share progress, e.g. by email, but preferably through Flowdock, Slack or Hipchat. It will increase the trust levels within the team. If I look at myself, when I work from home, I start to over-share what I'm doing and make sure the threshold for getting in contact is as low as possible.

Those are plenty of examples that I’ve seen in action, so I hope this helps. But regardless of how you approach your team members, don't forget the fundamental principles of effective communication: safety, respect, ownership and intention.

So what do you do to improve the level of collaboration between team members and teams? Let me know by commenting below or tweeting me at @ddoomen.

Sunday, March 22, 2015

Bringing the power of PowerShell to your build scripts

If you look back at the last couple of years, you'll notice an increasing attention for best practices that should make us more professional software developers. We design our classes using Test Driven Development, we review our code in pairs, and we apply all kinds of architectural principles such as those represented by the S.O.L.I.D. acronym. In other words, we really care about our code. But what about our build scripts? Do we care as much about those as we do for our regular code? I doubt it, but shouldn't we?

About a year ago we switched from Microsoft Team Foundation Server's build environment to JetBrains TeamCity, mostly because my client was moving to Git and GitHub. After being used to the cryptic MSBuild XML documents (I never bothered with the Windows Workflow Foundation stuff), being able to use TeamCity's elaborate build step system seemed like an attractive approach. But with that, we almost made the mistake of treating build scripts as something exotic again.

Luckily, my (then new) colleague Damian Hickey convinced us to look at how the open-source community solves the build problem. Quite a lot of these projects use either a .bat file that directly invokes the msbuild executable or use PowerShell to do the same thing. As the teams didn't have any prior experience with PowerShell, being able to use TeamCity's graphical user interface sounded like the way to go.

Tying the build process to TeamCity has some drawbacks however.
  • You can't run a build outside TeamCity. Being able to fully test whether your local changes are working correctly becomes an invaluable capability that you really start to appreciate once you have it.
  • A codebase evolves over time, potentially involving changes in the way the code is build, tested and/or deployed. You wouldn't want to have separate build definitions for different branches, or worse, align the build definition with the changes in the code-base.
  • Being able to treat your build scripts as first-class citizens of your code base also enables the capabilities you are used to such as merging changes from different contributions, supporting pull requests, and analyzing the history of your script. Although TeamCity supports auditing, it'll be a completely different experience.
  • Although TeamCity is a state-of-the-art build engine, especially compared to Microsoft Team Build, being able to switch to another build product (such as AppVeyor) is a big advantage.

So putting your build definition in source control, next the code base it builds, was a big advantage for us. And though we didn't have much prior PowerShell experience, being able to use .NET classes provides a lot of flexibility. We decided to use an open-source PowerShell library called PSake (pronounced as sake) that combines the concepts of the old make build language with the power of PowerShell.

After unzipping the release and adding the files to your source control repository, a simple default.ps1 file might look like this:
task default -depends Compile task Compile -description "Compiles the solution" { exec { msbuild /v:m /p:Platform="Any CPU" TestApplication.sln /p:Configuration=Release /t:Rebuild } }

Notice that both msbuild and exec are wrapper functions provided by PSake. The first ensures that the right version of msbuild for the appropriate .NET Framework is used. The second ensures that the last exit code of a DOS command-line is converted in the correct PowerShell exception. You can run this build script using psake.ps1 or psake.cmd, depending on whether you're running it from a PowerShell console or a Command Prompt.

Since the default behavior is to run the default task in the default.ps1 script, you don't need to provide any parameters. If you want, you can specify an explicit task to run. Just run psake.cmd -help or psake.ps1 -help to get the very extensive help page.

Now suppose you want to parameterize the project's solution configuration (from release to debug). PSake supports script properties for which you can pass values as part of the Psake.cmd call.
properties { $Configuration = "Release" } task default -depends Compile task Compile { exec { msbuild /v:m /p:Platform="Any CPU" TestApplication.sln /p:Configuration=$Configuration /t:Rebuild } }

Now when you run the following it'll build the debug version of the solution.

.\psake.ps1 -properties @{"Configuration"="Debug")

In most projects, your build script will consist of multiple tasks each depending on other tasks. In the previous example, the default task depends on the Compile task. Obviously, much more elaborate dependencies are possible, including conditions.

properties { $PackageName = "" } task default -depends Compile, BuildNugetPackage task Compile { } task BuildNugetPackage -depends Compile -precondition { $PackageName -ne "" } { }

So in addition to verifying that the $PackageName variable must be specified as a property, you can also run the BuildNugetPackage task directly. But since it depends on the Compile step, the net result is the same as just running the default task. It's good practice to add the essential dependencies. BTW, if you want to make that package name property mandatory, you can write the task like this:

task BuildNugetPackage -depends Compile -requiredVariables $PackageName { }

Notice that running this build script from a build product is just going to route the PowerShell output as the build output. However, both TeamCity and AppVeyor have ways to enable more deep integration. For instance, TeamCity will recognize certain phrases in the output and use that to interpret the phase of the build or the severity of certain build problems:

task Compile { "##teamcity[blockOpened name='Compilation']" # compilation steps "##teamcity[blockClosed name='Compilation']" }

But since PSake will scan and load PowerShell modules (.psm1 files) from under the Modules folder, you can make this a lot more readable by importing the TeamCity extensions provided by the Psake Contrib project:
task Compile { TeamCity-Block "Compiling the solution" { } }

AppVeyor offers PowerShell extensions to interact with their build system as well. And don't underestimate the capability of using PowerShell modules here. This is what will allow you to treat your PSake scripts as real code including opportunities for refactoring, code reuse, accepting open-source contributions, etc.

Anyway, this post was in no way meant to be a comprehensive discussion on Psake. Instead, I recommend you to checkout out the many examples from the GitHub repository or look at the build script that I'm using to build Fluent Assertions from CodeBetter's TeamCity environment.

So how do you organize your build environment? Does what I'm proposing here make sense? Let me know by commenting below or tweeting me at @ddoomen.