QCon New York Day 1: Micro-services, Team Latency and the Lean Enterprise

So now that I've got the unofficial part of this trip behind me and I managed to get through a very in-depth full day workshop on reactive extensions in JavaScript with my limited JavaScript experience, it is time for the main part of the conference. As is customary at QCon, each day always starts with a pretty boring introduction of all tracks by the track owners. I'm honestly wondering why they force those people to get on stage for such a large group. They are obviously very uncomfortable with it. Even the guy that opens up the day sounds like he doesn't actually want to talk about QCon.

Dart

Anyway, Gilad Bracha, Google employee and author of Newspeak brought us up-to-speed on web programming in the future. He promised not to make it too much of a marketing talk. But after several (not so succesful) demos on Dart he horribly failed on that promise. Why not invite Anders Hejlsberg and let him talk about TypeScript instead. That is a speaker with excellent speaking skills, has charisma and can tell a story.

Flight

In my current project we have historically used KnockoutJs, but we are in process of switching to AngularJS for the newer features. Twitter has been sharing their internal framework called Flight, whereas Facebook is actively pushing React, a JavaScript framework based on the concept of a virtual-DOM. To get a better understanding of the first, I decided to go to Kassandra Perch's (unexpectingly short) introduction. Well, either I didn't pay enough attention, or my JavaScript experience really is hampering me, but I totally didn't get what she was doing. I might be wrong, but those JS classes looked like she was coupling the DOM and the UI behavior within the same code. Call me a purist, but avoiding this is what EmberJS, AngularJS and KnockoutJs have been doing all along.

Oh, and one of the guys in the audience asked her how to select the appropriate framework for your project. She proposed to create a nice table and do a feature comparison of the various frameworks in the context of your project. I think that is stupid. You're not going to understand a framework's limitations if you don't actually use it in a real-life scenario. Just do a spike and see if you can rebuild an existing page using the framework of your choice.

Accidental architecture

Historically, the Open Space concept has been one of the best aspects of a QCon conference, so I decided to attend one about architectures you always wanted to learn about. Its painful to see how many people are facing the same problems over and over again, and how few opportunities you have to share experiences. Somebody was proposing the topic of accidental architecture but every a short discussion she actually concluded that they didn't have a known architecture at all and hardly any process. Each developer took a story, wandered off and did their own thing. In my own experience, authority and rules are not going to work, so I proposed to using an open space within the organization to find out what problems the developers themselves were facing. If they see the problems themselves, they are more likely to try to fix it themselves as well.

Micro-services

I also talked with somebody who has like a 100 micro-services running in production and is suffering from operational issues and data consistency problems. They were trying to solve the event up conversion the same way as we do in our Event Sourcing architecture and threw in all kinds of enterprisy products to help them. As far as I can judge, they were far ahead of what I've heard at QCon. We had a nice discussion and we both concluded that their next challenge is to accept that it will be impossible to keep all the micro-services in sync in terms of message and API versioning and that they should invest heavily in pro-active signaling and continuous delivery of those services.

More micro-services

Talking about micro-services, I also attended a talk by Yoni Goldberg on how Gilt, a US fashion outlet site, decided to break their monolithic website into about 50 micro-services each owned by a separate team. They used Docker to ease deployment of those services on shared Amazon VMs. According to the speaker they were very successful in that migration, but were now facing the more operational challenges of supporting so many dependent services. Apparently he has taken the same conclusion that tooling for supporting that, as well as continuous deployment, are becoming more important by the year. He concluded the talk with the advice that if you consider moving to micro-services, just start building every new feature as a micro-service and add the required tooling along the way.

Anyway, I'm not yet convinced about this micro-services thing. It solves certain problems quite nicely and is very scalable, but it also introduces a lot of new challenges around API/message versioning, finding the right bounded context and operational support such as monitoring and pro-active signaling. For now, nobody has really solved this entirely.

Team latency

Edmund Jorgensen delivered a refreshingly fun session on the latency of teams and how bringing down latency is best thing an organization can do. So many managers are focusing on the short-term needs rather than understanding how expensive any form of latency is. Think of technical debt, manual deployment steps, or anything else that prevents a team for speeding up. His basic approach is that reducing latency creates information earlier and early information is worth a lot of money. In fact, it saves money, so why do manager still persist on spending all their capacity on building features rather than having a good balance between functionality and reducing latency. Somehow, during his talk I had this sense of a déjà vu, like I've had heard the exact same story before. And then I realized that he was actually the author of this excellent article on speeding up your engineering organizations. Go make sure you read that.

The Lean Enterprise

The final session of the day was delivered by Trevor Owens, the author of Lean Enterprise. He showed us that a lot of the current successful startups were founded by prior Google developers. So why did they leave and became so successful? Well, according to his findings, using equity instead of salary as an incentive really gets people to innovate while most organizations have a rather traditional reward system. He also used a lot of fun facts from the world of the venture capitalists and how they never bet on a single horse. In fact, their entire investment strategy is based on the approach that one of the investments is going to be paying the losses made by the others. So he advices organizations to have multiple teams work on innovation at the same time. Or even better, let them leave the company, do their innovations autonomously and then buy them back, just like Google did. I'm definitely going to read that book.

I also attended the evening keynote prior to the conference party where Peter Wang tried to get us to focus on the bigger problems like early innovators did rather than on what technology stack to use. He's a great speaker, but by then my brain was already starting to shut down. Attending in-depth QCon sessions from 9 until 19 is not for the faint of heart.