Onboarding: Making the most of joining a new team
Joining a project can be tough. Some things are the same, but there is so much to learn: the domain of the application, the architecture, the environment (usually I am working on a client site), the names of all the people interacting with your team. Oh and maybe finding your way around the code is pretty important too!
Working for ThoughtWorks for a few years, plus some time spent prior to this on a few shortish contracts I have had to get skilled at getting skilled! I believe that a new team member should take a large part of the responsibility for onboarding themselves, rather than leaving all of the effort to the team itself. I present a few tips that help you get through this process with minimal fuss, confusion and broken bones.
Be patient. I don’t think you can hit the ground running on your first day on a project and be as super-productive as you were on the way out from your last project. There will be critical production issues that you won’t be able to help out with. There will be meetings and discussions you cannot reasonably contribute to. There will be code and new technologies you have never used before. This can be particularly frustrating but it’s important to give yourself time to settle in and to learn. In the meantime…
Exploit your newness. A new person on the team is a considerable asset and you should exploit that in your first few weeks and months. As project incumbents, we tend to get very accustomed to the code and the status quo - however much we try not to (and we really try not to!). A new team member can shake this up. I try and use my first month or so on a project to assess what I am seeing and how things work. If you have ideas and suggestions for improvement in what you see, that can be really useful but remember…
Don’t be too critical too early. Sometimes when you arrive on a new project, there may be things that do not make a lot of sense: a choice of tool, technology or practice that no-one appears to question but to you as an outsider seems substandard. It’s worth just hanging back on that criticism just for a couple of weeks. Ask your team mates for back story and try and get their opinions too. It’s alway worth digging around for the reasons first. Then you might be in the position to change things and make a positive contribution to the team.
Write things down. I have sometimes found it useful to write a journal (usually on the bus on the way home). This would generally include some brief descriptions of who I talked to, what they told me and my thoughts about what I was observing. Spending some time at the end of the day fitting pieces of information together and writing it in your own words is a great way of remembering the vast quantity of information you will pick up in those initial weeks.
Spread yourself around. On some teams, the responsibility of onboarding new team members will fall to its lead. This is generally good as they are most likely the ones with the best overall knowledge. However, it is worth trying to spend time with all of your team (perhaps through pairing if that’s what you are about). The benefit is that you take the pressure of your lead who may be pretty busy, and you get to know the project from a few different perspectives. The last new person to join the team, particularly if they joined fairly recently, can also be a good source of information as he/she is likely to be going through the same exercise as you.
Learn T-shaped. I am on a fairly large and long running project. Over time, the domain and architecture have gotten pretty big and complex. I could never hope to understand it all at once. So my strategy was to try and figure out the big picture and understand one vertical slice at a time (hence the T-shape). In my first week in the project, our lead gave me a great overview of the architecture, pen and whiteboard style. He covered the high level blocks of the architecture: applications, integration points, persistence etc. and then ran through some of the key concepts of our domain model. It was just enough information for me to make a start pairing with my fellow developers - admittedly with them doing most of the driving. I would use the down time (pre commit builds mainly) to ask my pair questions about what we were looking at and take another look at the code. Gradually these T-shapes started to get wider and overlap.
Get great at reading code. The ability to read and understand even the nastiest bits of code is a skill that benefits beyond onboarding. But in the initial stages of a project, it’s yet another great information source alongside your colleagues, whiteboards and any official documentation. Chances are it’s better than the official documentation.
From the other side
Although this article mainly focuses on the experience of the onboardee, there is obviously some effort required on the part of the onboarder. Most importantly, you need to give them time to get up to speed. It goes without saying that this needs to be accommodated in any kind of capacity planning you might do.
Find time to explain the important parts of the application but avoid over-long ‘induction’ routines. Ideally, give them the briefest of introductions to the architecture and domain and get them started working with the team immediately. Go into more depth once they have had a chance to get their hands dirty.
You should be approachable when the new team member has questions but try and avoid explaining everything. I always find it much more valuable to be shown how to find something out rather than be told precisely the information I need.
If the new recruit is following the advice I laid out above then they will be doing most of the hard work in figuring things out, but there is a responsibility on the incumbents to tell the story of the project - the history of the team, why certain decisions were made and the battles that were won and lost.
Why should we bring on new team members?
Onboarding is a time-consuming and sometimes difficult activity. It might take many months before the new team member is completely up to speed.
But the benefits should be considered. Building a diverse team of people is a great way of exposing the project to a wide ranging set of viewpoints, contexts and methods. Just as diversity of background and perspective brings valuable dimensions to a team, so can diversity in project experience. Just as much as you need the old guard who have been around a while and know how things work and why decisions were made like that in the first place, having new people join your team with a fresh perspective can give an influx of new ideas and experience that the team might not have experienced before. This will ultimately be of great benefit to the team and project alike.
clojure/conj 2011
clojure/conj 2011 was announced around about when I was preparing to move to Chicago and right after 3 full on days of clojure training. A combination of being massively enthusiastic about clojure and my relocation to the same country as the conj caused me to sign up as soon as I arrived.
The three days were incredibly intense but also incredibly enjoyable. I have come away having learnt a great deal - from the sessions and from the conversations that took place afterwards.
The following is an assortment of the highlights of the conference for me in no particular order.
Single track
Among the people I talked to this was widely recognised as a major asset. We’ve all been to conferences where we have made the wrong choice and missed something really interesting. Having a single track meant we all saw the same presentations and made the conference feel a great deal more intimate.
Variety of talks
For a conference focussing on a single language, there was a wide variety in subject matter. We covered a variety of topics - logic, logging, music, map/reduce, data structures, performance - in addition to deep looks at the language itself. I would say I managed to follow about 75% of the talks.
Logic programming and some of the more advanced talk about functional data structures was so new to me that I found it tough to follow. But importantly it’s made me want to know more and to understand these things. I don’t have a particularly academic background so I am coming into much of this stuff pretty cold, but what I really like about clojure and the community around it is it makes me feel comfortable and confident in learning. I mean, one day I actually might understand what monads are…
Real world usages of clojure
There were a handful of more practical applications of clojure too. I am sure I have missed a load of them out, but here are the ones I can remember:
Heroku’s treatment of logs as data described how they rethought the concept of logging to be concerned with the emitting and handling of events with associated data rather than being strings dumped to a file. Their use of pulse to emit events from rails applications and logplex to allow different consumers to subscrbe to these events realtime is a pattern I had never really considered for application events/log messages.
David McNeil spoke on how revelytix use concurrent stream processing helped them parallelize querys over multiple databases. They took inspiration from the way in which unix commands work: asynchronously pushing/pulling data with pipes and built a similar model in clojure. This allowed them to express trees of nodes and pipelines representing how they wanted the data from multiple sources to be selected, filtered and aggregated. This tree is then fed into a fork/join threadpool for execution.
The code is not something that they have made available (yet) and I wonder how similar their approach is to something like Aleph - David did mention that they started by using Lamina on which Aleph is based.
Nathan Marz gave an excellent talk on Cascalog (slides). Cascalog is a clojure library, built on top of the Java library Cascading, allowing processing and querying of big data in hadoop. The power and flexibility of clojure allows cascalog to be great abstraction layer on top of hadoop map/reduce. This presentation was very code-sample heavy, which brings me to the next highlight:
Code!
For a conference dedicated to a language, it is not suprising that we saw a significant amount of code samples and live demos. It made a great deal of sense for example in Anthony Grime’s Clojail presentation to demonstrate how clojail provides a sandbox for clojure execution. I particularly appreciated how he used rectangles to highlight individual parts of the screen as he explained what was happening.
Music
Sam Aaron’s Overtone talk was one of the most enjoyable of the week. Overtone is a clojure library written to interface with Supercollider: a client/server style audio synthesis platform. I had looked at Overtone before the conference (and had my ears nearly blown off by an excessively amplified sinewave!), but I was not aware of some of the more clever things it does under the hood. It was great to hear Sam’s explanations of the drivers behind Overtone: the desire to have a system of notation/language for electronic music in the same way as we have sheet music. I am glad they saved this talk until the end as it meant he could continue showing us demos of music written using overtone, using external input devices as well as creating music from the REPL. I hope the video of this become available soon as it is well worth watching. He got a standing ovation which was definitely well deserved.
Other links
During the conference, Phil Potter spent a lot of time hacking on Overtone and using the continuation monad to create loops and sequences. Last I heard, it was a little Radiohead, a little Aphex Twin with some added theremin on top.
The rest of the slides are available on Github.
QCon 2011 - Friday session - Node.js: Asynchronous I/O for Fun and Profit
Stefan Tilkov started this talk by discussing the difference between thread and event based I/O. I get the feeling that this might have been covered in the previous talk (or just old hat to most people) but I really appreciated the explanation as now I think I finally understand the difference between the two models. Let’s see shall we…
Most commonly used web servers - like Apache, IIS etc. use the thread based model. Each request is served by a dedicated thread. A request for a resource might look something like this:
The thread spends much more time sitting around waiting to read/write data and waiting for data from other collaborators (Database, File system, Client) than it does actually processing the request. While waiting for external events to happen, the thread blocks and does not do much apart from use a lot of memory. The number of threads you are going to need is going to scale linearly with the amount of concurrent requests you have (this is correct?), eating memory up and leading to thread starvation under a high load. The problem will be exacerbated by an upstream connection experiencing performance problems (like a dodgy database server): more threads will be hanging around doing nothing waiting for the upstream service to finish meaning there are less threads hanging around to serve new requests and thread starvation gets quicker.
Evented I/O based servers on the other hand will deal with I/O activities asynchronously and avoid blocking on I/O operations alltogether. They will make use of events and callbacks to pick up the processing work again (on whatever thread it happens to be invoked on). So if in our thread based model a thread blocks while waiting for a client request to be read from the socket, in the Evented I/O model the thread will call the corresponding async socket read function, passing in a callback to be called when the async operation is complete. As it does not have to wait for the I/O to complete, the calling thread can go off and do other things - like picking up the next request. Infact, at the heart of such a server you would probably see an event loop similar to GUI applications which will just cycle round waiting for incoming events.
The nginx webserver operates in an evented I/O model and Stefan showed some interesting comparisons of its performance with Apache (nginx_performance)
The evented I/O model is traditionally regarded as a low level and complicated pattern. There is support for it in most standard platforms (.NET IO completion ports and java.nio for example) but it is not widely used. Thread based is still very much the default.
JavaScript
Stefan then went on to talk about everyone’s favourite browser scripting language. Javascript has gone from being reguarded as a ‘toy’ language with some strange design features and browser compatibility issues to one with great framework support, ironing out dom compatibility issues. With things like the Crockford ‘Good Parts’ book being widely read, we as an industry have a good understanding of the language itself - as well as a good knowledge of the ‘bad’ parts. All fairly well-covered stuff but it lead him nicely to:
Evented I/O + JavaScript => Node.js
So we come on to Node.js - a framework for writing Evented I/O in JavaScript. Node.js runs on the V8 engine, free from the browser so you can use it to write performant networking programs - like web servers etc. As Stefan put it: ‘High-performance network runtime, using JavaScript as a high-level DSL’.
Carrying on with the car based puns, under the hood:
image_2
Other interesting parts are the low level C libraries:
- libev: event loop library
- libeio:async I/O
- c_ares:async DNS
- http_parser:superfast http request/response parser written especialy for node.js by its creator Ryan Dahl
From the code samples, it seems that getting started with basic programs is pretty easy:
var net = require("net");
var server = net.createServer(function (socket) {
socket.write("Echo server\r\n");
socket.setEncoding("ascii");;
socket.on("data", function(data) {
socket.write(data.toUpperCase());
});
});
server.listen(8124, "127.0.0.1");
This listing sets up a server using the net library. When an incoming socket is accepted, it binds to the data receive event and simply writes back to the socket the data it receives uppercase. Both I/O parts of this program - accepting an inbound connection and waiting to receive data on a socket - are handled asynchronously by passing in event handlers.
Stefan went through a few more simple code samples which he has put up on github: Stefan Tilkov node-samples.
Spaghetti code
One drawback of working in an asynchronous/callback world is that it tends to lead to tangled, spaghetti code. If you wanted to call two operations in a synchrnonous way, it might look like this:
jump_to_the_left();
step_to_the_right();
You can call each method procedurally in the order you desire. In node.js it might end up looking like this (presuming these were I/O operations):
jump_to_the_left(function(){
step_to_the_right();
});
Which presumably if it gets to any kind of complicated, the code would start to resemble:
jump_to_the_left(function(){
step_to_the_right(function(){
put_your_hands_on_your_hips(function(){
bring_your_knees_in_tight();
});
});
});
Then if you add in error handling to the mix, synchronously:
try{
jump_to_the_left();
step_to_the_right();
}
catch{
log("Failed to execute timewarp sequence");
}
… which wouldn’t work asynchronously, as the try/catch would not be surrounding the point at which the handlers are executed. A convention has been adopted to pass in the error status as the first parameter to a callback, leading to:
jump_to_the_left(function(err){
if(err){
log("Failed to jump to the left");
}
step_to_the_right();
});
It’s not hard to see that any program above the level of ‘trivial’ is going to get very complex and unreadable.
Libraries for node
Fortunately you can mitigate the complexity of working with evented I/O in node - as well as perform a number of other standard operations - using the multitude of libraries available for node. See this list of node libraries. These can be managed using its very own package manager npm. As node is primarily intended for web/http server usage, there are many libraries to deal with common HTTP/web tasks: templates, routing as well as libraries allowing you to run multiple nodes to explore multiple processors. You can even get an in-browser debugger: node-inspector.
Step is a particularly interesting library that attempts to smooth out some of the dodgy spaghetti code that arises from working asynchronously.
Stefan also mentioned that alongside all these modules there is also an active community around node.js. The node.js community are quite chummy with the NoSQL community too, so apparently you can generally expect good library support for your NoSQL technology of choice.
Summary
In conclusion, Stefan stated that node.js “Popularises the ‘right way’ of network programming.” and that it is easy to get started with and also fun to use (profitable too?). In the Q&A, it was interesting to note that he had more experience using node.js as a client rather than a server (I don’t think we write web servers every day): he might use node.js to quickly knock up a test client or utility program. I can really see the value in using node.js this way - as a quick and productive way of writing networking applications.
This was one of the most enjoyable talks I attended at qcon. Stefan is an engaging and laidback presenter and I particularly appreciated his explanation through code samples. I attended the talk to understand more about node.js (which was just defined in my head as ‘javascript-on-the-server-whats-the-point-of-that’) and left with a good understanding and eager to use it when I get a chance.
Static methods: feel the fear and do it anyway
When your tests go green, it’s often a good idea to think about tidying up the internals of the method you just added/changed. One of my favourite things to do here is to extract out meaningful methods from the implementation e.g
if(toolkit.Contains(Tools.Hammer))
{
var hammer = toolkit[x.Tool.Hammer];
Day.Morning.Do(hammer.Execute);
Day.Evening.Do(hammer.Execute);
ThisLand.AllOver.Do(hammer.Execute);
}
… At the click of an alt-shift-m becomes : i
if(IHadAHammer())
{
IdHammerInThe(Day.Morning);
IdHammerInThe(Day.Evening);
AllOverThisLand(); // OK I took that one a bit too far.
}
Very often, extracting methods will leave you with a helpful suggestion from our friend resharper: “This method could be made static”. I have often noticed a reticence from fellow developers for following this advice. Static methods tend to be considered harmful and seem to take us back to the bad old days of global methods (not to mention global state…). Added to that, the performance improvement you get is negligible and might smell like unnecessary optimisation. What tends to happen is that ReSharper is ignored (or the message is disabled) and we continue with something else. Thisc course of action does rather miss the point: the method makes no use of ‘this’. So even if you don’t put the static keyword in front of it, the method is still basically static anyway. Does that not tell you that maybe that method might not belong here? But where should it go? This is where the ‘do it anyway’ part comes in. Go make the method static. If there are several similar methods (and there usually are), go make all of them static and then look for patterns:
- A set of functions that take in a type you control:
public bool static HasExpiredPassword(User user) {...}
public bool static HasPermissionToEditResource(User user, Resource resource) {...}
Perhaps this reasoning could be part of the User class as instance methods.
- Methods which take in a system type as the first parameter, all with the same/similar name, for example:
public static bool IsValidEmailAddress(string emailAddress){...}
public static string GetEmailHostName(string emailAddress )
Maybe email address warrants a class of it’s own containing these methods. The EmailAddress class could end up being a magnet for many other similar methods dotted around other classes too.
- Either of the above but taking in a collection/IEnumerable of the same, e.g. :
static MailAddressCollection BuildSenderList(IEnumerable emailAddresses)
static IEnumerable GetAllUsersInGroup(IEnumerable users, Group group)
Here, extension methods are your friend for a quick fix (ReSharper
can totally help you out here). Chances are that if 'User' is part
of your domain model, then 'Users' should be too. So why not create
a type representing the collection - Users, EmailAddresses - and
migrate the static methods there.
Simply by following our refactoring tool’s suggestion, we have made the code more expressive and readable. The design of our code has fallen into place without us having to think too hard (a very good thing).
So what if you can’t find a way to move around those static methods? I would suggest that you should still go static to acknowledge that the methods are slightly disjunct from the class. Maybe as your codebase grows, you can revisit and find a place for these new concepts.
Google letter template
I was hunting for a decent letter template on google docs and could not for the life of me find one. I like to have the recipient address on the left and the sender on the right rather than all on one line. There is a lot of weird stuff in the public templates site! Anyway, here is the template:
… Enjoy
Spark codec for OpenRasta
Update
I haven’t really had the chance to support this too much due to various reasons (new job, laptop with vmware image dying, supreme laziness and attention span issues). Anyway, the project is now on github and (thanks to some work from Lee Henson) is the most up to date: http://github.com/jennifersmith/openrasta-sparkcodec Hope that helps you all out and thanks for your interest!
It all started after I went to Sebastian Lambla’s talk on OpenRasta at a VBUG event. I really got into the framework - it felt to me as a cleaner and lighter way to think about MVC when compared to ASP.NET MVC. At the same time, I really wanted to have look at the Spark View Engine as I think the syntax is a breath of fresh air and welcome relief from all the tagsoup we seem to get into when writing MVC views. When I asked ‘Is there a Spark codec for OpenRasta?’ I had the reply ‘No, but why don’t you write one’. Never one to back down from a challenge I did. And you can download it from GoogleCode (though I would prefer you grab the trunk cos the latest version is probably already out of date!).
OpenRasta meet Spark, Spark meet OpenRasta
If you are not familiar with either of these projects, then this project probably means nothing to you. If you get one and not the other then read on.
OpenRasta ‘is a Resource-Oriented framework to build MVC-style applications on asp.net 2 and above.’ (I copied this from here). Easiest way to get into it I think is to either hear a talk on it, or use this tutorial.
The Spark View Engine is written for ASP.NET MVC and provides a slightly alternative syntax for specifying views. Basically it uses an extended set of attributes and tags on top of plain old HTML to let you do crazy things like:
<viewdata products="IEnumerable[[Product]]"/>
<ul if="products.Any()">
<li each="var p in products">${p.Name}</li>
</ul>
<else>
<p>No products available</p>
</else>
Simple, readable and pretty cool I think, when your alternative is a big old mess of tagsoup. Out of all the alternatives, it is definitely the clearest and most logical I have seen so far. Any web-designers/front-end devs out there want to tell me if they agree?
Using the Codec from OpenRasta
In a later post I might choose to talk about my design approach, but when it came to OpenRasta, this approach was very much ‘Copy what the webforms codec does’. So to use the spark view engine, you start by adding this to the top of your configuration block:
ResourceSpace.Uses.SparkCodec();
This does some behind the scenes magic to register all the stuff you need to use the SparkCodec. Next, for the resources you want to render using spark, you configure along the lines of:
ResourceSpace.Uses.SparkCodec();
ResourceSpace.Has.ResourcesOfType()
.AtUri("/shoppinglist")
.HandledBy()
.AndRenderedBySpark("shoppingList.spark");
… note the ‘AndRenderedBySpark’ extension method there which passes in the name of the spark template you are going to use. The Spark lookup procedure takes in a root folder as part of its configuration and I have defaulted this to ‘Views’. So ShoppingList.spark must exist in the folder views/. There you are - you are all ready to start getting Sparky with your OpenRasta. Note that OpenRasta allows you to mix and match your codecs - so you can just have one or two views rendered using Spark if that is what you want.
Extensions to the Spark syntax
One of the things I liked about OpenRasta was some of the markup extensions that it contains. I.e. you can do something like:
Xhtml.TextBox(()=>MyResource.Name)
And through the magic of expressions, the output html is something like:
<input type="text" name="MyResourceTypeName.Name" value="Whatever the current value of the name is"/>
The base view class from which all spark views inherit (SparkResourceView) exposes the Xhtml interface so you can just use the extension method as is. However, I thought that seeing as the point of the Spark syntax is to hook tidily into html, I wanted to provide an alternative. So I have used the extensions facility in the Spark View Engine (which was simple once I got the hang of it), to add some custom attributes. Now if your view contains the following:
<viewdata resource="Customer"/>
<input for="resource.Name" type="text" anotherattribute="somethingelse"/>
The output becomes:
<input name="Customer.Name" value="Fred" type="text" anotherattribute="somethingelse"/>
Similarly to hook into the URI-resolving stuff in OpenRasta you can have something like:
<viewdata resource="Customer">
<a to="resource">Click here to view the customer</a>
<a totype="IEnumerable<Customer>">Click here to view all customers</a>
On render, this figures out the URIs based on what you set up for the resources in question as part of the configuration. Anyway, this syntax is far from complete - it works enough to just about power the demo application that I packaged with the codec source so I suggest you take a look at this for more pointers.
Futures
This started as a small pet project but if you want to give me a hand please please do catch up with me on twitter. The same goes for any other feedback (apart from ‘it’s crap’ - that would not be very nice). I have a small idea for an application that I wanted to have a go at with OpenRasta and as I get into implementing this I am sure I will come up with a few more extensions to the syntax. Now it is far too hot and I am going out to get an ice lolly.
Extension method assertions versus standard assertions
I have been playing around lately with assertions in my tests and figuring out the best (funnest?) way to work with them. Within the context of BDD style testing there seems to have emerged a style for using extension methods for asserts. As I found out at the SkillsMatter NBehave talk, NBehave comes with a set of extension method wrapped assertion methods so I guess that is where the link comes from perhaps.
What do I mean by extension method assertions?
OK so a normal vanilla unit test might check that the conditions of the test are met as follows:
Assert.IsEqual(foo, bar);
Assert.IsNotNull(foobar123);
Or if you have been bitten by the fluent bug:
Assert.That(foo, Is.EqualTo(bar));
Assert.That(foobar123, Is.NotNull());
Anyway, if you use the extension methods approach you will typically have defined somewhere something like:
public static class AssertionExtensions
{
public static T ShouldBe<T>(this object val)
{
Assert.That(val, Is.InstanceOf(typeof (T)));
return (T) val;
}
public static void ShouldBeNull(this Object val)
{
Assert.That(val, Is.Null);
}
}
Leaving your asserts looking something like:
foo.ShouldEqual(bar);
foobar123.IsNotNull();
You could even read that back and it might make some sort of sense: “Foo shoud equal bar”. “Foobar123 should not be null”. As far as C# goes, that is pretty damn readable (I see you Ruby-heads smirking at the back!).
Taking it a bit further
I have started to get into BDD style testing for project number one (well, the parts that aint smart-ui anyway), and I have got slightly carried away with my extension method asserts. As each test is testing a different outcome of the behaviour under test, I think the best tests only have one line of code in them. Something like:
[TestFixture]
public When_user_sends_article_to_a_friend
{
....
[Test]
public void Email_is_sent_to_the_friends_address_via_email_service()
{
EmailService
.EmailWasSent()
.To("Fred@flintstone.com");
}
}
(For the record my assertions have never got to more than one method, but it would be possible right!). Arguably you could just say this is a bit of syntactic sugar, but I think it hides the complexity of what it means to satisfy these requirements… which I believe is A Good Thing. At first glance, I should be able to read this code out loud and have half a chance at figuring out the intentions of what was supposed to be tested. I don’t need to worry whether EmailService is a stub/mock/stubbed-mock(!) - and indeed if it starts life as a mock then progresses to a concrete, this test doesn’t need to change.
It’s not for BAs
One argument I have heard for extension-method assertions is that it ‘makes it readable for business analysts’. Now, if your business analyst happens to be a former programmer, you might be on to something. However, that is not usually the case, our BAs can’t understand it and such a claim rather detracts from the usefulness of these methods. Really what we should be thinking about is the how readable extension method assertions are for us! If someone asks me ‘what does this function do’ and I wrote it about 10 days ago (this is about the maximum time I can remember anything about something I have created), I can find out easily and quickly. Similarly, the poor dev who comes after me could probably have a go at comprehending it too.