It can be found on Kindle or iBooks. 

Below are some screen-captures of passages I liked (sorry for the images as opposed to text):

Summary of the “3 Signs”

The book makes it clear that it doesn’t matter what the job is nor does higher compensation ultimately negate the need of these things. 

In the end, the manager is the person who has the biggest impact on a persons happiness in their job and the case is made as to the importance of a manager in the life of those he manages yet how most managers don’t realize this nor act on it – hence the misery of many in what would otherwise be fulfilling and satisfying jobs.

For me “immeasurement” is the one that hits closest to home as being something that affects me in my day to day life and that I’ve done poorly when I’ve managed others. 

Software development (where I personally have experience and can apply these lessons) in general is difficult in this regard – projects and code are never really done. Sure, something is deployed or ships, but then it’s on to fixing things, adding the next feature or whatever, and rarely does an individual person have full impact on a product or system such that they can measure their success with it, especially on large teams or products. 

Timelines or task completion are impossible to use as yardsticks as they vary wildly and change all the time – the very definition of “moving goalposts”. Don’t even get me started on “lines of code” – the worst possible form of measurement. Bugs (or lack thereof) don’t work either for many reasons that I won’t get into here. 

I have yet to see an objective way of measuring software development that works – any ideas?

Solitude and Leadership: an article by William Deresiewicz | The American Scholar

“solitude is one of the most important necessities of true leadership”

“what I saw around me were great kids who had been trained to be world-class hoop jumpers. Any goal you set them, they could achieve. Any test you gave them, they could pass with flying colors. They were, as one of them put it herself, ‘excellent sheep.’”

“excellence isn’t usually what gets you up the greasy pole. What gets you up is a talent for maneuvering. … Not taking stupid risks like trying to change how things are done or question why they’re done. Just keeping the routine going.”

“We have a crisis of leadership in this country, in every institution.”

“We have a crisis of leadership in America because our overwhelming power and wealth, earned under earlier generations of leaders, made us complacent, and for too long we have been training leaders who only know how to keep the routine going. Who can answer questions, but don’t know how to ask them. Who can fulfill goals, but don’t know how to set them. Who think about how to get things done, but not whether they’re worth doing in the first place. What we have now are the greatest technocrats the world has ever seen, people who have been trained to be incredibly good at one specific thing, but who have no interest in anything beyond their area of exper tise. What we don’t have are leaders.

What we don’t have, in other words, are thinkers. People who can think for themselves. People who can formulate a new direction: for the country, for a corporation or a college, for the Army—a new way of doing things, a new way of looking at things. People, in other words, with vision.”

“there are a lot of highly educated people who don’t know how to think at all.”

“what makes him [General David Petraeus] a thinker—and a leader—is precisely that he is able to think things through for himself. And because he can, he has the confidence, the courage, to argue for his ideas even when they aren’t popular.”

“true leadership means being able to think for yourself and act on your convictions”

“people do not multitask effectively. And here’s the really surprising finding: the more people multitask, the worse they are, not just at other mental abilities, but at multitasking itself.”

“Multitasking, in short, is not only not thinking, it impairs your ability to think.”

“You do your best thinking by slowing down and concentrating.”

“Leadership means finding a new direction, not simply putting yourself at the front of the herd that’s heading toward the cliff.”

“Once the situation is upon you, it’s too late. You have to be prepared in advance. You need to know, already, who you are and what you believe: not what the Army believes, not what your peers believe (that may be exactly the problem), but what you believe. How can you know that unless you’ve taken counsel with yourself in solitude?”

Some favorite portions:

“A mess is a reckless debt which results in crippling interest payments or a long period of paying down the principal.”

“The prudent debt to reach a release may not be worth paying down if the interest payments are sufficiently small – such as if it were in a rarely touched part of the code-base.”

“Not just is there a difference between prudent and reckless debt, there’s also a difference between deliberate and inadvertent debt. The prudent debt example is deliberate because the team knows they are taking on a debt, and thus puts some thought as to whether the payoff for an earlier release is greater than the costs of paying it off. A team ignorant of design practices is taking on its reckless debt without even realizing how much hock it’s getting into.”

“while you’re programming, you are learning. It’s often the case that it can take a year of programming on a project before you understand what the best design approach should have been. Perhaps one should plan projects to spend a year building a system that you throw away and rebuild, but that’s a tricky plan to sell. Instead what you find is that the moment you realize what the design should have been, you also realize that you have an inadvertent debt.”

According to Wikipedia, “Tom DeMarco has authored over nine books and 100 papers on project management and software development” and has been intimately involved over the past 40 years in the software industry and the establishment of “structured analysis”, software project management concepts and legendary software development projects such as the first large scale telephone exchange for Bell.

He recently wrote an article titled “Software Engineering:  An Idea Whose Time Has Come and Gone?“

The article is well written and there is no value in me repeating it verbatim, so read the original.

The following quotes however are worth me repeating along with my commentary:

“Software development is inherently different from a natural science such as physics and its metrics are accordingly much less precise.”

Software development is not like building a bridge – where the materials, laws of physics and chemistry, labor times etc have all been experienced and practiced thousands of times, can be accurately predicted and correctly planned for, then managed.

“Software development is and always will be somewhat experimental. The actual software construction isn’t necessarily experimental, but its conception is. And this is where our focus ought to be. It’s where our focus always ought to have been.”

If we are writing software, it’s because we’re trying to do something that hasn’t been done before or is different in some way from what already exists, otherwise we’ll purchase the existing software. We are building a solution to a problem – to create value of some kind.

If we’re doing something that hasn’t been done before – either with new technologies, new rules, new performance criteria or whatever it is that makes it unique – then we are doing “research and development” – experimentation. It is not as if a blueprint for a house is given and the construction team comes in to build it as they’ve done 1000 other houses in the same community. Software “architecture” doesn’t work that way. There is never a replicate – because it’s not physical.

Yet business and project management would love it to be – it makes it easier to control, schedule and most critically, budget for.

Tom DeMarco discusses “metrics” in the most detail in the article, about why we spend so much time trying to control software development projects, and particularly the timeline/cost aspect.

He says:

“For the past 40 years, for example, we’ve tortured ourselves over our inability to finish a software project on time and on budget. But as I hinted earlier, this never should have been the supreme goal. The more important goal is transformation, creating software that changes the world or that transforms a company or how it does business.”

After giving an example of two projects, one creating little value and one creating significant value he states:

“This leads us to the odd conclusion that strict control is something that matters a lot on relatively useless projects and much less on useful projects. It suggests that the more you focus on control, the more likely you’re working on a project that’s striving to deliver something of relatively minor value.“

Ironically, I’ve generally seen the opposite perspective from business – that the “expensive” projects, the ones typically capable of creating the most value, are the ones where the most control of timeline and cost is desired and therefore the most damage done by “management” as opposed to allowing the necessary experimentation, design, thought, research and development to occur to create the most value out of the software.

Being criticized of being “unable to delivery on time” in software development is not something I can really take personally – when the industry at large struggles so significantly with this, largely because it has tried so desperately to liken software development to a public works engineering project such as building a bridge, a building or a road.

Software development is more like art than most seem willing to accept.

It is driven by creativity, experimentation, research, design and artistic drive.

Is it any wonder that the “transformative” software projects most often come from startups (Apple apparently being the exception with their iPhone OS and demonstrating the amazing culture of design they allow to flourish) and then as they mature become stagnant once quarterly reports drive business as opposed to the development of a product? Or that opensource, where timelines mean nothing, can build great software such as Linux, Firefox and MySQL? Look at what has happened to MySQL over the past 2 years as business “management” has replaced “experimentation” and focus on quality — the key people who built MySQL over the years have left to work on something where strict control is not the focus.

As Tom DeMarco suggests, I hope to be focusing on development where the value is worth allowing the proper time, people, experimentation and development to occur – so that something of true value can be created. If management of “costs” is the most important aspect of the project, the greatest value to be created, I don’t even want to bother. I’d rather work on a project that has the ability to be transformative, and create greater value than the cost of the effort.

The document is approaching 150 pages and beginning to approach something that properly describes the vision and needs so that business and technical folks can have a solid understanding of scope, requirements, priorities and how it will work (plus general direction on how it will be built).

It takes a lot of effort and time to sit down and not only read, but understand the scope of this system. It’s not a simple thing. As much as we try to keep it conceptually simple, there’s a lot going on and a lot of details.

In a few weeks I’m to sit down with a partner team to review and collaborate. My plan was to send over the detailed document for review so we could have a productive meeting to truly discuss missing requirements, strategy, execution planning etc.

However, I’ve been told they don’t “want” to read the detailed version — just a summary — bullet points with perhaps 10-15 pages.

These are senior developers and architects. Not high level business folks.

Attention to detail is in my opinion an absolute requirement for technical people. Decision making without details is dangerous and fairly useless.

My opinion is not high for people who do not care or have the attention span to be detail oriented.

An executive who’s not directly involved in the operations of something – a summary makes sense.

A team or person who is supposed to directly impact the design and delivery of something and its ongoing operations – if the details aren’t part of their focus, they don’t deserve to be involved.

If they don’t have the time to be detail oriented, then either they shouldn’t be working on the project, their time needs to be re-prioritized, or the project isn’t worth doing.

• the ability, coming from one’s knowledge, practice, aptitude, etc., to do something well
• competent excellence in performance; expertness;

Trust is defined as:

• reliance on the integrity, strength, ability, surety, etc., of a person or thing; confidence.

Two common themes are ability defined as:

• competence in an activity or occupation because of one’s skill, training, or other qualification
• power or capacity to do or act physically, mentally, legally, morally, financially, etc.

My experience unfortunately is that skill and ability in people are not abundant – and therefore it is hard to have trust.

More problems are caused by lack of skill than anything else I run into.

Making mistakes and learning from them is part of life – it’s the human experience.

However, there is a reason world class organizations have “senior” people  and restrict the impact of “junior” people’s mistakes by limiting what they are allowed to do and affect until they have gained trust – by demonstrating their ability and skill.

Trust nowadays is given away much too freely – desire, eagerness, willingness do not replace ability and skill.

Trust should not be given unless someone has earned it. People should not expect to be trusted until they’ve earned it by showing they deserve it.

I had someone tell me (after failing on something) that I should trust them more and give them “opportunity” so they could succeed (by giving them the ‘keys to the kingdom’). This is backwards. Prove to me on small things incrementally and trust will be earned and greater opportunities given.

Trust does not work like “innocent until proven guilty”. Trust must be earned. Trust stems from skill, honesty, aptitude, competence and most importantly – time.

If business operations are failing, my opinion is it’s generally due to placing trust and unfair expectations on people who do not have the skill or ability to deliver.

Put the money and time on having the right people – and restrict the impact of “junior” people – and operations will improve.

The gist of it is that it’s very difficult to rely upon resumes, and that we as an industry seem to put up with “falsification” of skills and keep people onboard who really don’t have the skills they need to do their job.

It closes with the following statement:

“World class organizations are built on foundations where people can trust each other and don’t have to second guess the ability of the team. If you find yourself in a shop that doesn’t get it and you stay, maybe you are getting exactly what you deserve… ” – James McGovern

Continuing on this theme I have continued pondering what it takes to effectively do development with a remote office.

I use the term “distributed development” as opposed to “outsourcing” since the word outsourcing generally implies one company sending the work to a second company – not multiple teams of the same company in different geographic regions, but my thoughts more or less apply to both scenarios.

My thoughts are shaped greatly by my experience with a remote development team in Pakistan and myself based in the US. Therefore they may or may not be applicable to other regions such as India and China.

What I Feel is Necessary for Effective Remote Development

In order for remote development to be successful, I believe the following things are necessary:

• Senior leads “on the ground” with the team who have experience gained over years of development and most importantly in delivery of applications to customers and operating them in production.
• Frequent communication between the US and remote office with as much of it being in “real-time” as possible, not just “touching base” morning and night. Development is not a shift-work job like a factory. Continuity and comprehensive knowledge is critical.
• Human interaction through travel and face-to-face meetings. Nothing establishes team rapport like sharing a meal together. Rapport is what enables communication and difficult work to be done together in the long haul. Relations can be built remotely – but much more slowly, with much more effort and with far fewer people.
• The ratio of senior to junior must be healthy. Too few senior resources means the junior developers are not getting the mentoring and guidance they need, they make mistakes, code quality goes down, production issues arise, morale decreases. It’s my belief that a senior developer can handle no more than 5 junior developers if one expects proper mentoring, training and good code quality. This ratio may be able to change if there are solid ‘intermediate’ developers who can then have junior developers managed by them, but I have not seen this work very well. Intermediate generally means you’re not fresh (completely junior) but you still need guidance, direction and mentoring, and are therefore not ready to lead, mentor or manage others and therefore should not be setup to fail as such.
• The developers themselves must be consumers of technology products. If a developer writing an ecommerce application has never actually purchased a product online and had it shipped to their door, they have no common relationship to “how it should work” let alone “how it should feel”. Common sense is taken for granted – it is something learned through life experience.

Positives of Pakistan Remote Development

In 7 years of working with Pakistan developers, and likely seeing over 100 developers during that time, I have found the following to be true.

• Very intelligent.
• Very eager to work and grow.
• Amazing work ethic.
• English language skills (for being a non-english country) are quite good.

Of course, the extremely low cost is a final factor that can not be ignored and is the driving reason for having a team in Pakistan.

I have delivered some significant software and production systems with Pakistan development teams – myself being the only US based person. It can work and be very cost effective and rewarding.

Negatives of Pakistan Remote Development

There are definite drawbacks however to working remotely in Pakistan.

Time Zone

The time difference is 12 hours (from Mountain or Pacific US time depending on time of year) which makes real-time communication very difficult as it is limited to early morning or late night. It would be perfect for shift-work as you could do hand-offs every 12 hours – but development is not that type of work.

Cultural

Cultural difference means that “common sense” in each country is very different. Thought process is definitely different. Consumer experience with technology products is limited. This has an impact on understanding and requires much more detailed communication, requirements definitions and business reviews as nothing can be taken for granted. When it is, or something is “assumed”, the cost is wrong implementations and/or delayed timelines.

Perhaps another cultural difference and a side effect of the positive eagerness is not knowing what they don’t know. A desire to try something does not equate to knowing what or how to do it. Saying “yes” is great – except when that “yes” is not accompanied by a comprehension of what it entails. Commitments to deliverables can be rather misleading if in fact the person does not know what has been committed to or know how to deliver it, but wants to try it all the same – and therefore says yes.

Geopolitical

Let’s face it, Pakistan is not an ideal place in the world these days. It hasn’t been for years and likely won’t be for many more. My hopes and prayers are that the country will improve in its stability, infrastructure, economy, education and other matters, but today it is not a country many people desire to be in. This is a big part of why the cost is so low to hire people in the country. At the same time, it has a very significant impact on talent: people don’t stay.

Virtually anyone with a skill and the opportunity to leave will do so. It is evident to me in how many people I have seen emigrate away from Pakistan to Canada, US, Britain, Singapore and other countries over the years. It is more evident in the seemingly complete lack of “senior” talent in Pakistan.

It is virtually impossible to find and retain a “senior” level developer with 8+ years of experience, let alone 15+. Even 5+ is hard to find.

If the effort is made to train someone, after a few years as their skills and experience mature, the pull to leave the country increases and the company loses them.

I understand the pull. I work with 3 people in the US all who have immigrated from Pakistan. Incredibly smart, hard working people – very good at what they do. But they all left Pakistan instead of living their lives and careers there.

Impact on Cost and Productivity

Considering the extremely low cost of developers in Pakistan, doesn’t that outweigh the negatives?

Under certain conditions, it can work very well, but those conditions must be met or it falls apart, and in general I question the actual “low” costs.

Assuming communication and timezones are dealt with, and the cultural barriers addressed – one item can not be overcome through any amount of process, hard work or documentation. Geopolitical.

The geopolitical situation is beyond resolution for any one company or team, and it is a force that can not be ignored. To ignore it is to invite failure.

Long timelines, poor quality code, production issues and failed deliverables are all symptoms of one thing: a team that does not have the skills to deliver on what is being attempted.

What about mentoring, grooming and training? All of these are great – but if the people that are trained do not stick around longer than the training, then there is an incredible cost of training that is not recouped.

The senior developers spend their time training, not developing. Then the junior developers leave before they can become productive. The time of the senior developer is wasted as far as the company is concerned. The senior developer is seen to be “unproductive” because delivery is so slow – yet they have no control over the loss of the junior resources who were just getting good enough to work productively.

To make up for it, more junior developers are added – making the matter worse. A higher ratio of junior to senior means less mentoring and training for each developer. This means lower morale, lower skills, and poorer quality of delivery.

This catches up — bugs, production issues, delivery failures, long timelines. This then leads to morale problems and further turnover of resources, generally the ones with the most potential and skill.

Conclusion

Remote teams such as this can work very well if they are small and directly led by a senior developer lead.

Mentoring and grooming of developers in Pakistan (or other remote locations) can be done and can be successful. I’ve done it. It has worked well. It has only worked well though when the remote team has worked my hours and the team size was small — so that I had constant real-time communication. I did this for several years.

This however does not grow a business if those people don’t stick around. Nor does it scale.

Thus, I believe the actual cost of development for an outsourced development team with these traits is in fact much higher than the monthly salary of a developer in Karachi.

If a senior developer from the US is spending 75% of their time mentoring, training, grooming and managing junior developers – and those developers all leave within 2-3 years, what is the cost of that 75% of time compared to if it had been spent developing the products directly?

If a 4 month project takes 8 months because of junior talent, and 4 months of sales are lost because of this, are those 4 months of sales higher or lower than the cost of having a team of more senior talent?

To ignore these facts and not look at the actual costs is to set oneself up for failure.

No amount of documentation, process, hope or patriotism can overcome the geopolitical issues and lack of long-term talent.

So what is the solution?

The first thought is to have everything return to the US (or Europe, Canada etc, wherever the parent company resides) where developers expect to live their lives and have long careers, so talent can be groomed and retained.

Some business models just can’t afford this however. In that case, another location must be found where the following applies:

• cost of developers at least half of the US
• geopolitical situation where developers desire to stay, to live their full careers in their country
• good english communication skills
• intelligent and educated people
• preferably a culture exposed to technology driven consumerism
• preferably in a more similar timezone to the “parent company”, in this case, the US timezones
• safe and easy for North Americans and Europeans to travel to and stay for extended periods of time

I believe distributed development is here to stay, and creates significant value when done correctly. It is not an easy thing to do well though. Nor does ignoring the challenges of culture, distance, time and politics benefit the business.

I want to see a distributed team setup according to the thoughts I’ve described above and see it achieve the level of productivity and efficiency that is its potential.

A draft of a requirements document numbering in excess of 100 pages has very thoroughly covered the technical definitions of all of the various ‘features’ – what the system needs to contain. However, it reads like a dictionary definition and does not in any way help me understand the following:

• What is the business purpose of the product?
• What is the business purpose of a particular requirement?
• What are the use cases that requirement must serve?
• Who will use the specific requirement and for what?
• How does a given requirement fit into the overall system so I know its context?
• How will the product implement the feature that address the given requirement?
• Where in the system architecture should the developers implement the given feature?

I became aware while reading that most of the people for whom this document exists would not have a much better understanding of what needs to be built after reading the document than from before.

During my consideration I listened to a podcast on “requirements engineering” that interviewed Christof Ebert.

I think the recommendations and principles are valuable regardless of what process is followed — whether they are done upfront or at the beginning of each iteration with waterfall, agile or a mixture of both.

A few notes from the podcast:

What Requirements Should Define

• What the Customer Needs Are
• What the Product Should Do
• How the Product Should Do It

Common Requirement Problems

• Missing Requirements: Incorrect effort or tactics to seek out (elicit) all of the requirements.
• Wrong Requirements: Poor capture, communication or lack of validation with stakeholders.
• Changing Requirements: Requirements are never static, they always change.

3 Levels of Requirements That Need to be Addressed

Market (External)

• what user or client expects
• the “problem space”, defining the problem and what is desired to solve it
• what will make the product successful

Product (Internal)

• what the product will do
• how the product will address the market requirements
• the “solution space”, defining the solution that will meet the user (market) requirements
• define what functionality and requirements will be made into a product

Component (Software)

• lower level details of how to implement the requirements in software
• non-functional requirements such as quality, service levels, maintainability etc
• context of requirement in overall system

What is good enough?

Reality must be paid attention to. Generally the economics of a project can not justify the “ivory tower” of solutions for all aspects of a project.

What is good enough to serve the user – in both functional and non-functional requirements?

What is of high priority?

Almost always there is more work to be done than resources, so what is of priority?

What can and will be committed to for development, with what resources and by when?

What can users live without, what must they have?

My Final Thoughts

As I prepare to attempt a revision of the requirements I’m working on, I must remember the primary reason for the documents existence: to communicate to developers, QA, analysts and business the reason for the product, what value it’s creating and what use cases it will provide functionality for.

If it doesn’t leave the reader with comprehension of what they’ll end up with at the end of development – at least at a 10,000 foot level – and is just a list of features with no context or purpose, then it’s a failure, no matter the length or level of detail.

Over the past 7 years I have had very hands on experience with distributed development – working from the US with over 100 people in development, QA, operations and other teams in Karachi, Pakistan. It has included two personal trips to Pakistan, thousands of hours of MSN or Skype chatting and many odd working hours in order to accommodate a 12 hour time zone difference.

June 2006 trip to Karachi, Pakistan

All in all it has been a fruitful experience. It has definitely had its troubles, pains and inefficiencies, but products and services have been delivered cost effectively to our customers ranging from Fortune 100 on down. Applications have been built which support 10s of millions of dollars in revenue and millions of transactions per day.

In the past year however I have begun to see a different set of challenges emerge — the ability to scale.

By scale I mean 2 types of scale:

1) Scaling the size of the team.

By this I simply mean adding more people to the team in Karachi to accommodate increasing work load.

2) Scaling the size and complexity of projects, code bases and system architecture.

This refers to the ever increasing amount of code, projects and modules to manage and develop as years of projects become legacy or enter “operational support” status, as increased business success and growth demands more from its systems and feature sets become increasingly larger and as architectures require more advanced models such as SOA, distributed computing, distributed infrastructures etc to handle scale, volume and global distribution of business.

In short, recent experience is leading me to believe that a ratio of approximately 1:5 is the maximum scale that can be achieved in a distributed development model. 1 “local” senior developer to 5 “remote” junior/intermediate developers.

Successful Experience

A team of 5-10 junior/intermediate developers and QA working with leadership from 12 time zones away proved to work well when the following conditions were met:

- The Karachi team worked their nights to accommodate the business and customer schedule which was US Pacific time zone.

- Daily interaction between myself and the Karachi team accounting to several hours a day of direct real-time communication on design, planning, business, and code related subjects.

- Micro-level involvement of myself at the code and task levels involving review of virtually every line of code written, every test, every deliverable.

Three successes came out of the 2+ years of this type of model:

1) Successful and cost effective delivery of the projects.

2) Mentoring, training and grooming of junior developers into what eventually became the best developer talent pool of the company in later years.

One member of that original team in particular, after 3+ years of 10-12 hour work days, all night shift, has immigrated to the US and works as one of our key resources in the role of Application Architect. He had the raw materials, personal drive and competency, but the opportunity to progress from a fresh graduate to “senior” level skills and roles in a matter of years was obviously accelerated greatly by the virtually non-stop direct contact with a senior developer, high-profile projects and US customers.

3) Proof that the distributed development model can work (under certain conditions at least).

What a Junior Developer Needs

I have come to the conclusion that a junior or intermediate developer needs quick and constant communication with their lead — whatever we call that person (senior developer, architect, development manager etc).

The lead must be capable of providing timely and thorough responses, training, mentoring and guidance.

Nothing can replace working in the code together, explaining why decisions are made, how to refactor something, how to add a feature to an existing codebase, how to profile, tune, optimize, debug, test and deploy.

Generally a young developer has little to no experience with “production” – what it really means to build and deliver a system that can be deployed and operate 24/7 — by itself with little to no involvement from developers — or even more difficult, be handed off to an operations team for deployment and operations with no direct involvement of the development team.

The concepts of logging, configuration, performance, multi-threading, system interaction, hardware and infrastructure are all simple in dev. Until one has experienced true production, and more importantly — issues in production, it’s difficult to understand why certain design decisions are made which seem to not be that important in dev, but make a significant impact in production, or to the teams maintaining production, or to the teams maintaining the code a year later.

Raw intelligence and capability exists in many people in many locations, but the proximity and availability of skilled, experienced senior personnel is key to maturing the raw skills of the young inexperienced developers into productive “seasoned” developers

No different than how a fresh graduate from a college in the US is of limited value until several years of grooming and experience under senior guidance, a developer working remotely has the same challenges — but with some further hurdles:

- Language barriers: Educated in english does not equate to fluent or comfortable.

- Time differences: If 2 people are 12 hours apart, that means that unless they purposefully shift their schedules, they only have a short window of time early morning and late night to work together.

- Physical separation: Can’t just walk around the corner to ask a quick question and can’t easily judge the emotions of the person you’re working with.

- Cultural separation: Many things in North America and Europe are taken for granted, such as purchasing a product on Amazon.com and having it show up at your door 2 days later. If someone has never experienced this it’s much more difficult to relate to a US business requirement when they say “the user experience should be like Amazon.com”.

The concept of “common sense” is not universal – it applies to experiences gained in relation to when, where and how one has lived.

How I’ve Countered the Hurdles

Language

I have found that rarely is verbal communication a good thing, even when it appears that it’s going well.

“Yes, I understand” is not a trust worthy statement in many cases – written or verbal – but specifically in verbal with developers whose English is a 2nd or 3rd language primarily learned through school, textbooks and the internet.

Reading and writing on the other hand can generally have very high comprehension and actually be much faster.

Using something like a Skype chat (text not verbal) allows for a number of positive things:

- built in meeting notes that can be referred back to

- dialect and speed of talking do not impair comprehension, as everyone can read and re-read if necessary what has been written

- appropriate time to think (and translate thoughts to English if needed) is built in to the medium

- multiple conversations can be had in multiple windows at a time if needed

Time Difference

On the projects where I have been most successful, I eliminated the time difference barrier by having the Karachi team work US hours to align with our customers and US business.

This solved 2 critical issues:

- The team in Karachi could now participate in business and customer interaction and communicate directly with customers via phone calls, conference calls, Skype or MSN chats – instead of everything being done in emails with 12 hour intervals or routed back and forth through a single US contact

- The US team and Karachi team worked together as a true team with direct real-time communication all day. The Skype/MSN window became the office, talking almost as naturally via text as if we were next door.

Physical Separation

This is obviously the most difficult to solve, but handling the time difference by everyone working the same hours and having virtually permanent communication windows open at all times in many ways made this issue seem negligible. Yes whiteboarding is more challenging – and you all have to be fast typists, but it worked.

It did not work for all though. I had several senior US/Canadian people that didn’t work out because they couldn’t adjust to an environment where typing instead of talking face-to-face was the primary means of interaction and whiteboarding could not be done in an impromptu manner.

Culture

I have found no secret solution for this. I have had to learn to be extremely detailed in communication, take nothing for granted, think from their perspective and avoid colloquial use of the English language and all forms of sarcasm.

Challenge to Scaling Distributed Development

Growing a team from 5 to 15 or 20 developers, but still having a single senior developer leading them has not worked well.

Obviously the team of 20 is broken into smaller teams, and there are team leads.

The challenge is as follows:

If a “team lead” has 4-5 years of experience, and that experience itself is of only limited exposure to senior skillsets, then that person is being setup to fail as they do not yet have the skills, experience, maturity or understanding to do what they are being asked to do.

This person, who may be an excellent developer on a 5 person team led by 1 senior developer – when asked to lead the development of 5 others is now put in a position where they are struggling to deliver, they are not able to learn as they once did while working just as a developer with the senior developer and the 5 below them are learning even less and the quality of code and timelines of delivery fall thru the floor.

The solution on paper is simple: “Find senior developers in Karachi to lead the team in Karachi.”

In practice however this is far more difficult. Even with restricted immigration policies to US, Canada, UK and other candidate countries in recent years, senior skilled technical people seek out opportunities wherever they may be and are not generally available in Karachi by the time their skills have matured to what would be a “senior lead” or “architect”.

Finding and retaining what is considered a “lead”, “senior developer” or “application architect” in the US is not easy or cheap anywhere – and in “outsourcing” countries challenging to the point of “it’s probably not going to happen”. At least not in a way that can be counted on. When it does happen it’s great, but it’s not something one can plan for.

Process Doesn’t Replace Skill and Experience

I believe process and documentation is needed and has its place, especially as projects increase in size and complexity.

However, no amount of documentation and process can replace competence (see comments by James McGovern on the subject).

Junior developers do not become competent and capable through documentation and process – it comes through mentoring and experience, being groomed by those ahead of them and by applying their raw skills to educate and improve themselves.

I can be particularly verbose in writing when I want or need to be. I have written thousands of pages of documentation including hundreds of pages of visual wireframe mockups, use cases, requirements, workflow diagrams etc. The end result? Most of the people they were intended for don’t understand how it all fits together, how the business use cases apply, or how the user interfaces accomplish the use cases.

It has been a frustration for that to be the base — and I’ve attempted making documentation ever more detailed, having thorough review and feedback sessions and even getting to the point of spelling out exact implementation details of given requirements right down to where the code should be done in a given codebase.

Looking back over these experiences, the documentation worked very well for the few people on the team who had already worked with me for a few years and were more advanced in their skillsets and understanding. They would read the documents and question them very specifically. They would find mistakes, or complete design errors. They would find inconsistencies in use cases or workflows.

It was obvious they understood it – and contributed to improving and enabling the delivery. The other 80% of the team however had very little if anything to say.

Thus, documentation and process is important – they are tools to organize and enable large and complex systems to be built and maintained by skilled and competent people. They do not however enable just anybody with a degree in computer science to deliver.

Conclusion

Successful development depends on skilled, competent, experienced people taking ownership and delivering a project, product or system.

Ideally all people involved would be equally skilled and senior – but that’s a rare scenario that I’ve never experienced.

Thus, the mixture of senior and junior needs to be right so as to enable the senior members to still be productive in accomplishing business objectives while grooming and mentoring the next generation and increasing productivity on tasks that can be done by those with less experience with guidance.

It seems that the ratio can’t be much higher than about 1-to-5. The amount of time a senior developer must spend in mentoring, communicating, teaching, reviewing, and designing with the junior members, and then performing their own development, architecture and other roles limits the number of junior people a senior resource can effectively work with.

Regardless of location, whether a team be all physically together, distributed in cities across the US or distributed across 12 time zones like I’ve been, I believe this principle applies. Perhaps the ratio differs for different geographic placements, but in a distributed environment where one can’t walk around the corner to communicate for 5 minutes on a white-board there is a definite effort and time commitment to communication that limits the scale of a development team to a ratio of senior and junior developers that if surpassed dramatically deteriorates code quality, morale and timelines.