Requisite Organization in Software Development & Information Technology

I’d like to begin a discussion about requisite organization of software development groups. Gordon has raised this issue, which I have been meaning to write on but have avoided for some time. I invite both he and JMMJ to comment from the developmers’ perspective.

A Hierarchy of the Flat Organization

One of the very interesting things about working with developers is how quickly they establish pecking orders based on ability. Big Insurance Group (BIG), an old client of mine, had some amazing people whom everyone could identify. “You should talk to Mary X and Raja Y,” they would tell me. “They really know their stuff.” Others at the same level as Raja or Mary would get a nod towards their status within the organization chart but only as an afterthought. “Oh, and you may want to also see Tim Z and Yolanda A,” they would say, almost as if they had to mention them but I wouldn’t get much out of the discussion.

I have been fascinated by how accurate this pecking order always seemed to be. My judgments from working with them would normally line up with the recommendations. And the developers would normally try to pick their “real boss”, the other software geek whom they had to take real orders from. Others could be their manager (and at BIG, developers directly report to three or more of them) but only certain other software folks could provide them with direction.

I think this pecking order comes about through ability and performance. There were endless cases of developers that the managers didn’t like because they were difficult to control in meetings who nonetheless garnered admiration from the ranks as competent folks who “knows his sh*t” [sic]. Whenever developers meet, they go through elaborate tests to see who is more competent than someone else.

The Levels of Enterprise Development

Enterprise-level development groups often have four different types of work done:

Programming, the basic tasks of writing code according to standards
Development, a higher form of programming that involves tasks such as object-oriented coding
Design, the process of putting together the project work for a particular system
Artictecture, the process of putting together systems to fit together for the enterprise

Architecture can be divided up into different levels, too, depending on the size of the enterprise. You may have:

Systems Architecture, the process of putting systems together for a particular function, such as Finance
Enterprise Architecture, the process of putting the architectures of Systems Architectures together into a coherent whole for the entire enterprise

Each level up is an abstraction of the previous layer. Development abstracts Programming. Programmers often work from templates, such as when they create a stored procedure in COBOL for DB2 from a standard template for this type of stored procedure. Developers work at an abstraction of this, creating their own templates or working with abstract issues like Objects. I’ll deal with the problems created by Object Oriented Programming (OOP) in a later post. Design abstracts Development. Architecture is an abstraction of Design.

Mapping to Jaques’s Strata

Elliott Jaques discovered that if he asked people who their “real boss” was that the organization would break into some defined levels. When he looked at the length of time that each person had discretion over his work, the time before it needed to be checked (what Jaques called the Timespan of Discretion or TSD), he discovered that there were breaks in the continuum. People didn’t just need a boss who was capable of working a longer TSD than they were: their “real boss” was always an apparent layer above them. He later discovered that there are some thinking styles that seemed to change at these breaks. I’m translating it as “abstracting the lower level”, but Jaques described it more elaborately (see his book, Human Capability).

What I am calling the software development operation maps to Jaques “strata” of Complexity of Information Processing in the following way:

[Click here to open table in new window]

[UPDATE 2013: I have no idea what this was or if it was ever completed. Lost now completely.]

Comment on where you see problems with this.

Problem of terminology

Right now I should stop and redefine all the terms that I am using because they have been used in so many different ways, confusing the differences between them.

For example, if a developer (level 2) maps out the work he is about to do an a small application for a website, he may say that he has done Design work. He has designed the application, but the level of work wasn’t any bigger than coding the application. When developers get into true design work the difference becomes obvious. “Design” as I am using it refers to abstracting the development process.

The abstraction of the work is what’s important. I see software people gathering around these levels in their speech. “He’s a competent developer but he’s not in our league,” said one designer of a younger developer who wanted to join their group.

A group may not have all levels

Smaller operations (not necessarily smaller companies) may not have all the levels. Operations that are limited to one system, where systems do not have to interoperate, would have little need for what I am labelling “architecture”. Operations that have no real system but only a collection of applications would have little need for “design”. During the 1960s, many internal software groups did little more than “programming”: outside vendors such as DEC and IBM performed all of the “development” and higher work.

We should probably use the following terms to make things clear:

Programmer
Development programmer
Systems Designer
Systems Architect
Enterprise Architect

To Be Continued….

I will continue this in further posts.

UPDATE 2013: Which I don’t think I ever did. Odd, that. I’m not sure that I would agree with all of this any longer. The topic is still worth considering.

Read more