Typists. By Lewis W. Hine (1874-1940). ca. 1915. George Eastman House Collection.

Judgment of Capability (RO Work Level) Must Be On Work

Forrest ChristianTheory 15 Comments

I talk a lot about judgment and how one can get a good idea of someone’s current capacity (how big of a job that they can do) from an interview where they “do work”. Elliott Jaques and Kathryn Cason describe this in Human Capability. But it can’t be just any interview topic: you need to get the person talking about a topic that they have spent time thinking and learning about.

I started thinking about this when I watched the recent interview that Katie Couric, an American television news anchor, did with the US vice presidential candidate Sarah Palin, governor of the state of Alaska. (CBS has the Couric/Palin interview available online [Part 1 & Part 2].) Couric is asking probing questions to get to Gov. Palin’s opinions on economic issues. Gov. Palin seems quite flustered and doesn’t really include details.

The interview was widely held as a poor one on the part of Gov. Palin, even by Republican editorialists. (Democratic editorialists were unlikely to view the interview as a success regardless.) But you can’t use this interview to gauge Palin’s capacity for work because it did not engage her on issues that she worked on in the past.

Gov. Sarah Palin has breakfast and visits with Ramstein Air Base Airmen in Germany on July 26, 2007. Palin went on to visit wounded Airmen and Soldiers at Landstuhl Regional Medical Center, Landstuhl, Germany. Photo by Airman 1st Class Kenny Holston. Public Domain (work of US Federal Gov't)
Gov. Palin visiting US Airmen in Germany, July 2007

The Gov. Palin you see in the interview is very different from the self-assured, confident, intelligent Gov. Palin that you can see discussing topics for which she is better prepared. (The Newsweek forum on Women in Leadership is a good example and would be more useful for evaluation purposes.)

Gov. Palin is a smart and competent politician who has a strong accomplishment record. These were not topics that had previously been part of her political discussions and so was not a good interview to show her current capacity for work. Alaska has a different economy than the rest of the U.S. — e.g., they tend to do well with higher gas prices while we do poorly — so her economic concerns as governor are concentrated on natural resources issues. (See for example the NY Times article describing some of the issues with which Gov. Palin has been concerned.)

It may seem odd that a governor would not have a great deal of things to say about the financial crisis but there’s no crisis in Alaska: they have a US$2 billion surplus. Each resident gets a check that is worth a fifth of my pre-tax salary when I started working after university. They are awash in cash up there.

The same principle applies to listening for someone’s current capacity in conversation, which I know some of you do to provide you more information at work. You need to address issues that this person has spent some time thinking about. Issues that have not been important before are probably a bad idea. Using a conversation that is a “make or break your career” type is also probably counter-productive, since very few people do very well in these.

You can learn something about someone’s current capability on topics they don’t anything about, of course. Recall that capability is a function of one’s exercised capacity (how much of it you are currently using) and one’s knowledge and skills. If you tried to get me to do thinking work about carburetors you will quickly find out that I am a mechanical idiot. (I even had to look up how to spell “carburetor” and I confess I have no idea what they do.) You won’t learn much about my current capacity but you will learn something about my capability regarding motor engines.

Tonight, Gov. Palin will be debating Sen. Biden, the democratic candidate for vice president. Hopefully we will hear them both doing work in their answers. Especially useful would be to have Gov. Palin talk on issues involving natural resources, especially oil and gas, which are a focus of Alaska government and of great interest to the rest of the nation. (Sen. Biden’s loquaciousness represents its own problem, of course.)

So take this lesson:

You need to listen closely in CIP assessments, just like you need to look closely to judge a subordinate’s work effectiveness.

Yes, it is a personal judgment but it needs to be about work and not simple emotional perception. If you haven’t had something like PeopleFit’s class on conducting CIP evaluations and you are trying to use the general principles presented in Human Capability, please take care. Listen but check what you think you heard against your judgment of this person’s job performance.

You should also use the “bigger” trick: is this person bigger, smaller or the same size as this other person whom I feel comfortable I know? Good advice even if you have taken PeopleFit’s CIP class or use the Career Path Assessment instrument: you should never make a decision based on a single data point.

NOTE: If Jack Fallow is right (and there are indications in Warren Kinston’s unpublished research that he is on to something), and if we use Cason’s and Krause’s earlier studies of presidential races, then the likely Presidential candidate will be someone Str 6. If both are Str6 then the one with the most future potential (Mode) will win. Jack’s idea that we understand someone in a like quatrile is a big one. When Warren finishes his current work, you will see why.

Image Credit: Typists. Photo by Lewis W. Hine (1874-1940), ca. 1915. George Eastman House Collection. Public Domain image.

Gov. Sarah Palin has breakfast and visits with Ramstein Air Base Airmen in Germany on July 26, 2007. Photo by Airman 1st Class Kenny Holston. Public domain image of US Federal Government.

Comments 15

  1. Based on your summary Governor Palin is not qualified to be Vice President of the USA. While you argue that she has the capacity and that it simply is not visible because of the subject matter (not sure I agree) if we were to grant this by your own account she does not possess the skills and knowledge to do the work. I find it difficult to believe that any congressman or woman, senator or governor who not be versed on not only the domestic economy but also the broader global economic conditions. With no visible understanding of macro economics she is not qualified for the job she is running for. In fact I would assert that she has been dragging John McCain down in the polls. On complex matters she comes off as a buffoon.

  2. Post

    Remember that there are no intellectual or experiential qualifications to be a politician. It’s not that intellect doesn’t matter but that it’s not a prerequisite. Also, Gov. Palin is clearly more than what these bad showings would indicate.

    I also know that you disagree with Warren on this. Paul may be able discuss more about how this plays out better than I could. He has a good deal of experience in this.

    There is a problem in the way that American government is organized, in that it mixes the political and the organizational. The moves by the Bush administration to greatly politicize positions that should have been organizational is important. I just don’t know exactly how this plays out. Again, I think that Paul would have more intelligent things to say here.

  3. In “Judging Human Capability” Elliott and Kathryn uses examples from drug debates to exemplify levels of reasoning. So it should be possible to use political debate to assess capability. However, the issue is to separate how the person actually thinks from when the person repeats what somebody else has done. Which is why we in capability assessment prefer to work with concrete work examples.

    We also need to remember that Elliott wrote that in order to be able to do a job one needs knowledge/skills, experience, capability and the personal preferences/values wanting to actually do the work. If we only work with capability assessment it can be easy to overlook the other factors.

    No matter if the politician works with policies or is an executive it would be great if the person has capability as well knowledge/skills and experience.

    Just because a politician happens to be a very public person is no excuse for us who are knowledgeable in capability assessment to speculate and discuss the capability of politicians. A proper assessment can only be done in personal contact and in a deep interview. I find public speculation about capabilities highly unethical.

    The article published some years ago speculating about the capabilities of presidential capabilities was not only unethical, but also led to quite a lot of negative publicity as it was published so close to an election.

    Let us not forget that politics is not entirely rational, which was probably what led Mark Twain to write “I am quite sure now that often, very often, in matters concerning religion and politics a man’s reasoning powers are not above the monkey’s”

  4. Post

    Here in the States, we evaluate our politicians six ways to Sunday, and being suspicious people, labeling someone “smart” is not always a compliment. I don’t feel that speculating on a politician’s current level of Complexity of Information Processing (CIP) is unethical. We speculate on our candidates’ religiosity, marital fidelity, intelligence, willingness to fight, ability to “cross the aisle”, the beer they drink (or not), whether or not their stance on [ISSUE] is real or just to get votes, what type of family they have, whether that large campaign contribution from those Wall Street sources encouraged their bailout vote, and even whether or not they like broccoli. Other nations probably don’t do this, but it’s been like this for a long time.

    The problem with Cason and Krause’s piece was that it seemed to be put forth as more than speculation. I don’t believe that in America it is considered particularly unethical that they did it but it surely was unwise. It certainly didn’t help to legitimize their viewpoints nor did it help the Kerry campaign. It was considered uppity and almost anti-American. If lump it with the Smile Rating (most pleasant person in the debates wins) or the Height Rating (tallest guy wins), then it can be seen as part of a long and ignominious history of “expert” predictions.

    Ethical issues aside, it is certainly true as you point out that the validity of anything coming out of debates is questionable, for the reasons you cited. Interviews where the person ranges far and wide are more interesting, as are situations where the interviewer asks probing follow-up questions. They give you a general feel, especially of which is bigger, this person or that.

    But it still doesn’t tell you who is more qualified to govern or for whom people will vote. I have always voted for the war vet, which seems like quite an irrational thing to do. Others vote on single issues like gun ownership, abortion or ecology. Still others vote on race or sex. Democracy is not rational, thank God. Totalitarianism is rational.

    I’ll stick with the statement that there is no level of work prerequisite to get into office. In the US, the requirements for president are pretty simple: you have to native US citizen, over 35, resident of the US for the last 14 years, and get on enough state ballots that you can get the electoral votes necessary to win. After that it is just convincing people to vote for you. Our parties have little power and have gotten saddled with candidates the party powers didn’t really like.

    Nor do I think that the job itself has any real requirements of work level, knowledge or skills. I’m not even sure what skills or knowledge would be helpful or detrimental.

    But I’m getting pessimistic about the ongoing decline of the American democracy so maybe I’m just being cranky.

    Right now I’m simply impressed that so many Americans even consider that a man of color can be presidential quality. Another wonder was that Obama ran for the senate against another African-American, Alan Keyes; this when there had only been four black senators in American history.

  5. Politics is about policies – what will the party or candidate pursue when in office. Things get messed up when you vote for a person rather than a party and when that person is more of an executive than a policy-maker. In a party-system the debate is about policies, in a person-system the debate invariably moves to abilities and personality.

    I have a RO-colleague and friend who is involved in party-based Danish politics. He notes that the parties select people with high capabilities. The only times the system does not work is when a party expands too quickly for it’s talent pool.

    In both party and person systems the amount of spin has increased significantly over the years, and spin, particularly about persons is highly detrimental. I forever astonished about the amount of insult, lies and mud-slinging in politics, particularly in the US. I keep asking myself what drives a person to endure all that – are they narcissists or is politics so personally profitable?

    My personal opinion is that this drive away from policies to mud and spin is negative for democracy, as it probably drives sensible and capable people away from politics. I feel that the elections in the US has moved dangerously close to some mix of Gladiators, American Idol and Jerry Springer.

  6. I’ll promote that you haven’t yet made a convincing argument. While I would agree that both values and CIP come into evaluation by the electorate I wouldn’t so quickly discount the significance of a political leader’s CIP.

    The postulations of Kathryn Cason and Alison Brause with respect to presidential debates cannot be rejected out of hand. There is sufficient data to attract strong statistical inquiry. How do you otherwise account for the “coincidence”? If we entertained for a moment that CIP is significant in the evaluation of the electorate and that the majority of times the voting democratic populace gets the assessment right (we could cite examples of misjudgment of CIP in the workplace) then it stands to reason that the constituents would elect the most capable president. Granted some voters will vote party lines regardless, and some will vote the incument out for the sake of change, however the decisive swing vote would more often than not go to the most capable candidates. The swing voter is inclined to evaluate candidates and policy ahead of party lines and rhetoric. By their nature these voters may be among the most capable voters within the general populace.

    Incidentally, executives at some level promote policy. They put forward policies that support their strategies and foster intended cultures within their organizations. The distinction is only whether one is engaged in creating an organizational culture at work or a societal culture.

  7. Post

    I’m not arguing that people don’t make judgments about the candidates’ CIP (level of capability). I’ve said that I think that Jack is right, and that Americans currently want a L6 for president. They get some of that from the debates and the interviews. I think that Alison Brause was correct in her original assessment but not in the conclusions reached by her and KC in 2004. I imagine that smart campaigners will begin to give their lower or higher stratum candidates Str6 arguments. Because it’s not that the American people want a higher level person: they want a level 6 person.

  8. I am not sure that the work by Cason and Brause can be considered as good science.

    The original development curves, up to mode 6 have been verified in Jaques work, other studies at Bioss and by independent researchers. CPA has been verified as a method of measuring Current Level of Capability and in longitudinal studies has been verified in predicting Future Level of Capability. I am not aware of any published or independent research validating CIP methods.

    However, I do assume that CIP works. In order for Cason and Brause to make their claim they would need to point to some study where assessment of speeches is validated against interview-based CIPs of the same persons. Without that proof in place their postulations have to be rejected.

    However there is probably something in the statements that (some?) people make intuitive assessments of capability. In party-based systems members select and prioritize their listed candidates for elections. According to my Danish colleague, both politician and CPA-practitioners), selections are based on “electability”, making sense of policies and as a result many, or even most, of those selected as parliament candidates have level 4 capability. Very good for a small Nordic country.

  9. Post

    CORRECTION: I misspelled Dr. Brause’s name in the original post as “Krause”. We are discussing the PhD work of Alison Brause and a later study with Dr. Kathryn Cason of the debates by democratic candidates and later US presidential nominees in 2004. I regret the error.

    CORRECTION: I made it seem like Alison Brause was a part of the research of the 2004 election. That does not seem to be true. It appears that Dr. Cason did the work on 2004 and released it to the press without Dr. Brause’s assistance. I regret the error.

    If you’ve been following this, some links follow at the end.

    CIP has been shown to match with manager’s judgment in consulting work by PeopleFit and others. CIP interviews have been tested against Career Path Assessments by PeopleFit, too, and they have seen only small variance. It may be that none of these are measuring what is theorised, but it does seem that they are all (CIP interviews, CPAs, & managerial judgment) measuring the same thing.

    As for the science, it’s at least better than that for Myers-Briggs

    1. Press release on Alison Brause’s PhD work. Includes a link to a summary of her research.
    2. New Yorker “Talk of the Town” short article on Cason as the one saying Kerry will win. (Brause is not mentioned.)

      ason is prepared to make a scientific prediction, on behalf of the institute: John Kerry will beat George Bush. How confident is she of this? “One hundred per cent.”

    3. PeopleFit’s republication of Brause’s summary [PDF]
    4. Earlier post of mine on the Boston Globe article on the 2004 controversy
  10. Cason & Brause were published in a peer-reviewed journal which is why I feel free to question the scientific validity of using a method validated for use in an interview in a context of text analysis, without proving the validity in that context. That would be done by first doing an assessment based on a speech (debates or a document), then interviewing and ascertaining that the results are the same.

    Considering how modern presidential candidates seem to be “primed” before debates it must be difficult to assess what is true thinking from the candidates side and what is conditioned learning.

    Please note that I am not questioning the validity of CIP itself. As I am doing research in this area I am interested in published peer-reviewed articles validating methodology.

    I agree that MBTI does not prove anything, particularly as research has shown that respondents reply differently and get different results depending on the context they are doing it in.

    The 2004 controversy is regrettable and did not create positive interest in what we do. In retrospect it can only be seen as either ill-judged or a determined political act.

  11. Post

    Okay, so we all appreciate the work Brause did and find it interesting. And we appreciate the underlying principles of CIP/CMP interviewing. The rest is pretty important to discuss and I am looking forward to Paul’s publications.

    For an interesting view on why American politics is different, Michael Bates recently linked to an article in Asia Times that describes
    Hockey moms and capital markets

  12. Thanks for pointing to the article about the Asian view. I agree that it is an interesting and relevant view on one advantage of voting for persons rather that parties.

    In the small town where I live (50 000 inhabitants), we have two full-time politicians, i.e. they are paid by the local council. As the majority shifts at elections they change chairs, but we still have the same two guys election after election. I don’t think they are corrupt, just incompetent, so the corruption lies in getting a wage for that. At times I wish I could find such an easy income flow.

    At times I wish that we had more dynamics so that they either had a real fight instead of cosying up to each other or even better getting them both kicked out. But then on the other hand a town with 50 000 inhabitants does not have a great talent pool.

  13. With respect to presidential debates the assessment can be made with the sole benefit of the debate. There is no better forum for assessing CIP than the debate forum where the individual is heavily engrossed in a topic that he or she is passionate about. It is unnecessary to validate this assessment against a separate interview.

    Candidates cannot be adequately “primed” in rehearsal for a true debate. If we observed the candidates last night we can find evidence of this. One can observe the cause and effect relationships that are presented by the candidates and the interrelationships between social, economic and foreign and domestic policy. The candidates drew some complex relationships among sustainable energy, reliance on foreign petroleum products, and their effects with current economic conditions and domestic security.

    From a transcript of the debate last night one could derive an assessment of the CIP employed by the candidates. My intuition is that both Barack Obama and John McCain are Stratum VI candidates. I would suggest that Obama’s current capability is higher within that stratum than McCain’s and obviously Obama’s mode is higher than McCain’s due to the differences in age. I will venture that Obama is mode 8. Perhaps Cason and Brause will validate this intuitive assessment. Consequently, I would also predict that Obama will win the presidency.

  14. I happened across this thread and would love to see the conversation continue especially because of the zaniness of the current election cycle – or, can y’all direct me to another place where this is being discussed. I was surprised that the last comment placed McCain and Obama at Stratum VI. Regarding the current cycle, does it boil down to negative temperament? In that regard, I thought Louise Sunshine’s comments about simply using ear plugs vis-a-vis Trump’s noise (presumably she was saying that outrageous comments to maintain media coverage are not useful in assessing capability).

  15. Post

    I think that Requisite Organization International (http://requisite.org) may be the best place to get more information. Cason & Brause have led these efforts in the past.

    Warren Kinston has argued elsewhere that comparing politicians to business people is comparing entirely different work domains. The research into complexity thinking indicates that he is at least partly right: one can be able to parse a great deal of complexity in one domain but only very little in another. It may or may not be relevant in this election cycle. 

    Certainly times are interesting.

Leave a Reply

Your email address will not be published. Required fields are marked *