|
 |
|
 |
 |
 |
 |
Level of Analysis
The level of analysis for the Team Impact Builder is the executive team in its entirety. We assess the perceptions of the group executive team as to the functionality and performance of the executive team.
Firms Included in this Analysis
Several thousand executive teams are included in our assessment of reliability and validity generally. That is, however, not the case in regards to how Discrimiant and Convergent validity were established. Because of the need for multiple methods to be used to establish such validities, we have thousands of teams involved in these calculations. Over the last 15 years, the Executive Assessment Institute (EAI) has established high levels of reliability and validity of the Team Impact Builder tool and process. The next sections cover Face Validity, Reliability, Construct Validity, Discriminant and Convergent Validity, and Predictive Validity.
Face Validity
The Team Impact Builder has been developed based on previous research into executive teams at all management levels, within small to large companies, and across industries. Over the past 10 years we have researched 300 to 400 executive teams annually. And, over the past several years we have added over 1000 executive teams to the database. Today the database includes an assessment of over 10,000 teams which could be the largest database of executive teams in the United States, or perhaps the world. These teams represent the Fortune 1000, the Inc. 500, and key industry samples. The data collected from these samples is based on key research starting with the Upper Echelons perspective and ending with the most recent research on the impact of executive team dynamics on firm performance. Using this research as a foundation, and having two of the most active researchers in this area on the scientific board, has infused the tool and process with key thought leadership and an insightful empirical approach to understanding the key executive team factors that drive firm performance. However, our face validity is driven from key research. This research has created a compact list of critical variables such as heterogeneity, team trust, social integration, conflict management, team confidence, and shared cognition. These variables have been demonstrated in research as key variables and are on their face the difference between higher and lower firm performance.
Reliability
We have tested all of the constructs in the executive team financial impact system for test-retest, inter-rater, and inter-item reliabilities. In total our Chronbach’s Alphas range from .72 to .93 across all of the constructs in use. The test-retest correlations on the constructs ranged from .86 to .94. The inter-rater reliability was tested using Demoree and Wolfe’s (1984) Rwgj measure of the inter-correlations between members of the same team. The range for the Rwgj measures was from 0.69 to 0.91 which demonstrate acceptable general inter-rater agreement. Taken together we believe that we have the foundation for the development of validity through a firm foundation of inter-item, inter-rater, and test-retest reliability.
Construct Validity
Construct validity was tested in a two phase exploratory and confirmatory factor analysis approach. First, each major section of the TMTEvaluator was exploratory factor analyzed. We tested each of the dimensions using the more rigorous confirmatory factor analysis using LISREL 8.73. The loadings and fit indexes for each of the confirmatory factor analyses demonstrated that construct validity does exist. The fit indices of the final model was (GFI=0.94, AGFI=0.89, NFI=0.91, and RSMEA of 0.06). In general, this establishes across a large number of teams that model works and that the within and between statistical structures are holding.
Convergent and Discriminant Validity
The use of Multi-Trait-Multi-Method matrices means that there is a way using multiple research methods to establish additional understanding of true validity. We used interview data from just over 500 of our teams to develop matrices for each of our measures. Interview data was coded by a Delphi panel of Graduate students of a prestigious North Eastern University. These codings allowed us to develop tables of correlations between the codified interview data and the scale items for each construct. We are looking for two key indicators here: that measurements that should be correlated are correlated and that those that are not are not correlated. Our findings indicate that there are high correlations between the scale measures and the interview coded measures from the Delphi Panels. In addition, it is also clear that where there should not be correlations between different measures there is discrimination. Our research demonstrates clearly that the measures have demonstrated power of being independent constructs and that they measure what they are intended to measure. The interviews were carefully constructed and developed protocols and the high correlations between the interview data and the scalar items demonstrates that there is a clear linkage between peoples voiced perceptions and their scores on the scales.
Predictive Validity
It is our perspective that a range of TMT dynamics and compositional dimensions must be covered to fully understand the quality of the TMT. Therefore we have worked diligently over the last nine years to build the most predictive TMT variables that we could build, develop, or identify. Interestingly, most of the items that we use are items that we have either substantially altered for application to the TMT or developed completely from scratch. Table Two presents the effect sizes of each of the variables included in the TMTEvaluator. Simply, the effect size is the total percentage of the variance explained by the individual variable in the context of the model. This gives us an understanding of the importance of an individual variable to the model. Variables with higher effect sizes are deemed more important to the model. However, one should note that variables included in the model were significant and deemed important by most of the literature and our own research. Organizational size, strategy, structure, and industry or task environment are included only as control variables. These control variables allow us to develop a clearer understanding of the important of the TMT to the overall firm performance model. In total, this fully specified model gives us more accurate measures of the true influence of the TMT on the performance of the firms they manage. Taken together the range of prediction of the TMT variables on firm performance in the over 4000 firms on which the evaluator is built is nearly 35 percent. However, in earlier stage ventures, those only few years old or startups, the effect size of the TMT is found to predict as much as 60 percent of the variance in new venture financial performance. In larger more established firms the proportion of the firm’s financial performance that is predicated on the functionality of the executive team is much smaller. That is the context of the team is a key factor in how much prediction exists (i.e. the size of the firm).
Our research demonstrates that executive team composition and dynamics is a critical aspect of firm financial performance. The model demonstrates that it is possible with our patented process for understanding executive team dynamics to build key predictions as to the performance of business organizations. The prediction levels are high and range from upwards of 70 percent of future financial performance (in very young firms) to only 6 to 8 percent (in firms over 100 years and of a very large size). However, the consistency of the impact is real. Most 100 billion dollar firms are truly interested in the development of their organizations and would do a great deal to gain 5 percent in firm performance, such as EPS. These findings demonstrate a reliable and valid way to create such outcomes.
|
 |
 |
 |
 |
|
 |
|
| |
|
|
