|
 |
|
 |
 |
 |
 |
Level of Analysis
The level of analysis of the c360 is the leadership of any individual
Individuals Included in this Analysis
Almost 2,000 managers, including 1,000 CEOs, are included in this assessment of the validity of the instrument. The reliability and validity of the c360 has been assessed and established over the last 15 years. The next sections cover Face Validity, Reliability, Construct Validity, and Predictive Validity.
Face Validity
This tool assesses research-driven leadership behaviors that have been identified to correlate widely with performance across industries. Key researchers include Pearce et al (2001 & 2003) and Pearce, Ensley, and Hmieleski (2006) and Judge (2001) . Many of these constructs were included in the original Bass and Stogdill Ohio State University leadership studies. However, the exact form which this particular research is based comes from Tepper et al. (2007) and represents a critical step forward for leadership 360s. The difference is clear. Previous leadership assessments are built upon universal assumption of what “good” and “bad” leadership is in absence of the situation and context in which the leader is operating. Without understanding the context in which a leader is operating, 360 assessment reports present only part of the necessary data set to be able to identify relevant strengths, gaps and targeted development opportunities, The Context-Based 360 gathers respondent data on critical leadership behaviors, and matches their behavior with the situation in which they are leading. .
Reliability
We have tested all of the constructs in the c360 for inter-rater and inter-item reliabilities. In total our Chronbach’s Alphas range from .75 to .84 across all of the constructs in use. The inter-rater reliability was tested using Demoree and Wolfe’s (1984) Rwgj measure of the inter-correlations between subordinates from the same leader. The range for the Rwgj measures was from 0.74 to 0.91 which demonstrates acceptable general inter-rater agreement. Taken together we believe that we have the foundation for the development of validity through a firm foundation of inter-item and inter-rater reliability.
Construct Validity
Construct validity was tested using confirmatory factor analysis. We tested each of the dimensions by confirmatory factor analysis using LISREL 8.73. The loadings and fit indexes for each of the confirmatory factor analyses demonstrated that construct validity does exist. The fit indices of the final model were (GFI=0.95, AGFI=0.91, NFI=0.92, and RSMEA of 0.05). This model tested all dimensions from the four leadership super constructs of Directive, Transactional, Transformational, and Empowering with a sample of 1000 respondents. The findings demonstrate that in general, the within and between statistical structure of the instrument is holding.
Convergent and Discriminant Validity
The use of Multi-Trait-Multi-Method matrices means that there is a way using multiple research methods to establish additional understanding of true validity. We used interview data from just over 200 of our leadership assessments to develop matrices for each of our measures. Interview data was coded by a Delphi panel of Graduate students of a prestigious University in the state of New York, USA. These codings allowed us to develop tables of correlations between the codified interview data and the scale items for each construct. We were looking for two key indicators here: those measurements that should be correlated are correlated and that those that are not are not correlated. Our findings indicate that there are high correlations between the scale measures and the interview coded measures from the Delphi Panels. In addition, it is also clear that where there should not be correlations between different measures there is discrimination. Our research demonstrates clearly that the measures have demonstrated power of being independent constructs and that they measure what they are intended to measure. The interviews were carefully constructed and developed protocols and the high correlations between the interview data and the scalar items demonstrates that there is a clear linkage between peoples voiced perceptions and their scores on the scales.
Predictive Validity
Of greatest concern here is prediction. Research is the foundation of what we do at the Executive Assessment Institute. We are scholars first, and thus we have a need to develop the overall level of prediction. In the case of the c360 we had two prediction tasks. From a predictive standpoint the c360 is designed to accurately portray the current leader behaviors engaged in by those being assessed and therefore to predict future leader behaviors. As we believe that past behavior is the best predictor of future behavior it is critical we be able to accurately depict leader performance. To this end, we have found critical linkages between leader behavior of the executive group and the CEO in particular, and linkages with firm level financial performance. As the executive group is the model by which all leadership is judged in an organization then it is critical that they do so in the most effective way possible. Our studies of over 400 CEOs demonstrates that their leadership is strongly correlated with how decisions are made by their executive teams (R2 of 0.56 on key team decision making dynamics and a direct 0.11 R2 on firm financial performance). These key numbers indicate that the time lagged research finds a clear and strong relationship between CEO leader behavior, team dynamics, and firm financial performance. Taken together these statistics indicate strong predictive validity for the c360 leadership instrument.
|
 |
 |
 |
 |
|
 |
|
| |
|
|
