How is Insights Discovery Validated?

In 2005 the University of Westminster’s Business Psychology Centre performed an extensive study of the 'Insights Discovery Evaluator (IDE) English Version 3.0'. Key statistical analysis was used to study the IDE’s “item analysis”, “norm data”, “reliability” and “validity”.

For a more comprehensive review of the IDE’s properties, please read “An Overview of the Development, Validity and Reliability of the English Version 3.0 of the Insights Discovery Evaluator” produced at the University of Westminster’s Business Psychology Centre (BCP) which is available upon request.

The norm data for the IDE is of good quality, being segmented by the language of the evaluator completed; the country a respondent is based in; age (in ten-year bands) and occupation. The occupational norm data provides predictive validity. As expected but although not exclusive, those in charge of financial data tend to have a preference for objective thinking (either Red or Blue as a top two colour preference), using facts and performing detailed analysis. It should be noted whilst this data indicates that people in certain roles tend to have a preference for certain colours, it does not correlate or necessarily relate to how well they are doing their job or how capable they are in fulfilling that role. It also does not limit a certain colour energy from being good in a role that falls outside of this observation.

Cronback-Alpha coefficients, α, measure the error variance on the average inter-item correlations. When the error variance is low, which is desirable, the alpha coefficient approaches 1.00. A value of 0.70 is the commonly accepted inferior limit. Analysing 24,224 completed evaluators shows the four colours to have very high Cronbach- Alpha coefficients, providing evidence of reliability.

Test-retest reliability is determined through the administration of the same evaluator across time. It helps gauge how robust the items are. Such tests are generally expected to yield reliability coefficients ranging between 0.70 and 0.90. The results of the test-retest analysis performed on the four colour scores show very high reliability, translating into coefficients ranging from 0.81 to 0.86 for the Pearson correlation coefficients and 0.89 to 0.92 for the Cronbach-Alpha reliability coefficients for the same data. Validity Confirmatory Factor Analysis was used to test the hypothesised factor structure of the Insights Discovery model. Specifically, it is hypothesised that the four sets of 25 colour based items in the IDE should load onto the factors such that the polar opposite nature of the ‘Fiery Red’ vs. ‘Earth Green’ items is apparent and the polar opposite nature of the ‘Sunshine Yellow’ vs. ‘Cool Blue’ items is apparent. The four colours should load onto their appropriate factor only.

The constant evolution of language means that the Discovery Evaluator will continually evolve. The Insights Research Team monitors the Insights Discovery Evaluator in its 30+ languages on a daily basis. Examples of the Statistical Reliability and Validity tests can be obtained from Insights Learning & Development Ltd on request.