How the Organizational Health Assessment was Developed
The Ability to Identify and Quantify the Human Side of Your Enterprise
To help provide you some context around how our approach to organizational health will help you, following is a brief history of how the Work Systems Organizational Health Assessment was developed.
Work System Associates, Inc. was formed in 1980. Like many consulting companies it was founded by a University professor, Dr. Harold Resnick, who wanted to put his theories and experience about adult learning and organizational development (OD) into use outside the lecture room.Towards the end of the 1980s, as the company grew and the assignments became larger, Dr. Resnick started to recruit additional like-minded people from different backgrounds to undertake the growing book of business. Two of the people recruited, both founders of Work Systems Canada, Howard Corbett and Richard Jones were fortunate to be given the opportunity to work alongside Dr. Resnick. There are many others who have benefited from his coaching and counseling who have also gone on to be exceptional consultants in the Organizational Development field.
Dr. Resnick was not only a great teacher but also a visionary. He recognized that the work WSA was doing — principally developing organizations through running programs (called “interventions”) that expanded the abilities of the senior managers, managers and individuals — was important but not sufficient to create a truly world class productive working environment.
Dr. Resnick is a believer in the old adage, “What gets measured gets managed”. As a result he decided that Work Systems Associates would undertake a multi-year research project to see if it was possible to create a validated tool that would provide a “snapshot” of a company. The tool would enable accurate and consistent measurement of work force attitudes balanced against a simple model enabling all managers to understand the productivity of the organizations they led. This was a bold move for a small consulting company based in Marlborough, Massachusetts.
Work Systems’ Organizational Beliefs
Before initiating the research it was important to lay out the hypothesis we had for what constituted an outstanding corporation. This was in 1989 and 16 eminent MIT faculty members published a report after a two year study of North American business and industry. It was a stinging indictment of the current practices and suggested that greed, short term thinking and lack of investment in both people and equipment were the main culprits.
Against this backdrop we tried to simplify our thinking. It was as follows:
1. World class companies will have two ways of improving their standing.
2. Productivity can be increased in two ways:
- Technology – improved computer systems, robotics etc.
- People – increasing the productivity of every individual
People offer the greatest potential for improving productivity. To do this we need to know:
- What are the major factors?
- What are the baseline data that establish the standards against which productivity improvement can be measured?
- What tools could be used to measure the productivity of people in organizations?
How can these major factors be adjusted to increase workplace productivity?
The Research Plan
The plan had several phases and was designed to provide ample time to ensure the work of the company could profitably continue while the research continued.
Phase 1 – Creating the Model
The model was developed over the whole 3 years. It is the result of much iteration and is shown below. It was the result of a large literature search the bibliography for which covered over five pages.
Phase 2 – Validating the Model
This was done by an extensive search of the literature regarding productivity and performance. It culminated in a Symposium, sponsored by WSA, involving senior managers from a number of clients who met specifically to review the model and the variables in each factor.
The outcome was an expansion of our original model by one factor – to eight. This served as the basis for developing the instruments.
Phase 3 Developing the Instruments
The eight factors were expended into over 300 items felt necessary to fully explore and understand the productivity of a company. There were two types of questions:
- Factual statements presented for response on a seven point Likert scale.
- Pairs of words or phrases designed to force a choice response on a similar seven point scale.
The questions were reviewed and revised several times by WSA’s own consultants.
Phase 4 – Validation of the Instruments
The 300 items were then validated for:
- Importance of content
- Clarity of the statement
- Assignment to the correct factor in the model
This process was complex and lengthy. A “Validation Kit” was developed and sent to 50 senior managers across North America. They were asked to respond to the clarity, meaningfulness and assignment to the factor in the model.
They all rated each item on a five point scale where a 5 was the lowest value and 1 the highest. This was done in isolation and with no pre-cueing from the WSA research team.
To be accepted, an item it had to meet three criteria as follows:
- Importance: an average rating of 2 or lower
- Clarity: an average rating of 2 or lower
- Finally at least 50% of respondents had to place the item in the same factor.
The outcome was:
- Over 300 items were whittled down to 136 for inclusion in the instrument.
- Five factors had a name change
- One factor was eliminated and another split into two.
Phase 5 – Creating the Data Pool
The challenge was to create a North American standard for the responses to the 136 items. The criteria selected for inclusion in the list were representative of:
- A broad spectrum of companies in the US and Canada
- A variety of industries
- A variety of geographic locations
- A variety of sizes of organization
- A variety of growth rates
- A variety of organizational maturity
Several sources were used to create the list:
- Business Week
- Forbes 500
- Fortune 500
- Inc. Magazine
- WSA’s own client database
Each company was then approached and asked if between 45 and 70 individuals would participate. To provide a balance across the three levels at least 10 needed to be managers and 5 senior managers.
The response was most encouraging. Over 70 companies agreed to participate enabling the initial database to start with over 4,000 individuals, managers and senior managers from a wide variety of companies across North America.
Phase 6 – Beta Testing
The next task was to Beta test the instrument and again companies willingly involved themselves resulting in over 51 companies providing the completed instruments to WSA.
Phase 7 – Organization and Analysis of the Data
This was the most complex and most important part of the project. How did WSA validate what the item results were and know them to be consistent and repeatable? How could WSA be sure that every question was clear enough to measure what it was intended to measure?
It was determined that the research would be more meaningful if the influence of the different organizations could be equalized so that the impact of the number of the participants was the same and ensure each group (individuals, managers and senior managers) were normalized.
This was statistically complex and involved the use of multiple external experts who performed a battery of statistical analysis over several weeks. Some of the tests were:
- Alpha reliability testing
- Cronbach alpha coefficients
- Item to item correlation for all 136 items
- A principal component analysis
- A-Z score analysis was completed
Next, the factors were analyzed to create a mean and a standard deviation for factors as well as items. All the factors and 135 of the items passed the tests for purity, uniqueness and reliability.
The next task was to test for relationships across the factors. The results enabled WSA to substantiate the model, validate the instruments and understand the implications of the item and factor scores in enhancing the productivity of the organization.
In 2009 Work Systems Canada decided to completely overhaul how the data was presented. At the same time the system was re-labeled. The new name selected was the Organizational Health Assessment [OHA]. The format was also simplified and made easier for clients to understand the data.
The survey has been available for the past twenty years. However, the questions are still valid today. In fact, researchers today (Mark Huselid and Brian Becker – The Workplace Scorecard) reaffirm the importance of the 31 action items in the ODS. Added to this the importance of having an “MRI for the human side of enterprise” has become ever more important for today’s CEO. The rise of competitors in China and India put most North American companies in situations where the workforce must continue to improve productivity faster than ever.
The internet has also made data collection easier and more confidential. Reports can now be more easily customized and questions for individual companies can easily be added.
Since the update the survey has been in almost contiguous use with several clients. The ability to get to the “root cause” of issues has enabled more than one CEO to remark that they look at their company differently today as a result of the data platform the OHA provided.