How the Qualification Examination is Written

The writing of the qualification examinations is a strictly standardized process conducted by the measurement and evaluation consultants at the Commission de la construction du Québec (CCQ). All of these professionals have training in skill assessment and solid experience in exam design methodology. In collaboration with trades vocational subcommittees, experienced journeymen are recruited to work with the measurement and evaluation consultants at each step in the design of a new exam. They act as content experts. 

Formulation of the table of specifications

The first step in creating the exam consists of performing a detailed analysis of the vocation to be evaluated in order to produce a table of specifications. The measurement and evaluation consultant and experts in the trade then determine the main skills that will be measured by the exam questions. To do this, they refer, among other things, to the description of the trade written by a group of journeymen. These vocation analysis reports are available online. 

A vocational subcommittee, formed of representatives from the employer associations and the union associations for the trade, must validate the table of specifications. The committee is responsible for establishing the weighting of each skill. This is what determines the importance of each skill in the exam.

Question design

Analysis of the statistical data of results obtained in the past by exam takers gives good indicators about the degree of difficulty and relevance of the questions. This information is essential for production of new exams.

A dozen one-day meetings are needed to design the exam questions (including drawings, illustrations, diagrams, charts, tables, and plans). A large number of questions are produced in order to create several versions of a single exam.

The CCQ also gathers informational intelligence from expert journeymen regarding new notions or skills for each trade in order to produce questions reflecting the reality of the construction industry. For example, if changes are made to the Québec Construction Code, questions relating to the affected trades or specialties will quickly be revised.

Validation of questions

From six to eight expert journeymen, different from those who took part in the question-design step, are asked to validate these questions. This step makes it possible, if necessary, to clarify or correct the questions, and to simply delete those deemed not relevant. The questions are then translated into English. This translation is validated by a bilingual expert journeyman.

Pre-test

Fifty journeymen, qualified in recent years, are asked to test the exam. After the measurement and evaluation consultant analyzes the results of the pre-test, questions may be revised, if necessary, before the exam is implemented. The data obtained in the pre-test make it possible to design different versions of an exam that have a similar level of difficulty and test the same skills.

Examination

The qualification examination usually includes between 50 and 70 multiple-choice questions, depending on the trade or specialty.