Trade Chart / Red Seal

Analysis of Red Seal Exam Questions for Refrigeration and Air Conditioning Mechanic & Sheet Metal Worker


Prepared by the Centre for Canadian Language Benchmarks
For Human Resources and Skills Development Canada
March 2012


Executive Summary

The Centre for Canadian Language Benchmarks (CCLB) is the centre of expertise in support of the Canadian Language Benchmarks (CLB) and the Niveaux de compétence linguistique canadiens (NCLC). CLB and NCLC are recognized as the Canadian standards for describing, measuring and recognizing the language proficiency of adult immigrants and prospective immigrants.


CCLB has been working with Human Resources and Skills Development Canada (HRSDC) since 2006 to benchmark the language demands of Red Seal Trades. CCLB has also benchmarked some of the Red Seal tests following a rigorous and proven methodology.
In February 2012, HRSDC requested CCLB to benchmark questions for the Refrigeration and Air Conditioning Mechanic and Sheet Metal Worker Exams.  This report details the findings of the benchmarking.
The tests were benchmarked using the CLB and NCLC standards which are currently documented in the CLB 2000 and NCLC 2006. As revised versions of CLB and NCLC will be published in April 2012, CCLB ensured its benchmarking methodology included both the current and the revised standards. The intent was to ensure consistency between tests benchmarked in the past and those that may be benchmarked in the future. Previous experience in benchmarking Red Seal exams had determined that most questions should fall between CLB/NCLC 5-8.


The benchmarking process had the following objectives:


CCLB used two benchmarkers/language experts/test development consultants to perform the work. Anne Senior is a CLB expert who is also very knowledgeable of the NCLC standard while Hector Gauthier is a NCLC expert who is very knowledgeable of the CLB.  Both benchmarkers were thus able to make, or support, benchmarking decisions for both languages. They both also have expertise in the design of test questions and the use of plain language which further informed their recommendations.


The benchmarkers were very impressed with the clarity of the questions and the excellence of the translation. They found a great consistency between the English and French versions of both tests. All the questions fell in the expected range - between CLB/NCLC 5 and CLB/NCLC 8 - except for one question in French which, as it stands now, is a NCLC 9. A suggestion to rewrite it to a NCLC 8 has been provided in the report.  Two questions in Sheet Metal Worker were confusing in both languages and we were unable to provide a benchmark.


Except for the questions mentioned above there is parity between the English and French questions.


The benchmark level breakdown is as follows:

Refrigeration and Air Conditioning Mechanic - Benchmarking Summary

CLB Benchmark

# of questions at that Benchmark

NCLC  Benchmark

# of questions at that Benchmark

5

34

5

34

6

29

6

29

7

7

7

7

8

5

8

4

9

0

9

1

Sheet Metal Worker - Benchmarking Summary

CLB Benchmark

# of questions at that Benchmark

NCLC  Benchmark

# of questions at that Benchmark

5

36

5

36

6

27

6

27

7

8

7

8

8

2

8

2

Unable to assign a Benchmark

2

Unable to assign a Benchmark

2

The benchmarkers

The tests were generally clearly written and the translations were well done. There were only two questions with which we had any major issues (Sheet Metal work # 76941 and # 76421). Suggestions, comments and recommendations to further improve the exams are included in the report. These suggestions, comments and recommendations are synthesized below:

 

Date modified: