​​​Assessment of Registration Practices:
Progress and Problems in 2013–14

Message from the Commissioner

I am pleased to present the results of my office’s most recent assessment of licensing in Ontario’s regulatory bodies. The 2013–14 assessment was a rigorous and detail-oriented process that occupied my staff for well over a year.

As I have noted in the past, I do see progress. Regulators are removing unnecessary barriers and some applicants have faster, fairer and more efficient experiences getting their professional licences. Many regulators are implementing the recommendations we have made.

But improvement is slow – too slow. I am disappointed that this assessment unearthed serious problems.

The public debate about the fate of internationally educated professionals in Ontario is becoming more and more urgent. Taxi-driving engineers were heartbreaking a few years ago; now they are a cliché. Internationally educated nurses are frustrated with the licensing process here; American recruiters, knowing of obstacles in Ontario, are openly trying to attract them.

Informed observers are more and more impatient with the existing ways of doing things. Their frustration is understandable:

• The evidence about the unemployment and underemployment of internationally trained professionals is mounting.
• On the difficult issues, public policy seems to favour the status quo.
• Regulators for the professions are not fully grasping their legal obligations.

It is in the public interest that licensing in Ontario become fairer and faster.

I am convinced that our systematic assessments contribute to improvements that will enhance access to the professions. My staff and I are pursuing these findings with provincial ministers and the regulatory bodies themselves.

Hon. Jean Augustine, PC, CM
Fairness Commissioner

Executive summary

In 2013–14, the Office of the Fairness Commissioner (OFC) undertook its second cycle of assessments of the practices that Ontario’s regulatory bodies follow for licensing people to practise their professions in Ontario. These assessments are part of the OFC’s strategy for continuous improvement, designed to improve fair access to the professions for all qualified applicants.

These assessments consisted of targeted assessments, focusing on areas of weakness previously identified in the 2011–12 full assessments.

The OFC learned a great deal through the 2011–12 assessment process, and adjusted its strategy and its assessment methods. And regulators made many improvements, implementing hundreds of the OFC’s recommendations. Because of these changes, and because the 2013–14 assessments targeted specific areas, the OFC was positioned to probe more deeply, and to focus more strongly on the transparency of regulators’ practices and on their capacity to provide evidence.

The 2013–14 assessments showed progress in many areas. As demonstrated by the 167 commendable practices the OFC identified, regulators are working diligently to improve their practices.

However, the OFC also made 247 recommendations for improvement. Clearly, problems persist, especially in areas related to:

• fairness of registration requirements
• quality of assessments and registration decisions
• access to records

There may be some underlying reasons for the persistence of these problems, such as:

• inadequate understanding by regulators of their fair-access obligations
• inadequate strategies for getting and learning from feedback provided by applicants and the public
• lack of motivation to change
• differences in legislated oversight among the various regulators, and especially between health and non-health regulators
• lack of sustainable government funding for fair-access initiatives

The assessments confirmed the need for ongoing vigilance, continuing and rigorous assessment, and the OFC’s strategy for continuous improvement. The OFC will continue to advocate for necessary change, to help enable all qualified applicants to put their skills and knowledge into practice.

1. Introduction

Purpose of this report

From May 2013 to September 2014, the Office of the Fairness Commissioner (OFC) assessed the registration practices (also called licensing practices) of 35 regulatory bodies that fall under its mandate. Of these 35, 34 were subject to a targeted assessment of certain practices.

This report shows trends in progress and problems that the OFC identified through the 34 targeted assessments. It does not describe all findings.

Overall, the assessments showed significant progress, but the OFC notes continuing problems, and a slow pace of improvement by some regulatory bodies.

Data in this report is current to September 30, 2014.

Organization of this report

The report is organized as follows:

• Section 2 describes the background to this report, including the OFC’s strategy for continuous improvement, the assessments’ role in the strategy, the assessment process, and some results from the first assessment program.
• Sections 3 to 6 discuss the trends in regulators’ improvements seen in the 2013–14 assessments:
  • Section 3 identifies reductions in recommendations and identifies commendable work.
  • Section 4 describes how regulators have made more information more accessible, and have provided greater transparency.
  • Section 5 discusses the development of more flexible registration pathways.
  • Section 6 describes improvements in training.
• Sections 7 to 10 describe key areas where the 2013–14 assessments show that problems still exist:
  • Section 7 introduces the key areas and provides an overview of the assessment data in these areas.
  • Section 8 discusses problems related to the fairness of registration practices.
  • Section 9 discusses problems related to the quality of assessments and registration decisions.
  • Section 10 discusses problems related to applicants’ access to their records.
• Section 11 explores five possible explanations for the persistence of the problems described in Sections 7–10.
• Section 12 provides concluding remarks.

2. Background and methodology

OFC’s mandate

The Office of the Fairness Commissioner (OFC) has a legislated mandate  to oversee the licensing practices of 42 regulatory bodies, to ensure that the practices comply with Ontario’s fair-access legislation:

• Fair Access to Regulated Professions and Compulsory Trades Act (FARPACTA)¹
• associated amendments to the Regulated Health Professions Act (RHPA)²

Strategy for continuous improvement

To help fulfil this mandate, the OFC uses its Strategy for Continuous Improvement of Registration Practices (April 2013).³

Assessment program

In accordance with the Strategy, the OFC completes an assessment program it developed. The program includes, at fixed intervals (cycles), comprehensive assessments of regulatory bodies’ practices.

General and specific duties

The OFC’s assessments look objectively at the state of professional registration practices in Ontario at a point in time, measuring them against regulatory bodies’ general duty and specific duties:

• Regulators’ general duty is to have registration practices that are:
  • transparent
  • objective
  • impartial
  • fair
• Their specific duties are to provide the following:
  • information to applicants
  • internal review or appeal processes
  • information on appeal rights
  • information about the documentation that applicants must submit to prove their qualifications
  • transparent, objective, impartial and fair qualifications assessments
  • training to their staff
  • access for applicants to their own records

Regulatory bodies subject to FARPACTA also have a specific duty to provide timely decisions, responses and reasons.

The general-duty principles of transparency, objectivity, impartiality and fairness are not defined in the fair-access legislation. The OFC has published its interpretation of these principles.

Assessment guides

The OFC translated the fair-access legislation into a series of about 70 clearly defined practices in the general-duty and specific-duty categories listed above that help to illustrate the practical application of the law. The OFC published them in two revised assessment guides in April 2013 – one for health regulators⁴ and one for non-health regulators.⁵ The OFC assesses regulatory bodies against the practices in these guides.

Goals

The assessment program is designed with the following goals in mind:

• to help regulatory bodies focus on continuously improving the way they register people who apply for licences for professions or trades in Ontario, by identifying areas for improvement and commendable practices
• to require regulatory bodies to commit to action aligned with meeting their obligations in the fair-access legislation

Revision and reporting

Following the completion of each assessment cycle, the OFC reviews its assessment program. During this review, the OFC identifies whether the assessment program is meetings its goals and reports to the public about its findings.

To date, there have been two assessment cycles:

• Cycle 1 began in September 2011 and was completed in August 2012.
• Cycle 2 began in May 2013 and was completed in September 2014.

Reporting of Cycle 1 results and revising the program

A Fair Way to Go

The OFC reported its findings about the 2011–12 assessment results in A Fair Way to Go: Access to Ontario’s Regulated Professions and the Need to Embrace Newcomers in the Global Economy.⁶

Broadly speaking, the report found that regulators were working in good faith to follow their obligations under the fair-access law, but that they had a fair way to go.

The report noted 304 commendable practices among the various regulatory bodies and made 339 recommendations for improvement.

Revising the program

After the 2011–12 assessment, the OFC made various changes to the program, including recategorizing many of the positive practices in the April 2013 guides.

Achievements by regulatory bodies

In the cases where OFC made recommendations, regulatory bodies developed action plans to address the recommendations.

By the time of the 2013–14 assessments, the regulatory bodies had implemented 313 (92%) of the 339 recommendations that were made in 2011–12. The remaining 26 recommendations were carried over into the 2013–14 targeted-assessment results.

This achievement suggests that some of the areas for improvement identified in 2011–12 have been resolved. However, systemic change is slow and some issues persist.

The assessments also revealed:

• gaps in regulators' understanding of the law and fair-access principles
• underdeveloped ways of soliciting feedback from applicants
• a reluctant approach to transparency

Recognizing these weaknesses helped the OFC to position the 2013–14 assessments.

2013–14 Assessment program methodology

The OFC carried out its 2013–14 assessments from May 2013 to mid-September 2014. The assessment findings reflect the registration practices assessed at that point in time.

The OFC identified 35 regulatory bodies subject to participation. Of the 35, 34 were subject to a targeted assessment.⁷

The 2013–14 targeted assessments used a consistent process and the OFC’s assessment guides to assess regulatory bodies.

Targeted-assessment process 

The OFC varied the scope of assessments for each two-year cycle, alternating full assessments with targeted assessments. The 2011–12 assessments were full assessments.

The 2013–14 assessments targeted each regulator’s problem areas. That is, a regulator's targeted assessment focused on categories of specific and general duties (in the appropriate April 2013 assessment guide) related to the OFC’s 2011–12 recommendations.

The process for each assessment was iterative. At various points during an assessment, the OFC motivated the regulatory body to provide current evidence that it was demonstrating the assessment-guide practices it was being assessed on.

If a regulatory body did not show enough evidence of demonstrating a practice in the 2013–14 assessment, the OFC made a recommendation to improve. If the body demonstrated a program, activity or strategy that went beyond the minimum set by the OFC assessment guides, the OFC noted this as a commendable practice.

Deeper investigation

With the information learned through the 2011–12 assessments and the regulatory bodies’ improvements, and because the 2013–14 assessments were targeted assessments, the OFC was able to probe more precisely, more deeply, and more rigorously in the areas of weakness in the 2013–14 assessments, and with a stronger emphasis on transparency and the providing of clear evidence.

For similar reasons, and because of the revision of the assessment program and guides (including the recategorization of practices) after the 2011–12 assessment, the 2013–14 discussions are aligned differently from the discussions in A Fair Way to Go.

3. Progress: Commendable practices and recommendations

Progress noted in the 2013–14 targeted assessments

The 2013–14 assessments reflect important progress. Regulatory bodies assessed show noticeable improvements in the areas where the OFC made recommendations in 2011–12:

• In total, 7 of the 34⁸ (20%) regulatory bodies that were assessed in 2013–14 did not receive any recommendations at all:
  • Royal College of Dental Surgeons of Ontario
  • College of Early Childhood Educators
  • Certified Management Accountants of Ontario
  • College of Medical Laboratory Technologists of Ontario
  • College of Midwives of Ontario
  • College of Respiratory Therapists of Ontario
  • Ontario College of Teachers
• In every practice category in the April 2013 assessment guides where recommendations were made in 2013–14, some regulators that were assessed in that category received no recommendations for improvement (see Table 1). For example:
  • Information for Applicants: 17 regulatory bodies were assessed in this category but only 5 received recommendations for improvement.
  • Impartiality: 17 regulatory bodies were assessed but only 11 received recommendations.
• The OFC commended 31 regulatory bodies about 167 registration practices and added 44 of these practices to its Database of Exemplary Practices, to share these examples with regulatory bodies that wish to improve their practices.

These statistics demonstrate significant progress during the past two years.

For more information about the overall breakdown of 2013–14 targeted assessment data, see Table 1.

Table 1: 2013–14 targeted assessment summary data

Table Notes:

• ^a There are two assessment guides, with slight differences in their numbers of assessment practices and categories:
  • The Registration Practices Assessment Guide — For Regulated Professions and Trades⁹ is for assessing the non-health professions and trades that are within the Fairness Commissioner's mandate.
  • The Registration Practices Assessment Guide — For Health Regulatory Colleges¹⁰ is for assessing all health regulatory colleges.

  The numbers in column 2 show the numbers of practices listed in the non-health guide, followed by the number of health-guide practices in parentheses. For example, column 2, row 1 (“Information for Applicants”) shows 4 practices for the non-health guide, and 3 for the health guide.

  The health guide does not contain the category, “Timely Decisions, Responses and Reasons.”

• ^b Some regulatory bodies were assessed in more than one category. Some of these bodies received recommendations for improvement and/or had commendable practices in more than one category.

Three positive trends

The following three sections discuss some positive trends in important areas, related to some of the issues raised in A Fair Way to Go:

• more accessible information and greater transparency (Section 4)
• more flexible registration pathways (Section 5)
• better training (Section 6)

4. Progress: More accessible information and greater transparency

Many of the OFC’s 2011–12 recommendations addressed regulators’ incomplete or unclear information, and the transparency of their licensing processes. At that time, almost all regulators needed to improve their information for applicants about timelines, requirements, the licensing process, and the required documentation.

As a result, in 2013–14, 32 regulators were assessed on transparency and 17 were assessed on information for applicants (see Table 1).

The improvement is evident.

Types of improvement

Regulatory bodies are making more information available to applicants, and in a greater variety of ways. For example, many are:

• adding new information to their websites or reorganizing existing information to make it clearer and easier to find
• providing clearer information about how to request an appeal or review of a decision
• stating more clearly the rationales for their registration requirements
• improving the quality and quantity of information that they provide to all applicants, but especially those who are internationally educated

More regulatory bodies are also informing applicants about the steps in the registration process that can be completed outside Canada. Applicants from anywhere in the world are more likely to have access to clearer information about the registration process than they did in 2011–12.

Exemplary practices

Some regulators are using a variety of exemplary ways to provide information about:

• the registration process
• decisions, timelines, and documentation
• pre-arrival steps and other information specifically for internationally educated professionals
• resources designed to help applicants navigate the licensing steps

Transparency

The OFC’s interpretation of transparency requires a regulatory body to reveal its registration procedures so that others can see how the registration process operates.

Some regulators are being more open with applicants and the public about their registration practices.

Benefits of transparency

As regulators become more open about their processes, they are better able to manage the feedback they receive when they do not communicate effectively with applicants or give them enough information. A simple strategy for providing enough information can prevent time-consuming misunderstandings with applicants.

Being proactively open can also avoid any possibility of mistrust in a regulatory body’s ability to meet its obligations.

Also, as regulatory bodies become more open about their registration practices, they are more able to learn about the problems associated with weaknesses in their registration pathways, including:

• the rationales for registration requirements and assessment methods
• training
• appeals processes

5. Progress: More flexible registration pathways

“The OFC’s recommendations aim at ensuring that all requirements are clearly aligned with the regulator’s public-protection mandate, that training from non-accredited institutions is appropriately accommodated, and that assessment methods do not set unnecessary barriers to licensing or disproportionately affect specific groups of applicants.”

A Fair Way to Go​ (OFC), page 39.

Ontario’s fair-access legislation has led to increasing openness to considering alternative pathways that individuals can use to demonstrate the competencies (the skills and knowledge) required for professional practice. Flexible registration pathways help to ensure fairness in licensing practices, by putting the focus on the competencies required to practise the profession.

The OFC’s 2011–12 recommendations showed that regulatory bodies needed to carefully review their licensing requirements and ensure that their assessment criteria were directly linked to these requirements.

The 2013–14 assessments show that some regulators have improved their information about requirements and have developed some alternatives and exemptions.

Still, the OFC made recommendations for improvement to 14 of the 15 regulatory bodies that were assessed in 2013–14 on these fairness practices (see Table 1). Changes to licensing requirements are often slow-moving. In fact, the self-regulation model often leads to escalating requirements and protectionism.¹¹

Qualifications assessment

Qualifications assessment is another essential part of the licensing process, and one that frequently creates difficulties for internationally educated applicants. There are usually different ways to assess an applicant’s qualifications or competencies, and some alternatives provide greater flexibility than others.

In 2013–14, 26 regulatory bodies were assessed in their practices that relate specifically to assessment of qualifications. While this area remains challenging for most regulatory bodies, 7 of the 26 regulatory bodies did not receive any recommendations in this area (see Table 1). This means that these 7 regulators removed unnecessary barriers to applicants in the area of qualifications assessment.

However, issues remain. Having access to an appeal of assessment decisions is an important part of fairness. While regulators may be providing clear information about how to appeal, applicants may still experience barriers, such as a lack of transparency about how to obtain records to prepare for their appeal. These problems are described further in Section 10.

Commendable practices

Besides the OPFA, many other regulatory bodies are doing commendable work to support flexible registration pathways.

Perhaps one of the reasons that these practices make up such a large number of the commendable practices is that these practices cover a wide range of topics – that is, fairness in registration practices and the quality of assessment of qualifications practices (see Table 1).

More progress needed

Some regulators are improving registration pathways, but more can be done. In the 2013–14 assessments, the OFC made recommendations to 19 regulatory bodies that need to improve their assessment of qualifications practices (see Table 1). The OFC is still identifying persistent problems in areas related to flexible registration pathways. These problems will be discussed further in Section 8 and Section 9.

6. Progress: Better training

The fair-access law requires that those involved in the licensing of applicants be well-trained in their licensing roles.

Impartiality and objectivity

The OFC also assesses regulatory bodies on the training they provide to help decision-makers and assessors to be impartial and reduce bias.

Licensing involves high-stakes decisions that have a huge impact on an applicant’s career. Assessments and licensing decisions are often made by volunteer committees composed of members of the profession. As a result, there is a risk of real or perceived bias on the part of these decision-makers, who may not be trained as assessors and who may be acutely aware of their labour-market competition, particularly in challenging economic times.

In this context, it is important for regulatory bodies to offer rigorous training to assessors and decision-makers that reduces bias.

Improvements and commendable practices

The 2013–14 assessment findings show some continuous improvement in training.

Also, the OFC noted commendable practices about the training that regulatory bodies provide for their registration staff, assessors and decision-makers.

Need for further improvement

While some regulators are improving their training to reduce bias, there are still some persistent problems that relate to training to ensure the quality of assessment and registration decisions. These persistent problems are highlighted in Section 9.

7. Problems persist

Although important progress has been made toward fair access to the professions, progress is often slow, and too many barriers remain.

The 2013–14 assessments revealed three key areas that are sources of persistent problems:

• fairness of registration requirements (see Section 8)
• the quality of assessments and registration decisions (see Section 9)
• access to records (see Section 10)

Table 2 provides an overview of the data that relate to these problems most directly.

Table 2: Most persistent problems

Sections 8, 9 and 10 discuss the main persistent problems in these three areas.

8. Problem: Fairness of registration requirements

Fairness of registration requirements is a critical area of fair access, and it is one where the assessments show the need for significant improvement.

In particular, improvement is needed in:

• developing defensible rationales for registration requirements
• recognizing acceptable alternatives

Developing defensible rationales for registration requirements

The OFC has long identified the need for regulators to explain and justify the necessity and relevance of all registration requirements.

Differential requirements

In 2011–12, the OFC recommended that regulatory bodies provide public rationales for registration requirements that can have a different impact on certain groups of applicants. When regulatory bodies provided their rationales, the OFC was largely unconvinced that these rationales were defensible. The rationales that were particularly weak were those that required Canadian work experience or practical training.

Canadian experience

A Canadian-experience requirement means that applicants must undergo a period of practical training or work experience in Canada (or Ontario) related to the profession or trade, in order to be deemed eligible to apply for a licence to practise in Ontario. This requirement is a particular focus for the OFC because it contributes to the systemic disadvantage of internationally educated applicants applying for a licence to practise a profession or trade in Ontario.

Fourteen regulators¹² require applicants to undergo a period of practical experience in Ontario or Canada, usually ranging from 6 to 18 months. These requirements have changed very little between the 2011–12 and 2013–14 assessments:

• 13 of these regulators have an unchanged requirement for Canadian work experience or practical training.
• 1 (foresters) has developed an acceptable alternative.

Organizations such as the Ontario Human Rights Commission (OHRC) are also unconvinced about the rationale of requirements for Canadian work experience or practical training. In February 2013, the OHRC approved its Policy on Removing the “Canadian Experience” Barrier, which called on regulators to justify or remove Canadian-experience requirements, or to develop alternative ways to evaluate internationally obtained experience.¹³

In its 2013–14 assessments, the OFC made 25 recommendations – more pointed than the 2011–12 recommendations – for regulators to identify the competencies associated with existing requirements for Canadian work experience or practical training.

Multi-stakeholder involvement

For some regulators, altering a Canadian-experience requirement or developing flexible pathways for meeting it is tied to the agreement of multiple stakeholders.

Supply and demand and employers’ preferences

From a fair-access perspective, supply and demand should not dictate licensing requirements, nor should employer preferences. In the 2013–14 assessments, it is troubling to note a tendency by regulators to align licensing requirements with employer preferences, without engaging in a deeper analysis of the relevance and necessity of those requirements to practising the profession.

Professional self-interest

The role of regulatory bodies is to protect the public interest, not the interest of their profession or members. Nevertheless, professional self-interest (protectionism) is commonly identified as an inherent weakness in self-regulation. The examples above attest to the risk of protectionism by regulatory bodies. The examples also reflect the deep-rooted shift in perspective that is needed, from raising the stature of the profession to protecting the public.

The OFC expects all regulatory bodies to have rationales that will explain the necessity and relevance of each registration requirement for public protection. Even if a regulatory body has a rationale for its requirements, it may be weak. The OFC tests the quality of the regulatory bodies’ rationales, by requiring the regulatory bodies to identify the strength of the connection of their registration requirements to the competencies needed for safe and competent practice.

In addition to assuring applicants that regulators’ requirements are necessary and relevant to ensuring that safe and competent practitioners are licensed, the OFC assesses whether a regulator’s requirements for licensing will unnecessarily exclude certain groups of applicants.

Reviewing requirements

It is crucial that every regulatory body critically review the necessity and relevance of all registration requirements. Defensible requirements make a logical connection to specific competencies needed to practise the profession safely. By examining the connection, a regulatory body can also identify whether these requirements are still necessary or relevant.

Regular review can open up the possibility of developing other ways of demonstrating competencies. It is only when the regulatory body demonstrates the connection between the competencies and the registration requirement that it can identify acceptable alternatives to meeting the registration requirement.

In the 2013–14 assessments, the OFC was more vigilant about requirements than in the 2011–12 assessments. OFC assessors asked for rationales for requirements and for evidence where they assessed for fairness.

Recognizing acceptable alternatives

The 2013–14 assessments reinforced the 2011–12 findings about the need for acceptable alternatives.

It is important for regulatory bodies to establish and maintain flexible pathways for meeting their registration requirements. These pathways help to ensure that all people, regardless of where they were educated, have equal opportunity to demonstrate their qualifications and competencies. Where gaps exist in an applicant’s competencies, these alternative pathways may allow the applicant to more effectively achieve competencies without repeating a full program of academic training.

For example, some regulatory bodies offer pathways that enable applicants to take a series of assessments or certain exams as an alternative way to meet an education requirement.

Sustainability and accessibility

But these opportunities may disappear when a regulatory body cannot sustain them or offer them often enough, or if they become too costly for applicants.

For example, some pathways are too resource-intensive for a regulatory body to maintain.

Certain pathways may be difficult for some individuals to access because the pathway may be offered only once a year, or in only one location, or it may accept only a certain number of individuals.

Also, the pathway may be expensive for the applicant: Certain equivalency exams or bridging programs can cost applicants several thousand dollars, and may help them to meet only one requirement.

OFC advocacy

The OFC continues to influence regulatory bodies to develop accessible, affordable and sustainable alternative pathways for applicants to meet requirements for licensing. It also continues to advocate that the Ontario government provide guidance and oversight for regulatory bodies where there is a lack of financial resources to implement or sustain alternative pathways to licensing.

Bridging programs depend highly on project funding from the Ministry of Citizenship, Immigration and International Trade (MCIIT) and the Ministry of Training, Colleges and Universities (MTCU). The OFC continues to encourage these ministries to provide sustainable funding for bridging programs.

Applicants also depend highly on financial aid through the Ontario Bridging Participant Assistance Program (funded by the MCIIT). This program offers some support to applicants and does not address the costs of any prior learning assessments or exams. The OFC encourages the broadening of this program.

9. Problem: Quality of assessments and decisions

Regulatory bodies are accountable for all the decisions they make about an applicant’s qualifications for licensing. The quality of a regulatory body’s assessment and decision-making processes has a tremendous impact on whether an applicant will be licensed to practise or not.

Ensuring high-quality assessments and decisions involves:

• developing assessment criteria linked to standards or competencies for entry-to-practice
• ensuring the validity and reliability of assessment methods
• ensuring the impartiality of decision-making
• ensuring the accountability of qualifications assessment agencies

Developing assessment criteria linked to competencies

Importance

Regulatory bodies must connect their registration requirements to the competencies necessary to enter their professions (see Section 8). But regulatory bodies must also ensure that the methodology they use to evaluate competencies allows applicants to demonstrate those competencies. The methods must be appropriate.

Developing these various connections may require a regulatory body to map its assessments to its competencies or standards for entering the profession. Making these connections helps the body to determine whether applicants are demonstrating the same competencies over and over in different assessments, or whether each particular way of assessing is actually assessing a different competency.

Lack of connections

Based on the OFC’s 2013–14 assessments, the lack of clear connections between how regulatory bodies assess and the competencies necessary to practise remains an area of weakness.

Even though many regulatory bodies have developed their profession’s competency frameworks, not all of them are doing a good job of connecting competencies to the assessment methods and criteria.

If a connection is not evident or established:

• The assessment method may not provide accurate evidence of an applicant’s competencies.
        or
• The assessment itself may be unnecessary.

The OFC continues to challenge regulatory bodies to ensure clearly established connections between the competencies necessary for licensing and the assessment methods used to measure these competencies.

Streamlining

The OFC also always encourages the streamlining of qualifications assessment processes. Regulatory bodies should determine which competencies are being assessed in multiple ways, and justify this.

For example, communication competencies may be tested through a standardized language test, or informally during a written or practical exam, an interview, or a prior learning assessment.

How many times and in what ways must all applicants’ communication competencies be assessed to ensure public protection? Reviewing assessment methods with this question in mind may help to streamline qualifications assessment processes.

Ensuring the validity and reliability of assessment methods

The OFC’s understanding of the fair-access principle of “objectivity” involves validity and reliability.

This means that objective decision-making requires that not only the criteria but also the tools, training and procedures used enable a regulatory body to measure what it intends to measure, and to do so consistently.

Regulatory bodies need to have, and be able to demonstrate to the public and applicants, methods for assessing applicants that are valid and reliable.

Assessing an applicant’s competencies by using a method that has not been reviewed for validity or reliability is a huge risk to the public interest. The risk is that regulatory bodies are making decisions based on assessment outcomes that may not be:

• reliably reflecting an applicant’s competencies
• validly measuring the competencies that the regulatory body is actually hoping to measure

If the assessments are not reliable or valid:

• The public may be at risk. Assessment methods are mechanisms to protect the public, by requiring applicants to demonstrate their competencies to practise the professions safely.
• Qualified applicants may not get the recognition they deserve or may undergo assessments that are not needed.

Valid and reliable ways of assessing applicants’ qualifications protect the public from incompetent practitioners and ensure fair access.

Barriers to improvement

Changing assessment methods and procedures is a very demanding exercise that may require substantial resources and stakeholder engagement. As a result, making changes may prove challenging for regulatory bodies with limited resources. Further, regulatory bodies may prefer the comfort of the status quo and may feel discomfort when external agencies, such as the OFC or psychometric experts, suggest changes to their assessment criteria or procedures.

But despite a lack of resources or organizational discomfort, it is crucial for all regulatory bodies to continuously assess the validity and reliability of the assessment methods they use to determine licensing.

Ensuring the impartiality of decision-making

Impartial decision-making requires that decision-makers be adequately trained and that strategies be in place to address any potential for bias.

During the 2013–14 assessments, many regulatory bodies found it difficult to explain the steps they took to ensure that they were implementing impartial practices.

Certain assessment methods are more vulnerable to allowing bias than others. For example, the OFC is concerned about the heightened risk of bias and discrimination associated with oral assessment methods, especially unstructured interviews.

Structuring interviews

Where interviews are used to arrive at an assessment or registration decision, regulators should consult with an assessment expert, who can:

• review whether the interview method is appropriate
• advise on ways of structuring the interview to ensure that the questions, process, and scoring are transparent, objective, impartial and fair

Training

In the 2013–14 assessments, deeper probing revealed that training for impartiality is critical. To minimize the risk of bias, assessors, staff, and decision-makers should be trained to:

• be aware of sources of bias
• identify their own potential for bias, and take appropriate action if they identify themselves as biased
• understand the criteria, tools, processes and procedures that are used in their roles

While most regulators have training procedures for addressing potential bias resulting from conflict of interest, that is only one source of bias. This is why the OFC continues to challenge regulatory bodies to identify sources of bias and develop strategies to reduce bias in assessment and decision-making processes.

Ensuring the accountability of qualifications assessment agencies

Most regulatory bodies outsource various licensing steps to external organizations (third parties), such as World Education Services or a national body for the profession. These organizations:

• assess academic credentials
• administer exams
• assess competency or prior learning

These third parties operate outside of the OFC’s direct oversight. However, the fair-access law holds regulatory bodies accountable for their decision to use third-party processes.²¹ Regulators bear legal responsibility for licensing decisions, and must ensure that any third parties they rely on comply with fair-access principles. The OFC requires all regulatory bodies to ensure that the third parties they use have transparent, objective, impartial and fair assessment practices.

Using third parties can be very beneficial. It can:

• facilitate labour mobility
• improve the quality of assessments
• streamline licensing costs

However, without effective strategies in place to ensure that third parties are complying with province-specific laws, Ontario regulators may subject applicants to unnecessary hurdles in par​​ts of the licensing process.

This is why the OFC continues to challenge regulatory bodies to develop effective measures to ensure that third parties’ practices comply with fair-access principles.

The fair-access law requires qualifications assessments that are transparent, objective, impartial and fair

 

Regulators must ensure that any third-party assessments are transparent, objective, impartial and fair

 

The OFC assesses regulators on whether they have effective measures in place to do this

10. Problem: Access to records

Providing access to records should be a fairly easy practice for regulators to demonstrate to the OFC. However, the OFC is still making a substantial number of recommendations in this transparency-related area. In 2013–14, the OFC made recommendations for improvement to 12 regulatory bodies that needed to enhance transparency regarding document retention and applicants’ access to their records.

This may suggest either that these regulatory bodies do not see this practice as a priority or that they may not have procedures in place to enable access to records. Without a structure or procedures in place, it may be difficult for a regulatory body to provide information about how applicants may access their records. A regulatory body’s reluctance to provide this type of information may indicate other gaps in its procedures.

When an applicant has difficulty gaining access to the information needed to understand procedures or reasons for limiting access to their records, he or she may think a regulatory body is trying to hide unfair practices.

It is an important part of transparent and fair registration practices that applicants can access their records, including their application and supporting documents. Access to these records can help an applicant to understand a decision, build an appeal case, and maintain access to documents that are difficult or costly to obtain (for example, official translations, credential or education documents, prior learning assessments, competency-gap reports, or portfolio submissions).

For these reasons, the OFC continues to make recommendations about enhancing access to records.

11. Explanations for persistent problems

Following are five possible explanations for the persistence of the problems outlined in Sections 7–10.

1. Inadequate understanding

If regulators do not understand the OFC’s interpretation of the law and its expectations for regulatory bodies, they may not see a need to change.

Others may be clear about the expectations but may not know what actions they could take to improve their processes. For example, some regulatory bodies may understand that they are accountable for ensuring that their third parties are transparent, objective, impartial and fair, but may not be sure how to monitor this or how to go about developing a good accountability mechanism.

Currently, training opportunities about the fair-access law are limited. For this reason, the OFC is developing online learning modules to ensure that regulatory bodies have greater access to the required training. The OFC expects that if regulatory bodies have greater access to training in the objectives of the fair-access law, more of them will be able to demonstrate fair practices.

The OFC will continue to educate regulators, through resources or activities, to help them understand the OFC’s expectations.

2. Inadequate public-participation strategies

Some regulators may not solicit feedback about their processes from their applicants or the broader public.

By limiting the opportunities for applicants to provide feedback on their experience, regulators may be limiting their own understanding of the problems in their registration processes.

Participation by the broader public is also valuable, enabling a regulator to understand public perspectives. Consultation can also enable public trust. In addition, if members of the public are more involved, they will be more alert to regulatory actions that are not in the public interest.²²

The OFC will continue to challenge regulatory bodies to collect feedback from applicants about their experience in the licensing process.

3. Lack of motivation to change

A small number of regulatory bodies are not motivated to change their registration practices, even though their practices may create unnecessary hurdles for applicants. Some regulatory bodies challenge some of the OFC’s fair-access recommendations, or even its right to suggest or demand changes.

Motivating change in organizational behaviour and enforcing compliance presents challenges, given the current licensing system.

4. Differences in legislated oversight

The OFC does not have the authority to enforce compliance evenly among the regulatory bodies it oversees. There are differences in external appeals mechanisms, public reporting, timeliness expectations, and records and transparency requirements between the health and non-health regulators. Although the government has committed to addressing some of these differences, the changes are not yet in place.

The fair-access law requires that all regulatory bodies ensure that their registration practices adhere to the same four principles of transparency, objectivity, impartiality and fairness. The regulatory bodies must work in the public’s interest.

However, there are differences in the checks and balances in place to ensure that each regulatory body is acting in the public interest. For example, some regulatory bodies have very few public representatives on their councils. Also, different regulatory bodies have different levels of oversight by the ministries they are responsible to. These differences may increase the risk of self-interested behaviours in the regulatory bodies.

An important role for government lies in strengthening checks on self-interested action that may occur in self-regulating professions.²³ The fair-access law serves as an important check on self-interested action, but this does not mean that challenges related to government oversight will fade away.

Some regulatory bodies are ambivalent about the fair-access law. Some may still hold an organizational belief that a regulatory body is completely autonomous. In fact, a self-regulated profession is only autonomous within the limits of its legislated role to act in the public interest.

The OFC will continue to express its concerns and propose recommendations that the Ontario government:

• ensure that each regulator subject to the fair-access law faces equivalent consequences for inaction and non-compliance
• resolve differences in external appeals mechanisms, public reporting, timeliness expectations, and records and transparency requirements between the health and non-health regulators
• require a larger representation of members of the public in the regulatory bodies’ decision-making processes

5. Lack of sustainable government funding for fair-access initiatives

Some regulators may not have enough resources to address fair-access problems on their own. Implementing changes to address persistent problems may be expensive, time-consuming and complicated.

Alternative pathways often bring needed flexibility to the licensing process, but only if these pathways are sustainable over a long term for regulators, and affordable and accessible for applicants.

Despite access to a range of acceptable alternatives for meeting experience or academic requirements, internationally educated professionals still experience delays and costs as they work to demonstrate equivalency of qualifications. Delays may occur because of a change in the sustainability of a pathway. For example, bridging programs designed to reduce hurdles are often subject to inconsistent or unreliable funding.

The OFC will continue to advocate that the government fund more alternative pathways to licensing, such as bridging programs.

12. Conclusions

Continuing improvement and persistent problems

The 2013–14 assessment findings show that regulatory bodies continue to improve their practices. They provide clearer information about registration requirements, alternative pathways, timelines, documents required, and qualifications’ assessment practices. Some have developed new alternative pathways and have publicly documented their rationales for registration requirements.

Overall, regulatory bodies were better positioned to be assessed with greater scrutiny in 2013–14, because they had improved their practices since 2011–12.

As a consequence, the OFC had a higher expectation for transparency and for the evidence it required.

A higher expectation for transparency in registration practices uncovered more complex and persistent problems (see Sections 7 to 10).

Wide range of licensing approaches

To date, there are no established standards or preferred approaches for licensing that all regulatory bodies adhere to. As a result, there is great variety among the various regulators in areas such as:

• the initial application process
• requirements for the format and content of educational documents that applicants need to submit
• the scope of an appeal
• the approach to orienting decision-makers

As a result of this variety, there are also many ways in which duplication and barriers to licensing can arise. Some of these barriers are complex, and addressing them may require time, resources and multi-stakeholder partnerships.

Learning from commendable practices

Regulatory bodies look to each other to learn different approaches to licensing practices and strategies for their work. The OFC continues to provide information about commendable or exemplary work that regulatory bodies are doing, to help other regulators that want to improve their own practices.

Using multi-stakeholder strategies

Strategies that include multi-stakeholder partnerships may have the most potential for streamlining and reducing unnecessary burdens for applicants. Many of the persistent problems – from mapping competencies to implementing alternative pathways – require regulators, government and other partners to help implement change successfully.

Looking forward

What the OFC has learned from the 2013–14 assessments will help inform its approach to the next cycle of assessments.

The 2013–14 assessment of registration practices confirmed the importance to the OFC of ongoing vigilance, rigorous assessment, and the value of its strategy for continuous improvement.

The OFC maintains its resolve to work with all stakeholders – regulators, government, third-party agencies, applicants, and the public – toward registration practices that are transparent, objective, impartial and fair for all applicants in Ontario.

Notes and references