Main menu

Prioritizing user feedback from our D-ROL initiative

By: Erin Whittingham, Product Development Analyst

As part of our ongoing efforts to ensure that CaRMS Online continues to meet the evolving needs of our user community, CaRMS launched the Development – Rank Order List (D-ROL) initiative last November.

The first phase of the initiative involved in-person focus groups with faculty users at the majority of Canada’s medical schools. These focus groups, which concluded in mid-March, yielded a tremendous amount of feedback that we will use to optimize our ongoing investment in CaRMS Online and ensure that future development priorities reflect both how the platform is used by our clients and our internal business needs. Now that our team has had a chance to synthesize all of the data we collected, I am pleased to be able to share some of our findings.

National priorities

At the outset of the D-ROL initiative, we set out to understand how programs use CaRMS Online in six key areas: the landing page, global and custom filters, the view application section, scoring, the requirement to print application documents, and file review groups. Focus group participants also identified other areas of interest that factored into our discussions: bulk entry, reports, applicant portal features, program administrator accounts, interview management tools, and communication tools.

At the conclusion of phase 1, we visited 13 schools and spoke with a total of 115 PGME managers, program directors, program administrators and file reviewers from programs of every size. Our analysis of consultation data allowed us to create the following list of top five priorities from a national perspective:

  1. Enhancing the information provided on the landing page and enabling quick access to applications for file reviewers.
  2. Adding the functionality for user-created file review groups to enable more efficient review of applications.
  3. Creating a centralized scoring form that can be filled out directly within CaRMS Online to eliminate the need to manually compile scores.
  4. Enhancing the view application section to enable quick access to documents and applicant status information, as well as better management of applicant lists.
  5. Adding a bulk entry functionality that would allow users to enter applicant status information for all applicants on one page.

Our current task is to map these identified national priorities onto our development roadmap, taking into account each item’s feasibility, the logical sequence of development and deployment time.