About Ryerson

November 6, 2006

Both Maclean’s and The Globe and Mail published their annual rankings of Canadian universities the week of October 30th, 2006.  There are significant differences between the two publications with respect to methodology, presentation of results and clustering of institutions. Maclean’s includes Ryerson with “Primarily Undergraduate,” while the Globe and Mail includes Ryerson under “Large Universities.”  The result is that these publications provide starkly different views of the University.

This year, Ryerson joined with many other Canadian universities in withdrawing from Maclean’s annual rankings issue. It was not a decision taken lightly, and we appreciate what Maclean’s has done to put the focus on Canadian universities. In the end, however, the methodology of the rankings reduced the validity of the statistics to such a degree that we could not justify participating.

The Globe and Mail’s University Report Card is based on their own on-line survey of university students who used studentawards.com student aid website.  The survey included questions on the quality of education and teaching, equipment and library resources, faculty knowledge and interaction with students, reputation, career preparation, student services, facilities, residences, and student support. There are weaknesses in the methodology.  The sample of students is not necessarily representative of each university’s student body nor is it random.  The Globe and Mail acknowledges some of the limitations of the survey.

We believe deeply in accountability to our students and the public, and we recognize that students make comparisons. However, the information must be useful.  For the past several months universities in Ontario have been working together to develop a new online tool, called Common University Data Ontario – or CUDO – which will release, in a common format, information that is useful to students about programs, admissions policies, student life, expenses, entering grades, and other data.  CUDO will provide potential students and their families with a deeper understanding of the wide range of choices available at the university level in Ontario.  This should help potential students make more informed choices about the path that will help them best meet their own unique educational goals.

At Ryerson, CUDO will be launched within a broad accountability page that gathers, in one location, many of the performance indicators and measures that we already use, along with additional information about Ryerson. We will continue to develop the site after launch to provide more information as it becomes available.

Ryerson has for some time published a range of performance indicators, including the National Survey of Student Engagement and the Canadian Undergraduate Survey Consortium. Provincial accountability agreements and key performance indicators are other examples of publicly available information about our performance.  We intend to continue our efforts to develop new and more sophisticated measurement tools and make them widely available.

Below are brief analyses of the methodologies used for both the Maclean’s and Globe and Mail rankings, from a Ryerson perspective. You can see the Maclean’s rankings at www.macleans.ca/universities and The Globe and Mail University Report Card at www.theglobeandmail.com/reportcard.

Sheldon Levy
President

Maclean’s Magazine University Rankings 2006

Maclean’s Magazine 2006 University Rankings follows past years’ ranking approach with rankings for 23 input indicators related to the student body, classes, faculty, finances, library and alumni, along with 4 reputation indicators based on a reputation survey. 

In addition to the traditional rankings, it published for 16 universities, including Ryerson, partial results of 4 non-core questions (i.e. the questions used by Maclean’s are not part of the core National Survey of Student Engagement (NSSE) benchmarks) from the 2004 or 2005 NSSE survey.  It published for 26 universities, including Ryerson, partial results for 3 questions from the 2005 Canadian Undergraduate Survey Consortium (CUSC) survey.  It published for 23 institutions partial results for 3 questions from the 2006 Maclean’s Graduate Survey.

The information behind this year’s rankings is inconsistent from institution to institution.  It should be noted that 26 of 47 universities did not participate in this year’s Maclean’s rankings (Alberta, Brandon, UBC, Brock, Calgary, Carleton, Concordia, Dalhousie, Lakehead, Laurentian, Lethbridge, Manitoba, McMaster, Moncton, Montreal, UNB, Nipissing, Ottawa, Queen’s, Ryerson, Simon Fraser, Toronto, Trent, Western, Windsor and York).   For the 21 institutions that did participate, Maclean’s used their submitted 2005-06 data.  For the universities that did not, Maclean’s used 2005-06 data from the three federal research granting councils (2 indicators), 2004-05 Canadian Association of University Business Officers financial data (5 indicators), and old information submitted for the 2005 Maclean’s rankings for the remaining indicators.

The 7 student survey questions used from the 200-plus questions found in the two surveys in which Ryerson participated misrepresent the intent and purpose of the surveys and survey outcomes (note: the results of each of these surveys have been presented to the Board of Governors and Academic Council and summary reports can be found on the University Planning Office website ryerson.ca/upo/uporeports.html).

Maclean’s has added an on-line Personalized University Ranking Tool at www.macleans.ca/universities for people to create their own customized evaluation of universities.

Maclean’s University Rankings was the second “ranking” publication released the week of October 30th.  Earlier The Globe and Mail released its University Report Card.  There are significant differences between the two publications with respect to methodology, presentation of results and clustering of institutions. Maclean’s includes Ryerson with “Primarily Undergraduate,” while the Globe and Mail includes Ryerson under “Large Universities.”  The result is that these publications provide starkly different views of the University.

The Globe and Mail University Report Card 2006

The Globe and Mail’s University Report Card is based on a survey of university students who used studentawards.com student aid website.  The survey included questions on the quality of education and teaching, equipment and library resources, faculty knowledge and interaction with students, reputation, career preparation, student services, facilities, residences, and student support. Details can be found at www.globeandmail.com/reportcard.

Ryerson is included in the ‘Large’ category among universities with enrolment greater than 22,000 students (Alberta, UBC, Calgary, Concordia, Laval, Manitoba, McGill, Montreal, Ottawa, UQaM, Ryerson, Toronto, Waterloo, Western, York).   There are three other categories: Medium 12,000 – 22,000; Small 4,000 – 12,000; Very Small less than 4,000.  Based on the students’ responses, each university is given a letter grade from D to A+ for each question. 

Relative to other universities, Ryerson did well with respect to quality of education, average on most other questions and very good on the quality of career preparation. 

The Report Card has more editorial content than in the past, providing generally positive coverage of Canadian universities and stories on individual students, faculty and presidents.  The Globe has also introduced the University Navigator, a web-based search tool that allows prospective students to compare universities using their own criteria from the data amassed for the Report Card.

There are weaknesses in the methodology used.  The sample of students is not necessarily representative of each university’s student body nor random.  The Globe and Mail cautions that as a result of sampling error there may not be statistically significant differences between universities assigned different letter grades.  The National Survey of Student Engagement (NSSE) is a more reliable survey instrument.  Ryerson will be providing the results of the 2006 survey in the near future.

Given the methodology used by the Globe and Mail, we must be careful not to draw significant results from the survey.  However, overall, the survey reaffirms many of the observations from other student surveys we have conducted.

For a young university and the only young one in the large institutions’ category – Ryerson did well in this survey.