ࡱ>   #` bjbjmm paDpppRqLq2spt@NtNtNtNtht tt  $ΰh6/{NtNt{{/NtNt{XNtNt { R٥Nts 0SgNpӃD5M02SƳ$Ƴ<٥٥Ƴ`|tyLf| q|t|t|t//;X|t|t|t2{{{{DK$K Developing and Applying Rubrics Mary J. Allen,  HYPERLINK "mailto:mallen@csub.edu" mallen@csub.edu, February 2008 Scoring rubrics are explicit schemes for classifying products or behaviors into categories that vary along a continuum. They can be used to classify virtually any product or behavior, such as essays, research reports, portfolios, works of art, recitals, oral presentations, performances, and group activities. Judgments can be self-assessments by students; or judgments can be made by others, such as faculty, other students, fieldwork supervisors, and external reviewers. Rubrics can be used to provide formative feedback to students, to grade students, and/or to assess programs. There are two major types of scoring rubrics: Holistic scoring one global, holistic score for a product or behavior Analytic rubrics separate, holistic scoring of specified characteristics of a product or behavior Rubric Examples Holistic Critical Thinking Scoring Rubric (Facione & Facione) Holistic Critical Thinking Rubric (Portland State University) Critical Thinking Rubric (Northeastern Illinois University) Scoring Guide for Critical Thinking (California State University, Fresno) Information Competence (CA State University) Holistic Critical Thinking Scoring Rubric Facione and Facione 4Consistently does all or almost all of the following: Accurately interprets evidence, statements, graphics, questions, etc. Identifies the salient arguments (reasons and claims) pro and con. Thoughtfully analyzes and evaluates major alternative points of view. Draws warranted, judicious, non-fallacious conclusions. Justifies key results and procedures, explains assumptions and reasons. Fair-mindedly follows where evidence and reasons lead. 3Does most or many of the following: Accurately interprets evidence, statements, graphics, questions, etc. Identifies relevant arguments (reasons and claims) pro and con. Offers analyses and evaluations of obvious alternative points of view. Draws warranted, non-fallacious conclusions. Justifies some results or procedures, explains reasons. Fair-mindedly follows where evidence and reasons lead. 2Does most or many of the following: Misinterprets evidence, statements, graphics, questions, etc. Fails to identify strong, relevant counter-arguments. Ignores or superficially evaluates obvious alternative points of view. Draws unwarranted or fallacious conclusions. Justifies few results or procedures, seldom explains reasons. Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions. 1Consistently does all or almost all of the following: Offers biased interpretations of evidence, statements, graphics, questions, information, or the points of view of others. Fails to identify or hastily dismisses strong, relevant counter-arguments. Ignores or superficially evaluates obvious alternative points of view. Argues using fallacious or irrelevant reasons, and unwarranted claims. Does not justify results or procedures, nor explain reasons. Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions. Exhibits close-mindedness or hostility to reason. (c) 1994, Peter A. Facione, Noreen C. Facione, and The California Academic Press. 217 La Cruz Ave., Millbrae, CA 94030. Permission is hereby granted to students, faculty, staff, or administrators at public or nonprofit educational institutions for unlimited duplication of the critical thinking scoring rubric, rating form, or instructions herein for local teaching, assessment, research, or other educational and noncommercial uses, provided that no part of the scoring rubric is altered and that "Facione and Facione" are cited as authors. Retrieved September 2, 2005 from http://www.insightassessment.com/pdf_files/rubric.pdf Portland State University Studies Program Holistic Critical Thinking Rubric* Inquiry and Critical Thinking Rubric Students will learn various modes of inquiry through interdisciplinary curriculaproblem posing, investigating, conceptualizingin order to become active, self-motivated, and empowered learners. 6 (Highest)Consistently does all or almost all of the following: Accurately interprets evidence, statements, graphics, questions, etc. Identifies the salient arguments (reasons and claims) pro and con. Thoughtfully analyzes and evaluates major alternative points of view. Generates alternative explanations of phenomena or event. Justifies key results and procedures, explains assumptions and reasons. Fair-mindedly follows where evidence and reasons lead. Makes ethical judgments. 5Does most the following: Accurately interprets evidence, statements, graphics, questions, etc. Thinks through issues by identifying relevant arguments (reasons and claims) pro and con. Offers analysis and evaluation of obvious alternative points of view. Generates alternative explanations of phenomena or event. Justifies (by using) some results or procedures, explains reasons. Fair-mindedly follows where evidence and reasons lead. 4Does most the following: Describes events, people, and places with some supporting details from the source. Make connections to sources, either personal or analytic. Demonstrates a basic ability to analyze, interpret, and formulate inferences. States or briefly includes more than one perspective in discussing literature, experiences, and points of view of others. Takes some risks by occasionally questioning sources or by stating interpretations and predictions. Demonstrates little evidence of rethinking or refinement of ones own perspective. 3Does most or many of the following: Respond by retelling or graphically showing events or facts. Makes personal connections or identifies connections within or between sources in a limited way. Is beginning to use appropriate evidence to back ideas. Discusses literature, experiences, and points of view of others in terms of own experience. Responds to sources at factual or literal level. Includes little or no evidence of refinement of initial response or shift in dualistic thinking. Demonstrates difficulty with organization and thinking is uneven. 2Does many or most the following: Misinterprets evidence, statements, graphics, questions, etc. Fails to identify strong, relevant counter arguments. Draws unwarranted or fallacious conclusions. Justifies few results or procedures, seldom explains reasons. Regardless of the evidence or reasons, maintains or defends views based on self-interest or preconceptions. 1 (lowest)Consistently does all or almost all of the following: Offers biased interpretations of evidence, statements, graphics, questions, information, or the points of view of others. Fails to identify or hastily dismisses strong, relevant counterarguments. Ignores or superficially evaluates obvious alternative points of view. Argues using fallacious or irrelevant reasons and unwarranted claims. Does not justify results or procedures, nor explain reasons. Exhibits close-mindedness or hostility to reason. XNo basis for scoring. (Use only for missing or malfunctioning portfolios.) *taken verbatim from Stevens, D. D., & Levi, A. J. (2005). Introduction to Rubrics. Sterling, VA: Stylus, pp. 122-123 Northeastern Illinois University General Education Critical Thinking Rubric Retrieved 3/2/05 from  HYPERLINK "http://www.neiu.edu/~neassess/gened.htm#rubric" http://www.neiu.edu/~neassess/gened.htm#rubric Quality Macro Criteria No/Limited Proficiency (D&E) Some Proficiency (C) Proficiency (B) High Proficiency (A) 1. Identifies & Explains Issues Fails to identify, summarize, or explain the main problem or question. Represents the issues inaccurately or inappropriately. Identifies main issues but does not summarize or explain them clearly or sufficiently Successfully identifies and summarizes the main issues, but does not explain why/how they are problems or create questions Clearly identifies and summarizes main issues and successfully explains why/how they are problems or questions; and identifies embedded or implicit issues, addressing their relationships to each other. 2. Distinguishes Types of Claims Fails to label correctly any of the factual, conceptual and value dimensions of the problems and proposed solutions. Successfully identifies some, but not all of the factual, conceptual, and value aspects of the questions and answers. Successfully separates and labels all the factual, conceptual, and value claims Clearly and accurately labels not only all the factual, conceptual, and value, but also those implicit in the assumptions and the implications of positions and arguments. 3. Recognizes Stakeholders and Contexts Fails accurately to identify and explain any empirical or theoretical contexts for the issues. Presents problems as having no connections to other conditions or contexts. Shows some general understanding of the influences of empirical and theoretical contexts on stakeholders, but does not identify many specific ones relevant to situation at hand. Correctly identifies all the empirical and most of theoretical contexts relevant to all the main stakeholders in the situation. Not only correctly identifies all the empirical and theoretical contexts relevant to all the main stakeholders, but also finds minor stakeholders and contexts and shows the tension or conflicts of interests among them. 4. Considers Methodology Fails to explain how/why/which specific methods of research are relevant to the kind of issue at hand. Identifies some but not all methods required for dealing with the issue; does not explain why they are relevant or effective. Successfully explains how/why/which methods are most relevant to the problem. In addition to explaining how/why/which methods are typically used, also describes embedded methods and possible alternative methods of working on the problem. 5. Frames Personal Responses and Acknowledges Other Perspectives Fails to formulate and clearly express own point of view, (or) fails to anticipate objections to his/her point of view, (or) fails to consider other perspectives and position. Formulates a vague and indecisive point of view, or anticipates minor but not major objections to his/her point of view, or considers weak but not strong alternative positions. Formulates a clear and precise personal point of view concerning the issue, and seriously discusses its weaknesses as well as its strengths. Not only formulates a clear and precise personal point of view, but also acknowledges objections and rival positions and provides convincing replies to these.  California State University, Fresno General Education Scoring Guide for Critical Thinking Retrieved 3/2/05 from  HYPERLINK "http://www.csufresno.edu/cetl/assessment/CTScoring.doc" http://www.csufresno.edu/cetl/assessment/CTScoring.doc Scoring LevelInterpretationAnalysis & EvaluationPresentation4 - AccomplishedAnalyzes insightful questionsExamines conclusionsArgues succinctlyRefutes biasUses reasonable judgmentDiscusses issues thoroughlyCritiques contentDiscriminates rationallyShows intellectual honestyExamines inconsistenciesSynthesizes dataJustifies decisionsValues informationViews information criticallyAssimilates information3 - CompetentAsks insightful questionsFormulates conclusionsArgues clearlyDetects bias.Recognizes argumentsIdentifies issuesCategorizes content.Notices differencesAttributes sources naturallyIdentifies inconsistenciesEvaluates dataSuggests solutionsRecognizes context Seeks out information Incorporates information2 - DevelopingIdentifies some questionsIdentifies some conclusionsMisconstructs arguments Notes some biasSees some argumentsGeneralizes issuesRecognizes basic contentIdentifies some differencesCites sourcesStates some inconsistenciesParaphrases dataPresents few optionsSelects sources adequatelyAssumes information validOverlooks some information1 - BeginningFails to question dataFails to draw conclusionsOmits argumentIgnores biasSees no argumentsMisrepresents issuesMisses major content areasOverlooks differencesExcludes dataDetects no inconsistenciesRepeats dataDraws faulty conclusionsChooses biased sourcesOmits researchShows intellectual dishonesty Rubrics for Assessing Information Competence in the California State University ACRL StandardBeginningProficientAdvanced1. Determine the Extent of the Information NeededStudent is unable to effectively formulate a research question based on an information need.Student can formulate a question that is focused and clear. Student identifies concepts related to the topic, and can find a sufficient number of information resources to meet the information need. Question is focused, clear, and complete. Key concepts and terms are identified. Extensive information sources are identified in numerous potential formats.2. Access the Needed Information Effectively and EfficientlyStudent is unfocused and unclear about search strategy. Time is not used effectively and efficiently. Information gathered lacks relevance, quality, and balance.Student executes an appropriate search strategy within a reasonable amount of time. Student can solve problems by finding a variety of relevant information resources, and can evaluate search effectiveness.Student is aware and able to analyze search results, and evaluate the appropriateness of the variety of (or) multiple relevant sources of information that directly fulfill an information need for the particular discipline, 3. Evaluate Information and its Sources CriticallyStudent is unaware of criteria that might be used to judge information quality. Little effort is made to examine the information locatedStudent examines information using criteria such as authority, credibility, relevance, timeliness, and accuracy, and is able to make judgments about what to keep and what to discard.Multiple and diverse sources and viewpoints of information are compared and evaluated according to specific criteria appropriate for the discipline. Student is able to match criteria to a specific information need, and can articulate how identified sources relate to the context of the discipline.4. Use Information Effectively to Accomplish a Specific PurposeStudent is not aware of the information necessary to research a topic, and the types of data that would be useful in formulating a convincing argument. Information is incomplete and does not support the intended purpose. Student uses appropriate information to solve a problem, answer a question, write a paper, or other purposesStudent is aware of the breadth and depth of research on a topic, and is able to reflect on search strategy, synthesize and integrate information from a variety of sources, draw appropriate conclusions, and is able to clearly communicate ideas to others5. Understand the Economic, Legal, and Social Issues surrounding the Use of Information, and Access and Use Information Ethically and LegallyStudent is unclear regarding proper citation format, and/or copies and paraphrases the information and ideas of others without giving credit to authors. Student does not know how to distinguish between information that is objective and biased, and does not know the role that free access to information plays in a democratic society.Student gives credit for works used by quoting and listing references. Student is an ethical consumer and producer of information, and understands how free access to information, and free expression, contribute to a democratic society. Student understands and recognizes the concept of intellectual property, can defend him/herself if challenged, and can properly incorporate the ideas/published works of others into their own work building upon them. Student can articulate the value of information to a free and democratic society, and can use specific criteria to discern objectivity/fact from bias/propaganda.*Prepared by the CSU Information Competence Initiative, October 2002, based on the 2000 ACRL Information Literacy Competency Standards For Higher Education. For more information, see http://www.calstate.edu/LS/1_rubric.doc. Online GE Rubrics % Bowling Green University (http://folios.bgsu.edu/assessment/Rubrics.htm). Links to six general education rubrics for assessing connection, investigation, leadership, participation, presentation, and writing. % CSU Information Competence Initiative (http://www.calstate.edu/LS/1_rubric.doc). An analytic information competence rubric based on the 2000 ACRL Information Literacy Competency Standards for Higher Education. % California State University, Long Beach ( HYPERLINK "http://www.csulb.edu/divisions/aa/personnel/fcpd/resources/ge/" http://www.csulb.edu/divisions/aa/personnel/fcpd/resources/ge/). A holistic and an analytic writing rubric. % California State University, Fresno (http://www.csufresno.edu/cetl/assessment). Links to four general education rubrics for assessing critical thinking (CTScoring.doc), integration (ICScoring.doc), integrative science (IBScoring.doc), and writing (WritingScoring.doc). % California State University System (http://www.calstate.edu/acadaff/sloa/links/rubrics.shtml). Links to a wide variety of rubrics that could be adapted for general education assessment. % Johnson County Community College (http://www.jccc.net/home/depts/S00015/site/plan/). Links to rubrics for culture and ethics, mathematics, modes of inquiry, problem solving, speaking, and writing. % Northeastern Illinois University (http://www.neiu.edu/~neassess/gened.htm#rubric). Links to a writing rubric and long and short versions of a critical thinking rubric. % Palomar College (http://www.palomar.edu/alp/benchmarks_for_core_skills.htm). Links to holistic rubrics assessing communication (listening, speaking, reading, and writing), cognition (problem solving, creative thinking, quantitative reasoning, and transfer of knowledge and skills to a new context), information competency (technical competency), social interaction (teamwork), and personal development and responsibility (self-management and respect for diverse people and cultures). % State University of New York College at Geneseo (http://gened.geneseo.edu/pdfs/assess_tools_revised.pdf). Links to rubrics assessing numeric and symbolic reasoning, critical writing and reading, humanities, social science, fine arts, basic research, U.S. history, non-western traditions, natural science, and oral discourse outcomes. % University of Arkansas at Fort Smith ( HYPERLINK "http://www.uafortsmith.edu/Learning/GeneralEducationCompetenciesAndRubrics#BookmarkRubrics" http://www.uafortsmith.edu/Learning/GeneralEducationCompetenciesAndRubrics#BookmarkRubrics). Links to rubrics assessing analytical skills, communication skills, computer literacy, creativity, global and cultural perspectives, information literacy, personal responsibility, quantitative reasoning, scientific and technological literacy, and social interaction. % University of South Carolina (http://ipr.sc.edu/effectiveness/assessment/criteria). Links to seven general education rubrics for assessing electronic, humanities/cultural, math, oral communication, science, social/behavioral sciences, and writing outcomes. % Washington State University ( HYPERLINK "http://wsuctproject.wsu.edu/ctr.htm" http://wsuctproject.wsu.edu/ctr.htm). An analytic critical thinking rubric. Rubrics have many strengths: Complex products or behaviors can be examined efficiently. Developing a rubric helps to precisely define faculty expectations. Well-trained reviewers apply the same criteria and standards. Rubrics are criterion-referenced, rather than norm-referenced. Raters ask, Did the student meet the criteria for level 5 of the rubric? rather than How well did this student do compared to other students? This is more compatible with cooperative and collaborative learning environments than competitive grading schemes and is essential when using rubrics for program assessment because you want to learn how well students have met your standards. Ratings can be done by students to assess their own work, or they can be done by others, e.g., peers, fieldwork supervisions, or faculty. Rubrics can be useful for grading, as well as assessment. Rubrics can be useful for grading, as well as assessment. For example, points can be assigned and used for grading, as shown below, and the categories can be used for assessment. Faculty who share an assessment rubric might assign points in different ways, depending on the nature of their courses, and they might decide to add more rows for course-specific criteria or comments. Notice how this rubric allows faculty, who may not be experts on oral presentation skills, to give detailed formative feedback to students. This feedback describes present skills and indicates what they have to do to improve. Effective rubrics can help faculty reduce the time they spend grading and eliminate the need to repeatedly write the same comments to multiple students. Analytic Rubric for Grading Oral PresentationsBelow ExpectationSatisfactoryExemplary Score OrganizationNo apparent organization. Evidence is not used to support assertions. (0-4)The presentation has a focus and provides some evidence which supports conclusions. (5-6)The presentation is carefully organized and provides convincing evidence to support conclusions. (7-8)ContentThe content is inaccurate or overly general. Listeners are unlikely to learn anything or may be misled. (0-8)The content is generally accurate, but incomplete. Listeners may learn some isolated facts, but they are unlikely to gain new insights about the topic. (9-11)The content is accurate and complete. Listeners are likely to gain new insights about the topic. (12-13)DeliveryThe speaker appears anxious and uncomfortable, and reads notes, rather than speaks. Listeners are largely ignored. (0-5)The speaker is generally relaxed and comfortable, but too often relies on notes. Listeners are sometimes ignored or misunderstood. (6-7)The speaker is relaxed and comfortable, speaks without undue reliance on notes, and interacts effectively with listeners. (8-9)Total Score Suggestions for Using Rubrics in Courses Hand out the grading rubric with the assignment so students will know your expectations and how they'll be graded. This should help students master your learning outcomes by guiding their work in appropriate directions. Use a rubric for grading student work and return the rubric with the grading on it. Faculty save time writing extensive comments; they just circle or highlight relevant segments of the rubric. Some faculty include room for additional comments on the rubric page, either within each section or at the end. Develop a rubric with your students for an assignment or group project. Students can then monitor themselves and their peers using agreed-upon criteria that they helped develop. Many faculty find that students will create higher standards for themselves than faculty would impose on them. Have students apply your rubric to some sample products before they create their own. Faculty report that students are quite accurate when doing this, and this process should help them evaluate their own products as they are being developed. The ability to evaluate, edit, and improve draft documents is an important skill. Have students exchange paper drafts and give peer feedback using the rubric, then give students a few days before the final drafts are turned in to you. You might also require that they turn in the draft and scored rubric with their final paper. Have students self-assess their products using the grading rubric and hand in the self-assessment with the product; then faculty and students can compare self- and faculty-generated evaluations. Rubric Category Labels Unacceptable, Developing, Acceptable, Exemplary Unacceptable, Marginal, Meets Expectations, Exceeds Expectations Novice, Developing, Proficient, Expert Beginner, Developing, Accomplished, Mastery Below Basic, Basic, Proficient, Advanced (AAC&U Board of Directors, Our Students Best Work, 2004) Creating a Rubric Adapt an already-existing rubric. Analytic Method Expert-Systems Method Steps for Creating a Rubric: Analytic Method Identify what you are assessing, e.g., critical thinking. Identify the characteristics of what you are assessing, e.g., appropriate use of evidence, recognition of logical fallacies. Describe the best work you could expect using these characteristics. This describes the top category. Describe the worst acceptable product using these characteristics. This describes the lowest acceptable category. Describe an unacceptable product. This describes the lowest category. Develop descriptions of intermediate-level products and assign them to intermediate categories. Ask colleagues who were not involved in the rubrics development to apply it to some products or behaviors and revise as needed to eliminate ambiguities. Steps for Creating a Rubric: Expert-Systems Method Have experts sort sample documents into piles with category labels. Determine the characteristics that discriminate between adjacent piles. Use these characteristics to describe each category. Ask colleagues who were not involved in the rubrics development to apply it to some products or behaviors and revise as needed to eliminate ambiguities. Managing Group Readings One reader/document. Two independent readers/document, perhaps with a third reader to resolve discrepancies. Paired readers. Before inviting colleagues to a group reading, Develop and pilot test the rubric. Select exemplars of weak, medium, and strong student work. Develop a system for recording scores. Consider pre-programming a spreadsheet so data can be entered and analyzed during the reading and participants can discuss results immediately. Inter-Rater Reliability Correlation Between Paired Readers Discrepancy Index Scoring Rubric Group Orientation and Calibration Describe the purpose for the review, stressing how it fits into program assessment plans. Explai /0TUdegtu 3 4 ' 9 @ C b c u ˽˳uhAIh4;5CJ\aJ h4;CJ hr^!CJhr^!CJOJQJ hd[hr^!hr^!hr^!hr^!5 hr^!hr^!hCJOJQJhCJOJPJQJaJhOJPJQJaJhhU[ hrnhrnjhrnUhrnhh5CJ$\aJ$) uvw 4 ' & F 7$8$H$gdr^! $7$8$H$a$gd4;&d P ]gd & Fdgd & Fdgdgddgd$a$gd$&d P ]a$gdrn$&d P ]a$gd ' c    ! # Y L$7$8$H$If^`Lgd F $7$8$H$Ifgd F 7$8$H$gd4; $7$8$H$a$gdr^!&d P ]gdr^! $7$8$H$a$gd4; & F t"]gdr^!    ! " # qrvwx-.24:\]^_G,-/01:K(½ЭУМУУУ h4;5>* hlnh4;hlnh4;5>*h)y_h4;5hlnh4;5 h4;5 hh4; hAIh4;h4;hAIh4;CJ aJ h4;CJaJhAIh4;5CJ\aJh4;5CJ\aJ: (` k]]]]] $7$8$H$Ifgd F}kd$$Ifl0,"<\ t0644 layt FL$7$8$H$If^`Lgd F O;rstuvxoaaaa $7$8$H$Ifgd F}kde$$Ifl0,"<\ t0644 layt F$7$8$H$If^gd F xW./01]}kd$$Ifl0,"<\ t0644 layt FL$7$8$H$If^`Lgd F $7$8$H$Ifgd F 124j/vfL$7$8$H$If^`Lgd F $7$8$H$Ifgd F ^_GIxxxxxpkpkb #gd4;gd4;$a$gd4; 7$8$H$gd4;}kd/$$Ifl0,"<\ t0644 layt F I[-/K1kVX8 & F h !h^hgd4; #gd4;(8> ? A B C T e !!!!!!!!###A$X$}$~$$$$$%%%M%N%P%X%Y%h%i%%%%%%%%%((**,,..1ƻƭƻƟhlh4;CJaJhlh4;CJaJhlh4;5CJ\aJjhlh4;Ujhlh4;U hlh4;hlh4;5 h4;CJh)y_h4;6hBqh4;5 h4;5>*h4;hlnh4;5>*68uj> A e !D!!!!n""E####$$  !gd4; & F h#h^hgd4; #gd4; & F h !h^hgd4;$$$|$}$~$$O%P%Y%i%%%%% $Ifgd F $Ifgd F$a$gd4; $  a$gd4;gd4;  !gd4;%%%.&f&&9'(LCCCCCC $Ifgd Fkd$$IfTlr >>>(@i`RRRRR  !$Ifgd F $Ifgd Fkd$$Ifl\&6$40t44 layt F(@)@i@x@GAABi`RRRR  !$Ifgd F $Ifgd FkdO$$Ifl\&6$40t44 layt FBBACD{E|EFi`RRRR  !$Ifgd F $Ifgd Fkd$$Ifl\&6$40t44 layt FBACFFTGGGGGHJHKKKL@LLLXXYYY=Z>ZF^H^^^^^^J_L_P_R_V__к{unjah5CJ\hr^! hr^!5\ hCJj#hUj "hUj hUjhU h6hhsQh5 h4;5h4;CJaJhd[h4;6CJ]aJhd[h4;CJaJhsh4;5CJaJhsh4;CJ\aJhsh4;5CJaJ&FFGGGILKM8OPia\\OOOOO h^h`gdgd4;$a$gd4;kd$$Ifl\&6$40t44 layt F P>RSUXL[^J_L_N_P_T_V__`H``Ibbbb & Fdgdgdr^!&d P ]gdr^!gd&d P ]gd $da$gd h^h`gd__``G`H```HbIbbbbbc f fffqfiijkkll nnQoRo q q#q$q%q-rCrKrLrMr^r_r`rrrrrrr뿳ݟֈֈΚ݈ hz'|hhz'|h5\h,h6 h5h,h5hfFOJPJQJaJhOJPJQJaJh5OJPJQJ\aJhOJQJ h5\ hCJhPJaJh h5CJOJPJQJ\aJ2bbccdd f f ff=f>f?fQf|s $Ifgd Fakd$$$Ifl4%t%0t%64 laf4yt F $$Ifa$gd F&d P ]gdgd Qf^fhfifofpfqf=kd%$$Ifl4rH!% ` ` 0t%64 laf4yt F $Ifgd F $$Ifa$gd Fqf~fffff!g"g#g)gggg $$Ifa$gd F $Ifgd F ggghhhRI;;;  !$Ifgd F $Ifgd FkdV&$$Ifl4rH!% ` ` 0t%64 laf4yt Fh hhh i i iii $$Ifa$gd F $Ifgd F$ !$Ifa$gd Fiiiiij!jRII=I= $$Ifa$gd F $Ifgd Fkd+'$$Ifl4rH!% ` ` 0t%64 laf4yt F!jjjjjjj=kd($$Ifl4rH!% ` ` 0t%64 laf4yt F $$Ifa$gd F $Ifgd Fjjjjjjjjkl}lg}gQQ & F hh7$8$H$^hgdgd&d P ]gdgdtkd($$Ifl40H!%!0t%64 laf4yt F $Ifgd F lnRoHp q q q$q%qUqqqqKrLrMr_r`rr & F hh^hgd $]a$gd & F hh^hgd$a$gdgd&d P ]gd & F hh7$8$H$^hgdrrrrrrrrrrssslttuuuuuu & F hh^hgd]gdgd&d P ]gd & F hh^hgdrrsssssssktltttuuuuv=w>w?wWwwwxx(x)x*x+xcxdxexfxxxxxyyy6y?y@yjylymynyz  _ j     ľľľĞоhJU hr^!hr^!hr^!hr^!5hlh5 h5CJ hCJhhlh\ h5\hlh5\hr^!hlhPJaJ hlhhlhPJ9u&vnvv=w>w?wWwXwmwwwwwx]gd & F hph^hgd & F hh^hgdgdgd&d P ]gd & F hh^hgd & F hh]^hgdx*xexxyyy6yYykylymynyyy x^x`gdgd&d P ]gdr^! & F hh]^hgdr^!]gdr^!gd&d P ]gd & F hh]^hgdn that the purpose is to assess the program, not individual students or faculty, and describe ethical guidelines, including respect for confidentiality and privacy. Describe the nature of the products that will be reviewed, briefly summarizing how they were obtained. Describe the scoring rubric and its categories. Explain how it was developed. Explain that readers should rate each dimension of an analytic rubric separately, and they should apply the criteria without concern for how often each category is used. Give each reviewer a copy of several student products that are exemplars of different levels of performance. Ask each volunteer to independently apply the rubric to each of these products, and show them how to record their ratings. Once everyone is done, collect everyones ratings and display them so everyone can see the degree of agreement. This is often done on a blackboard, with each person in turn announcing his/her ratings as they are entered on the board. Alternatively, the facilitator could ask raters to raise their hands when their rating category is announced, making the extent of agreement very clear to everyone and making it very easy to identify raters who routinely give unusually high or low ratings. Guide the group in a discussion of their ratings. There will be differences, and this discussion is important to establish standards. Attempt to reach consensus on the most appropriate rating for each of the products being examined by inviting people who gave different ratings to explain their judgments. Usually consensus is possible, but sometimes a split decision is developed, e.g., the group may agree that a product is a 3-4 split because it has elements of both categories. You might allow the group to revise the rubric to clarify its use, but avoid allowing the group to drift away from the learning outcome being assessed. Once the group is comfortable with the recording form and the rubric, distribute the products and begin the data collection. If you accumulate data as they come in and can easily present a summary to the group at the end of the reading, you might end the meeting with a discussion of five questions: Are results sufficiently reliable? What do the results mean? Are we satisfied with the extent of student learning? Who needs to know the results? What are the implications of the results for curriculum, pedagogy, or student or faculty support services? How might the assessment process, itself, be improved?     PAGE  Scoring Rubrics - PAGE 1 PAGE  Scoring Rubrics - PAGE 6 y ZS      |      gd F&d P ]gd7&d P ]gd  !gd & F XpX^X`gd & F hh^hgd                             hJh&i0JmHnHuh&i h&i0Jjh&i0JUh}Fcjh}FcU$                 &d P ]gd7 &`#$gd'< &`#$gd F &`#$gdJgd F21h:p4;/ =!"#$% 51h0:p4;= /!"#$% 51h0:p4;= /!"#$% 21h:p4;/ =!"#$% c$$If!vh5<5\#v<#v\:Vl t65<5\yt Fc$$If!vh5<5\#v<#v\:Vl t65<5\yt Fc$$If!vh5<5\#v<#v\:Vl t65<5\yt Fc$$If!vh5<5\#v<#v\:Vl t65<5\yt F5DyK /http://www.neiu.edu/~neassess/gened.htm#rubricyK Phttp://www.neiu.edu/~neassess/gened.htmrubric$$If!vh54 5L 5( 5L 5L #v4 #vL #v( #vL :V l0@865 / 4yt FT$$If!vh54 5L 5( 5L 5L #v4 #vL #v( #vL :V l0@865 / 4yt FT$$If!vh54 5L 5( 5L 5L #v4 #vL #v( #vL :V l0@865 / 4yt FT$$If!vh54 5L 5( 5L 5L #v4 #vL #v( #vL :V l0@865 / 4yt FT$$If!vh54 5L 5( 5L 5L #v4 #vL #v( #vL :V l0@865 / 4yt FT$$If!vh54 5L 5( 5L 5L #v4 #vL #v( #vL :V l0@865 / 4yt FTQDyK 7http://www.csufresno.edu/cetl/assessment/CTScoring.docyK nhttp://www.csufresno.edu/cetl/assessment/CTScoring.doc$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l 6P /6,5 5< 5` 5\ / 4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l 6P /6,5 5< 5` 5\ /  4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ / 4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l 6P /6,5 5< 5` 5\ /  4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4V 6P /6+,5 5< 5` 5\ / 4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l 6P /6,5 5< 5` 5\ /  4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ / 4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l 6P /6,5 5< 5` 5\ /  4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4 6P /6+,5 5< 5` 5\ /  4f4yt F$$If!vh5 5< 5` 5\ #v #v< #v` #v\ :V l4V 6P /6+,5 5< 5` 5\ / 4f4yt F$$If!vh5555#v#v#v#v:V l0t455554yt F$$If!vh5555#v#v#v#v:V l0t455554yt F$$If!vh5555#v#v#v#v:V l0t455554yt F$$If!vh5555#v#v#v#v:V l0t455554yt F$$If!vh5555#v#v#v#v:V l0t455554yt F$$If!vh5555#v#v#v#v:V l0t455554yt FqDyK ?http://www.csulb.edu/divisions/aa/personnel/fcpd/resources/ge/yK ~http://www.csulb.edu/divisions/aa/personnel/fcpd/resources/ge/DyK [http://www.uafortsmith.edu/Learning/GeneralEducationCompetenciesAndRubrics#BookmarkRubricsyK http://www.uafortsmith.edu/Learning/GeneralEducationCompetenciesAndRubricsBookmarkRubricsDyK $http://wsuctproject.wsu.edu/ctr.htmyK Hhttp://wsuctproject.wsu.edu/ctr.htm$$If!vh5t%#vt%:V l40t%65t%4f4yt F$$If!vh5 55` 5` 5#v #v#v` #v:V l40t%65 55` 54f4yt F$$If!vh5 55` 5` 5#v #v#v` #v:V l40t%65 55` 54f4yt F$$If!vh5 55` 5` 5#v #v#v` #v:V l40t%65 55` 54f4yt F$$If!vh5 55` 5` 5#v #v#v` #v:V l40t%65 55` 54f4yt F$$If!vh5!5#v!#v:V l40t%65!54f4yt F@@@ NormalCJ_HaJmH sH tH >@> Heading 1$$@&a$CJ`V@V 4; Heading 3$<@&5CJOJQJ\^JaJ@@@ Heading 4$$@&a$5\H@H J Heading 6 <@&5CJ\aJDA@D Default Paragraph FontRi@R  Table Normal4 l4a (k(No List8B@8 Body Text$a$CJ84@4 Header  !LC@L Body Text Indent^`>"@> Caption5OJPJQJaJ6U@16  Hyperlink >*B*ph4 @B4 Footer  !.)@Q.  Page Numberj@cj 4; Table Grid7:V04>@r4 4;Title$a$5aJJJ@J 4;Subtitle] &P#$/ 5CJaJZY@Z : Document Map-D M CJOJQJ^JaJ~)?p4k uvw4'c  !#Y(` O;rstuvx W . / 0 1 2 4 j / v f  ^_GI[-/K1kVX8uj>AeDnE|}~OPYi.f9  ' !e!"";"""#$$$%{%%I&&&-''())))************+$+9+K+L+M+Z+s++++++++++,,,,,,I,a,b,c,d,e,f,g,u,,,,,,,,,,,--4-5-6-Q-`-s-t-u--------------..../.C.V.W.X.q.........../1/2/3/4/5/6/7/E/\/v//////////////0&0?0@0A0X0g0000000011711[22263n334555D6666(8)8i8x8G99::A;<{=|=>>???@AxBCCD EEGHJKLLLLLLLL/MmM0OOOOOOOtQuQRRRR$S%S&S8SESOSPSVSWSXSeSSSSST T TTqTwTxTyTTTTTTUUUUUUUUVyVVWWWWWWWWWWWWWWWXYZ9\/]]]] ^ ^<^}^^^2_3_4_F_G_i_y__________{``Saaabbbbb cUcc$d%d&d>d?dTddddddeLeseffff@fRfSfTfUfffggAhhik:nnfoooocpppppppppppppppppppppppppq0000000 0 00000 0 0 0 0 000000000000 0000000 0 0000 0000000 0 0000 0000000 0 0000 00000000 0 00000000000 0 0 0 0 0 0 000 0 0 0 0 0 0 00 0 0 0 0 0 000 0 0 0 0 0 000 0 0 0 0 000 0 0 0 0! 0"0000000000000 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 00 00 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 000 0 0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 000 0 0 0 00 0 0 0 0 0 00 0 0 00>0>0>0>0>0>0>0>0>0>0>0>0>0>0>0>0>0>0>0> 0> 0> 0> 0> 0>0>0>0>0>0>0>0>0>0>0>0>0> 0> 0> 0> 0> 0>0> 0>0> 0> 0> 0>0>0>0> 0>0>0>0> 0>0> 0> 0> 0> 0>0>0>0> 0>0> 0>0>0>0> 0> 0> 0> 0>0> 0>0> 0>0>0> 0> 0> 0> 0> 0> 0>0>0>0>0> 0> 0> 0> 0> 0> 0>0>0>0>0> 0> 0> 0> 0> 0>0>0>0>0> 0> 0> 0>0>0>0>0>0>0>0> 0> 0> 0> 0> 0> 0> 0>0>0>0>0> 0> 0> 0> 0>0>0>00&d 0&d 0&d 0&d0&d0&d0&d 0d&d 0d&d 0d&d 0d&d0&d0&d0&d 0&d 0&d0&d0&d0&d80&d0Uf 0Uf 0Uf 0Uf 0Uf 0Uf 0Uf 0Uf 0Uf 0Uf 0nUf 0nUf 0nUf 0nUf 0nUf0Uf0Uf0@0000@0000@0000@0000@0@0@0@0@0@00@01@0@0@0@0@000 uvw4'c(`;rs . / f  |}O  ""$$&&'())))*****K+L+++++,,a,b,f,g,,,,,4-5-s-t-----..V.W.....1/2/6/7///////?0@0A0X0g000000011711[22263n334555D6666(8)8i8x8G99::A;<{=|=>>?KLLLLLLLL/MmM0OOOOOOOtQuQRRRR$S%S&S8SESOSPSVSWSXSeSSSSST T TTqTwTxTyTTTTTTUUUUUUUUVyVVWWWWWWWWWWWWWWWXYZ9\/]]]3_______{``Saaabbb%d&d>d?dTddddddeLesefff@fRfSfTfUfffggAhhik:nnfooocppppq030@0_@0_@0_@0_@0_@ 0_@ 0_@00 0K00K000 0 0 00 0K00K00@00 0K00+K00K00K00+K00K00QK00K00K00+K00K0 0 QK0 0K0 0 QD4@ CK00QK00K00QK00K00K00K00K00K000 0!00 00 0@0@0K00K00j@0K00j@0K00@0K000K00 @0K0n0oK0n0K0n0K0 0@0@0@0K00 @0_ K00 @0_ K00 @0_ K00 @0_ K0 0 @0_ K0 0 @0_ K00 @0_ K00 @0_ K00 @0_ K00 @0_ K00 @0_ K00 @0_ K00 @0_ K00 @0_ K00 @0_ K0 0! @0_ K0"0# @0_ K0$0% @0_ K0&0' @0_ K0(0) @0_ K0*0+ @0_ K0,0-`@0_ K0.0/`@0_ K0001`@0_ K0{0|K0{0K0{0K020 K00u @0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0K0Q390000@0@0@0@0@0@0@ 0@ 0@ 0@ 0@ 0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@0@ 0@ 0@ 0@ 0@ 0@ 0@0@0@0@0@0 @0@0"@ 0"@ 0"@ 0"@ 0"@ 0"@ 0"@ 0"@0@0@0"@0" @0@0@ 0@ 0@ 0@0!@0!@0!A 0"!A 0"!A 0"!@0"@0"0101@0"@0"@0@0:@0@0J@ 0J@ 0J@ 0J@ 0J@ 0J@ 0J@ 0J@ 0J@ 0JA 0JA 0JA 0JA 0J@0J@00 66666666666A`````````c (1s5B_r =AHQ^qt' x1I8$%(*,.122K3334a4f44445s5556V6661767777?889:=(@BFPbQfqfghi!jjlruxy >@BCDEFGIJKLMNOPRSTUVWXYZ[\]_`abcdefghijklmnoprsuvwxyz{|}~?/TdM0*v**A BIBI}II L=LaLpXXXXXX )026=SZ\c!!!! OLE_LINK1g,qt,qT6L:7L:8L:,9L:l:L:;L:L:l?L:@L:lAL:lBL:,CL:DL:EL:lFL:,GL:HL:IL:lJL:,KL:LL:ML:lNL:,OL:PL:QL:lRL:,SL:TL:UL:lVL:,WL:XL:YL:lZL:,[L:\L:l]L:^L:_L:,`L:laL:bL:cL:,dL:leL:fL:gL:,hL:liL:jL:kL:,lL:lmL:nL:loL:,pL:lqL:rL:,sL:ltL:uL:vL:, wL:l xL: yL: zL:, {L:l |L:,}L:~L:,L:lL:L:lL:L:L:,L:lL:L: L: L:   NNW   $$'ZZd))))0000???AAAAAzBBBBBCCEDEDMDTD E EE"EEEEHHHHIIJJJKKKK5d5dq     ! "#%$&')(*+,-./0123456789:<;=>@?ABCDEFGHIJKLMNPOQRS%%Vaa    **&))bff))))0000?@@AAAAABBBBBCCLDSDeDeDE!E,E,EEEEHHHIIIJKKKKLL=d=dq  ! "#%$&')(*+,-./0123456789:<;=>@?ABCDEFGHIJKLMNPOQRS (:urn:schemas:contacts middlename>?*urn:schemas-microsoft-com:office:smarttags PostalCodeB*urn:schemas-microsoft-com:office:smarttagscountry-region9E*urn:schemas-microsoft-com:office:smarttagsState8I*urn:schemas-microsoft-com:office:smarttagsCity9T*urn:schemas-microsoft-com:office:smarttagsplace=R*urn:schemas-microsoft-com:office:smarttags PlaceType=S*urn:schemas-microsoft-com:office:smarttags PlaceName;C*urn:schemas-microsoft-com:office:smarttagsaddress:B*urn:schemas-microsoft-com:office:smarttagsStreet <1TSRRTSRSRRTITRRETCBIE?TSRR:T:ITETSRSTRRSTRRSTRSRRITSRRITETSTRRSTSRSTRTRSITRTSSTRRITpppppppppppppq]b.8 UaAY66;7H7x8}8==~???6@7@@@AAAALBMBwBxBBBCCBDCDDD E E^EEFFGGHHHHIIJJ>?R$S%S&S8SESPSWSXSeSSTwTxTyTTTUUUUVVWWWWWWWqeOeO333}@l:?@ABCDEGHIJKLfp`@`H`J`L`N`P`R`@`V`X`@`\`^`@`@UnknownGz Times New Roman5Symbol3& z Arial3z Times5& zaTahoma?5 z Courier New;Wingdings"1h_9_9!4dpdp 2qHX ?2Developing and Applying Rubrics Mary Allen Mary Allen<         Oh+'0 ( H T `lt| Developing and Applying Rubrics Mary AllenNormal Mary Allen2Microsoft Office Word@F#@gN@gN_՜.+,D՜.+,P  hp  CSUB9dp'  Developing and Applying Rubrics Title 8@ _PID_HLINKSA$|0$http://wsuctproject.wsu.edu/ctr.htmF  Khttp://www.uafortsmith.edu/Learning/GeneralEducationCompetenciesAndRubricsBookmarkRubricsbe ?http://www.csulb.edu/divisions/aa/personnel/fcpd/resources/ge/%y7http://www.csufresno.edu/cetl/assessment/CTScoring.docRW(http://www.neiu.edu/~neassess/gened.htmrubricuGmailto:mallen@csub.edu  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~   Root Entry F0 gNData v)1TableWordDocumentSummaryInformation(DocumentSummaryInformation8CompObjq  FMicrosoft Office Word Document MSWordDocWord.Document.89qRoot Entry F\Data v)1TableWordDocument՜.+,D՜.+,P  hp  CSUB9dp'  Developing and Applying Rubrics Title 8@ _PID_HLINKSA$|0$http://wsuctproject.wsu.edu/ctr.htmF  Khttp://www.uafortsmith.edu/Learning/GeneralEducationCompetenciesAndRubricsBookmarkRubricsbe ?http://www.csulb.edu/divisions/aa/personnel/fcpd/resources/ge/%y7http://www.csufresno.edu/cetl/assessment/CTScoring.docRW(http://www.neiu.edu/~neassess/gened.htmrubricuGmailto:mallen@csub.eduSummaryInformation(DocumentSummaryInformation8CompObjq