We propose to evaluate the research process using the NPTAI rubric of Ferrés, Marbà and Sanmartí (2015) which is an adaptation of the well-known PTAI (Practical Test Assessment Inventory) rubric of Tamir et al (1982).
RUBRIC FOR EVALUATION OF INQUIRY PAPERS NPTAI
IDENTIFICATION OF RESEARCHABLE PROBLEMS
Points
0 Does not identify problems or does not pose problems or poses unapproachable problems.
1 Raises problems with ambiguous or generic formulation or poorly formulated problems.
2 Identifies appropriate research problems and specifies questions.
FORMULATION OF HYPOTHESES
Points
0 Does not formulate hypotheses or does not identify hypotheses, or formulates hypotheses without meaning.
1 Raises hypotheses unrelated to the problem or objectives.
2 Formulates ambiguous hypothesis or with errors in logic or poorly formulated or only makes predictions.
3 Formulation of hypotheses in the form of deductions that fit the research problems.
4 Pose hypotheses that fit with the research problem and describe them in the form of deduction and with reference to the model: «If we think that… then if… we will observe that...»
IDENTIFICATION OF VARIABLES
Points
0 The design should contemplate variables and does not take them into account.
1 Does not identify IV or DV or does not know how to specify them despite having considered them in the design.
2 Confuses IV and DV or proposes IV and DV that do not fit with the hypotheses formulated.
3 Identifies IV and DV but in an unspecific or imprecise manner.
4 Identifies and defines appropriate IVs and DVs that fit the hypotheses.
RESEARCH PLANNING
Points
0 There is no experimental or methodological design or there is one but it is not identified.
1 The methodological design does not allow testing the hypotheses.
2 The methodological design only allows a partial testing of the hypotheses.
3 The methodological design offers adequate hypothesis testing, but does not propose replications or explicit controls, or the control is incomplete, or incomplete description of the design.
4 The methodological design provides adequate hypothesis testing, with replication and control.
DATA COLLECTION AND PROCESSING
Points
0 Failed to collect research data: neither generated in experiments or observations, nor obtained from data sources.
1 Incomplete data collection, with lack of accuracy, or with deficits in the application of techniques and measurements, inadequate or incomplete data processing, graphs without titles or with inadequate titles, and calculations with inaccuracies.
2 Data collection with errors or inaccuracies or showing lack of understanding of the procedures and/or with evidence of lack of relationship between the data and the hypotheses tested, but with adequate treatment of the data and graphical representation.
3 Methodical data collection, with good understanding and execution of the techniques and measurements, providing data related to the hypotheses, with good mathematical and graphical treatment, but with no replications and insufficient control.
4 Methodical, adequate and sufficient data collection with good understanding and execution of the techniques and measurements, good mathematical and graphical treatment of the data, and with replications and controls.
DATA ANALYSIS AND DRAWING CONCLUSIONS
Points
0 No data analysis.
1 Poor analysis and conclusions not supported by data.
2 Conclusions very similar to the results, without interpretation or data analysis. Failure to coordinate theoretical justifications with empirical evidence.
3 Incomplete or poorly supported analysis of data or based on unreliable data, «simplistic»….
4 Well-supported data analysis and evidence-based conclusions. Coordinates theoretical justifications with empirical evidence.
META-REFLECTION
Points
0 Does not know how to describe the characteristics of scientific inquiry processes.
1 Incomplete description of characteristics of an inquiry process or with confusion of concepts, purely inductivist ideas and little or no reference to scientific concepts.
2 Good description of the processes of inquiry, with reference to scientific concepts both to formulate hypotheses and in the analysis of data and the argumentation of conclusions, which do not arise simply from induction processes.
Based on the results of the rubric, and as proposed by its authors, the Levels of Inquiry Competence or NCI of the students (or work groups) can be calculated according to 5 levels:
SCORE RANK
14 to 16
11 to 13
8 to 10
6 to 7
0 to 5
CATEGORY
INQUIRERS
INSECURE INQUIRERS
INCIPIENT INQUIRERS
PRE- SCIENTISTS
UNSCIENTIFIC
Great job, you know what it’s like to inquire!
Good job, although you need to polish some aspects!
Not bad, you’re beginning to understand what inquiry is!
You have some notion, but you still have a long way to go!
Inquiry is not your thing!