220 likes | 336 Views
Writing Proposals with Strong Methodology and Implementation. Kusum Singh, Virginia Tech Gavin W. Fulmer, National Science Foundation. Goals. Encourage you to seek funding from NSF for your research.
E N D
Writing Proposals with Strong Methodology and Implementation Kusum Singh, Virginia Tech Gavin W. Fulmer, National Science Foundation
Goals • Encourage you to seek funding from NSF for your research. • Help you develop rigorous methodology, data collection and analysis plans that will make your proposal competitive. • Help you consider the level of detail appropriate for implementation projects.
Expectations for Methods in DRL • The DRL Programs welcome research using a variety of evidence. • The program is open to qualitative, quantitative, and mixed methods. • Methods must be rigorous and appropriate to the proposed research questions or hypotheses. • Design, methods, and analytic techniques should have a coherent and logical link. • Research methods should be described in adequate detail.
Details of Methods to Include – 1 • Provide a rationale for your research design • Make it clear how the research design and analyses answer the research questions (RQs) • Include a description of study population and sampling method, sample size, expected effect size • Power analysis should inform sample size decision
Details of Methods to Include – 2 • Instruments or protocols to be used • Validity, reliability, and triangulation of measures • Reviewers are cautious about development of new measures • Data analysis plans • Statistical Models, procedures for analysis of text/video/observation data • All of these need to have a rationale for them that connects to your RQs
Quantitative research • Research design (e.g. experimental, quasi-experimental and non-experimental designs, issues of internal & external validity) • Measurement (e.g. data to be collected, constructs, measures, validity & reliability of measures) • Data analysis (e.g. statistical decisions, models & procedures)
Qualitative Research • Identify the methodology as a systematic research design (e.g. case study, discourse analysis etc.) • Describe how and what data will be collected • Consider issues of validity, and triangulation • Include plans for analysis of textual data (coding scheme, themes etc.) • Find good balance between planned approach to analysis and flexibility to respond to findings
Find the Expertise You Need • Content experts are not necessarily methods experts; so partner with research methodologists • Sooner is better than later (in proposal writing stage) • Especially necessary if design is complex or you use innovative methods • Find a colleague • As co-PI or as consultant
Common Missteps in Methods -1 • Overly generic language and description • “We will use constant comparative methods.” • “We will use HLM.” • Lack of consistent link between the theory, the RQs, the data collected, and the analyses • Reviewers will notice. • Methods and planned analyses inadequate to answer RQs. • Try developing a matrix of RQs, data/measures, and analyses – even if only for you during planning
Common Missteps in Methods -2 • Too little or too much data without clear analysis plan • Reviewers will wonder if you understand the task. • Method is novel and not well understood in field • Needs more detail, examples and citations to justify that it is appropriate
Summary of Main Points • Articulate clearly your research questions or research hypotheses • Think about the most appropriate and rigorous methods to answer your research questions • Give a clear and concise description of the research methods • Include your rationale for research design decisions • Include a research methods expert in your team • Articulate clearly why your research is important and how it would contribute to theory and practice
Details of Implementation • There are important implementation issues that need to be addressed if your project includes • Curriculum development • Professional development • Interventions
For All Implementation Projects • Consider the method(s) used to gauge the quality of the implementation • Whether as “Fidelity of Implementation” (FOI), Intended/Enacted Curriculum, or other approaches • Be specific on the STEM content, ages/grades, settings • Be clear on the roles of the team • Who will lead PD or curriculum, who will oversee implementation? • Who will collect evaluative data on implementation?
Issues for Curriculum Development • Specify the STEM content of interest and age range(s) for which you are developing curriculum • Specify the role(s) of the PI team, outside experts, participating teachers, or others • Identify the process for development, revision, and field-testing • Provide justification for the design process you will use • Make sure the measures match the materials/curriculum under development
Issues for Professional Development • Be specific on the professional development (PD) • STEM content, grades, and school settings • Role(s) of the PI team, outside experts, participating teachers, or others • Format of professional development (e.g., online, workshops) • Duration and location of PD • Evaluation • Identify the model for PD you will use • Train-the-trainer • Master teacher • Professional Learning Community • Provide justification for the model, the format, and your team’s expertise
Issues for Intervention • Describe development history and its prior use • Provide evidence, if any, for intervention’s potential effects • Describe in detail: • Population and sample; • Setting, duration, and content; • Design process, if the intervention will be revised iteratively
Consider Generalizability • If you are developing a new curriculum/PD model: • How will the intervention, curriculum, or the professional development developed in your setting apply to new settings that may differ from the study? • If you are applying an intervention, PD model, or curriculum adopted from another setting: • How well does that intervention apply to your setting? • Will promising prior results be replicable in this project?
Evaluation Plan • Evaluation should be useful for improving the research project • Design and content of the plan should be appropriate to what would enhance or benefit the project • Formative or summative, internal or external may be appropriate, depending on the project. • For example, advisory committees are appropriate for the evaluation of projects. • Go to specific session on Project and Program Evaluation later in the conference for more details.
Any Questions?? Don’t be shy.
Thank you! Feel free to contact Kusum Singh for follow-up and tips for finding a good methodologist: ksingh@vt.edu