Frequently Asked Questions
The following questions address Section VIII., Government Performance & Results Act Reporting:
- How do I enter teacher prestest and posttest scores in the electronic spreadsheet?
- What are the assumptions about the pre- and post-tests when the t-statistic is used?
- Should the same test be used for pre- and post-tests?
- When should pre- and post-tests be administered?
- How many questions/points should be on each test?
- Do the pre- and post-tests have to have the same number of points?
- What if pre- and post-tests have a different number of total points?
- Is there a place to enter the total number of points on the teacher content knowledge assessment? Is that important?
- If the same teacher takes one test more than two times (e.g., pre-test, immediate post-test, and delayed post-test), how do these get entered and counted?
- If the same teacher takes more than one pre- and post-test in one professional development program, are all tests entered?
- Can/should students of participants be associated with more than one teacher? In particular, should the students of special education/inclusion teachers who work with classroom teachers be included if both teachers participate in the PD? (This would count the same students twice.)
- What should a project do if all data to be reported are not available when the report is due?
1. How do I enter teacher pretest and posttest scores in the electronic spreadsheet?
In section VIII. Government and Results Act Reporting, A. Teachers, you should click on the link to the electronic spreadsheet.
You can either save the spreadsheet file or open it directly. When you do, the following window should appear:
If the window appears, then select the "Enable Macros" button in the pop-up window.
If the Window does not appear, open Excel and try these instructions to reset the Excel macro security from High to Medium:
- From the Tools menu, select Options.
- Click the Security tab.
- Press the Macro Security button (lower right), which brings up another window.
- Select the Medium security level and press OK.
- Press OK to close the window.
Then attempt to open the spreadsheet again.
If you still do not see the window with the "Enable Macros" button, then check with your computer staff or internet provider for help getting the spreadsheet past any firewall or anti-virus software that may be blocking the macros.
2. What are the assumptions about the pre- and post-tests when the t-statistic is used?
The paired-samples t test assumes that each subject's score is independent of any other subject's score and that the difference scores are normally distributed. The assumption of independent observations between subjects is controlled by the study design. When the assumption of normality is not met, the paired-samples t test can be invalid. For each evaluation program, a sample size of 30 or more teachers should be large enough to meet the normality assumption and the paired-sample t test can be used validly. For sample sizes with fewer than 30 teachers, the spreadsheet uses a nonparametric statistical test called the Wilcoxon signed ranks test.
3. Should the same test be used for pre- and post-tests?
Yes. The pre-test and post-test should be the same test.
4. When should pre- and post-tests be administered?
The pre-test should be administered before or at the beginning of the intervention; and the post-test should be administered after the intervention has ended.
5. How many questions/points should be on each test?
This depends on the test used.
6. Do the pre- and post-tests have to have the same number of points?
Yes, because they should be the same test.
7. What if pre- and post-tests have a different number of total points?
The pre-test and post-test should be the same test. Using the same test allows for the measurement of change/gain.
8. Is there a place to enter the total number of points on the teacher content knowledge assessment? Is that important?
Technically, the total number of points is not necessary for calculating the significance of gains as assessed on a particular instrument.
9. If the same teacher takes one test more than two times (e.g., pre-test, immediate post-test, and delayed post-test), how do these get entered and counted?
The first test should always be entered first since it is the pre-test, and the score serves as the baseline. One way to handle subsequent tests is to average the test scores and enter the average score as the post-test.
10. If the same teacher takes more than one pre- and post-test in one professional development program, are all tests entered?
Grantees who administer more than one test to the same group of teachers should select the most relevant or important test and report that one. For each individual MSP teacher participant, there should be only one set of pretest, posttest, and gain score information.
11. Can/should students of participants be associated with more than one teacher? In particular, should the students of special education/inclusion teachers who work with classroom teachers be included if both teachers participate in the PD? (This would count the same students twice.)
Generally, it is best to assume one teacher per classroom. However, in this case, we would want to capture the experience of both teachers and any over-counting due to the presence of special education students would not be large.
12. What should a project do if all data to be reported are not available when the report is due?
Submit what data you have available by December 31.