From Data to Action: Continuous Improvement
Identifying a discipline disparity is not the end of the analysis. This lesson examines how school and district teams can move from a documented pattern to a focused improvement strategy using the Plan–Do–Study–Act process, implementation data, outcome indicators, and repeated cycles of inquiry.
Read Report Pages 15–21
These pages contain the guide's full discussion of using discipline data within a continuous improvement process. Read the general Plan–Do–Study–Act framework on pages 15–19 and continue through the district example on pages 19–21. The example is important because it demonstrates how the individual phases operate together rather than as isolated steps.
As you read, focus especially on the following:
- Figure 6 and the four phases of the Plan–Do–Study–Act cycle: Plan, Do, Study, and Act.
- Why the process is described as a continuous cycle rather than a one-time sequence that ends after an intervention is implemented.
- During Plan, notice the emphasis on identifying root causes, developing goals, creating an intervention plan, and selecting indicators that will be used to track progress.
- Notice that the guide recommends gathering additional information before choosing an intervention when the existing data are not sufficient to explain the problem.
- Examine the examples of additional data that may help refine a discipline problem: racial or ethnic patterns in suspension rates, average days of suspension, behaviors associated with suspension, and locations of discipline incidents.
- Pay attention to the guide's emphasis on investigating root causes and circumstances contributing to disproportionate discipline before selecting an intervention.
- Notice how an intervention should be connected to the specific problem identified through the data rather than selected simply because it is familiar or available.
- During Do, examine the recommendation to implement and test the plan while continuing to collect data.
- Pay particular attention to the suggestion that teams may start small by piloting an intervention with a limited group of students or schools before wider implementation.
- Identify the reasons the guide gives for starting small, including the opportunity to examine effectiveness, make adjustments, and identify professional development or implementation support needs.
- Distinguish between collecting data on student outcomes and collecting data on implementation fidelity. Both are necessary to interpret results.
- During Study, notice that progress indicators should represent both desired outcomes and the quality of implementation.
- Examine the characteristics the guide recommends for progress indicators: they should be practical to collect, understandable, useful, and capable of providing sufficiently accurate information to guide decisions.
- Review the behavior reflection room example and notice the additional information the school collected: students using the room, referring teachers, time of day, reasons for use, student perspectives, suspensions, and instructional time missed.
- Consider why monitoring a specific intervention can require new data that were not part of the original discipline database.
- During Act, notice the decision rule when progress is limited: first investigate whether the intervention was implemented as intended before deciding that the intervention itself is ineffective.
- Identify the two possible directions after reviewing results: revise an intervention that is not producing the desired outcome, or expand/continue a successful strategy and begin another cycle of improvement.
- Read the district example on pages 19–21 as an integrated case. Identify how the district defined its concerns, selected strategies, monitored discipline outcomes, reviewed implementation, and used what it learned to guide subsequent action.
A Disparity Identifies a Problem Area, Not an Intervention
Previous lessons focused on determining whether differences in discipline outcomes exist and selecting indicators that describe those differences. The next analytic task is fundamentally different. Once a team determines that disproportionality is a concern, it must decide what to do about it.
Moving directly from a relative rate ratio or composition difference to a preferred intervention creates a serious problem: the indicator identifies an outcome pattern, but it does not necessarily identify the conditions producing that pattern. Two schools can display similar disparities for very different reasons and therefore require different improvement strategies.
A school psychologist can help the team transform a broad statement such as “our suspension data are disproportionate” into a problem statement precise enough to guide action. The goal is to identify where the pattern is concentrated, what circumstances accompany it, and which variables may be alterable through school-based practice.
The Plan–Do–Study–Act Cycle
The guide uses the Plan–Do–Study–Act process as a framework for linking problem identification, intervention, implementation monitoring, outcome evaluation, and revision. Each phase has a distinct purpose, but the phases should be interpreted as parts of one repeating improvement process.
Define the Problem Before Selecting the Solution
Planning begins with a deceptively simple question: what are we trying to accomplish? A team that cannot answer that question precisely is not ready to evaluate whether an intervention works.
The guide emphasizes that teams may need additional data during planning. Discipline indicators can establish that a disparity exists, but intervention planning requires information about the conditions surrounding that disparity. The team may examine rates across groups, the average number of suspension days, behaviors associated with suspension, locations of incidents, or other variables that help make the concern more specific.
This inquiry supports root-cause analysis. Root-cause work should not be confused with assigning blame. Its purpose is to identify the conditions, processes, or practices that may be maintaining the problem and that are sufficiently actionable to inform intervention.
Planning also requires specifying the desired outcome and deciding which indicators will show whether progress is occurring. The indicator should correspond to the goal. If the goal is to reduce repeated exclusion, for example, monitoring only the total number of discipline referrals may not be sufficient.
Implement, Test, and Monitor Fidelity
In the Do phase, the team puts the intervention plan into practice. The guide recommends considering a small-scale pilot rather than assuming that a new strategy should immediately be implemented schoolwide or districtwide.
Starting small allows a team to learn. A pilot can reveal whether the strategy is feasible, whether staff need additional training or support, whether the selected indicators can actually be collected, and whether implementation problems should be addressed before broader expansion.
The Do phase also introduces a distinction that is critical for interpreting later results: outcome data and implementation data are not the same.
| Type of Information | Question It Answers | Examples |
|---|---|---|
| Outcome indicators | Are the desired discipline outcomes changing? | Suspension rates, repeated exclusions, days of missed instruction, relative disparities |
| Implementation indicators | Was the intervention carried out as intended? | Staff participation, use of the new procedure, completion of intervention steps, access to needed support |
Use Data to Learn, Not Merely to Judge Success or Failure
The Study phase involves regularly reviewing indicators to determine whether the desired outcomes are changing and whether the intervention is being implemented with sufficient quality. The guide emphasizes selecting indicators that teams can realistically collect, understand, and use.
A useful indicator must do more than exist in a database. It should provide sufficiently accurate and meaningful information to guide the next decision. Depending on the intervention, this may require creating new data-collection procedures.
This example illustrates an important principle: intervention evaluation may require data that were not needed to identify the original disparity. Outcome monitoring asks whether the desired result is occurring. Improvement work also asks what the implementation process is teaching the team.
Decide What the Evidence Justifies Doing Next
During the Act phase, the team determines what should happen based on the information gathered during implementation and study. When there is little or no progress, the guide recommends investigating implementation fidelity before concluding that the intervention itself should be abandoned.
If implementation was not sufficiently faithful, the appropriate response may involve additional training, coaching, clarification, resources, or modification of the implementation plan. If implementation was adequate and outcomes still did not improve, the team has stronger reason to reconsider the intervention itself.
If the intervention is successful, the team may expand the intervention or identify another problem of practice. Either way, the process returns to planning. Continuous improvement does not end with a single positive result.
What the District Example Demonstrates
The district example on pages 19–21 is useful because it shows the complete improvement process operating in a real system. The district did not treat disproportionality as a statistic to report once. It used its discipline data to identify concerns, developed a coordinated improvement response, implemented changes, monitored both exclusionary discipline and missed instructional time, and reviewed results over multiple years.
The example also illustrates that improvement work can occur at multiple levels. Changes may involve district policy, school procedures, professional learning, leadership teams, staff support, and methods for monitoring student outcomes. A discipline disparity can therefore be approached as a systems problem rather than only as a student-level behavior problem.
For school psychologists, this systems orientation is important. Expertise in measurement, consultation, problem analysis, behavioral principles, and progress monitoring can contribute at several points in the cycle—from selecting defensible indicators to helping teams distinguish intervention failure from implementation failure.
Use the Continuous Improvement Process
The following activities require you to make decisions about sequence, implementation, and interpretation rather than simply identify the names of the four phases.
Sequence the Improvement Decisions
A district has confirmed a persistent discipline disparity. Arrange the following actions into the strongest sequence for beginning and evaluating an improvement cycle. Use the arrow buttons to move items.
Respond to a Stalled Intervention
Work through the scenario by selecting the strongest next decision at each point.
Decision 1
A middle school implements a new classroom response procedure intended to reduce exclusionary discipline. After eight weeks, suspension rates have not meaningfully changed. What should the team do first?
Build a Monitoring Plan
A school is piloting a behavior reflection room intended to reduce suspensions and instructional time lost. Classify each measure according to its primary role in the improvement plan.
Continuous Improvement Connects Data to Action
Discipline data are most useful when they change what teams do. Identifying disproportionality establishes a problem that requires attention, but the disparity measure alone usually does not specify the appropriate intervention. Teams need to move from broad identification to focused investigation.
During Plan, the team clarifies the problem, investigates contributing conditions, develops goals, selects an intervention that matches the identified need, and decides how progress will be measured. Planning is therefore an analytic process rather than simply an intervention-selection process.
During Do, the team implements the plan and simultaneously creates the information needed to interpret later results. Outcome data alone are insufficient because a team cannot meaningfully evaluate a strategy without knowing whether it was implemented as intended.
During Study, the team examines both outcomes and implementation. It asks not only whether the desired result occurred but what the data reveal about the intervention process, the students and settings involved, and the assumptions underlying the original plan.
During Act, evidence determines the next step. Weak implementation may call for training or support. Strong implementation with weak outcomes may call for a different intervention. Positive outcomes may justify continued implementation or careful expansion. Each decision creates the starting point for another cycle.
For school psychologists, this framework provides a way to integrate measurement, consultation, systems thinking, behavioral problem solving, and progress monitoring. The professional contribution is not simply identifying a disparity or recommending a program. It is helping the team build a defensible chain from data → problem definition → intervention → implementation → outcomes → next decision.
Apply Continuous Improvement to a School Discipline Scenario
Apply the continuous improvement principles from this lesson to the professional scenario below. Consider how the team should move from an identified disparity to a sufficiently specific, monitored, and revisable improvement strategy.