返回文章列表
Interrupted Time SeriesProgram EvaluationCausal Inference
🔬

Did the New Opening Hours Change Attendance?

A library opening-hours example shows how choice of baseline, level, slope and comparison locations changes an interrupted-series claim.

iBuidl Research2026-11-1817 min 阅读

TL;DR: An interrupted time series evaluates a dated change against a clearly defined continuation of the pre-intervention trajectory. Separate an immediate level shift from a change in trend, account for repeated measurement and seasonality, and investigate other events that could have changed the same outcome at the same time.

A library extends its weekday opening hours. The next monthly report shows more visits, and the board credits the new schedule. The visits chart has been rising for two years. It also shows a jump every time schools begin a new term. A board member asks the question that makes the claim testable. What would the same branches probably have recorded during those weeks under their old schedule?

The plot annotations below use an invented library system called Cedar County. Every visit count, period and coefficient is hypothetical. An interrupted time series uses repeated measurements around a dated intervention. Methodological work by Lopez Bernal, Cummins and Gasparrini explains the distinction between level and slope changes, potential autocorrelation, seasonal trends and time-varying causes. This example applies those ideas to a nonmedical service decision. Interrupted time series regression tutorial.

Set the calendar seam at the first full week

Write down when the public could actually use the new hours. The announcement, first evening session, complete rollout and first reliable count might sit on different dates. If the intervention is a gradual ramp across branches, its implementation exposure has a trajectory rather than one abrupt boundary. A first full week can act as a useful breakpoint when the change is actually delivered consistently at that time.

Cedar County schedules an extra evening opening on four weekdays. An interim rollout in one branch begins two weeks before the full schedule. The evaluator can mark those weeks separately, or model exposure through the documented introduction. Calling them ordinary pre-intervention weeks could put early effects into the baseline. Calling them fully treated could water down the modeled level change. Write down which population and dates receive each definition before interpreting results.

The definition of the outcome also needs a place on the chart. A door counter records physical entries. A member who leaves for lunch and returns can count twice. Wireless visits from people who use an outdoor area may be absent. A positive result can describe more entry events without proving that more distinct residents used the library. If the outcome of interest is unique access, identify what legitimate data can measure that concept and how incomplete coverage will be handled.

A weekly total is one natural service outcome. Visits per open hour is another. The new policy changes the number of hours in the denominator and can lower the rate while increasing total access. Suppose a branch moved from 500 visits across twenty-five hours to 560 across thirty-five hours. Total visits rose by 60. Visits per hour declined from twenty to sixteen. These results can coexist under the same policy, and each matters to a different budget decision.

The board defines the primary question as additional visits per branch-week among branches that implemented the opening change. It keeps staffing costs and visit concentration by hour as secondary outcomes. If it wants evidence about attendance per open hour, a count model with an exposure term or a carefully defined rate may answer that different question. Using opening hours as an adjustment variable in the main total-visits analysis could remove part of the mechanism the policy intentionally created.

Draw the unbroken line before plotting the result

A simple segmented model includes a time index, an indicator for the active intervention, and time elapsed after the intervention. Set elapsed time to zero in the first treated week. Then the indicator coefficient describes an estimated immediate change at that boundary, while the elapsed-time coefficient describes how the post-period slope differs from the earlier trend under the model. The displayed effect at a later time is their combined implication.

Give the baseline at the first complete week a fitted level of 520 visits if the old linear trend continued. Assume the old slope is five extra visits per elapsed week, the immediate level estimate is thirty additional visits, and the incremental slope is two. At elapsed week zero the fitted treated level is 550. At week ten, the continuation of the old line is 570. The fitted treated value is 520 plus fifty from baseline trend, thirty from the shift and twenty from accumulated slope change, or 620. The modeled difference at that date is fifty.

The level estimate and slope estimate answer questions about the fitted trajectory. A stable thirty-visit shift would create the same difference every later week under a linear model with no slope change. A policy that takes time to become known may move demand gradually, with a small immediate difference and a delayed increase. A temporary novelty effect could spike and fade. Such choices should reflect the service theory and evidence before a result is fit, with alternatives reported as sensitivity analysis.

The model's reference time matters for interpretation. Coding a general time variable and interacting it with the post-policy indicator can produce a coefficient whose intercept difference is defined at time zero of the complete series, rather than at the policy date. Using an elapsed post-intervention term with the date aligned makes the level easier to interpret. If the model is specified differently, calculate the actual fitted differences using the parameterization, rather than reading a coefficient label as a literal change.

The old line is a counterfactual estimated from data and assumptions. There is no direct observation of the implementing branches in the same weeks under the prior schedule. A design can support that line with a stable measurement process, plausible baseline trajectory, comparison series and inquiry into other events. It should make departures from those assumptions visible. Confidence limits on the fitted difference alone cannot repair a disconnect between the counterfactual and what the service would plausibly have done.

Consider whether a single linear path makes sense over the intended window. If the county opened another branch during the baseline, the first half may represent a structurally different service. A nonlinear baseline or explicit event date can be appropriate if supported by the evidence and analysis plan. A fitted curve selected mainly because it makes the post-policy effect large is weak support. Plot the actual observations, pre-period fit and counterfactual across the measured horizon before describing precision.

Mark the week school opened in two colors

Usage of library facilities can follow seasons, holidays and school schedules. A schedule expansion at the start of a new term arrives at the same time as other demand influences. Several baseline cycles can help estimate recurring variation. Depending on data and design, model seasonal periods or include a comparable calendar effect based on the way the service operates. A plot of deviations across the year can show why the chosen approach matters.

Use the right scale. A short weekly record may include only one term transition before treatment, giving little evidence about recurring jumps. Adding a complex time pattern to a short series can consume the variation needed to distinguish trend. Having a later term transition after the policy can show repeatability under the changed service mix, but cannot by itself reconstruct what this term would have been without the policy. The report describes how much historical information actually supports the chosen pattern.

Other events can coincide with a policy change without a regular cycle. The county may launch a citywide reading campaign, a branch may reopen a classroom, or a nearby event venue may close. The analyst makes a dated landscape of such changes and decides whether they would predict a shift for the same branches and outcome. A coincident improvement in every location leaves multiple explanations if no eligible comparison could separate them.

Measurement changes deserve the same scrutiny. In the example a new door counter is installed on the weekend between the old and new schedules. Its placement counts people coming through a side entrance that older records omitted. A level change could reflect more complete measurement. One solution uses overlap with a hand count or another independently stable measure at both times. If no reliable comparison is available, state that the measurement transition limits attribution rather than assuming the device only improved technical quality.

A change in missing data can be material. Missing counts from a closed facility and a valid observed zero describe distinct states. A broken counter left blank during a busy week should not be recoded as empty through convenience. The dataset can record outages and report sensitivity to their treatment. If the intervention required newly opened hours in which counting occasionally fails, the missingness may be related to the very effect the analyst is estimating.

A holiday can illustrate calendar selection. Suppose the inclusive first treated week contains a holiday when only two extra evenings actually ran. A chart records that deviation from the fully scheduled exposure. Delaying the time origin until the next full week makes the estimand clearer for a standard implementation, but the first calendar week still had partial exposure. Keeping it in the dated record makes the decision reproducible. It can also be useful to report calendar-week effects relevant to the actual launch.

Put a neighboring branch on the same chart

A similar branch with continued ordinary hours can supply a comparison series. Its outcome should be measured on a compatible schedule and subject to relevant shared external influences. It may face the same school opening and general publicity. If the extended hours draw readers from that branch, its outcome is partly affected by treatment. That movement can increase or decrease an estimated contrast depending on the direction of exchange. A suitable comparison needs reasons to believe it represents a plausible unexposed path.

Suppose Cedar County's implementing group has a fitted immediate increase of thirty attributable to the combination of changes visible at the boundary under that model. Its comparison group has a fitted ten-visit increase relative to its own baseline. A corresponding adjusted boundary difference is twenty in a simplified comparison at compatible units. That arithmetic can remove a common change only if it estimates what would plausibly have also affected the implementing group. Different trends and scales need explicit attention in an actual controlled design.

Plot pre-period values for both series and compare patterns, seasonal timing and the effects of known disruptions. Matching groups on one preceding week provides little reassurance about future trajectories. A smaller branch may have an equivalent change in percent but a different change in counts. The analysis chooses the outcome scale before deciding what constitutes a shared shift. It also states whether totals, rates or branch-weighted averages match the decision.

Some library systems roll out extended hours in stages. Other branches can supply comparison time while they remain under old schedules. A branch selected early because demand was already accelerating may differ from a later branch. The analysis needs to address selection and the outcomes observed over each period. Repeated observations give evidence of dynamics. They do not remove the lack of random assignment by themselves.

A broader geography can reveal displacement. Total county visits could remain stable while more readers visit the newly open central branch and fewer visit satellite branches. If the public goal is flexible access for people with evening jobs, redistribution with a new time of use may still carry value. If the goal is additional total visits across the whole system, it is a different outcome. State the benefit the original service policy intended before judging the observed changes.

One comparison branch is itself a small population. A one-day outage there can materially change the fitted difference when all other branch patterns remain stable. Add a second comparison or inspect whether results depend on an unusual unit if data allow. Present changes in precision and the point estimate across compatible analyses. Picking the most convenient neighbor after seeing its results creates another path for researcher discretion to influence the claim.

Inspect the space between dots before reporting uncertainty

Weekly visit counts usually share influences across nearby periods. Weather, local popularity and staffing issues can last more than one week. A regression procedure that assumes independent errors can present inappropriate uncertainty when the residuals are correlated. The chosen model or variance procedure should address dependence relevant to the time scale and design. This is particularly important when only a small post-policy series is available.

Counts can vary more than a simple model expects. A large special event can create a sharp excess visit count in one week. Showing the actual points can reveal that one episode dominates the average. The analyst can prespecify a reasonable count or continuous model on the relevant scale and diagnose misfit. Strongly overlapping moving averages are excellent visual aids but give successive plotted points shared underlying observations. Using them as independent data would distort an uncertainty calculation.

Adding more entries per weekly count does not automatically create more independent weeks. One thousand visits can make a count precise for the covered people, while the causal design still has the same limited number of time positions. A short follow-up poses an identification concern for a sustained trend change. Consider an immediate outcome with sufficient follow-up requirements. Label slope estimates from a few weeks as understanding in progress.

Use residual and fit diagnostics that match the model. Look at remaining seasonality, dependence and influences of specific time windows. An analysis selected after seeing many candidate models should acknowledge that uncertainty. A planned robust comparison can show whether results hold over reasonable baseline lengths and timing assumptions. A response to a visible lack of fit should have an operational explanation where possible. Altering the model merely to improve apparent significance leaves the design fragile.

Report an appropriate interval for the comparison at chosen follow-up points, including covariance between level and slope terms when combining them. An interval for an estimated mean difference and a prediction range for an individual future week answer different questions. A board deciding staffing next month needs an idea of actual volume variability as well as the estimated change. A policy analyst evaluating effectiveness may emphasize the fitted average trajectory and assumptions.

Try reading the old part as if the outcome were still hidden

Before using the treated trajectory, Cedar County can test how plausible the baseline model looks at earlier historical boundaries. Fit the pre-policy method on an earlier span and examine predictions for a reserved block of still-untreated weeks. This is a check of forecast behavior under known operating conditions. It gives the team specific information about recurring timing errors, but a successful pre-period forecast cannot prove that no new event occurred alongside the actual schedule change.

The chosen baseline can be poor at holiday weeks and useful at ordinary ones. An analyst can ask whether the intended follow-up begins in a period similar to the evaluated forecast horizon. Differences in enrollment, temporary closures or data recording should be documented before treating the historical holdout as equivalent. Forecast horizons matter too. A plausible projection two weeks ahead may have little support across a full school year.

Suppose the baseline forecast frequently undercounts the first school week by thirty-five visits but tracks the rest of the term closely. A new modeled level increase of thirty precisely at that week needs to be read beside this diagnostic. Rather than silently removing the unfavorable point, the analysis can try a previously justified calendar term and describe its effect. The actual schedule response may still be positive. The information available about one event overlapping another now limits how exactly the report can attribute it.

The evaluator can also record a plan for a delayed response. In a model that prespecifies two weeks of gradual uptake, there may be no instantaneous step at opening. If the actual use change emerges when the library begins advertising its evening workshops, implementation evidence distinguishes late discovery from a generic time fit. Showing the observed staffing and communication dates shares the mechanism readers need to evaluate that alternative.

Finally, specify which revisions to counting errors occurred while outcomes were blinded. A data recovery team can validate a door-counter outage using maintenance records independent of the study hypothesis. If the model was already known when a week was corrected or excluded, document that sequence. Integrity depends on retaining enough information for another analyst to understand why the curve changed. A decision log prevents a simple choice of data cleaning date from looking like a property of the policy itself.

Write the conclusion as a dated annotation

At week ten the county can say the illustration estimates fifty more weekly visits than the modeled continuation at that point, with an uncertainty statement and evidence notes in a real analysis. It can identify which branches, whether the comparison was direct or adjusted, and how many complete observation weeks support the result. It should separate any claim about changed visits from claims about access for particular groups or the cost of each visit.

The report can also show cumulative effects if that is the target decision. Adding fitted weekly intervention differences across a declared horizon gives an estimate of extra visits during that horizon under the model. It differs from multiplying the week-ten gap by the number of weeks when the gap changes over time. Under a modeled increase of thirty at week zero rising by two each week through week ten, the eleven weekly differences form a series from thirty to fifty. Their sum is 440. That number is an illustration of what the chosen slope implies, not independent evidence of additional visits.

The total access question needs a consistent accounting of any new time windows and displaced visits. A counter can segment visits by evening and daytime before aggregating. If some readers shift toward late hours and other users take the freed daytime capacity, distinct changes can sit behind one favorable total. A researcher can choose a follow-up survey or other supported data for audiences whose accessibility was part of the program's purpose. The original count remains meaningful within its scope.

On the last plot add marks for future policy decisions. When the school year changes, collect the same definitions through the next comparable period. When another branch extends hours, distinguish staged implementation from a new general trend. Share data definitions and model assumptions so someone repeating the analysis checks which instruments, hours and populations stayed comparable. A credible analysis should still make sense when its original author has moved on.

Give the service board the decision variables in natural language. How many additional visits does the evidence support at named times? What changes in the measurement process limit that conclusion? Which upcoming observations would materially revise it? The line on the printed chart is evidence of a plausible comparison with a dated counterfactual. The analysis earns its place in the decision when it tells the board why that particular line is a responsible benchmark.

Sources

更多文章