返回文章列表
aiNovember Releasesopenai-background-completion-windows
🧠

Would You Pay Less for an AI Report That Arrives Later?

OpenAI added discounted two-hour and eight-hour background options. A service-level exercise weighs deadline targets, elapsed workflow time, and the recovery cost of missing an operating handoff.

iBuidl Research2026-11-1816 min 阅读

TL;DR

OpenAI added discounted two-hour and eight-hour completion windows to the Responses API public beta on November 17. The discounts are 30 and 40 percent below Standard model token rates on eligible requests. The selected window targets an initial API response after acceptance. An operational report may still need source gathering, tool follow-ups, review, and a publication slot. Choose the window using the deadline for the usable report and the cost of a miss.

A distribution supervisor needs a morning summary of overnight exceptions. The analyst can ask the model for a report as soon as the records are ready, or take a lower price in return for allowing a longer response target. The choice appears to have an obvious answer when staffing starts in eight hours. It becomes less obvious when a source file arrives late and the final report has to be checked thirty minutes before the shift.

That distinction defines the product question created by OpenAI's November 17 update. The API offers a default soonest option and discounted two-hour and eight-hour target windows for supported models. A response window is an available pricing and scheduling choice. A business deadline is an agreement about when a person can act on the result. Span both with one budget before sending the request. OpenAI API changelog and background mode guide

Envelope A is an operating incident with a live reader

In the first fictional request, a department supervisor wants a summary of the last hour of exception codes twenty minutes before a staffing decision. A validated aggregate is already available from the line-of-business system. A model can reorganize it into a short explanation. The value of the summary mostly disappears if it reaches the supervisor after they allocate staff. Save the decision time and deduct the time required for checking and delivery before choosing how long the request itself may take.

If the supervisor can comfortably inspect a final response in four minutes and it takes a minute to publish it, the model and data preparation together have about fifteen minutes inside the twenty-minute lead time. A two-hour response target does not fit that planning envelope. An ordinary faster path with a brief answer can be reasonable if observed completion supports it. The application can also show the existing validated aggregate directly at the deadline. That outcome is meaningful even if the narrative takes longer.

This example favors a modest output. A user who needs to allocate people can act on a clearly labeled list of queues and observed counts. Preparing a full explanation of last week's volume at the same time might waste the short margin for error. The report owner can split immediate evidence and supporting analysis into separately scheduled items, each with a permission and audience appropriate to its purpose. An asynchronous rate reduction can then apply to the analysis if its real deadline permits.

The fallback has its own source and display rules. It identifies the data timestamp and names what can be decided from the available counts. If a later narrative completes, it can be added to the record after the staffing decision with its own date. It cannot silently become the original basis for a decision that happened earlier. The timing contract helps the operations team answer an audit question about which evidence supported its decision in the room.

Envelope B has a real two-hour response budget

The second team receives a set of exception categories at 10:00 and holds a review at 13:00. Source exports normally take fifteen minutes to validate. There are twenty additional minutes reserved for review and five for publishing a usable packet. The effective response budget is about 140 minutes in the planned case. A two-hour initial response target fits into that budget with a further twenty minutes of slack if the API accepts the request when preparation finishes.

For this workload the response is independently sufficient. The initial request contains the allowed documents and data. The application checks formatting and a reviewer verifies the fields used in the decision. No follow-up lookup is expected during the normal path. A two-hour target could be a sensible option because the chosen window leaves time for the other measured stages. An eight-hour target would make the modeled plan depend on completion far ahead of its target.

Suppose the validated exports arrive forty minutes later than usual. At 10:55, the API acceptance and two-hour target would imply a response around 12:55 at the edge of that target. Review and publication would then run beyond the meeting. The scheduler can recalculate slack at submission. It may use a faster path or deliver the verified source records in time while keeping the narrative pending. Its chosen route changes because the prepared input arrived later, even if the type of analysis remains the same.

A production test could record these events for a few cycles and characterize both the planned and observed budget. The delivered packet timestamp comes from storage or publication, and a separate API event supplies response readiness. A calculation based on only the latter can overstate success when the model completes before the cutoff and the report is stuck waiting for someone to review it. The business objective is measured where its recipient can actually use the result.

The team also considers whether exported documents should include the full day to date. A source freeze before submission ensures that the evidence packet describes a reproducible snapshot. If the next hour is material to the meeting, a later update can show the new snapshot and deltas. A job that silently revises its time range while working may take longer and change what its facts represent. Separate, dated views let recipients understand exactly which records were included.

Envelope C must fit two replies into an overnight cycle

The third team prepares fifteen document reviews for the start of the next day. It often has ten hours between acceptance of the job and handover. The first API response might request an applicable revision from an authorized source tool. Once the application retrieves it, a follow-up response summarizes the evidence. Review happens after both replies. The workflow still sounds like one report, although two different responses consume time.

OpenAI's current guide specifies that the selected completion target for the initial response starts at API acceptance and applies to that response. Reusing a matching continuation identifier, model, window, and project can make follow-ups eligible for greater queue priority under the documented setup. The documentation describes scheduling eligibility. The application should budget separately for the tool stage and the time needed for the follow-up. The report acceptance condition arrives only at the end. Responses API background mode and continuation workflow

For an example with made-up times, the request is accepted at 21:00. Its initial response arrives at 04:00 and asks for an authorized document lookup. The scheduled application retrieves the document in fifteen minutes. The follow-up result arrives at 06:40. Review takes twenty minutes. The final handover at 07:00 fits a 07:30 deadline. Change only the initial response to 05:00, hold the later stage durations fixed, and the handover moves to 08:00. Both initial responses fit within eight hours of acceptance. Only the first entire workflow meets the fictional business deadline.

A team with a hard early cutoff can change the plan in several ways. It can fetch the core source before the initial model call, making the response self-contained. It can choose the two-hour target when the workflow's later steps are variable. Or it can run separately accepted jobs for distinct documents with independent results when the final assembly tolerates partial delivery. The actual choice should be tested on recorded cycles with comparable evidence and review quality. It is common for source selection to yield a larger scheduling gain than changing a request option.

Some overnight jobs have genuine dependencies. The first answer may identify one of several relevant attachments based on a policy question. Fetching everything beforehand could increase token usage and send unnecessary content. The owner can instead prepare a bounded local index so the model receives less text and its subsequent lookup resolves promptly. The team tests whether that approach saves time after accounting for missed documents or added clarification. A shorter path deserves adoption when its accepted results preserve the evidence the user needs.

The guide also distinguishes the programmatically chosen continuation ID from the response IDs used for retrieval. Save them in separate fields. If a previous response expires before a follow-up, the guide describes sending the appropriate saved history instead of referring to the unavailable response, with the prior items in order and the required tool result. The design may require application-side retention. An operations team should evaluate that requirement against its own data rules before it relies on rebuilding old state. OpenAI continuation and response retrieval documentation

Price the late branch as well as the expected branch

The discount applies to eligible model token rates. Start the budget with the Standard token charge that the actual request would incur, plus separately billed or internally measured costs. If the Standard model charge for an invented report is $1.00, the two-hour rate gives $0.70 on the same billed token quantities and the eight-hour rate gives $0.60. That ten cents is the initial difference between the discounted paths. Token quantities, costs for other components, caches, and follow-up requests can change the final bill. OpenAI API pricing

Consider sixty such reports in a rehearsal period. If each requires only that request, switching from two hours to eight saves six dollars in modeled token charges. Suppose seven miss an internal deadline and trigger a one-dollar rerun on the faster path under an assumed policy used only in this example. The reruns add seven dollars. The slower path would now cost one dollar more than the two-hour path for tokens alone, assuming the two-hour path needs no such reruns. Staff time and the consequence of a missed deadline would raise the difference in affected cases.

The break-even calculation can be done per report. If the slower option saves ten cents and an extra rerun costs one dollar, five additional reruns per hundred reports leave half of the original savings. Ten consume it completely. If the actual replacement costs two dollars per event, five additional reruns per hundred consume it. The relevant rate is the incremental rework compared with the alternative. A team can compare these inputs with observed recovery counts rather than assume that the provider's advertised percentage is its cost to deliver.

For a job with a less strict deadline, waiting can be cheap. A completed report may still be useful to a trend reviewer if it arrives after a particular morning. The model can mark the amount of value lost or reassign the report's downstream use. A job whose destination only accepts today's manifest has much higher cost after expiration. Grouping reports by the value they retain after delay lets a manager allocate the cheapest paths without making the whole queue dependent on one tolerance.

Another common financial surprise comes from minimizing model token rates and inflating billed work. The provider documents that ordinary Standard requests outside discounted windows cannot reuse prompt cache entries created by the discounted requests. A team that expects a rapid rerun should account for how its allowed request will actually be billed. It can preserve useful prepared inputs and avoid regenerating expensive context blindly. A budget with the original and rerun usage attached to the same application job helps reconcile the actual spend.

Before delivery, the scheduler needs evidence of what it already submitted

Implement the ordinary state transitions around the response. Once the API accepts a request, save its response identifier and the application job it belongs to. The guide describes waiting for an event or polling the saved identifier while a discounted request remains queued or in progress, and automatic retries for eligible capacity errors under the stated conditions. A poll timeout at an application process records an observation gap. It does not justify classifying the underlying run as failed before observing the provider status.

The webhook path deserves an independent test. The provider advises signature verification and handling longer work outside the acknowledgment. The application verifies the message and connects a terminal event with its saved response ID. Duplicate deliveries should not create duplicate publications or tool work. A report generated with the old snapshot cannot replace the result of a newly accepted fallback simply because it arrives later. Tie publication to a current revision and let operators inspect other versions if needed. OpenAI background mode guide

A timed rehearsal can be run without any external publication. Feed the service a terminal event recorded before the cutoff, then an equivalent event received afterward. The first one can enter the ordinary validation queue. The second case lets the team evaluate whether stale results get annotated, stored, or withheld under the normal workflow. An app with a finite retrieval period also needs a state for a valid notification whose referenced content is no longer available. The agreed response to that case is part of the readiness test.

The data contract matters alongside timing. The guide says background mode uses temporary storage of response data to run the request and permit retrieval, including for examples involving store: false or Zero Data Retention, and discusses encryption and data controls. An application owner should confirm the particular processing and retention terms that govern the deployment and the kinds of documents involved. A fallback to local summarization or prepared aggregates can keep an operational report possible when a needed document cannot enter the named service.

The guide also says discounted windows use Standard processing, require nonstreaming in the described configuration, and draw from a rate limit shared with Batch requests. The default soonest window uses the regular limits. A team that submits a large batch plus overnight deadline jobs should check what is available to each at the time of submission. Record the acceptance time and distinct application deadline because waiting to obtain admission can consume budget before the provider's target starts. Discount eligibility itself depends on using an eligible model and tier.

Work sharing can move the actual start time

Imagine the overnight analyst schedules an independent batch of one hundred archived summaries for 21:00. The summaries can be ready at any point the following day. The deadline-sensitive document reviews arrive at the same time. The background guide says the discounted requests and Batch API jobs share an input limit. If the bulk job consumes available admission capacity first, the time before the deadline reviews can be accepted can grow. A scheduler that starts the eight-hour clock at local dispatch would conceal the effect.

Record the token quantities submitted, the API acceptance time, and the deadline of each job class. The capacity owner can leave margin for a known overnight event or schedule the flexible batch at a different time. The application can make that decision using actual usage and documented limits without guessing which processing path the provider used. A test should include a heavy submission period since an apparently successful low-volume run might never exercise admission pressure.

The same principle applies to human reviewers. If all fifteen reports arrive late in a narrow cluster, each one may have an individually reasonable review allowance. The reviewing team might still be able to check only five before handover. A scenario that measures readiness at the provider ignores this last shared queue. Give accepted administrative work an explicitly scheduled review window, or review records incrementally where independent sections allow it. The number of completed API responses per hour is a service component, and the number of approved packets per hour is a business outcome.

For a toy team with one reviewer taking six minutes per report, fifteen simultaneously ready packets require ninety minutes of service. If all become ready thirty minutes before handover, the reviewer can handle up to five in the simplest uninterrupted plan. Changing the queue order based on actual required use can preserve five material outcomes. Moving input preparation earlier, using a shorter route for certain documents, or staffing a second reviewer can change the necessary capacity. Choosing among those options requires knowing which constraint the pilot actually reached.

Set the source and recipient clocks in the same time zone

Store the deadline as an instant and carry the intended local handover time with it. A local meeting can recur at the same wall time even while one system shifts its relationship to UTC. A job started eight elapsed hours before it usually runs could therefore miss the new cutoff if the application uses an assumed offset. The rehearsal uses event timestamps and a local example with a shifted calendar deadline. The goal is to ensure an actual late event lands in the fallback path when its displayed clock time appears deceptively similar to a successful one.

The timer should also record why it stopped. There are separate cases when the owner cancels the need for the report, a source becomes invalid, the API returns an incomplete result, or the job simply passes its business deadline. Keeping those causes distinct makes future price comparisons more useful. A canceled meeting has no missed delivery and can still incur some work. An incomplete response can consume capacity and require recovery. A correct report that becomes stale before use tells the owner more about scheduling than model quality.

Finally include all started application jobs when evaluating timely handover. Reporting median latency on only the successfully delivered packets can reward a path that abandons the slowest requests. Show the completion and usable-by-deadline rates with that median. The team can then learn whether the advertised discount improves its own operating budget while accounting for the work that disappeared from a successful-responses-only chart.

Rehearse an arrival that misses by one minute

A team ends its trial with a replay in which a fully correct explanation reaches the review queue one minute after the final accepted handover. A text-quality score can still give the response full credit. The operations score records a missed handoff. The owner now knows the actual cost of the path and which grade belongs to which decision. A stable trial can include cases that arrive just before the cutoff to verify that review is always budgeted before calling a report ready.

The metrics can be simple. The team counts accepted reports that reach their recipient on time, reports that were useful after the planned cutoff, and jobs that had to replace the report with verified source data. It records how much of the elapsed time occurred before API acceptance, inside one or more responses, and during follow-up or review. Price is measured across all attempts and recovery runs. The process yields a comparable cost per usable result under each planning envelope.

The final decision can be made independently for each type of work. Use the lowest modeled cost supported by the true delivery window and observed completion distribution, with a clear deadline fallback. Select a shorter or more self-contained path when a later answer has little operational value. The November update adds options to the API. Its value to a specific team shows in cheaper reports that still reach the person waiting for them.

Sources

更多文章