TL;DR: An empty search establishes that a query found no matching records in the sources it could examine. It does not establish that an exposed credential was unused. Investigate coverage before interpreting silence: account, project, Region, event class, collection period, retention, permissions, and identity representation all matter. Then combine provider activity, application records, resource changes, and billing into findings whose limits remain visible. Sources were checked on October 28, 2026.
Railway's preliminary report says that its investigation concerned compromised internal systems and a subset of stored secrets. Its customer guidance recommends reviewing activity at the services those secrets could access. The provider's own forensic work and a customer's downstream investigation answer different questions; neither should be treated as a complete substitute for the other. Railway preliminary report
This memorandum uses an invented company, Alder Archive, with an AWS storage account and a Google Cloud processing project. It has contained identified credentials and now needs to assess possible misuse. No event, cost, identifier, or finding in the case is an observation from the Railway incident. The aim is a defensible investigation method, not a prewritten verdict about any customer.
The memorandum begins with three competing explanations
The first hypothesis is that the suspect credential was never used by an unauthorized party. The second is that unauthorized activity occurred and is present in available records. The third is that unauthorized activity occurred outside the records the investigator can currently observe. An empty dashboard search is compatible with both the first and third hypotheses.
The investigator writes down what would distinguish them. A provider request attributed to the credential and inconsistent with authorized work supports a misuse investigation. Confirmed coverage of a particular event class makes silence more informative for that class. Missing collection or expired retention leaves uncertainty even if every available search is empty.
This framing prevents a common inversion of evidence. The team should not require proof of misuse before preserving logs or containing suspected authority. Nor should it label every unfamiliar event malicious merely because a breach notice arrived. The response can act conservatively while the investigation tests explanations carefully.
The memorandum also distinguishes exposure from use. A secret can be exposed without an observed downstream request, and a request can be authorized by a legitimate service running from an unfamiliar address. These states have different implications. Clear language lets security, application, and finance teams discuss the same evidence without silently changing the claim.
Build the coverage map before the search query
Alder lists the accounts and projects reachable through each suspect identity. The list includes cross-account roles, delegated access, and resources outside the application's usual Region where permitted. A billing account or organizational console can aggregate some views without granting the investigator access to every underlying log source.
Next it lists event classes. Administrative changes, object reads, function invocations, authentication events, application requests, and billing records are different observations. A service might record some automatically and require explicit configuration for others. The investigator checks the controlling provider documentation rather than interpreting the generic label audit log as a universal recording promise.
AWS CloudTrail Event history covers recent management events by account and Region; its ordinary view does not include data events. AWS separately documents data-event collection and notes that trails and event data stores do not log those events by default. A successful search of Event history therefore does not answer every question about object access. CloudTrail Event history CloudTrail data events
Google Cloud distinguishes Admin Activity and other audit categories from Data Access logging. Data Access logs are generally disabled by default, with documented exceptions and service-specific behavior. The investigator checks the actual service and collection configuration, including what the investigating identity is permitted to read. One broad sentence about Google logging cannot replace that examination. Google Cloud audit categories
The map includes start and end times for collection and retention. Enabling a missing log today can improve future visibility but cannot reconstruct every unrecorded request from yesterday. A policy that is enabled now is not evidence that it was enabled during the exposure window. Historical configuration evidence belongs beside the current setting.
The first exhibit is a table of observable questions
The following table is Alder's investigation worksheet. It names evidence needs without assuming that the evidence exists.
| Question | Potential evidence | Limitation to establish |
|---|---|---|
| Was a new identity or permission created? | Administrative events and current configuration | Historical coverage, deletion, and legitimate change context |
| Were stored objects read? | Relevant object access records | Data-event selection, resource scope, and collection period |
| Was processing capacity created? | Resource events, inventory, and billing | Creation versus actual use and delayed cost reporting |
| Did the known application make the request? | Application traces and provider request references | Trace sampling, missing failures, and clock alignment |
| Was another identity impersonated? | Session and delegation evidence | Whether the relevant fields and services are recorded |
| Were logs changed or removed? | Logging configuration history and preserved artifacts | Collection gaps and available integrity mechanisms |
The worksheet prevents the team from using the easiest source to answer every question. Billing can reveal unexpected processing cost without identifying all data accessed. An application trace can explain one legitimate request without covering direct calls made outside the application. Each source contributes to a particular claim.
The investigator assigns an owner to unresolved coverage. A missing permission to read a log is an access problem; missing collection is an observation problem; missing retention is a historical evidence problem. Their immediate remedies differ. Combining them into logs unavailable makes it harder to decide what can still be recovered.
Translate identity before searching by its printed name
Alder initially searches for its long-lived key identifier and finds ordinary service activity. That is useful but incomplete if the identity could assume a role or obtain a temporary credential. Later requests may be represented through the resulting session rather than the original key's familiar label.
CloudTrail's userIdentity documentation explains fields for different identity types and temporary sessions, including session issuer information. The investigator uses those documented relationships to follow identity transitions. A principal name is a search input, not a guarantee that every relevant event contains that exact string in the same field. CloudTrail identity fields
The investigator retains the raw provider record and a separate normalized view. The raw record preserves original field names and values. The normalized view helps compare sources. If a parser drops session details or merges several principals under one display name, the normalized view can conceal precisely the distinction the investigation needs.
Identity attribution also has a limit. A record showing that a credential authorized a request does not identify the human who held it. The investigation can associate the event with a credential, session, or account under the provider's semantics. Human attribution requires additional evidence and should not be inferred from a convenient account label.
The team searches for newly created authority as well as the old identity. A new key, role binding, application authorization, or delegated account can persist after the original credential is revoked. The inquiry follows what the exposed authority could create, then checks whether those changes fit legitimate operations.
A request time and a discovery time are different
The incident ledger records four useful times where available: when an action happened, when the provider recorded it, when the record became available, and when the investigator retrieved it. Those times can differ. A delayed record is not automatically a clock error, and a late discovery does not establish that the action occurred after containment.
Alder normalizes its working chronology to UTC while preserving original offsets. A source that contains a local time without a zone remains ambiguous until resolved. The investigator does not silently choose the offset that makes the narrative look tidy. A few minutes of false alignment can move an event from before revocation to after it.
The search window begins broadly enough to account for uncertainty and is narrowed with evidence. A public disclosure time is a communication milestone, not necessarily the start of exposure. Railway's reported incident chronology is relevant context, but a customer's own credential may have a different creation, storage, or revocation history.
The team records boundary assumptions explicitly. If the old key was added to the host only after a documented point, that fact may narrow one exposure route. It does not prove there were no other routes through developer machines, repositories, or another platform. The investigation should answer the stated incident question without allowing one timeline to certify the credential's entire lifetime.
Follow an unfamiliar event into its business context
The first candidate is an object-read event at an unusual hour. A developer recognizes the object as part of a scheduled export. Recognition is a lead, not a completed explanation. The investigator looks for the corresponding job identity, execution record, expected request range, and output destination.
The job's application trace contains a provider request reference matching the event. Its schedule and authorized destination also fit. Alder classifies the event as explained legitimate activity and preserves that reasoning. The classification does not establish that every nearby event is legitimate. Evidence explaining one request should not spread through a whole time window by association.
Another candidate involves a permission change with no matching change request. The identity owner checks whether an automated provisioning system could have made it. If no authorized explanation exists, the event remains suspicious and the team considers containment of the resulting authority. The investigator does not wait for an invoice to confirm that a permission change matters.
For each candidate, the memo states what was observed, what could explain it, and what evidence changed the classification. This is more useful than a list of events marked bad by intuition. A later reviewer can see whether the conclusion rests on a stable fact or on an assumption that still needs verification.
Source addresses help rank questions, not settle identity
A request from an unfamiliar address deserves investigation. It can also arise from a changed egress route, a provider-managed service, a developer's approved diagnostic session, or another legitimate consumer. Shared hosting addresses can make two unrelated users appear similar, while one authorized application can use several addresses over time.
An apparent match to an expected address also does not prove legitimacy. A stolen credential used from a compromised workload may share the application's normal network location. An address allowlist can reduce exposure under its actual design, but an observed address is not a complete explanation of who controlled the request.
Alder combines address information with identity, operation, resource, timing, and application evidence. It preserves the address as an attribute rather than replacing the event's entire identity with a label such as Railway traffic. That approach avoids both overconfidence in familiar addresses and automatic accusations against unfamiliar ones.
The report uses careful wording: outside the documented application egress set, unexplained after owner review, or consistent with a known job. Those statements describe evidence. A statement naming an attacker or a specific organization requires a stronger basis than network resemblance.
Billing is a sensor with a different clock
Finance reports a higher daily bill. The investigator asks which service, project, Region, resource, and usage interval produced the change. An aggregate cost increase can come from ordinary demand, a pricing change, delayed reporting, or unauthorized consumption. The increase is a reason to investigate, not a complete attribution.
In an illustrative calculation, a project spends six hundred currency units in a comparison period and nine hundred in the incident period. The difference is three hundred, or fifty percent of the earlier amount. That percentage describes a cost change. It does not establish that three hundred units were fraudulent or that all unwanted activity generated a charge.
The team reconciles unusual resources against deployment and ownership records. An unexpected compute instance may be visible in configuration history even before its full cost appears. A data read may matter even if its incremental cost is small. Security consequence and billing magnitude are related only through the actual service and operation.
Finance preserves the relevant billing detail and request references for any provider inquiry. The investigation does not promise a refund or assume that a hosting provider can reverse another provider's charges. The organization's response separates evidence collection, containment, business accounting, and any contractual support process.
Preserve artifacts before repairing their consequences
If the investigator finds an unauthorized resource, immediately deleting it can remove evidence needed to understand its use. Continuing to run it can create cost or harm. The appropriate response coordinates containment and evidence preservation rather than treating preservation as a reason to leave active abuse untouched.
Alder records the resource identity, relevant configuration, provider events, and any appropriately captured evidence before the authorized owner removes or restricts it. Sensitive content is retained only through an approved evidence process. The incident channel receives references and conclusions rather than raw data dumps containing customer information or credentials.
For delivered CloudTrail files, AWS offers an integrity-validation mechanism involving signed digest records. That can help assess whether delivered files changed or disappeared under the documented setup. It does not create records for event categories that were never collected. Integrity and coverage remain different properties of the evidence. CloudTrail log-file integrity validation
The team retains the query and retrieval context alongside exported results. A file of matching events without its account, time range, filters, and retrieval date is difficult to interpret later. A second investigator should be able to understand what the search examined without reconstructing the first person's browser session.
Turn a negative finding into a precise sentence
After review, the investigator has found no unexplained administrative changes in the examined accounts and Regions during the defined window. That sentence is useful because it names the action class and scope. It is stronger than we checked the logs and weaker than no unauthorized activity occurred. Its precision is a benefit, not an admission that the investigation failed.
The object-access question remains partially unresolved because one resource lacked the relevant historical data-event collection. The memo says that available evidence cannot establish whether particular reads occurred through that route. It lists any independent application or service records that narrow the uncertainty without claiming they fill the entire gap.
The final findings separate observed misuse, explained legitimate activity, unresolved candidates, and unobservable questions. Counts belong within those categories. A total of zero confirmed misuse events should not conceal several unresolved candidates, just as a large count of failed authentication attempts should not automatically be described as successful access.
The team also states whether investigation changed containment. A newly discovered delegated role might require additional revocation. An explained export might remove one suspected case from the queue. Linking findings to actions lets the response evolve while preserving why each decision was made.
Test the investigation path without manufacturing an incident
Once immediate response permits, Alder rehearses a harmless, authorized reference operation in a nonproduction environment whose logging policy represents the relevant design. It knows the request time, identity, operation, and resource. The investigator then attempts to find that event through the documented search path.
Finding the reference event demonstrates something limited but valuable: under that test's collection, permissions, and query configuration, the path can retrieve the expected observation. It does not prove historical collection was identical or that every operation type is covered. The test report retains those boundaries rather than treating one visible event as certification of the whole logging estate.
Failure to find the reference event is actionable. The team checks whether it chose the wrong event class, looked in the wrong account, lacked permission, or expected a record the service does not produce. Repairing that path makes a future negative search more interpretable. It also prevents the organization from buying more log storage while leaving its query assumptions untested.
The exercise should avoid using a suspected compromised credential merely for convenience. An approved clean identity can validate the observation path without extending use of the old one. Where a credential-specific negative test is necessary, it requires a separate authorized purpose and safe handling. Investigative curiosity is not a reason to circulate a secret.
Finally, the team measures the time needed to answer one concrete question, such as whether a particular service identity created a resource. A dashboard containing many feeds can still be difficult to investigate. The rehearsal turns logging readiness into an observable capability: a person can find, interpret, and preserve the evidence needed for a defined decision.
The result set needs its own completeness check
Alder exports a search that returns exactly one thousand rows. The round number prompts a question: is that the total number of matches or the interface's display limit? A large result can appear convincing while omitting the events beyond the first page. The investigator checks the retrieval contract and retains any continuation information needed to obtain the rest.
The same issue applies to filters. A broad query can be narrowed accidentally by a remembered resource selection, a console's current project, or an inherited time setting. The report records the effective scope rather than only the text entered in the search box. Another authorized investigator can inspect that record and identify a missing account or an unintended exclusion.
For a large export, the team reconciles the number of retrieved records with the source's reported result count where that count has a documented meaning. It checks whether retries duplicated pages and whether interrupted downloads produced a partial file. A spreadsheet row count is not enough if the original export contained nested events or the import silently rejected malformed records.
Deduplication also needs care. Two records describing the same provider event can appear through different collection paths, while two legitimate distinct requests can share a timestamp and operation name. Alder uses a documented event identity where available and preserves provenance across copies. It does not delete every similar-looking row merely to make the timeline easier to read.
The investigator keeps raw data separate from annotations. A note marking an event explained should not alter the original operation field or timestamp. If a parser is corrected, the normalized view can be regenerated from the preserved source. This separation makes later disagreement productive: reviewers can challenge an interpretation without wondering whether the underlying evidence was edited to match it.
These checks do not require an elaborate forensic platform. They require an explicit answer to a basic question: did we retrieve and interpret the records we claim to have examined? Observation coverage can be adequate while retrieval is incomplete. Treating those as separate failure modes prevents a technical export defect from becoming an unsupported conclusion about the incident.
The memorandum remains open only where a question remains
Alder's report closes explained candidates, escalates supported misuse, and names the evidence gaps that remain. Each open question has a responsible owner and a plausible next source. The organization avoids keeping an incident permanently open through a list of generic possibilities that no available action can resolve.
Where uncertainty cannot be removed retrospectively, the owner makes a bounded risk decision and improves prospective collection. That decision does not rewrite the historical finding. Future logs can make the next investigation stronger while the current report continues to state what was and was not observable during this event.
The practical standard is a conclusion another investigator can assess. It identifies the authority examined, the questions asked, the records available, the events explained, and the limitations of silence. A quiet dashboard becomes useful evidence only after the organization understands which part of the world that dashboard could actually observe.