返回文章列表
Capture RecaptureCoverageRecord Linkage
🔬

How Many Organizations Are Missing From Both Directories?

Two directories become a worked example in population-coverage estimation, matching errors and dependence between sources.

iBuidl Research2026-11-1817 min 阅读

TL;DR: Two lists reveal how many known organizations they share. Estimating the unseen population requires assumptions about consistent identity, population boundaries, coverage and dependence. Treat the result as a model-based estimate, test plausible linkage and overlap changes, and give readers the observed union separately.

On the sorting desk sit two directories of workshop associations. One comes from a municipal grant program and another from a newsletter survey. Each covers many organizations the other seems to have missed. The funding director wants a number for how many groups neither directory reached. That is an estimation problem about an unseen cell, not a harder version of adding names.

The directories, organizations and totals in this article are invented. A standard two-sample capture-recapture idea estimates population from overlap under assumptions about the samples. A USGS field report uses the Chapman modification of the Lincoln-Petersen estimator in ecological sampling. William Link's published work explains difficulties in identifying population size when detection probabilities differ. The same logical issues can appear in directory coverage, with institutional rather than ecological causes. USGS application of the Chapman estimator, Link on heterogeneity and population size.

The clerk puts a card in each of three drawers

Define the target as independent workshop associations that were active in the same locality on one reference date. The first directory contains 120 distinct qualifying organizations. The second contains 100. After adjudicated entity matching the team counts 40 present in both. These counts already presume a resolved unit of analysis. Contact entries and organization aliases have been reconciled under the example's rules.

The desk now sees 80 first-list-only organizations, 60 second-list-only organizations and 40 in both. The observed union is 180. A fourth drawer represents groups absent from both lists and has no cards the analysts can count. Reporting 180 known groups is a direct result of the resolved lists. Estimating the fourth drawer requires a model relating capture to underlying population membership.

Recall the visit to a small grant desk. Say about one third of all groups appear in the second list, and the first list covers organizations with the same probability of appearing in the second as the whole target population. About forty of the first list's 120 also appear in the second. The sample overlap estimates that one-third share. A list of 100 covering about one third suggests 300 groups in total. Subtracting the observed 180 leaves 120 under the illustrative simple independence model.

The calculation can also be expressed by multiplying 120 and 100 then dividing by 40. A basic two-list formula rests on assumptions such as appropriate sampling, stable membership and the relevant independence or equal-capture structure. The Chapman adjustment uses increments in the sample counts to address small-sample behavior in its model. Multiplying 121 by 101, dividing by 41 and subtracting one gives about 297.1. Estimated unseen groups under that calculation are about 117.1. These are model results and rounding them to a whole-organization count does not create new observed groups.

Observe how a link carries weight on both sides. If five questionable cross-list links are rejected while within-list totals stay unchanged, overlap becomes 35. The observed union grows to 185. The raw simple estimator rises to about 342.9. Its estimate of unseen groups becomes about 157.9. If five new links instead increase overlap to 45, the observed union falls to 175 and the raw estimate falls to about 266.7. Estimated unseen groups then number about 91.7 under the same structure.

This is a decision about evidence for matching, rather than a justification for choosing the desired result. Near an agency's coverage target, resolving a handful of difficult records could change a public conclusion. The report can provide a sensitivity around plausible linkage outcomes and retain the underlying disputed cases. A confidence interval calculated as if those overlaps were known with certainty would leave an important uncertainty out.

Application paperwork has a different identity than an entry in a newsletter

One association applies for a grant with its legal name and sits in the newsletter under an abbreviated operating name. A second lists a regional chapter and the same general email address as its parent group. The target definition determines whether parent and chapter count separately, with evidence of independence or shared ownership as specified. A person cannot infer the unit from similarity of spellings alone.

The matching team starts inside each list. Several rows can describe one association after office relocation or officer change. A separate record can identify which raw rows were joined and why. Cross-list overlap is calculated only once the source's relevant units are resolved. Counting several contacts for one association as several organizations would change the sample size and model denominator.

False links and missed links affect the worksheet differently. An erroneous link can move records that should occupy separate single-list drawers into one both-lists card. That raises apparent agreement and tends to lower estimated unseen population in the simple model. A missed link places two records for one group in the two single-list drawers. That decreases overlap and tends to inflate the estimated population. Review evidence can focus on uncertain groups without presuming which direction produces a more flattering coverage figure.

Document how missing contact attributes are handled. A small voluntary organization may lack a public address or registered label. Such groups could be harder to link and less likely to appear in directories at all. Ignoring them because their records are difficult changes both the model and the target population. The report can give an explicit limited-scope subtotal for groups that satisfy verification criteria, while preserving a clear description of the evidence gap elsewhere.

A reviewer can independently read a bounded sample of matched and unmatched candidates, including likely difficult regions. Sampling only cases a matching rule already judged strongly may fail to detect systematic misses outside those candidates. The goal is to understand how the data generation and linkage decisions relate to the chosen target. Supporting information should be used within the institution's legitimate purpose and scope, with records of why a link was accepted.

Time becomes part of entity adjudication. A group can dissolve, open, merge or move between source dates. One file may show the pre-merger association and the other may show a successor. The team asks which entity existed at the common reference date. The final dataset preserves lifecycle evidence needed for this choice. A broad rule that treats every successor as identical can collapse legitimate membership changes that matter to the estimate.

A shared grant invitation reaches the third drawer twice

An administrator later discovers that many of the newsletter invitations were sent to organizations on the grant list. This creates a straightforward dependence. Membership in one list directly made capture in the other more likely. A large observed overlap can then occur even when both lists omit many smaller groups that never hear about either collection.

Put the four drawers into a simple table. The visible cell counts are 40 both, 80 first only, and 60 second only. If the unseen population is 120, multiplying the both cell and the unseen cell gives 4,800, which also equals 80 times 60. This is the ordinary independence relationship in the table. With 220 unseen groups, the same three visible cells instead imply stronger positive association between appearing in the two lists. Sixty unseen gives a different negative association. The observed cells are compatible with these different underlying situations.

A positive association tends to produce more overlap than the simple independence model expects for a given population under comparable margins and can cause that model to miss unseen groups. A negatively related mailing design can do the reverse. Imagine one outreach office purposely contacts associations absent from a known grant list. It generates a small overlap by design. Using that small overlap as evidence of a huge unseen population would misread the process that created it.

Even teams that share no mailing lists can face dependence arising from visibility. Large institutions maintain public sites and stable records. Small associations may appear mostly in informal local channels, making them less likely to be detected by both sources. That variation can generate positive association in the aggregated table through group differences. It also motivates reporting visibility and eligibility information before giving a simple point estimate.

Suppose the director obtains reliably comparable size classifications and separates the illustrative counts. Large groups contribute 60 in the first list and 70 in the second with overlap 35. The raw estimate in that stratum is 120. Small groups contribute 60 first and 30 second with overlap five. Their raw estimate is 360. Together the stratified estimates total 480, versus 300 from pooling the counts. The observed union across strata stays 180 under the assumed linkage.

The aim of these invented margins is to show how modeling heterogeneity changes the arithmetic. The small group estimate with five overlaps is especially uncertain. Within-stratum independence and complete assignment remain substantive assumptions. If the invisible groups lack any usable stratum membership evidence, the categories themselves can be model dependent. Extra subdivisions can also create groups with little or zero measured overlap, leaving each estimate unstable. Balance representation of real coverage differences against the information available.

Count a period's population before fitting the model

The funding director first asks about active groups in October. The grant list catalogs every application accepted since January, while the newsletter contains groups responding in November. Some names belong to a new group and some to recently inactive groups. The target is neither database as delivered. Specify the geographic and activity criteria, then decide how to determine status as of the shared date using reliable evidence.

A meaningful claim about a closed population requires that membership be sufficiently stable over the sampling frame or that population dynamics be handled with a suitable design. For organizations, eligibility can change without a name change. The relevant date might be current operations or program eligibility over a defined period. One classification cannot be inferred from the other based on convenience. Explain how a group with seasonal activity would be treated.

The two sources also have different opportunities to detect a group. Assume a grant register accepts only organizations with a registered fiscal sponsor. The target now includes informal associations that cannot appear on that list. The ordinary model's coverage assumptions are strained. Further investigation could narrow the estimand to eligible groups, add a genuinely different source with relevant coverage, or report a partial estimate and separately note that hidden group type. Which choice fits depends on the intended policy.

A group outside both collection mechanisms is especially important. Link's work on detection heterogeneity gives formal reasons population estimates can be strongly dependent on assumptions. A practically unreachable set cannot be counted just by manipulating existing overlap. A separate design could obtain evidence about that population, such as targeted field collection or a broader frame. Analysts should make a limitation visible when their evidence gives no empirical leverage over it.

Another issue is entries acquired indirectly. A regional office aggregates several neighborhood submissions into a central file while the newsletter counts each named activity. A target based on independent organizations needs both datasets translated to that level. If a source's internal grouping masks individual members, the source is not merely incomplete in a one-row-per-entity sense. Its observation mechanism differs. The analyst can quantify what is visible and define an alternate sampling route.

Read uncertainty in the cell that drives the estimate

Sampling variation matters even assuming simple random samples and compatible identification. In a designed capture-recapture experiment an interval can reflect uncertainty about how many shared entities would appear under repeated sampling. A directory pair chosen through administrative opportunity may not behave like that designed experiment. A narrow interval premised on random independent lists gives little reassurance about the most serious assumptions in the directory application.

An estimate with no observed overlap exposes the sensitivity. The raw division gives no finite number. Some adjusted formulas may produce a number algebraically, while the overlap evidence supplies very weak information about the unseen size. Analysts can report the scope of what is observed, why linkage and coverage may be limited, and the next evidence needed. A result chart should not imply that an arbitrary finite upper value has become observed fact.

One extra list can help only when analysts understand its capture process. If a third directory copies the second, the extra column contributes little independent information about unseen groups. More sources with credible varied coverage can support models of overlap patterns. The choice of allowed dependencies and detectability still matters. Document which relationships have a practical cause. A fitted statistical explanation cannot compensate for records that the population could never have generated.

A new designed survey can answer specific unknowns. For example, pick locations from a complete eligible geographic frame and inventory workshops there through a documented method. The evidence can estimate coverage among groups reachable by that approach or inform assumptions about unserved neighborhoods. It supplies independent questions that overlap alone could not settle. Sampling effort can be aimed at the least understood parts of the population.

If a decision depends on a coverage threshold, show how uncertain inputs affect it. The observed union of 180 is 60 percent of the raw 300 in the pooled illustration. It is 37.5 percent of the 480 in the simple stratified example. Both percentages have the same observable numerator. A reader who sees a decisive percentage should know which model and population definition supplied its denominator.

Price the value of one new directory before buying it

One way to communicate structural sensitivity keeps the three visible cells steady and varies only the assumed strength of the association. In the illustrated table, the odds ratio for the two list-membership indicators uses the product of 40 and the unseen cell divided by the product of 80 and 60. Setting the odds ratio to one implies 120 unseen. Setting it to one half implies sixty. Setting it to two implies 240. The hypothetical total over that range runs from 240 to 420.

The range is a consequence of specified alternatives, not an estimate of which assumptions are likely. Real guidance for a dependence range can come from collection practices, a study with controlled independent sampling, or externally credible bounds on use of the same contact lists. Without that guidance, display scenarios as a description of evidence needed. A user comparing uncertainty intervals should know that sampling and dependence are separate questions.

A design team can sometimes obtain a partial view directly. Suppose six of sixty reasonably comparable districts are randomly selected, and an independent canvass within those districts finds fifty active associations in the defined period. The fifty observations under that fictional even district selection correspond to a design-based estimate of 500 under full and correct canvassing within selected districts, since each district had one tenth selection probability. Precision would depend strongly on the variability across districts and whether the implementation actually reached the target. Equal probability of selecting districts does not imply equal precision in their observed totals.

If twenty of those fifty canvassed groups were absent from both directories, the canvass can also inform the combined list coverage under its scope. The thirty observed directory members represent sixty percent of those fifty cases in the realized sample. Scaling them as estimated totals using the assumed design yields 300 listed and 200 absent in that particular exercise. These are fresh sample-based quantities from a different fictional scenario. Comparing them with the original pair would require compatible targets and verification of why sampled totals differed.

Independent canvassing can still miss quiet organizations and inherit location biases. It may depend on public venues and discover only groups meeting there. The team can add legitimate local records, use multiple reasonable contact paths, and retain unsettled eligibility where a group cannot be confirmed. A deputy director's wish to treat a canvass as ground truth should yield to the recorded uncertainty about its own observation mechanism. An additional source strengthens the study when it reaches a new part of the population, not merely when a third spreadsheet arrives.

Coverage reporting should distinguish entities wholly absent from both sources and poorly described entities already counted. Spending all field hours following existing linked organizations can improve those records and leave the total's biggest uncertainty untouched. A split in the next budget can reserve enough time for new districts or other independently selected locations. Sampling creates a contrast a later report can measure, including the chance of finding groups that share neither directory's usual participation route.

The field team also reports dates and activity criteria against the program's intended purpose. A venue organizing a single fair may satisfy an annual participation measure and be inactive by October. A group relaunching under a new name might be the same population unit under another rule. A short review of ambiguous cases before collection creates consistent labels across sources, so discovered differences in coverage are traceable to how records entered the sample.

Each new list therefore receives a short source card. Name the collection owner, inclusion process, effective date, relation to the other sources, and methods for resolving responsible entity scope. Retain only relationship details appropriate to the coverage purpose. The evidence gap can be expressed as a concrete question about which groups one route reaches. That is a better purchasing criterion than the advertised number of rows in a prospective database.

Prepare the note the director can actually sign

The report names the target group and date, the sources and construction rules, within-list deduplication and cross-list linkage. It provides the observed cells. Then it describes the overlap estimate and conditional assumptions, plus the results of notable sensitivity checks. A reader who wants to improve coverage tomorrow learns which visible or predicted group is most relevant rather than being handed a single mathematically exact total.

Treat a changed process as evidence when comparing years. If the newsletter borrowed the grant mailing list in the second year, overlap can rise sharply even with similar total populations. A trend in estimated unseen abundance may reflect a change in dependence. First compare collection methods, entity definitions and effective dates. Then assess whether a time-series comparison needs a revised design.

Document separate limits around the total and the named membership. A statistical estimate that another hundred groups may exist gives a rough size under a model. It cannot specify which individual groups are absent. The program can use independent outreach to discover names while retaining appropriate handling of information. A department that needs invitations issued tomorrow needs a different output from a department allocating funding to unobserved demand.

The desk's most reliable artifact may be the resolved view of its three visible drawers. The fourth is an explicitly modeled estimate. Someone who disagrees about linkage, eligibility or list dependence can see exactly which parts of the result would move. In the next round, the director can put resources into collecting evidence about those parts rather than buying another directory with the same coverage mechanism.

Sources

更多文章