Evidence Verification and Robust Comparison in Investment Research

I. Feature attribution and the value of verification

SHAP splits a model prediction into additive feature contributions through its local accuracy property. Conditional Shapley methods also account for dependence among features, and Rashomon-set methods check how stable explanations are across near-optimal models (SHAP; Aas et al.; Marx et al.). None of them establishes whether the corporate inputs $x$ are factually correct, and in investment diligence that is an open question, because self-reported disclosures may contain AI-polished greenwashing. So an analyst also needs to know which corrections to the evidence could change a decision, and my framework is built to answer that question.

The first failure case I derive concerns verification priorities based on raw attribution magnitudes. Take a linear scoring model with a fixed marginal background distribution $Q$. The SHAP attribution for feature $j$ is $\phi_j = \beta_j(x_j - E_Q X_j)$. If the feature has unresolved factual uncertainty bounded by $|\delta_j| \le d_j$, its sensitivity to verification is $|\beta_j|d_j$, and over a box of admissible evidence errors the exact worst-case change in the company score is

The upper bound comes from the triangle inequality, and it is attained by picking the endpoint of each error interval so that every perturbation pushes the score in the same direction. Apart from the coefficient, that sum depends on how wide the errors are, whereas the attribution depends on where the feature sits relative to the background. Attribution magnitude and verification value therefore need separate calculations. Two cases show the difference:

• A financial metric may have a large SHAP attribution $|\phi_j|$ and come from audited filings that have already been verified ($d_j \approx 0$). Further checks of that field then do little to reduce score uncertainty.

• A self-reported ESG or operating metric may lie near the sector mean ($x_j \approx E_Q X_j \implies \phi_j \approx 0$) and so receive a near-zero SHAP attribution. Substantial unresolved reporting error ($d_j \gg 0$) may still be enough to reverse a company ranking.

The second failure case concerns conventional resampling: redrawing existing records cannot, by itself, identify a shared reporting or AI-marketing bias. Suppose $n$ reported observations satisfy $X_i = \mu + b + \varepsilon_i$, where $\varepsilon_i \sim \mathcal{N}(0, \sigma^2)$ are independent errors and $b \neq 0$ is an unmodeled bias common to all records, as with systematic overstatement across a sector or coordinated AI-assisted public relations. A conventional nominal 95% normal confidence interval centered on the sample mean $\bar{X}$ covers the true parameter $\mu$ with probability

where $z = \Phi^{-1}(0.975) \approx 1.96$. With a fixed nonzero common bias, a larger sample ($n \to \infty$) reduces random variance while the interval's coverage of the true parameter tends to zero. Bootstrapping the same biased records adds no information that identifies $b$. Bounding the common bias takes external evidence, and the ranking analysis then has to retain the resulting uncertainty.

II. Joint uncertainty over models and evidence

I represent model multiplicity and unresolved evidence jointly to assess whether an ESG or investment ranking stays stable under admissible changes to either.

On the model side, I fix the prediction target, input definitions and output scale, and let $\mathcal{F}_0$ contain $M$ candidate scoring models specified before examining the validation data. The validation set consists of $n$ i.i.d. observations from the target evaluation distribution, independent of model construction, and the loss is bounded in $[0, 1]$. For a failure probability $\alpha \in (0, 1)$, Hoeffding's inequality and a union bound give

Let $f^*$ minimize population loss within $\mathcal{F}_0$. I retain every model whose empirical validation loss is within $2t_n$ of the empirical minimum, which defines an empirical Rashomon set:

On the simultaneous concentration event, $\hat{L}_n(f^*) \le L(f^*) + t_n \le \min_g \hat{L}_n(g) + 2t_n$, so $f^* \in \mathcal{F}_\alpha$ and every retained model has population excess loss at most $4t_n$ (Marx et al.).

On the evidence side, $\mathcal{X}(E)$ represents the questions still unresolved under the current diligence state $E$: entity boundaries, reporting periods, physical units, measurement definitions, shared third-party sources and suspected greenwashing claims. Each $X \in \mathcal{X}(E)$ specifies a joint input configuration for all candidate companies at once, and that is how error dependence across companies and fields is preserved. Over the joint space $\Omega(E) = \mathcal{F}_\alpha \times \mathcal{X}(E)$, the exact worst-case bounds on the score difference between companies $i$ and $k$ are

If $\underline{\Delta}_{ik} > 0$, the ordering $i \succ k$ holds for every retained model in $\mathcal{F}_\alpha$ and every admissible resolution of the evidence in $\mathcal{X}(E)$. Turning that into a probability statement takes an assumption about the evidence set. Suppose calibration establishes that $\mathcal{X}(E)$ contains the true inputs with probability at least $1 - \beta$ at the specified certification stage. A union bound then gives joint inclusion of $f^*$ and the true inputs with probability at least $1 - \alpha - \beta$. The unconditional probability of issuing a certificate that is false for $f^*$ at the true inputs is therefore at most $\alpha + \beta$. The union bound needs no independence between model validation and evidence calibration.

That guarantee covers the score ordering of the best model in the prespecified class at the true inputs. The adequacy of that class and realized investment outcomes lie outside it. If certification is adaptive and repeated over several stages, the calibration has to be jointly valid across those stages.

III. Computing comparison bounds with support functions

I derive the joint comparison bounds with convex support functions, which make them tractable (Ben-Tal and Nemirovski). Let $u \in U$ collect the unresolved factual quantities, including common benchmark errors and company-specific allowances for reporting error, and write company inputs as $x_i(u) = \bar{x}_i + B_i u$. For an affine scoring model, the pairwise difference is $\Delta_f(u) = m_f + a_f^\top u$, where $a_f = (B_i - B_k)^\top \beta_f$, and the support function $h_U(a) = \sup_{u \in U} a^\top u$ gives exact comparison bounds:

For the ellipsoidal uncertainty set $U = \{Cz : \|z\|_2 \le r\}$, the Cauchy-Schwarz inequality gives $h_U(a_f) = r\|C^\top a_f\|_2$. For an $\ell_\infty$ box centered at zero, $h_U(a_f)$ reduces to a weighted sum of absolute values.

Keeping everything in one joint representation lets shared errors cancel exactly in a pairwise comparison. Let $S$ index common-source coordinates on which the two companies have identical loadings, so that $(B_i-B_k)_{:,S}=0$. Under the same linear scoring model,

The common-source contribution cancels, and company-specific uncertainty stays in the support-function bound. If every uncertain direction cancels, $a_f=0$ and $h_U(\pm a_f)=0$. Subtracting intervals constructed separately for companies $i$ and $k$ loses this, because it lets a shared source move in opposite directions at the same time. That combination is incompatible with the joint error structure, and it produces an unnecessarily wide difference bound.

For a smooth nonlinear model, set $m_f=\Delta_f(0)$ and $a_f=\nabla\Delta_f(0)$. If $\Delta_f(u)$ is twice continuously differentiable on $\|u\|_2 \le r$ and its Hessian has operator norm bounded by $K_f$, Taylor's theorem gives

Nonsmooth tree ensembles get their bounds from leaf-region partitioning or mixed-integer linear programming. The implementation spends computation first on company pairs whose difference bounds are close to the zero decision boundary.

IV. Selecting queries by the regret they can remove

When $\underline{\Delta}_{ik} \le 0 < \overline{\Delta}_{ik}$, the ranking is unresolved, since a factual correction or a resolution of ambiguous evidence could change the decision. The system then prioritizes the next verification query $q$, such as reviewing a supplier contract, requesting original utility bills, or checking a regulatory filing.

Let $A$ be a finite set of institutional actions, such as advancing company $i$ or company $k$ to final diligence, or deferring a rating. Let $U(a, \omega)$ be the prespecified utility of an action in scenario $\omega \in \Omega$, where $\Omega$ is nonempty and utilities are uniformly bounded. The baseline minimax regret over deterministic actions is

A query $q$ returns an observation $o \in O_q$ and restricts the compatible scenarios to a nonempty subset $\Omega_{q,o} \subseteq \Omega$. I score the query by its guaranteed reduction in worst-case decision regret:

Nonnegativity follows from $\Omega_{q,o} \subseteq \Omega$: restricting the scenario set cannot increase worst-case regret for any fixed action, and minimizing over $a \in A$ preserves that inequality (Robust Active Preference Elicitation). Calculating $V(q)$ uses the set of compatible scenarios and requires no prior probability that a disclosure is exaggerated. At a fixed evidence state, maximizing $V(q)$ over affordable queries minimizes worst-case regret after one query. With positive, unequal costs $c_q$, the ratio $V(q)/c_q$ is a one-step efficiency heuristic. Complementary or sequential queries require joint or adaptive optimization under the total diligence budget $B$.

V. Propagating factual corrections through scores and attributions

When diligence resolves a query and the verified evidence vector changes from $x$ to $x'$, the system follows versioned dependencies to recompute the affected scores, peer rankings and SHAP explanations. With the scoring function $f$, background reference $Q$ and rubric version held fixed, Shapley efficiency gives the exact identity (SHAP local accuracy):

Approximate Shapley outputs meet this only up to a residual. Suppose their efficiency residuals are $|\phi_0+\sum_j\phi_j-f(x)|\le\eta$ and $|\phi_0+\sum_j\phi'_j-f(x')|\le\eta'$ under the same baseline $\phi_0$. The triangle inequality then gives the recomputation check $|\sum_j \Delta \phi_j - \Delta f| \le \eta + \eta'$. A pass verifies aggregate conservation. The accuracy of individual attributions and the truth of the evidence are separate questions.

Every database update records (field version, evidence document ID, model version, background Q, rubric version). With these identifiers, a change caused by a factual correction can be told apart from one caused by a model update or a change to the background distribution.

VI. Reproducible quantitative comparisons

I tested the mechanisms in the analytical benchmarks below, under explicit model and parameter settings.

1. Confidence interval coverage under common reporting bias.

• Set the shared bias to $b = 0.2$ and independent noise to $\sigma = 1$. If the common bias $b$ is ignored, a nominal 95% interval has actual coverage of 48.40% at $n = 100$ records and 0.000637% at $n = 1,000$. Once targeted verification shows $|b| \le 0.2$, raising the half-width to $0.2 + 1.96/\sqrt{n}$ gives 97.50% actual coverage.

2. Cancellation of shared errors in pairwise rankings.

• Two companies have a nominal score difference of 0.6$0.6$ points and share an additive sector-benchmark error in $[-1, 1]$. Subtracting separately constructed company intervals gives $[-1.4, 2.6]$, which cannot certify the ordering, whereas the joint support-function calculation cancels the shared error and gives $[0.6, 0.6]$, which certifies the ranking.

3. Minimax-regret verification compared with SHAP magnitude.

• Consider a single query, with the same cost for each query option. The benchmark has 8 scenarios, which come from 2 candidate models in the Rashomon set ($s \in \{-1, 1\}$) and 4 unresolved evidence configurations ($z_1 \in \{-0.4, 0.4\}$, $z_2 \in \{-0.2, 0.2\}$). The pairwise score difference is $\Delta_s = 0.15s + z_1 + 0.3s z_2$. A further shared feature has a large SHAP attribution but is already verified ($d_0 = 0$). The action is to select company $i$ or company $k$, and regret is measured in model-score units.

Enumerating the eight scenarios shows that rechecking the verified field with a large SHAP attribution produces no regret reduction ($V(q) = 0$), while verifying $z_1$ takes worst-case regret from 0.61 to zero. So in this benchmark, verification value depends on how unresolved evidence can affect the decision. The Rashomon model set, support-function bounds and minimax-regret criterion make that relationship computable under the specified uncertainty model, and versioned data updates carry the result into the research workflow.