Method
What ASSAY measures, what it refuses to measure, and the three mistakes the engine has published.
What it measures
ASSAY takes a trade-level record and runs seven attacks against what that record claims about itself. The result is how much of the claim is still standing after scrutiny, published alongside the coverage it was measured under.
An attack that cannot look does not vote zero. It abstains and leaves the denominator. The difference between failed the test and could not be tested is the whole product.
What it refuses to measure
Future returns. Probability of success. Whether the idea is any good. Harvey, Liu and Zhu (2016) document that most published strategies do not survive implementation, and no engine reading a CSV can tell you which ones will. A full score is not investment advice.
Every threshold is measured
Every cut-off the engine applies is measured against the whole corpus: how many verdicts it moves, and whether it earns its place. Most move nothing. One of them is not justified and is recorded as a choice rather than a finding — stated, not smoothed over. The values themselves stay inside the engine: a published cut-off tells a dishonest seller exactly how far to move a record to clear it.
Three mistakes, with the commit that fixed each
A method that only shows its hits cannot be checked. These were found by the engine, against real data, and corrected in public.
| 63aedc1 | A false terminal verdict, already published in a report. | Found against real data, retracted, and the report that carried it withdrawn. The commit is the record. |
| e942303 | The configuration attack accused every honestly executed record. | It now abstains on records it cannot judge fairly, and says so in the report instead of scoring them. |
| 501696c | The tail was measured in currency. | Corrected to the unit the claim is actually made in. The earlier reading confused size with quality. |
What is not settled
The engine has never examined a profitable strategy. Everything in the corpus loses money, so its ability to clear a healthy record is asserted, not demonstrated. That line stays here until a clean external record proves otherwise.