2.3.0

Two releases since the last post. 2.1.0 introduced provenance — recording which source each value came from. 2.2.0 and 2.3.0 are about making that recording mean something: deciding which source wins when they disagree, and letting you watch the dataset improve day by day.

Sources now have to earn a value

Recording where a value came from is only half the problem. The other half is what happens when two sources say different things — which, for botanical data, is most of the time.

Until now the answer was: whichever import ran last. That is not an answer, it is an accident. Sources are now ranked, and several rules follow from it:

• A tie never overwrites. If two sources of equal standing disagree, the value already there stays, and the newcomer is recorded as a competing claim. Crawl order stops deciding anything.
• Older values are no longer defenceless. Values that predate provenance recording carry no source, so nothing spoke for them and anything could replace them. They now hold a rank of their own: only a genuinely stronger source may correct them.
• Implausible values are refused at the door. A plant ten kilometres tall, a pH of 47, a minimum temperature above the maximum. The attempt is kept on record rather than quietly dropped, so a source that keeps publishing nonsense is visible.
• Some fields have a closed vocabulary. nitrogen_fixation accepts None, Low, Medium or High and nothing else — a convention that lives in the contract rather than in someone's memory.

Taxonomy is arbitrated too

This is the change most likely to affect what you read.

Whether a name is accepted or doubtful, who authored it, when it was published — all of that was still on last-write-wins. So a status from Kew could be replaced by one from an aggregator whose taxonomy is assembled automatically, with nothing recorded.

That was backwards, and it is fixed. Nomenclature is arbitrated like everything else, and a disagreement about a taxon's status is now something you can read rather than something that silently resolved itself.

The scientific name itself is deliberately left out: renaming a species affects its URL, its grouping and how it is found in search, so it needs a deliberate decision rather than an automatic one.

The provenance trail says where, not just who

GET /api/v1/species/{id}/facts used to tell you a value came from POWO. It now tells you which POWO record:

{
  "attribute_name": "average_height_cm",
  "source": "powo",
  "source_record_id": "urn:lsid:ipni.org:names:857916-1",
  "value": "1500",
  "evidence_type": "reported"
}

That works for refused values too — the ones you most want to trace back.

Month and part fields also became readable. They used to come out as the raw numbers we store them with (bloom_months: "28"), which nobody outside our codebase could interpret. They now read "mar|apr|may".

Watching the dataset improve

Data quality is measured every day now, not every week, and the management dashboard shows the progression rather than only today's figure — including which fields gained or lost over the period. Regressions are shown next to progress, because a field that loses values is the thing worth noticing first.

We measure the share of measurable cells that are filled rather than an average completeness score. The average moves whenever we change which fields are relevant to which plants, and a number that shifts for reasons unrelated to the data cannot be compared across days.

Smaller fixes worth knowing

• Boolean filters no longer guess. filter[edible]=no used to be read as yes, because the value was coerced rather than checked. Unrecognised values now return a 400 listing what is accepted.
• Images with no identified part are grouped under an explicit unknown key instead of an empty one.
• Cultivation fields are expected only of cultivated species. Recording that a foraged thistle has edible roots used to make it look less complete, by unlocking sowing and spacing fields nobody would ever fill for a weed.

What we are working on

Filling the gaps, with sources that can be cited. Nomenclature is in good shape; botanical traits are largely empty, and we would rather say so than fill them with plausible-looking estimates. If a field matters to your work and is consistently blank, tell us — priorities are currently set by how often a species is looked up, and that is a poor proxy for what people actually need.