Emerging Narrative Detection

WHAT IT DOES. Extracts recurring word phrases from recent crypto headlines, requires at least two INDEPENDENT sources per phrase, and scores how much more each appears in the last 48 hours against a preceding baseline.

THE CLASSICAL FRAME IS BURST DETECTION. Kleinberg (2002) formalised the problem of identifying bursts of activity in document streams, modelling a stream as switching between states of higher and lower rate so that a burst is a statistically distinguishable regime rather than a busy patch. The essential discipline it encodes is the one that matters here: a rise must be judged against the series' OWN baseline variability, not against a fixed number, or the detector fires on ordinary fluctuation.

THE TWO SOURCES REQUIREMENT IS THE STRONGEST GUARD, and worth appreciating. News is heavily syndicated: one press release becomes twenty articles, and a phrase-frequency spike from a single origin measures ONE event reported many times. Requiring independent sources is what turns a repetition count into something closer to an evidence count - the same non-independence problem that runs through every synthesis analysis, solved here at the input rather than apologised for at the output.

THE SCORE is recent articles squared, divided by prior articles plus one, rounded to one decimal. Recent is the last 48 hours; prior is the earlier part of the loaded 9-day news window. These are raw counts, not time-normalised rates. At least 3 recent articles and 2 distinct publisher domains across the window are required. When prior is zero the score is the recent count squared; it adds no ordering beyond recent coverage when no phrase in a run has prior articles. Each phrase carries its own baseline. Distinct publisher domains are not proof of independent reporting.

WHAT PHRASE FREQUENCY CANNOT TELL YOU. It has no sentiment and no direction - a term surging because of a hack and one surging because of a launch look identical. It has no measure of importance, only of repetition. And journalism has its own dynamics: coverage begets coverage, so a phrase can burst because it became newsworthy TO JOURNALISTS rather than because anything changed in the market.

THE MULTIPLE-COMPARISONS PROBLEM APPLIES, as it does to any scan. Scanning many phrases for an unusual rise produces unusual rises by chance, so the count of detections is uninformative without knowing how many candidates were examined.

AND THE CENTRAL CLAIM DESERVES SCRUTINY. 'Before they go mainstream' asserts a lead over the market. A headline scan reads published journalism, which is the same public information everyone else has, and news frequently FOLLOWS the move it describes. Establishing a lead requires a time-ordered test against subsequent returns, which this does not perform.