In topic modeling, many algorithms that guarantee identifiability of the topics have been developed under the premise that there exist anchor words - i.e., words that only appear (with positive probability) in one topic. Follow-up work has resorted to three or higher-order statistics of the data corpus to relax the anchor word assumption. Reliable estimates of higher-order statistics are hard to obtain, however, and the identification of topics under those models hinges on uncorrelatedness of the topics, which can be unrealistic. This paper revisits topic modeling based on second-order moments, and proposes an anchor-free topic mining framework. The proposed approach guarantees the identification of the topics under a much milder condition compared to the anchor-word assumption, thereby exhibiting much better robustness in practice. The associated algorithm only involves one eigen-decomposition and a few small linear programs. This makes it easy to implement and scale up to very large problem instances. Experiments using the TDT2 and Reuters-21578 corpus demonstrate that the proposed anchor-free approach exhibits very favorable performance (measured using coherence, similarity count, and clustering accuracy metrics) compared to the prior art.
|Original language||English (US)|
|Number of pages||9|
|Journal||Advances in Neural Information Processing Systems|
|State||Published - 2016|
|Event||30th Annual Conference on Neural Information Processing Systems, NIPS 2016 - Barcelona, Spain|
Duration: Dec 5 2016 → Dec 10 2016
Bibliographical noteFunding Information:
This work is supported in part by the National Science Foundation (NSF) under the project numbers NSF-ECCS 1608961 and NSF IIS-1247632 and in part by the Digital Technology Initiative (DTI) Seed Grant, University of Minnesota.