Topic Models

Supported Data Formats

Topic models can be applied to any dataset that has group structure.

Supported Learning Algorithms

  • FiniteTopicModel supports VB, soVB, moVB
  • HDPTopicModel supports VB, soVB, and moVB. * with birth/merge/delete moves for moVB

Possible Implementations

  • FiniteTopicModel: stuff here
  • HDPTopicModel: more stuff here

There are two types of mixture model supported. Both define the model in terms of a global parameter vector \(\beta\), where \(\beta_k\) gives the probability of topic k, and local assignments \(z\), where \(z_n\) indicates which state {1, 2, 3, ... K} is assigned to data item n.

The FiniteMixtureModel has a generative process:

\[\begin{split}[\beta_1, \beta_2, \ldots \beta_K] \sim \mbox{Dir}(\gamma, \gamma, \ldots \gamma) \\ z_n \sim \mbox{Discrete}(\beta)\end{split}\]

while the DPMixtureModel has generative process:

\[\begin{split}[\beta_1, \beta_2, \ldots \beta_K \ldots] \sim \mbox{StickBreaking}(\gamma_0) \\ z_n \sim \mbox{Discrete}(\beta)\end{split}\]

If we let K grow to infinity, these two models converge if \(\gamma = \gamma_0 /K\).