Variational with birth and merge proposals for DP mixtures of Gaussians

How to train a DP mixture model.

We’ll show that despite diverse, poor quality initializations, our proposal moves that insert new clusters (birth) and remove redundant clusters (merge) can consistently recover the same ideal posterior with 8 clusters.

import bnpy
import numpy as np
import os

from matplotlib import pylab
import seaborn as sns
# sphinx_gallery_thumbnail_number = 2

FIG_SIZE = (3, 3)
pylab.rcParams['figure.figsize'] = FIG_SIZE

Read dataset from file.

dataset_path = os.path.join(bnpy.DATASET_PATH, 'AsteriskK8')
dataset = bnpy.data.XData.read_npz(
    os.path.join(dataset_path, 'x_dataset.npz'))

Make a simple plot of the raw data

pylab.plot(dataset.X[:, 0], dataset.X[:, 1], 'k.')
pylab.gca().set_xlim([-2, 2])
pylab.gca().set_ylim([-2, 2])
pylab.tight_layout()
../../_images/sphx_glr_plot-03-demo=vb+proposals-model=dp_mix+gauss_001.png

Setup: Function for visualization

Here’s a short function to show the learned clusters over time.

def show_clusters_over_time(
        task_output_path=None,
        query_laps=[0, 1, 2, 5, 10, None],
        nrows=2):
    ''' Read model snapshots from provided folder and make visualizations

    Post Condition
    --------------
    New matplotlib plot with some nice pictures.
    '''
    ncols = int(np.ceil(len(query_laps) // float(nrows)))
    fig_handle, ax_handle_list = pylab.subplots(
        figsize=(FIG_SIZE[0] * ncols, FIG_SIZE[1] * nrows),
        nrows=nrows, ncols=ncols, sharex=True, sharey=True)
    for plot_id, lap_val in enumerate(query_laps):
        cur_model, lap_val = bnpy.load_model_at_lap(task_output_path, lap_val)
        # Plot the current model
        cur_ax_handle = ax_handle_list.flatten()[plot_id]
        bnpy.viz.PlotComps.plotCompsFromHModel(
            cur_model, Data=dataset, ax_handle=cur_ax_handle)
        cur_ax_handle.set_xticks([-2, -1, 0, 1, 2])
        cur_ax_handle.set_yticks([-2, -1, 0, 1, 2])
        cur_ax_handle.set_xlabel("lap: %d" % lap_val)
    pylab.tight_layout()

Training from K=1 cluster

Using 1 initial cluster, with birth and merge proposal moves.

K1_trained_model, K1_info_dict = bnpy.run(
    dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
    output_path='/tmp/AsteriskK8/trymoves-K=1/',
    nLap=100, nTask=1, nBatch=1,
    sF=0.1, ECovMat='eye',
    K=1, initname='randexamples',
    moves='birth,merge,shuffle',
    m_startLap=5, b_startLap=2, b_Kfresh=4)

show_clusters_over_time(K1_info_dict['task_output_path'])
../../_images/sphx_glr_plot-03-demo=vb+proposals-model=dp_mix+gauss_002.png

Out:

Dataset Summary:
X Data
  total size: 5000 units
  batch size: 5000 units
  num. batches: 1
Allocation Model:  DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data  Model:  Gaussian with full covariance.
Obs. Data  Prior:  Gauss-Wishart on mean and covar of each cluster
  E[  mean[k] ] =
   [ 0.  0.]
  E[ covar[k] ] =
  [[ 0.1  0. ]
   [ 0.   0.1]]
Initialization:
  initname = randexamples
  K = 1 (number of clusters)
  seed = 1607680
  elapsed_time: 0.0 sec
Learn Alg: memoVB | task  1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=1/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
    1.000/100 after      0 sec. |    131.5 MiB | K    1 | loss  1.105578508e+00 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 4 states. 1/1 succeeded. 0/1 failed eval phase. 0/1 failed build phase.
    2.000/100 after      0 sec. |    131.6 MiB | K    5 | loss  7.390275336e-01 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 12 states. 3/4 succeeded. 1/4 failed eval phase. 0/4 failed build phase.
    3.000/100 after      1 sec. |    131.6 MiB | K   17 | loss  2.819339824e-02 |
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/12 succeeded. 10/12 failed eval phase. 2/12 failed build phase.
    4.000/100 after      2 sec. |    131.6 MiB | K   17 | loss -1.240759022e-02 | Ndiff   80.474
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 17 too busy.
MERGE @ lap 5.00 : 4/30 accepted. Ndiff 98.67. 10 skipped.
    5.000/100 after      2 sec. |    131.6 MiB | K   13 | loss -2.742336056e-02 | Ndiff   80.474
BIRTH @ lap 6.000 : None attempted. 0 past failures. 0 too small. 13 too busy.
MERGE @ lap 6.00 : 3/23 accepted. Ndiff 767.62. 5 skipped.
    6.000/100 after      3 sec. |    131.6 MiB | K   10 | loss -4.626686932e-02 | Ndiff   80.474
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 10 too busy.
MERGE @ lap 7.00 : 0/18 accepted. Ndiff 0.00. 0 skipped.
    7.000/100 after      3 sec. |    131.6 MiB | K   10 | loss -4.686532682e-02 | Ndiff    2.930
BIRTH @ lap 8.00 : Added 0 states. 0/2 succeeded. 2/2 failed eval phase. 0/2 failed build phase.
MERGE @ lap 8.00 : 2/2 accepted. Ndiff 0.09. 5 skipped.
    8.000/100 after      3 sec. |    131.6 MiB | K    8 | loss -4.769229049e-02 | Ndiff    2.930
BIRTH @ lap 9.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 9.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
    9.000/100 after      3 sec. |    131.6 MiB | K    8 | loss -4.787962424e-02 | Ndiff    0.091
BIRTH @ lap 10.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 10.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
   10.000/100 after      4 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 11.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 11.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
   11.000/100 after      4 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 12.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 12.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
   12.000/100 after      4 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
MERGE @ lap 13.00: No promising candidates, so no attempts.
BIRTH @ lap 13.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
   13.000/100 after      4 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 14.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 14.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   14.000/100 after      4 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 15.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 15.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
   15.000/100 after      4 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 16.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 16.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
   16.000/100 after      5 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 17.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 17.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
   17.000/100 after      5 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
MERGE @ lap 18.00: No promising candidates, so no attempts.
BIRTH @ lap 18.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
   18.000/100 after      5 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 19.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 19.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   19.000/100 after      5 sec. |    131.6 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
... done. converged.
SKIPPED 1 comps with size below 0.00
SKIPPED 2 comps with size below 0.00

Training from K=4 cluster

Now using 4 initial clusters, with birth and merge proposal moves.

K4_trained_model, K4_info_dict = bnpy.run(
    dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
    output_path='/tmp/AsteriskK8/trymoves-K=4/',
    nLap=100, nTask=1, nBatch=1,
    sF=0.1, ECovMat='eye',
    K=4, initname='randexamples',
    moves='birth,merge,shuffle',
    m_startLap=5, b_startLap=2, b_Kfresh=4)

show_clusters_over_time(K4_info_dict['task_output_path'])
../../_images/sphx_glr_plot-03-demo=vb+proposals-model=dp_mix+gauss_003.png

Out:

Dataset Summary:
X Data
  total size: 5000 units
  batch size: 5000 units
  num. batches: 1
Allocation Model:  DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data  Model:  Gaussian with full covariance.
Obs. Data  Prior:  Gauss-Wishart on mean and covar of each cluster
  E[  mean[k] ] =
   [ 0.  0.]
  E[ covar[k] ] =
  [[ 0.1  0. ]
   [ 0.   0.1]]
Initialization:
  initname = randexamples
  K = 4 (number of clusters)
  seed = 1607680
  elapsed_time: 0.0 sec
Learn Alg: memoVB | task  1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=4/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
    1.000/100 after      0 sec. |    130.2 MiB | K    4 | loss  9.001549009e-01 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 8 states. 2/4 succeeded. 2/4 failed eval phase. 0/4 failed build phase.
    2.000/100 after      0 sec. |    130.2 MiB | K   12 | loss  6.597743621e-03 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 0 states. 0/9 succeeded. 9/9 failed eval phase. 0/9 failed build phase.
    3.000/100 after      1 sec. |    130.2 MiB | K   12 | loss -2.949554282e-02 | Ndiff   83.491
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/3 succeeded. 3/3 failed eval phase. 0/3 failed build phase.
    4.000/100 after      2 sec. |    130.2 MiB | K   12 | loss -3.255430235e-02 | Ndiff   18.789
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 12 too busy.
MERGE @ lap 5.00 : 2/25 accepted. Ndiff 0.00. 5 skipped.
    5.000/100 after      2 sec. |    130.2 MiB | K   10 | loss -3.311213517e-02 | Ndiff   18.789
BIRTH @ lap 6.000 : None attempted. 1 past failures. 0 too small. 9 too busy.
MERGE @ lap 6.00 : 2/17 accepted. Ndiff 417.46. 3 skipped.
    6.000/100 after      2 sec. |    130.2 MiB | K    8 | loss -4.787665915e-02 | Ndiff   18.789
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 7.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
    7.000/100 after      3 sec. |    130.2 MiB | K    8 | loss -4.787962431e-02 | Ndiff    0.020
BIRTH @ lap 8.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 8.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
    8.000/100 after      3 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
MERGE @ lap 9.00: No promising candidates, so no attempts.
BIRTH @ lap 9.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
    9.000/100 after      3 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 10.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 10.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   10.000/100 after      3 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 11.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 11.00 : 0/8 accepted. Ndiff 0.00. 0 skipped.
   11.000/100 after      3 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 12.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 12.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   12.000/100 after      4 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 13.000 : None attempted. 5 past failures. 0 too small. 3 too busy.
MERGE @ lap 13.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
   13.000/100 after      4 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
MERGE @ lap 14.00: No promising candidates, so no attempts.
BIRTH @ lap 14.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
   14.000/100 after      4 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 15.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 15.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   15.000/100 after      4 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 16.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 16.00 : 0/8 accepted. Ndiff 0.00. 0 skipped.
   16.000/100 after      4 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 17.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 17.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   17.000/100 after      4 sec. |    130.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
... done. converged.
SKIPPED 2 comps with size below 0.00

Training from K=8 cluster

Now using 8 initial clusters

K8_trained_model, K8_info_dict = bnpy.run(
    dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
    output_path='/tmp/AsteriskK8/trymoves-K=8/',
    nLap=100, nTask=1, nBatch=1,
    sF=0.1, ECovMat='eye',
    K=8, initname='randexamples',
    moves='birth,merge,shuffle',
    m_startLap=5, b_startLap=2, b_Kfresh=4)

show_clusters_over_time(K8_info_dict['task_output_path'])
../../_images/sphx_glr_plot-03-demo=vb+proposals-model=dp_mix+gauss_004.png

Out:

Dataset Summary:
X Data
  total size: 5000 units
  batch size: 5000 units
  num. batches: 1
Allocation Model:  DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data  Model:  Gaussian with full covariance.
Obs. Data  Prior:  Gauss-Wishart on mean and covar of each cluster
  E[  mean[k] ] =
   [ 0.  0.]
  E[ covar[k] ] =
  [[ 0.1  0. ]
   [ 0.   0.1]]
Initialization:
  initname = randexamples
  K = 8 (number of clusters)
  seed = 1607680
  elapsed_time: 0.0 sec
Learn Alg: memoVB | task  1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=8/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
    1.000/100 after      0 sec. |    132.3 MiB | K    8 | loss  6.479391366e-01 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 12 states. 3/8 succeeded. 5/8 failed eval phase. 0/8 failed build phase.
    2.000/100 after      1 sec. |    132.4 MiB | K   20 | loss  7.345925210e-02 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 0 states. 0/13 succeeded. 8/13 failed eval phase. 5/13 failed build phase.
    3.000/100 after      2 sec. |    132.4 MiB | K   20 | loss  1.384405895e-02 | Ndiff   78.701
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/12 succeeded. 8/12 failed eval phase. 4/12 failed build phase.
    4.000/100 after      3 sec. |    132.4 MiB | K   20 | loss  1.057016337e-02 | Ndiff   14.549
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 20 too busy.
MERGE @ lap 5.00 : 4/41 accepted. Ndiff 635.65. 6 skipped.
    5.000/100 after      4 sec. |    132.4 MiB | K   16 | loss -3.619883330e-03 | Ndiff   14.549
BIRTH @ lap 6.000 : None attempted. 0 past failures. 0 too small. 16 too busy.
MERGE @ lap 6.00 : 4/26 accepted. Ndiff 459.69. 10 skipped.
    6.000/100 after      5 sec. |    132.4 MiB | K   12 | loss -2.198388303e-02 | Ndiff   14.549
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 12 too busy.
MERGE @ lap 7.00 : 3/21 accepted. Ndiff 366.43. 7 skipped.
    7.000/100 after      5 sec. |    132.4 MiB | K    9 | loss -4.222694604e-02 | Ndiff   14.549
BIRTH @ lap 8.000 : None attempted. 1 past failures. 0 too small. 8 too busy.
MERGE @ lap 8.00 : 1/16 accepted. Ndiff 75.14. 1 skipped.
    8.000/100 after      5 sec. |    132.4 MiB | K    8 | loss -4.787951351e-02 | Ndiff   14.549
BIRTH @ lap 9.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 9.00 : 0/7 accepted. Ndiff 0.00. 0 skipped.
    9.000/100 after      6 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.002
BIRTH @ lap 10.00 : Added 0 states. 0/4 succeeded. 4/4 failed eval phase. 0/4 failed build phase.
MERGE @ lap 10.00 : 0/3 accepted. Ndiff 0.00. 0 skipped.
   10.000/100 after      6 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 11.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 11.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
   11.000/100 after      6 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 12.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 12.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
   12.000/100 after      6 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 13.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 13.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
   13.000/100 after      6 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 14.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 14.00 : 0/7 accepted. Ndiff 0.00. 0 skipped.
   14.000/100 after      7 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 15.000 : None attempted. 5 past failures. 0 too small. 3 too busy.
MERGE @ lap 15.00 : 0/3 accepted. Ndiff 0.00. 0 skipped.
   15.000/100 after      7 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 16.000 : None attempted. 5 past failures. 0 too small. 3 too busy.
MERGE @ lap 16.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
   16.000/100 after      7 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 17.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 17.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
   17.000/100 after      7 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 18.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 18.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
   18.000/100 after      7 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 19.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 19.00 : 0/7 accepted. Ndiff 0.00. 0 skipped.
   19.000/100 after      7 sec. |    132.4 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
... done. converged.
SKIPPED 3 comps with size below 0.00
SKIPPED 1 comps with size below 0.00

Training from K=25 cluster

Now using 25 initial clusters

K25_trained_model, K25_info_dict = bnpy.run(
    dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
    output_path='/tmp/AsteriskK8/trymoves-K=25/',
    nLap=100, nTask=1, nBatch=1,
    sF=0.1, ECovMat='eye',
    K=25, initname='randexamples',
    moves='birth,merge,shuffle',
    m_startLap=5, b_startLap=2, b_Kfresh=4)

show_clusters_over_time(K25_info_dict['task_output_path'])
../../_images/sphx_glr_plot-03-demo=vb+proposals-model=dp_mix+gauss_005.png

Out:

Dataset Summary:
X Data
  total size: 5000 units
  batch size: 5000 units
  num. batches: 1
Allocation Model:  DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data  Model:  Gaussian with full covariance.
Obs. Data  Prior:  Gauss-Wishart on mean and covar of each cluster
  E[  mean[k] ] =
   [ 0.  0.]
  E[ covar[k] ] =
  [[ 0.1  0. ]
   [ 0.   0.1]]
Initialization:
  initname = randexamples
  K = 25 (number of clusters)
  seed = 1607680
  elapsed_time: 0.0 sec
Learn Alg: memoVB | task  1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=25/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
    1.000/100 after      0 sec. |    132.2 MiB | K   25 | loss  4.392317798e-01 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 14 states. 4/18 succeeded. 11/18 failed eval phase. 3/18 failed build phase.
    2.000/100 after      2 sec. |    132.2 MiB | K   39 | loss  1.219944497e-01 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 0 states. 0/15 succeeded. 3/15 failed eval phase. 12/15 failed build phase.
    3.000/100 after      3 sec. |    135.2 MiB | K   39 | loss  8.910722771e-02 | Ndiff   56.472
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/15 succeeded. 9/15 failed eval phase. 6/15 failed build phase.
    4.000/100 after      5 sec. |    135.2 MiB | K   39 | loss  8.017398425e-02 | Ndiff   34.554
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 39 too busy.
MERGE @ lap 5.00 : 8/63 accepted. Ndiff 338.97. 32 skipped.
    5.000/100 after      7 sec. |    135.2 MiB | K   31 | loss  5.747324378e-02 | Ndiff   34.554
BIRTH @ lap 6.000 : None attempted. 0 past failures. 0 too small. 31 too busy.
MERGE @ lap 6.00 : 10/39 accepted. Ndiff 1032.92. 38 skipped.
    6.000/100 after      8 sec. |    135.2 MiB | K   21 | loss  1.309182046e-02 | Ndiff   34.554
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 21 too busy.
MERGE @ lap 7.00 : 5/35 accepted. Ndiff 323.06. 14 skipped.
    7.000/100 after      9 sec. |    135.2 MiB | K   16 | loss -8.888342290e-03 | Ndiff   34.554
BIRTH @ lap 8.000 : None attempted. 0 past failures. 0 too small. 16 too busy.
MERGE @ lap 8.00 : 4/24 accepted. Ndiff 443.50. 12 skipped.
    8.000/100 after      9 sec. |    135.2 MiB | K   12 | loss -3.261743488e-02 | Ndiff   34.554
BIRTH @ lap 9.000 : None attempted. 0 past failures. 0 too small. 12 too busy.
MERGE @ lap 9.00 : 3/22 accepted. Ndiff 185.39. 7 skipped.
    9.000/100 after     10 sec. |    135.2 MiB | K    9 | loss -4.523238984e-02 | Ndiff   34.554
BIRTH @ lap 10.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 10.00 : 1/13 accepted. Ndiff 1.72. 1 skipped.
   10.000/100 after     10 sec. |    135.2 MiB | K    8 | loss -4.770137591e-02 | Ndiff   34.554
BIRTH @ lap 11.00 : Added 0 states. 0/3 succeeded. 3/3 failed eval phase. 0/3 failed build phase.
MERGE @ lap 11.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
   11.000/100 after     10 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.015
MERGE @ lap 12.00: No promising candidates, so no attempts.
BIRTH @ lap 12.00 : Added 0 states. 0/3 succeeded. 3/3 failed eval phase. 0/3 failed build phase.
   12.000/100 after     11 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 13.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 13.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
   13.000/100 after     11 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 14.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 14.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
   14.000/100 after     11 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 15.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 15.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   15.000/100 after     11 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 16.000 : None attempted. 4 past failures. 0 too small. 4 too busy.
MERGE @ lap 16.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
   16.000/100 after     11 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
MERGE @ lap 17.00: No promising candidates, so no attempts.
BIRTH @ lap 17.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
   17.000/100 after     11 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 18.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 18.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
   18.000/100 after     11 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 19.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 19.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
   19.000/100 after     12 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 20.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 20.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
   20.000/100 after     12 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
BIRTH @ lap 21.000 : None attempted. 4 past failures. 0 too small. 4 too busy.
MERGE @ lap 21.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
   21.000/100 after     12 sec. |    135.2 MiB | K    8 | loss -4.787962433e-02 | Ndiff    0.000
... done. converged.
SKIPPED 4 comps with size below 0.00
SKIPPED 3 comps with size below 0.00

Total running time of the script: ( 0 minutes 32.509 seconds)

Generated by Sphinx-Gallery