Note
Click here to download the full example code
Variational with birth and merge proposals for DP mixtures of Gaussians¶
How to train a DP mixture model.
We’ll show that despite diverse, poor quality initializations, our proposal moves that insert new clusters (birth) and remove redundant clusters (merge) can consistently recover the same ideal posterior with 8 clusters.
# SPECIFY WHICH PLOT CREATED BY THIS SCRIPT IS THE THUMBNAIL IMAGE
# sphinx_gallery_thumbnail_number = 2
import bnpy
import numpy as np
import os
from matplotlib import pylab
import seaborn as sns
FIG_SIZE = (3, 3)
pylab.rcParams['figure.figsize'] = FIG_SIZE
Read dataset from file.
dataset_path = os.path.join(bnpy.DATASET_PATH, 'AsteriskK8')
dataset = bnpy.data.XData.read_npz(
os.path.join(dataset_path, 'x_dataset.npz'))
Make a simple plot of the raw data
pylab.plot(dataset.X[:, 0], dataset.X[:, 1], 'k.')
pylab.gca().set_xlim([-2, 2])
pylab.gca().set_ylim([-2, 2])
pylab.tight_layout()
Setup: Function for visualization¶
Here’s a short function to show the learned clusters over time.
def show_clusters_over_time(
task_output_path=None,
query_laps=[0, 1, 2, 5, 10, None],
nrows=2):
''' Read model snapshots from provided folder and make visualizations
Post Condition
--------------
New matplotlib plot with some nice pictures.
'''
ncols = int(np.ceil(len(query_laps) // float(nrows)))
fig_handle, ax_handle_list = pylab.subplots(
figsize=(FIG_SIZE[0] * ncols, FIG_SIZE[1] * nrows),
nrows=nrows, ncols=ncols, sharex=True, sharey=True)
for plot_id, lap_val in enumerate(query_laps):
cur_model, lap_val = bnpy.load_model_at_lap(task_output_path, lap_val)
# Plot the current model
cur_ax_handle = ax_handle_list.flatten()[plot_id]
bnpy.viz.PlotComps.plotCompsFromHModel(
cur_model, Data=dataset, ax_handle=cur_ax_handle)
cur_ax_handle.set_xticks([-2, -1, 0, 1, 2])
cur_ax_handle.set_yticks([-2, -1, 0, 1, 2])
cur_ax_handle.set_xlabel("lap: %d" % lap_val)
pylab.tight_layout()
Training from K=1 cluster¶
Using 1 initial cluster, with birth and merge proposal moves.
K1_trained_model, K1_info_dict = bnpy.run(
dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
output_path='/tmp/AsteriskK8/trymoves-K=1/',
nLap=100, nTask=1, nBatch=1,
sF=0.1, ECovMat='eye',
K=1, initname='randexamples',
moves='birth,merge,shuffle',
m_startLap=5, b_startLap=2, b_Kfresh=4)
show_clusters_over_time(K1_info_dict['task_output_path'])
Dataset Summary:
X Data
total size: 5000 units
batch size: 5000 units
num. batches: 1
Allocation Model: DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data Model: Gaussian with full covariance.
Obs. Data Prior: Gauss-Wishart on mean and covar of each cluster
E[ mean[k] ] =
[0. 0.]
E[ covar[k] ] =
[[0.1 0. ]
[0. 0.1]]
Initialization:
initname = randexamples
K = 1 (number of clusters)
seed = 1607680
elapsed_time: 0.0 sec
Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=1/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
1.000/100 after 0 sec. | 202.5 MiB | K 1 | loss 1.105578508e+00 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 4 states. 1/1 succeeded. 0/1 failed eval phase. 0/1 failed build phase.
2.000/100 after 0 sec. | 202.5 MiB | K 5 | loss 7.390275336e-01 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 12 states. 3/4 succeeded. 1/4 failed eval phase. 0/4 failed build phase.
3.000/100 after 1 sec. | 202.5 MiB | K 17 | loss 2.819339824e-02 |
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/12 succeeded. 10/12 failed eval phase. 2/12 failed build phase.
4.000/100 after 2 sec. | 202.5 MiB | K 17 | loss -1.240759022e-02 | Ndiff 80.474
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 17 too busy.
MERGE @ lap 5.00 : 4/30 accepted. Ndiff 98.67. 10 skipped.
5.000/100 after 2 sec. | 202.5 MiB | K 13 | loss -2.742336056e-02 | Ndiff 80.474
BIRTH @ lap 6.000 : None attempted. 0 past failures. 0 too small. 13 too busy.
MERGE @ lap 6.00 : 3/23 accepted. Ndiff 767.62. 5 skipped.
6.000/100 after 3 sec. | 202.5 MiB | K 10 | loss -4.626686932e-02 | Ndiff 80.474
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 10 too busy.
MERGE @ lap 7.00 : 0/18 accepted. Ndiff 0.00. 0 skipped.
7.000/100 after 3 sec. | 202.5 MiB | K 10 | loss -4.686532682e-02 | Ndiff 2.930
BIRTH @ lap 8.00 : Added 0 states. 0/2 succeeded. 2/2 failed eval phase. 0/2 failed build phase.
MERGE @ lap 8.00 : 2/2 accepted. Ndiff 0.09. 5 skipped.
8.000/100 after 3 sec. | 202.5 MiB | K 8 | loss -4.769229049e-02 | Ndiff 2.930
BIRTH @ lap 9.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 9.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
9.000/100 after 4 sec. | 202.5 MiB | K 8 | loss -4.787962424e-02 | Ndiff 0.091
BIRTH @ lap 10.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 10.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
10.000/100 after 4 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 11.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 11.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
11.000/100 after 4 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 12.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 12.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
12.000/100 after 4 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
MERGE @ lap 13.00: No promising candidates, so no attempts.
BIRTH @ lap 13.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
13.000/100 after 4 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 14.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 14.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
14.000/100 after 4 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 15.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 15.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
15.000/100 after 5 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 16.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 16.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
16.000/100 after 5 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 17.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 17.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
17.000/100 after 5 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
MERGE @ lap 18.00: No promising candidates, so no attempts.
BIRTH @ lap 18.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
18.000/100 after 5 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 19.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 19.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
19.000/100 after 5 sec. | 202.5 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
... done. converged.
SKIPPED 1 comps with size below 0.00
SKIPPED 2 comps with size below 0.00
Training from K=4 cluster¶
Now using 4 initial clusters, with birth and merge proposal moves.
K4_trained_model, K4_info_dict = bnpy.run(
dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
output_path='/tmp/AsteriskK8/trymoves-K=4/',
nLap=100, nTask=1, nBatch=1,
sF=0.1, ECovMat='eye',
K=4, initname='randexamples',
moves='birth,merge,shuffle',
m_startLap=5, b_startLap=2, b_Kfresh=4)
show_clusters_over_time(K4_info_dict['task_output_path'])
Dataset Summary:
X Data
total size: 5000 units
batch size: 5000 units
num. batches: 1
Allocation Model: DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data Model: Gaussian with full covariance.
Obs. Data Prior: Gauss-Wishart on mean and covar of each cluster
E[ mean[k] ] =
[0. 0.]
E[ covar[k] ] =
[[0.1 0. ]
[0. 0.1]]
Initialization:
initname = randexamples
K = 4 (number of clusters)
seed = 1607680
elapsed_time: 0.0 sec
Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=4/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
1.000/100 after 0 sec. | 204.8 MiB | K 4 | loss 9.001549009e-01 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 8 states. 2/4 succeeded. 2/4 failed eval phase. 0/4 failed build phase.
2.000/100 after 0 sec. | 204.8 MiB | K 12 | loss 6.597743618e-03 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 0 states. 0/9 succeeded. 9/9 failed eval phase. 0/9 failed build phase.
3.000/100 after 1 sec. | 204.8 MiB | K 12 | loss -2.949554282e-02 | Ndiff 83.491
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/3 succeeded. 3/3 failed eval phase. 0/3 failed build phase.
4.000/100 after 2 sec. | 204.8 MiB | K 12 | loss -3.255430235e-02 | Ndiff 18.789
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 12 too busy.
MERGE @ lap 5.00 : 2/25 accepted. Ndiff 0.00. 5 skipped.
5.000/100 after 2 sec. | 204.8 MiB | K 10 | loss -3.311213517e-02 | Ndiff 18.789
BIRTH @ lap 6.000 : None attempted. 1 past failures. 0 too small. 9 too busy.
MERGE @ lap 6.00 : 2/17 accepted. Ndiff 417.46. 3 skipped.
6.000/100 after 2 sec. | 204.8 MiB | K 8 | loss -4.787665915e-02 | Ndiff 18.789
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 7.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
7.000/100 after 3 sec. | 204.8 MiB | K 8 | loss -4.787962431e-02 | Ndiff 0.020
BIRTH @ lap 8.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 8.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
8.000/100 after 3 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
MERGE @ lap 9.00: No promising candidates, so no attempts.
BIRTH @ lap 9.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
9.000/100 after 3 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 10.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 10.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
10.000/100 after 3 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 11.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 11.00 : 0/8 accepted. Ndiff 0.00. 0 skipped.
11.000/100 after 3 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 12.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 12.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
12.000/100 after 4 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 13.000 : None attempted. 5 past failures. 0 too small. 3 too busy.
MERGE @ lap 13.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
13.000/100 after 4 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
MERGE @ lap 14.00: No promising candidates, so no attempts.
BIRTH @ lap 14.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
14.000/100 after 4 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 15.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 15.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
15.000/100 after 4 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 16.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 16.00 : 0/8 accepted. Ndiff 0.00. 0 skipped.
16.000/100 after 4 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 17.000 : None attempted. 0 past failures. 0 too small. 8 too busy.
MERGE @ lap 17.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
17.000/100 after 4 sec. | 204.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
... done. converged.
SKIPPED 2 comps with size below 0.00
Training from K=8 cluster¶
Now using 8 initial clusters
K8_trained_model, K8_info_dict = bnpy.run(
dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
output_path='/tmp/AsteriskK8/trymoves-K=8/',
nLap=100, nTask=1, nBatch=1,
sF=0.1, ECovMat='eye',
K=8, initname='randexamples',
moves='birth,merge,shuffle',
m_startLap=5, b_startLap=2, b_Kfresh=4)
show_clusters_over_time(K8_info_dict['task_output_path'])
Dataset Summary:
X Data
total size: 5000 units
batch size: 5000 units
num. batches: 1
Allocation Model: DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data Model: Gaussian with full covariance.
Obs. Data Prior: Gauss-Wishart on mean and covar of each cluster
E[ mean[k] ] =
[0. 0.]
E[ covar[k] ] =
[[0.1 0. ]
[0. 0.1]]
Initialization:
initname = randexamples
K = 8 (number of clusters)
seed = 1607680
elapsed_time: 0.0 sec
Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=8/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
1.000/100 after 0 sec. | 206.8 MiB | K 8 | loss 6.479391366e-01 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 12 states. 3/8 succeeded. 5/8 failed eval phase. 0/8 failed build phase.
2.000/100 after 1 sec. | 206.8 MiB | K 20 | loss 7.345925210e-02 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 0 states. 0/13 succeeded. 8/13 failed eval phase. 5/13 failed build phase.
3.000/100 after 2 sec. | 206.8 MiB | K 20 | loss 1.384405895e-02 | Ndiff 78.701
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/12 succeeded. 8/12 failed eval phase. 4/12 failed build phase.
4.000/100 after 3 sec. | 206.8 MiB | K 20 | loss 1.057016337e-02 | Ndiff 14.549
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 20 too busy.
MERGE @ lap 5.00 : 4/41 accepted. Ndiff 635.65. 6 skipped.
5.000/100 after 4 sec. | 206.8 MiB | K 16 | loss -3.619883330e-03 | Ndiff 14.549
BIRTH @ lap 6.000 : None attempted. 0 past failures. 0 too small. 16 too busy.
MERGE @ lap 6.00 : 4/26 accepted. Ndiff 459.69. 10 skipped.
6.000/100 after 4 sec. | 206.8 MiB | K 12 | loss -2.198388303e-02 | Ndiff 14.549
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 12 too busy.
MERGE @ lap 7.00 : 3/21 accepted. Ndiff 366.43. 7 skipped.
7.000/100 after 5 sec. | 206.8 MiB | K 9 | loss -4.222694604e-02 | Ndiff 14.549
BIRTH @ lap 8.000 : None attempted. 1 past failures. 0 too small. 8 too busy.
MERGE @ lap 8.00 : 1/16 accepted. Ndiff 75.14. 1 skipped.
8.000/100 after 5 sec. | 206.8 MiB | K 8 | loss -4.787951351e-02 | Ndiff 14.549
BIRTH @ lap 9.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 9.00 : 0/7 accepted. Ndiff 0.00. 0 skipped.
9.000/100 after 5 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.002
BIRTH @ lap 10.00 : Added 0 states. 0/4 succeeded. 4/4 failed eval phase. 0/4 failed build phase.
MERGE @ lap 10.00 : 0/3 accepted. Ndiff 0.00. 0 skipped.
10.000/100 after 6 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 11.00 : Added 0 states. 0/1 succeeded. 1/1 failed eval phase. 0/1 failed build phase.
MERGE @ lap 11.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
11.000/100 after 6 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 12.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 12.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
12.000/100 after 6 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 13.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 13.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
13.000/100 after 6 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 14.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 14.00 : 0/7 accepted. Ndiff 0.00. 0 skipped.
14.000/100 after 6 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 15.000 : None attempted. 5 past failures. 0 too small. 3 too busy.
MERGE @ lap 15.00 : 0/3 accepted. Ndiff 0.00. 0 skipped.
15.000/100 after 7 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 16.000 : None attempted. 5 past failures. 0 too small. 3 too busy.
MERGE @ lap 16.00 : 0/2 accepted. Ndiff 0.00. 0 skipped.
16.000/100 after 7 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 17.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 17.00 : 0/5 accepted. Ndiff 0.00. 0 skipped.
17.000/100 after 7 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 18.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 18.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
18.000/100 after 7 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 19.000 : None attempted. 1 past failures. 0 too small. 7 too busy.
MERGE @ lap 19.00 : 0/7 accepted. Ndiff 0.00. 0 skipped.
19.000/100 after 7 sec. | 206.8 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
... done. converged.
SKIPPED 3 comps with size below 0.00
SKIPPED 1 comps with size below 0.00
Training from K=25 cluster¶
Now using 25 initial clusters
K25_trained_model, K25_info_dict = bnpy.run(
dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
output_path='/tmp/AsteriskK8/trymoves-K=25/',
nLap=100, nTask=1, nBatch=1,
sF=0.1, ECovMat='eye',
K=25, initname='randexamples',
moves='birth,merge,shuffle',
m_startLap=5, b_startLap=2, b_Kfresh=4)
show_clusters_over_time(K25_info_dict['task_output_path'])
Dataset Summary:
X Data
total size: 5000 units
batch size: 5000 units
num. batches: 1
Allocation Model: DP mixture with K=0. Concentration gamma0= 1.00
Obs. Data Model: Gaussian with full covariance.
Obs. Data Prior: Gauss-Wishart on mean and covar of each cluster
E[ mean[k] ] =
[0. 0.]
E[ covar[k] ] =
[[0.1 0. ]
[0. 0.1]]
Initialization:
initname = randexamples
K = 25 (number of clusters)
seed = 1607680
elapsed_time: 0.0 sec
Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/AsteriskK8/trymoves-K=25/1
BIRTH @ lap 1.00: Disabled. Waiting for lap >= 2 (--b_startLap).
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
1.000/100 after 0 sec. | 209.8 MiB | K 25 | loss 4.392317798e-01 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 2.00 : Added 14 states. 4/18 succeeded. 11/18 failed eval phase. 3/18 failed build phase.
2.000/100 after 2 sec. | 201.9 MiB | K 39 | loss 1.219944497e-01 |
MERGE @ lap 3.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 3.00 : Added 0 states. 0/15 succeeded. 3/15 failed eval phase. 12/15 failed build phase.
3.000/100 after 4 sec. | 203.2 MiB | K 39 | loss 8.910722771e-02 | Ndiff 56.472
MERGE @ lap 4.00: Disabled. Waiting for lap >= 5 (--m_startLap).
BIRTH @ lap 4.00 : Added 0 states. 0/15 succeeded. 9/15 failed eval phase. 6/15 failed build phase.
4.000/100 after 5 sec. | 203.2 MiB | K 39 | loss 8.017398425e-02 | Ndiff 34.554
BIRTH @ lap 5.000 : None attempted. 0 past failures. 0 too small. 39 too busy.
MERGE @ lap 5.00 : 8/63 accepted. Ndiff 338.97. 32 skipped.
5.000/100 after 7 sec. | 203.2 MiB | K 31 | loss 5.747324378e-02 | Ndiff 34.554
BIRTH @ lap 6.000 : None attempted. 0 past failures. 0 too small. 31 too busy.
MERGE @ lap 6.00 : 10/39 accepted. Ndiff 1032.92. 38 skipped.
6.000/100 after 8 sec. | 203.2 MiB | K 21 | loss 1.309182046e-02 | Ndiff 34.554
BIRTH @ lap 7.000 : None attempted. 0 past failures. 0 too small. 21 too busy.
MERGE @ lap 7.00 : 5/34 accepted. Ndiff 323.06. 15 skipped.
7.000/100 after 9 sec. | 203.2 MiB | K 16 | loss -8.888342290e-03 | Ndiff 34.554
BIRTH @ lap 8.000 : None attempted. 0 past failures. 0 too small. 16 too busy.
MERGE @ lap 8.00 : 4/25 accepted. Ndiff 443.50. 14 skipped.
8.000/100 after 9 sec. | 203.2 MiB | K 12 | loss -3.261743488e-02 | Ndiff 34.554
BIRTH @ lap 9.000 : None attempted. 0 past failures. 0 too small. 12 too busy.
MERGE @ lap 9.00 : 2/21 accepted. Ndiff 184.97. 7 skipped.
9.000/100 after 9 sec. | 203.2 MiB | K 10 | loss -4.513446764e-02 | Ndiff 34.554
BIRTH @ lap 10.000 : None attempted. 0 past failures. 0 too small. 10 too busy.
MERGE @ lap 10.00 : 2/14 accepted. Ndiff 2.56. 5 skipped.
10.000/100 after 10 sec. | 203.2 MiB | K 8 | loss -4.787873354e-02 | Ndiff 34.554
BIRTH @ lap 11.00 : Added 0 states. 0/3 succeeded. 3/3 failed eval phase. 0/3 failed build phase.
MERGE @ lap 11.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
11.000/100 after 10 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
MERGE @ lap 12.00: No promising candidates, so no attempts.
BIRTH @ lap 12.00 : Added 0 states. 0/4 succeeded. 4/4 failed eval phase. 0/4 failed build phase.
12.000/100 after 10 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 13.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 13.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
13.000/100 after 11 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 14.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 14.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
14.000/100 after 11 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 15.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 15.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
15.000/100 after 11 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 16.000 : None attempted. 4 past failures. 0 too small. 4 too busy.
MERGE @ lap 16.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
16.000/100 after 11 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
MERGE @ lap 17.00: No promising candidates, so no attempts.
BIRTH @ lap 17.000 : None attempted. 8 past failures. 0 too small. 0 too busy.
17.000/100 after 11 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 18.000 : None attempted. 3 past failures. 0 too small. 5 too busy.
MERGE @ lap 18.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
18.000/100 after 11 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 19.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 19.00 : 0/11 accepted. Ndiff 0.00. 0 skipped.
19.000/100 after 11 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 20.000 : None attempted. 2 past failures. 0 too small. 6 too busy.
MERGE @ lap 20.00 : 0/9 accepted. Ndiff 0.00. 0 skipped.
20.000/100 after 12 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
BIRTH @ lap 21.000 : None attempted. 4 past failures. 0 too small. 4 too busy.
MERGE @ lap 21.00 : 0/4 accepted. Ndiff 0.00. 0 skipped.
21.000/100 after 12 sec. | 203.2 MiB | K 8 | loss -4.787962433e-02 | Ndiff 0.000
... done. converged.
SKIPPED 4 comps with size below 0.00
SKIPPED 3 comps with size below 0.00
Total running time of the script: ( 0 minutes 30.909 seconds)