How delete moves can be more effective than merges.

In this example, we show how merge moves alone may not be enough to reliably escape local optima. Instead, we show that more flexible delete moves can escape from situations where merges alone fail.

```
import bnpy
import numpy as np
import os
from matplotlib import pylab
import seaborn as sns
FIG_SIZE = (3, 3)
pylab.rcParams['figure.figsize'] = FIG_SIZE
```

Create toy dataset of many points drawn from standard normal

```
prng = np.random.RandomState(42)
X = prng.randn(100000, 1)
dataset = bnpy.data.XData(X, name='StandardNormalK1')
```

Make a simple plot of the raw data

```
pylab.hist(dataset.X[:, 0], 50, normed=1)
pylab.xlabel('x')
pylab.ylabel('p(x)')
pylab.tight_layout()
```

```
merge_kwargs = dict(
m_startLap=10,
m_pair_ranking_procedure='total_size',
m_pair_ranking_direction='descending',
)
delete_kwargs = dict(
d_startLap=10,
d_nRefineSteps=50,
)
```

```
def show_clusters_over_time(
task_output_path=None,
query_laps=[0, 1, 2, 10, 20, None],
nrows=2):
'''
'''
ncols = int(np.ceil(len(query_laps) // float(nrows)))
fig_handle, ax_handle_list = pylab.subplots(
figsize=(FIG_SIZE[0] * ncols, FIG_SIZE[1] * nrows),
nrows=nrows, ncols=ncols, sharex=True, sharey=True)
for plot_id, lap_val in enumerate(query_laps):
cur_model, lap_val = bnpy.load_model_at_lap(task_output_path, lap_val)
cur_ax_handle = ax_handle_list.flatten()[plot_id]
bnpy.viz.PlotComps.plotCompsFromHModel(
cur_model, dataset=dataset, ax_handle=cur_ax_handle)
cur_ax_handle.set_xlim([-4.5, 4.5])
cur_ax_handle.set_xlabel("lap: %d" % lap_val)
pylab.tight_layout()
```

Unfortunately, no pairwise merge is accepted. The model is stuck using 5 clusters when one cluster would do.

```
gamma = 5.0
sF = 0.1
K = 5
m_trained_model, m_info_dict = bnpy.run(
dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
output_path=('/tmp/StandardNormalK1/' +
'trymoves-K=%d-gamma=%s-ECovMat=%s*eye-moves=merge,shuffle/' % (
K, gamma, sF)),
nLap=100, nTask=1, nBatch=1,
gamma0=gamma, sF=sF, ECovMat='eye',
K=K, initname='randexamplesbydist',
moves='merge,shuffle',
**dict(**merge_kwargs))
show_clusters_over_time(m_info_dict['task_output_path'])
```

Out:

```
Dataset Summary:
X Data
total size: 100000 units
batch size: 100000 units
num. batches: 1
Allocation Model: DP mixture with K=0. Concentration gamma0= 5.00
Obs. Data Model: Gaussian with full covariance.
Obs. Data Prior: Gauss-Wishart on mean and covar of each cluster
E[ mean[k] ] =
[ 0.]
E[ covar[k] ] =
[[ 0.1]]
Initialization:
initname = randexamplesbydist
K = 5 (number of clusters)
seed = 1607680
elapsed_time: 0.0 sec
Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/StandardNormalK1/trymoves-K=5-gamma=5.0-ECovMat=0.1*eye-moves=merge,shuffle/1
MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset.
1.000/100 after 0 sec. | 143.8 MiB | K 5 | loss 1.547776525e+00 |
MERGE @ lap 2.00: Disabled. Waiting for lap >= 10 (--m_startLap).
2.000/100 after 0 sec. | 143.8 MiB | K 5 | loss 1.446186110e+00 | Ndiff 251.672
MERGE @ lap 3.00: Disabled. Waiting for lap >= 10 (--m_startLap).
3.000/100 after 0 sec. | 143.8 MiB | K 5 | loss 1.434177601e+00 | Ndiff 126.605
MERGE @ lap 4.00: Disabled. Waiting for lap >= 10 (--m_startLap).
4.000/100 after 0 sec. | 143.8 MiB | K 5 | loss 1.429642157e+00 | Ndiff 81.076
MERGE @ lap 5.00: Disabled. Waiting for lap >= 10 (--m_startLap).
5.000/100 after 0 sec. | 143.8 MiB | K 5 | loss 1.427329685e+00 | Ndiff 57.141
MERGE @ lap 6.00: Disabled. Waiting for lap >= 10 (--m_startLap).
6.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.425951747e+00 | Ndiff 42.744
MERGE @ lap 7.00: Disabled. Waiting for lap >= 10 (--m_startLap).
7.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.425047267e+00 | Ndiff 33.297
MERGE @ lap 8.00: Disabled. Waiting for lap >= 10 (--m_startLap).
8.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.424412975e+00 | Ndiff 26.709
MERGE @ lap 9.00: Disabled. Waiting for lap >= 10 (--m_startLap).
9.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.423946233e+00 | Ndiff 21.904
MERGE @ lap 10.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
10.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.423589980e+00 | Ndiff 18.275
MERGE @ lap 11.00: No promising candidates, so no attempts.
11.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.423310120e+00 | Ndiff 15.457
MERGE @ lap 12.00: No promising candidates, so no attempts.
12.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.423085107e+00 | Ndiff 13.219
MERGE @ lap 13.00: No promising candidates, so no attempts.
13.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.422900698e+00 | Ndiff 11.408
MERGE @ lap 14.00: No promising candidates, so no attempts.
14.000/100 after 1 sec. | 143.8 MiB | K 5 | loss 1.422747120e+00 | Ndiff 9.920
MERGE @ lap 15.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
15.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422617464e+00 | Ndiff 8.680
MERGE @ lap 16.00: No promising candidates, so no attempts.
16.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422506710e+00 | Ndiff 7.635
MERGE @ lap 17.00: No promising candidates, so no attempts.
17.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422411133e+00 | Ndiff 6.745
MERGE @ lap 18.00: No promising candidates, so no attempts.
18.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422327908e+00 | Ndiff 6.095
MERGE @ lap 19.00: No promising candidates, so no attempts.
19.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422254864e+00 | Ndiff 5.547
MERGE @ lap 20.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
20.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422190300e+00 | Ndiff 5.057
MERGE @ lap 21.00: No promising candidates, so no attempts.
21.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422132869e+00 | Ndiff 4.617
MERGE @ lap 22.00: No promising candidates, so no attempts.
22.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422081490e+00 | Ndiff 4.220
MERGE @ lap 23.00: No promising candidates, so no attempts.
23.000/100 after 2 sec. | 143.8 MiB | K 5 | loss 1.422035287e+00 | Ndiff 3.860
MERGE @ lap 24.00: No promising candidates, so no attempts.
24.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421993542e+00 | Ndiff 3.534
MERGE @ lap 25.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
25.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421955661e+00 | Ndiff 3.236
MERGE @ lap 26.00: No promising candidates, so no attempts.
26.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421921151e+00 | Ndiff 2.964
MERGE @ lap 27.00: No promising candidates, so no attempts.
27.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421889597e+00 | Ndiff 2.714
MERGE @ lap 28.00: No promising candidates, so no attempts.
28.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421860649e+00 | Ndiff 2.558
MERGE @ lap 29.00: No promising candidates, so no attempts.
29.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421834007e+00 | Ndiff 2.440
MERGE @ lap 30.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
30.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421809418e+00 | Ndiff 2.331
MERGE @ lap 31.00: No promising candidates, so no attempts.
31.000/100 after 3 sec. | 143.8 MiB | K 5 | loss 1.421786661e+00 | Ndiff 2.230
MERGE @ lap 32.00: No promising candidates, so no attempts.
32.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421765547e+00 | Ndiff 2.137
MERGE @ lap 33.00: No promising candidates, so no attempts.
33.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421745910e+00 | Ndiff 2.050
MERGE @ lap 34.00: No promising candidates, so no attempts.
34.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421727607e+00 | Ndiff 1.969
MERGE @ lap 35.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
35.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421710511e+00 | Ndiff 1.894
MERGE @ lap 36.00: No promising candidates, so no attempts.
36.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421694512e+00 | Ndiff 1.824
MERGE @ lap 37.00: No promising candidates, so no attempts.
37.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421679511e+00 | Ndiff 1.758
MERGE @ lap 38.00: No promising candidates, so no attempts.
38.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421665422e+00 | Ndiff 1.696
MERGE @ lap 39.00: No promising candidates, so no attempts.
39.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421652166e+00 | Ndiff 1.638
MERGE @ lap 40.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
40.000/100 after 4 sec. | 143.8 MiB | K 5 | loss 1.421639676e+00 | Ndiff 1.583
MERGE @ lap 41.00: No promising candidates, so no attempts.
41.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421627889e+00 | Ndiff 1.532
MERGE @ lap 42.00: No promising candidates, so no attempts.
42.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421616749e+00 | Ndiff 1.483
MERGE @ lap 43.00: No promising candidates, so no attempts.
43.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421606208e+00 | Ndiff 1.437
MERGE @ lap 44.00: No promising candidates, so no attempts.
44.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421596220e+00 | Ndiff 1.393
MERGE @ lap 45.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
45.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421586745e+00 | Ndiff 1.351
MERGE @ lap 46.00: No promising candidates, so no attempts.
46.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421577745e+00 | Ndiff 1.324
MERGE @ lap 47.00: No promising candidates, so no attempts.
47.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421569187e+00 | Ndiff 1.299
MERGE @ lap 48.00: No promising candidates, so no attempts.
48.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421561041e+00 | Ndiff 1.275
MERGE @ lap 49.00: No promising candidates, so no attempts.
49.000/100 after 5 sec. | 143.8 MiB | K 5 | loss 1.421553280e+00 | Ndiff 1.252
MERGE @ lap 50.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
50.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421545876e+00 | Ndiff 1.229
MERGE @ lap 51.00: No promising candidates, so no attempts.
51.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421538809e+00 | Ndiff 1.206
MERGE @ lap 52.00: No promising candidates, so no attempts.
52.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421532055e+00 | Ndiff 1.184
MERGE @ lap 53.00: No promising candidates, so no attempts.
53.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421525596e+00 | Ndiff 1.163
MERGE @ lap 54.00: No promising candidates, so no attempts.
54.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421519413e+00 | Ndiff 1.143
MERGE @ lap 55.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
55.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421513490e+00 | Ndiff 1.122
MERGE @ lap 56.00: No promising candidates, so no attempts.
56.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421507811e+00 | Ndiff 1.103
MERGE @ lap 57.00: No promising candidates, so no attempts.
57.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421502363e+00 | Ndiff 1.084
MERGE @ lap 58.00: No promising candidates, so no attempts.
58.000/100 after 6 sec. | 143.8 MiB | K 5 | loss 1.421497133e+00 | Ndiff 1.065
MERGE @ lap 59.00: No promising candidates, so no attempts.
59.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421492107e+00 | Ndiff 1.047
MERGE @ lap 60.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
60.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421487275e+00 | Ndiff 1.029
MERGE @ lap 61.00: No promising candidates, so no attempts.
61.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421482626e+00 | Ndiff 1.012
MERGE @ lap 62.00: No promising candidates, so no attempts.
62.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421478151e+00 | Ndiff 0.995
MERGE @ lap 63.00: No promising candidates, so no attempts.
63.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421473840e+00 | Ndiff 0.979
MERGE @ lap 64.00: No promising candidates, so no attempts.
64.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421469686e+00 | Ndiff 0.963
MERGE @ lap 65.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
65.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421465679e+00 | Ndiff 0.948
MERGE @ lap 66.00: No promising candidates, so no attempts.
66.000/100 after 7 sec. | 143.8 MiB | K 5 | loss 1.421461813e+00 | Ndiff 0.933
MERGE @ lap 67.00: No promising candidates, so no attempts.
67.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421458080e+00 | Ndiff 0.918
MERGE @ lap 68.00: No promising candidates, so no attempts.
68.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421454475e+00 | Ndiff 0.904
MERGE @ lap 69.00: No promising candidates, so no attempts.
69.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421450991e+00 | Ndiff 0.890
MERGE @ lap 70.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
70.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421447622e+00 | Ndiff 0.877
MERGE @ lap 71.00: No promising candidates, so no attempts.
71.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421444363e+00 | Ndiff 0.867
MERGE @ lap 72.00: No promising candidates, so no attempts.
72.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421441210e+00 | Ndiff 0.881
MERGE @ lap 73.00: No promising candidates, so no attempts.
73.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421438156e+00 | Ndiff 0.894
MERGE @ lap 74.00: No promising candidates, so no attempts.
74.000/100 after 8 sec. | 143.8 MiB | K 5 | loss 1.421435198e+00 | Ndiff 0.907
MERGE @ lap 75.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
75.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421432332e+00 | Ndiff 0.920
MERGE @ lap 76.00: No promising candidates, so no attempts.
76.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421429553e+00 | Ndiff 0.931
MERGE @ lap 77.00: No promising candidates, so no attempts.
77.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421426858e+00 | Ndiff 0.942
MERGE @ lap 78.00: No promising candidates, so no attempts.
78.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421424243e+00 | Ndiff 0.953
MERGE @ lap 79.00: No promising candidates, so no attempts.
79.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421421705e+00 | Ndiff 0.963
MERGE @ lap 80.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
80.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421419241e+00 | Ndiff 0.973
MERGE @ lap 81.00: No promising candidates, so no attempts.
81.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421416847e+00 | Ndiff 0.983
MERGE @ lap 82.00: No promising candidates, so no attempts.
82.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421414521e+00 | Ndiff 0.991
MERGE @ lap 83.00: No promising candidates, so no attempts.
83.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421412261e+00 | Ndiff 1.000
MERGE @ lap 84.00: No promising candidates, so no attempts.
84.000/100 after 9 sec. | 143.8 MiB | K 5 | loss 1.421410063e+00 | Ndiff 1.008
MERGE @ lap 85.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
85.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421407925e+00 | Ndiff 1.016
MERGE @ lap 86.00: No promising candidates, so no attempts.
86.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421405845e+00 | Ndiff 1.023
MERGE @ lap 87.00: No promising candidates, so no attempts.
87.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421403820e+00 | Ndiff 1.031
MERGE @ lap 88.00: No promising candidates, so no attempts.
88.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421401849e+00 | Ndiff 1.038
MERGE @ lap 89.00: No promising candidates, so no attempts.
89.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421399930e+00 | Ndiff 1.044
MERGE @ lap 90.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
90.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421398060e+00 | Ndiff 1.050
MERGE @ lap 91.00: No promising candidates, so no attempts.
91.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421396238e+00 | Ndiff 1.056
MERGE @ lap 92.00: No promising candidates, so no attempts.
92.000/100 after 10 sec. | 143.8 MiB | K 5 | loss 1.421394462e+00 | Ndiff 1.062
MERGE @ lap 93.00: No promising candidates, so no attempts.
93.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421392731e+00 | Ndiff 1.068
MERGE @ lap 94.00: No promising candidates, so no attempts.
94.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421391043e+00 | Ndiff 1.077
MERGE @ lap 95.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
95.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421389396e+00 | Ndiff 1.087
MERGE @ lap 96.00: No promising candidates, so no attempts.
96.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421387789e+00 | Ndiff 1.096
MERGE @ lap 97.00: No promising candidates, so no attempts.
97.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421386221e+00 | Ndiff 1.105
MERGE @ lap 98.00: No promising candidates, so no attempts.
98.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421384690e+00 | Ndiff 1.114
MERGE @ lap 99.00: No promising candidates, so no attempts.
99.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421383195e+00 | Ndiff 1.122
MERGE @ lap 100.00 : 0/10 accepted. Ndiff 0.00. 0 skipped.
100.000/100 after 11 sec. | 143.8 MiB | K 5 | loss 1.421381735e+00 | Ndiff 1.130
... done. not converged. max laps thru data exceeded.
```

More flexible delete moves *are* accepted.

```
d_trained_model, d_info_dict = bnpy.run(
dataset, 'DPMixtureModel', 'Gauss', 'memoVB',
output_path=('/tmp/StandardNormalK1/' +
'trymoves-K=%d-gamma=%s-ECovMat=%s*eye-moves=delete,shuffle/' % (
K, gamma, sF)),
nLap=100, nTask=1, nBatch=1,
gamma0=gamma, sF=sF, ECovMat='eye',
K=K, initname='randexamplesbydist',
moves='delete,shuffle',
**dict(delete_kwargs))
show_clusters_over_time(d_info_dict['task_output_path'])
```

Out:

```
Dataset Summary:
X Data
total size: 100000 units
batch size: 100000 units
num. batches: 1
Allocation Model: DP mixture with K=0. Concentration gamma0= 5.00
Obs. Data Model: Gaussian with full covariance.
Obs. Data Prior: Gauss-Wishart on mean and covar of each cluster
E[ mean[k] ] =
[ 0.]
E[ covar[k] ] =
[[ 0.1]]
Initialization:
initname = randexamplesbydist
K = 5 (number of clusters)
seed = 1607680
elapsed_time: 0.0 sec
Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/StandardNormalK1/trymoves-K=5-gamma=5.0-ECovMat=0.1*eye-moves=delete,shuffle/1
DELETE @ lap 1.00: Disabled. Cannot delete before first complete lap, because SS that represents whole dataset is required.
1.000/100 after 0 sec. | 151.4 MiB | K 5 | loss 1.547776525e+00 |
DELETE @ lap 2.00: Disabled. Waiting for lap >= 10 (--d_startLap).
2.000/100 after 0 sec. | 151.4 MiB | K 5 | loss 1.446186110e+00 | Ndiff 251.672
DELETE @ lap 3.00: Disabled. Waiting for lap >= 10 (--d_startLap).
3.000/100 after 0 sec. | 151.4 MiB | K 5 | loss 1.434177601e+00 | Ndiff 126.605
DELETE @ lap 4.00: Disabled. Waiting for lap >= 10 (--d_startLap).
4.000/100 after 0 sec. | 151.4 MiB | K 5 | loss 1.429642157e+00 | Ndiff 81.076
DELETE @ lap 5.00: Disabled. Waiting for lap >= 10 (--d_startLap).
5.000/100 after 0 sec. | 151.4 MiB | K 5 | loss 1.427329685e+00 | Ndiff 57.141
DELETE @ lap 6.00: Disabled. Waiting for lap >= 10 (--d_startLap).
6.000/100 after 1 sec. | 151.4 MiB | K 5 | loss 1.425951747e+00 | Ndiff 42.744
DELETE @ lap 7.00: Disabled. Waiting for lap >= 10 (--d_startLap).
7.000/100 after 1 sec. | 151.4 MiB | K 5 | loss 1.425047267e+00 | Ndiff 33.297
DELETE @ lap 8.00: Disabled. Waiting for lap >= 10 (--d_startLap).
8.000/100 after 1 sec. | 151.4 MiB | K 5 | loss 1.424412975e+00 | Ndiff 26.709
DELETE @ lap 9.00: Disabled. Waiting for lap >= 10 (--d_startLap).
9.000/100 after 1 sec. | 151.4 MiB | K 5 | loss 1.423946233e+00 | Ndiff 21.904
DELETE @ lap 10.00: 1/1 accepted. Ndiff 43526.74.
10.000/100 after 3 sec. | 151.4 MiB | K 4 | loss 1.421389609e+00 | Ndiff 21.904
DELETE @ lap 11.00: 1/1 accepted. Ndiff 49884.65.
11.000/100 after 5 sec. | 151.4 MiB | K 3 | loss 1.421157450e+00 | Ndiff 21.904
DELETE @ lap 12.00: 1/1 accepted. Ndiff 43419.31.
12.000/100 after 6 sec. | 156.6 MiB | K 2 | loss 1.420888889e+00 | Ndiff 21.904
DELETE @ lap 13.00: 1/1 accepted. Ndiff 42527.97.
13.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 21.904
DELETE @ lap 14.00: Ineligible. Did not find >= 2 UIDs in entire model.
14.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 15.00: Ineligible. Did not find >= 2 UIDs in entire model.
15.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 16.00: Ineligible. Did not find >= 2 UIDs in entire model.
16.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 17.00: Ineligible. Did not find >= 2 UIDs in entire model.
17.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 18.00: Ineligible. Did not find >= 2 UIDs in entire model.
18.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 19.00: Ineligible. Did not find >= 2 UIDs in entire model.
19.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 20.00: Ineligible. Did not find >= 2 UIDs in entire model.
20.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 21.00: Ineligible. Did not find >= 2 UIDs in entire model.
21.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 22.00: Ineligible. Did not find >= 2 UIDs in entire model.
22.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 23.00: Ineligible. Did not find >= 2 UIDs in entire model.
23.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
DELETE @ lap 24.00: Ineligible. Did not find >= 2 UIDs in entire model.
24.000/100 after 6 sec. | 156.6 MiB | K 1 | loss 1.420559635e+00 | Ndiff 0.000
... done. converged.
```

```
pylab.plot(
m_info_dict['lap_history'][1:],
m_info_dict['loss_history'][1:], 'k.-',
label='vb_with_merges')
pylab.plot(
d_info_dict['lap_history'][1:],
d_info_dict['loss_history'][1:], 'b.-',
label='vb_with_deletes')
pylab.legend(loc='upper right')
pylab.xlabel('num. laps')
pylab.ylabel('loss')
pylab.tight_layout()
```

**Total running time of the script:** ( 0 minutes 20.276 seconds)