.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/03_faithful/plot-02-demo=merge_and_delete-model=dp_mix+gauss.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_03_faithful_plot-02-demo=merge_and_delete-model=dp_mix+gauss.py: ======================================================================== Variational with merge and delete proposals for DP mixtures of Gaussians ======================================================================== How delete moves can be more effective than merges. In this example, we show how merge moves alone may not be enough to reliably escape local optima. Instead, we show that more flexible delete moves can escape from situations where merges alone fail. .. GENERATED FROM PYTHON SOURCE LINES 12-22 .. code-block:: default import bnpy import numpy as np import os from matplotlib import pylab import seaborn as sns FIG_SIZE = (3, 3) pylab.rcParams['figure.figsize'] = FIG_SIZE .. GENERATED FROM PYTHON SOURCE LINES 23-24 Load dataset from file .. GENERATED FROM PYTHON SOURCE LINES 25-31 .. code-block:: default dataset_path = os.path.join(bnpy.DATASET_PATH, 'faithful') dataset = bnpy.data.XData.read_csv( os.path.join(dataset_path, 'faithful.csv')) .. GENERATED FROM PYTHON SOURCE LINES 32-33 Make a simple plot of the raw data .. GENERATED FROM PYTHON SOURCE LINES 34-41 .. code-block:: default pylab.plot(dataset.X[:, 0], dataset.X[:, 1], 'k.') pylab.xlabel(dataset.column_names[0]) pylab.ylabel(dataset.column_names[1]) pylab.tight_layout() data_ax_h = pylab.gca() .. image-sg:: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_001.png :alt: plot 02 demo=merge and delete model=dp mix+gauss :srcset: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 42-44 Setup: Determine specific settings of the proposals --------------------------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 44-57 .. code-block:: default merge_kwargs = dict( m_startLap=10, m_pair_ranking_procedure='total_size', m_pair_ranking_direction='descending', ) delete_kwargs = dict( d_startLap=20, d_nRefineSteps=50, ) .. GENERATED FROM PYTHON SOURCE LINES 58-60 Setup: Helper function to display the learned clusters ------------------------------------------------------ .. GENERATED FROM PYTHON SOURCE LINES 61-86 .. code-block:: default def show_clusters_over_time( task_output_path=None, query_laps=[0, 1, 2, 10, 20, None], nrows=2): ''' ''' ncols = int(np.ceil(len(query_laps) // float(nrows))) fig_handle, ax_handle_list = pylab.subplots( figsize=(FIG_SIZE[0] * ncols, FIG_SIZE[1] * nrows), nrows=nrows, ncols=ncols, sharex=True, sharey=True) for plot_id, lap_val in enumerate(query_laps): cur_model, lap_val = bnpy.load_model_at_lap(task_output_path, lap_val) cur_ax_handle = ax_handle_list.flatten()[plot_id] bnpy.viz.PlotComps.plotCompsFromHModel( cur_model, dataset=dataset, ax_handle=cur_ax_handle) cur_ax_handle.set_title("lap: %d" % lap_val) cur_ax_handle.set_xlabel(dataset.column_names[0]) cur_ax_handle.set_ylabel(dataset.column_names[1]) cur_ax_handle.set_xlim(data_ax_h.get_xlim()) cur_ax_handle.set_ylim(data_ax_h.get_ylim()) pylab.tight_layout() .. GENERATED FROM PYTHON SOURCE LINES 87-91 *DiagGauss* observation model, without moves -------------------------------------------- Start with too many clusters (K=25) .. GENERATED FROM PYTHON SOURCE LINES 92-108 .. code-block:: default gamma = 5.0 sF = 5.0 K = 25 diag1_trained_model, diag1_info_dict = bnpy.run( dataset, 'DPMixtureModel', 'DiagGauss', 'memoVB', output_path=('/tmp/faithful/' + 'trymoves-K=%d-gamma=%s-lik=DiagGauss-ECovMat=%s*eye-moves=none/' % ( K, gamma, sF)), nLap=1000, nTask=1, nBatch=1, convergeThr=0.0001, gamma0=gamma, sF=sF, ECovMat='eye', K=K, initname='randexamplesbydist', ) show_clusters_over_time(diag1_info_dict['task_output_path']) .. image-sg:: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_002.png :alt: lap: 0, lap: 1, lap: 2, lap: 10, lap: 20, lap: 79 :srcset: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_002.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none Dataset Summary: X Data total size: 272 units batch size: 272 units num. batches: 1 Allocation Model: DP mixture with K=0. Concentration gamma0= 5.00 Obs. Data Model: Gaussian with diagonal covariance. Obs. Data Prior: independent Gauss-Wishart prior on each dimension Wishart params nu = 4 beta = [ 10 10] Expectations E[ mean[k]] = [ 0 0] E[ covar[k]] = [[5. 0.] [0. 5.]] Initialization: initname = randexamplesbydist K = 25 (number of clusters) seed = 1607680 elapsed_time: 0.0 sec Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952 task_output_path: /tmp/faithful/trymoves-K=25-gamma=5.0-lik=DiagGauss-ECovMat=5.0*eye-moves=none/1 1.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 3.062238769e+00 | 2.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 3.029485836e+00 | Ndiff 3.204 3.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.988929671e+00 | Ndiff 3.849 4.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.934114089e+00 | Ndiff 3.694 5.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.864320498e+00 | Ndiff 3.057 6.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.796068518e+00 | Ndiff 2.511 7.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.755200911e+00 | Ndiff 2.108 8.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.724352077e+00 | Ndiff 2.158 9.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.714818175e+00 | Ndiff 2.034 10.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.710015995e+00 | Ndiff 1.890 11.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.704686401e+00 | Ndiff 2.337 12.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.698284253e+00 | Ndiff 2.931 13.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.684190801e+00 | Ndiff 3.692 14.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.673906693e+00 | Ndiff 4.579 15.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.665323886e+00 | Ndiff 5.389 16.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.654124178e+00 | Ndiff 5.757 17.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.640063487e+00 | Ndiff 5.488 18.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.617231862e+00 | Ndiff 4.911 19.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.586775359e+00 | Ndiff 4.515 20.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.569671629e+00 | Ndiff 4.553 21.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.545014854e+00 | Ndiff 4.407 22.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.525424540e+00 | Ndiff 3.380 23.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.509251223e+00 | Ndiff 3.026 24.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.493302635e+00 | Ndiff 3.039 25.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.488984785e+00 | Ndiff 2.950 26.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.484950693e+00 | Ndiff 2.825 27.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.480608059e+00 | Ndiff 2.770 28.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.467505075e+00 | Ndiff 2.837 29.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.465130737e+00 | Ndiff 3.079 30.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.463555210e+00 | Ndiff 3.395 31.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.461714070e+00 | Ndiff 3.730 32.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.459545432e+00 | Ndiff 4.089 33.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.456958951e+00 | Ndiff 4.479 34.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.453823064e+00 | Ndiff 4.899 35.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.449961599e+00 | Ndiff 5.330 36.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.445166898e+00 | Ndiff 5.712 37.000/1000 after 0 sec. | 212.3 MiB | K 25 | loss 2.439248319e+00 | Ndiff 5.926 38.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.432174168e+00 | Ndiff 5.797 39.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.424387029e+00 | Ndiff 5.157 40.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.416911819e+00 | Ndiff 4.062 41.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.409264356e+00 | Ndiff 2.986 42.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.392900067e+00 | Ndiff 2.341 43.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.379189152e+00 | Ndiff 2.183 44.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.375964695e+00 | Ndiff 2.106 45.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.373574138e+00 | Ndiff 1.965 46.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.371010870e+00 | Ndiff 1.705 47.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.368474850e+00 | Ndiff 1.323 48.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.366360106e+00 | Ndiff 0.879 49.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.364898037e+00 | Ndiff 0.531 50.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.363200338e+00 | Ndiff 0.358 51.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.354385944e+00 | Ndiff 0.162 52.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192787e+00 | Ndiff 0.035 53.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192415e+00 | Ndiff 0.021 54.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192256e+00 | Ndiff 0.014 55.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192185e+00 | Ndiff 0.009 56.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192152e+00 | Ndiff 0.007 57.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192136e+00 | Ndiff 0.005 58.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192129e+00 | Ndiff 0.003 59.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192124e+00 | Ndiff 0.003 60.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192122e+00 | Ndiff 0.002 61.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192120e+00 | Ndiff 0.001 62.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192119e+00 | Ndiff 0.001 63.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192119e+00 | Ndiff 0.001 64.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192118e+00 | Ndiff 0.001 65.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192118e+00 | Ndiff 0.001 66.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192117e+00 | Ndiff 0.001 67.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192117e+00 | Ndiff 0.000 68.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192117e+00 | Ndiff 0.000 69.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192117e+00 | Ndiff 0.000 70.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192117e+00 | Ndiff 0.000 71.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 72.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 73.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 74.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 75.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 76.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 77.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 78.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 79.000/1000 after 1 sec. | 212.3 MiB | K 25 | loss 2.350192116e+00 | Ndiff 0.000 ... done. converged. SKIPPED 2 comps with size below 0.00 .. GENERATED FROM PYTHON SOURCE LINES 109-114 *DiagGauss* observation model -------------------------------------- Start with too many clusters (K=25) Use merges and deletes to reduce to a better set. .. GENERATED FROM PYTHON SOURCE LINES 115-133 .. code-block:: default gamma = 5.0 sF = 5.0 K = 25 diag_trained_model, diag_info_dict = bnpy.run( dataset, 'DPMixtureModel', 'DiagGauss', 'memoVB', output_path=('/tmp/faithful/' + 'trymoves-K=%d-gamma=%s-lik=DiagGauss-ECovMat=%s*eye-moves=merge,delete,shuffle/' % ( K, gamma, sF)), nLap=100, nTask=1, nBatch=1, gamma0=gamma, sF=sF, ECovMat='eye', K=K, initname='randexamplesbydist', moves='merge,delete,shuffle', **dict(list(delete_kwargs.items()) + list(merge_kwargs.items()))) show_clusters_over_time(diag_info_dict['task_output_path']) .. image-sg:: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_003.png :alt: lap: 0, lap: 1, lap: 2, lap: 10, lap: 20, lap: 31 :srcset: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_003.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none Dataset Summary: X Data total size: 272 units batch size: 272 units num. batches: 1 Allocation Model: DP mixture with K=0. Concentration gamma0= 5.00 Obs. Data Model: Gaussian with diagonal covariance. Obs. Data Prior: independent Gauss-Wishart prior on each dimension Wishart params nu = 4 beta = [ 10 10] Expectations E[ mean[k]] = [ 0 0] E[ covar[k]] = [[5. 0.] [0. 5.]] Initialization: initname = randexamplesbydist K = 25 (number of clusters) seed = 1607680 elapsed_time: 0.0 sec Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952 task_output_path: /tmp/faithful/trymoves-K=25-gamma=5.0-lik=DiagGauss-ECovMat=5.0*eye-moves=merge,delete,shuffle/1 MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset. DELETE @ lap 1.00: Disabled. Cannot delete before first complete lap, because SS that represents whole dataset is required. 1.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 3.051961291e+00 | MERGE @ lap 2.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 2.00: Disabled. Waiting for lap >= 20 (--d_startLap). 2.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 3.022209522e+00 | Ndiff 2.772 MERGE @ lap 3.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 3.00: Disabled. Waiting for lap >= 20 (--d_startLap). 3.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 2.984471596e+00 | Ndiff 3.323 MERGE @ lap 4.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 4.00: Disabled. Waiting for lap >= 20 (--d_startLap). 4.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 2.926643058e+00 | Ndiff 2.598 MERGE @ lap 5.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 5.00: Disabled. Waiting for lap >= 20 (--d_startLap). 5.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 2.862469660e+00 | Ndiff 2.755 MERGE @ lap 6.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 6.00: Disabled. Waiting for lap >= 20 (--d_startLap). 6.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 2.779705996e+00 | Ndiff 3.164 MERGE @ lap 7.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 7.00: Disabled. Waiting for lap >= 20 (--d_startLap). 7.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 2.734025386e+00 | Ndiff 3.344 MERGE @ lap 8.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 8.00: Disabled. Waiting for lap >= 20 (--d_startLap). 8.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 2.705925412e+00 | Ndiff 3.188 MERGE @ lap 9.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 9.00: Disabled. Waiting for lap >= 20 (--d_startLap). 9.000/100 after 0 sec. | 216.2 MiB | K 25 | loss 2.695114165e+00 | Ndiff 2.809 DELETE @ lap 10.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 10.00 : 11/21 accepted. Ndiff 96.55. 39 skipped. 10.000/100 after 0 sec. | 216.2 MiB | K 14 | loss 2.588387948e+00 | Ndiff 2.809 DELETE @ lap 11.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 11.00 : 6/10 accepted. Ndiff 57.31. 23 skipped. 11.000/100 after 0 sec. | 216.2 MiB | K 8 | loss 2.418907387e+00 | Ndiff 2.809 DELETE @ lap 12.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 12.00 : 4/5 accepted. Ndiff 24.39. 14 skipped. 12.000/100 after 0 sec. | 216.2 MiB | K 4 | loss 2.344906523e+00 | Ndiff 2.809 DELETE @ lap 13.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 13.00 : 1/4 accepted. Ndiff 0.00. 2 skipped. 13.000/100 after 0 sec. | 216.2 MiB | K 3 | loss 2.342439735e+00 | Ndiff 2.809 MERGE @ lap 14.00: No promising candidates, so no attempts. DELETE @ lap 14.00: Disabled. Waiting for lap >= 20 (--d_startLap). 14.000/100 after 0 sec. | 216.2 MiB | K 3 | loss 2.341617749e+00 | Ndiff 1.311 DELETE @ lap 15.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 15.00 : 0/2 accepted. Ndiff 0.00. 0 skipped. 15.000/100 after 0 sec. | 216.2 MiB | K 3 | loss 2.341288596e+00 | Ndiff 0.753 DELETE @ lap 16.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 16.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 16.000/100 after 0 sec. | 216.2 MiB | K 3 | loss 2.341147153e+00 | Ndiff 0.467 DELETE @ lap 17.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 17.00 : 0/2 accepted. Ndiff 0.00. 0 skipped. 17.000/100 after 0 sec. | 216.2 MiB | K 3 | loss 2.341080073e+00 | Ndiff 0.312 MERGE @ lap 18.00: No promising candidates, so no attempts. DELETE @ lap 18.00: Disabled. Waiting for lap >= 20 (--d_startLap). 18.000/100 after 0 sec. | 216.2 MiB | K 3 | loss 2.341044964e+00 | Ndiff 0.220 DELETE @ lap 19.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 19.00 : 0/2 accepted. Ndiff 0.00. 0 skipped. 19.000/100 after 0 sec. | 216.2 MiB | K 3 | loss 2.341025070e+00 | Ndiff 0.163 MERGE @ lap 20.00: No promising candidates, so no attempts. DELETE @ lap 20.00: 1/1 accepted. Ndiff 169.12. 20.000/100 after 0 sec. | 216.2 MiB | K 2 | loss 2.330925760e+00 | Ndiff 0.163 DELETE @ lap 21.00: 0/1 accepted. Ndiff 0.00. MERGE @ lap 21.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 21.000/100 after 0 sec. | 216.2 MiB | K 2 | loss 2.330925543e+00 | Ndiff 0.009 MERGE @ lap 22.00: No promising candidates, so no attempts. DELETE @ lap 22.00: 0/1 accepted. Ndiff 0.00. 22.000/100 after 0 sec. | 216.2 MiB | K 2 | loss 2.330925529e+00 | Ndiff 0.002 MERGE @ lap 23.00: No promising candidates, so no attempts. DELETE @ lap 23.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 23.000/100 after 0 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.001 MERGE @ lap 24.00: No promising candidates, so no attempts. DELETE @ lap 24.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 24.000/100 after 0 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 MERGE @ lap 25.00: No promising candidates, so no attempts. DELETE @ lap 25.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 25.000/100 after 0 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 DELETE @ lap 26.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. MERGE @ lap 26.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 26.000/100 after 1 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 MERGE @ lap 27.00: No promising candidates, so no attempts. DELETE @ lap 27.00: 0/1 accepted. Ndiff 0.00. 27.000/100 after 1 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 MERGE @ lap 28.00: No promising candidates, so no attempts. DELETE @ lap 28.00: 0/1 accepted. Ndiff 0.00. 28.000/100 after 1 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 MERGE @ lap 29.00: No promising candidates, so no attempts. DELETE @ lap 29.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 29.000/100 after 1 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 MERGE @ lap 30.00: No promising candidates, so no attempts. DELETE @ lap 30.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 30.000/100 after 1 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 DELETE @ lap 31.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. MERGE @ lap 31.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 31.000/100 after 1 sec. | 216.2 MiB | K 2 | loss 2.330925528e+00 | Ndiff 0.000 ... done. converged. .. GENERATED FROM PYTHON SOURCE LINES 134-139 *Gauss* observation model ------------------------- Start with too many clusters (K=25) Use merges and deletes to reduce to a better set. .. GENERATED FROM PYTHON SOURCE LINES 140-155 .. code-block:: default full_trained_model, full_info_dict = bnpy.run( dataset, 'DPMixtureModel', 'Gauss', 'memoVB', output_path=('/tmp/faithful/' + 'trymoves-K=%d-gamma=%s-lik-Gauss-ECovMat=%s*eye-moves=merge,delete,shuffle/' % ( K, gamma, sF)), nLap=100, nTask=1, nBatch=1, gamma0=gamma, sF=sF, ECovMat='eye', K=K, initname='randexamplesbydist', moves='merge,delete,shuffle', **dict(list(delete_kwargs.items()) + list(merge_kwargs.items()))) show_clusters_over_time(full_info_dict['task_output_path']) .. image-sg:: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_004.png :alt: lap: 0, lap: 1, lap: 2, lap: 10, lap: 20, lap: 31 :srcset: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_004.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none Dataset Summary: X Data total size: 272 units batch size: 272 units num. batches: 1 Allocation Model: DP mixture with K=0. Concentration gamma0= 5.00 Obs. Data Model: Gaussian with full covariance. Obs. Data Prior: Gauss-Wishart on mean and covar of each cluster E[ mean[k] ] = [0. 0.] E[ covar[k] ] = [[5. 0.] [0. 5.]] Initialization: initname = randexamplesbydist K = 25 (number of clusters) seed = 1607680 elapsed_time: 0.0 sec Learn Alg: memoVB | task 1/1 | alg. seed: 1607680 | data order seed: 8541952 task_output_path: /tmp/faithful/trymoves-K=25-gamma=5.0-lik-Gauss-ECovMat=5.0*eye-moves=merge,delete,shuffle/1 MERGE @ lap 1.00: Disabled. Cannot plan merge on first lap. Need valid SS that represent whole dataset. DELETE @ lap 1.00: Disabled. Cannot delete before first complete lap, because SS that represents whole dataset is required. 1.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.959097158e+00 | MERGE @ lap 2.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 2.00: Disabled. Waiting for lap >= 20 (--d_startLap). 2.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.937002302e+00 | Ndiff 3.003 MERGE @ lap 3.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 3.00: Disabled. Waiting for lap >= 20 (--d_startLap). 3.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.912622033e+00 | Ndiff 3.709 MERGE @ lap 4.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 4.00: Disabled. Waiting for lap >= 20 (--d_startLap). 4.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.880654959e+00 | Ndiff 3.092 MERGE @ lap 5.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 5.00: Disabled. Waiting for lap >= 20 (--d_startLap). 5.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.829532053e+00 | Ndiff 3.072 MERGE @ lap 6.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 6.00: Disabled. Waiting for lap >= 20 (--d_startLap). 6.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.769025659e+00 | Ndiff 2.422 MERGE @ lap 7.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 7.00: Disabled. Waiting for lap >= 20 (--d_startLap). 7.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.725273863e+00 | Ndiff 1.750 MERGE @ lap 8.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 8.00: Disabled. Waiting for lap >= 20 (--d_startLap). 8.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.675946971e+00 | Ndiff 1.849 MERGE @ lap 9.00: Disabled. Waiting for lap >= 10 (--m_startLap). DELETE @ lap 9.00: Disabled. Waiting for lap >= 20 (--d_startLap). 9.000/100 after 0 sec. | 218.2 MiB | K 25 | loss 2.647374823e+00 | Ndiff 1.998 DELETE @ lap 10.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 10.00 : 11/18 accepted. Ndiff 107.08. 41 skipped. 10.000/100 after 1 sec. | 218.2 MiB | K 14 | loss 2.556490112e+00 | Ndiff 1.998 DELETE @ lap 11.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 11.00 : 6/7 accepted. Ndiff 88.56. 27 skipped. 11.000/100 after 1 sec. | 218.2 MiB | K 8 | loss 2.396890018e+00 | Ndiff 1.998 DELETE @ lap 12.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 12.00 : 4/5 accepted. Ndiff 35.75. 11 skipped. 12.000/100 after 1 sec. | 218.2 MiB | K 4 | loss 2.315013331e+00 | Ndiff 1.998 DELETE @ lap 13.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 13.00 : 2/3 accepted. Ndiff 6.24. 3 skipped. 13.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.282002142e+00 | Ndiff 1.998 MERGE @ lap 14.00: No promising candidates, so no attempts. DELETE @ lap 14.00: Disabled. Waiting for lap >= 20 (--d_startLap). 14.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268874488e+00 | Ndiff 1.018 DELETE @ lap 15.00: Disabled. Waiting for lap >= 20 (--d_startLap). MERGE @ lap 15.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 15.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268112275e+00 | Ndiff 0.390 MERGE @ lap 16.00: No promising candidates, so no attempts. DELETE @ lap 16.00: Disabled. Waiting for lap >= 20 (--d_startLap). 16.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268077838e+00 | Ndiff 0.076 MERGE @ lap 17.00: No promising candidates, so no attempts. DELETE @ lap 17.00: Disabled. Waiting for lap >= 20 (--d_startLap). 17.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076838e+00 | Ndiff 0.013 MERGE @ lap 18.00: No promising candidates, so no attempts. DELETE @ lap 18.00: Disabled. Waiting for lap >= 20 (--d_startLap). 18.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076812e+00 | Ndiff 0.002 MERGE @ lap 19.00: No promising candidates, so no attempts. DELETE @ lap 19.00: Disabled. Waiting for lap >= 20 (--d_startLap). 19.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 DELETE @ lap 20.00: 0/1 accepted. Ndiff 0.00. MERGE @ lap 20.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 20.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 21.00: No promising candidates, so no attempts. DELETE @ lap 21.00: 0/1 accepted. Ndiff 0.00. 21.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 22.00: No promising candidates, so no attempts. DELETE @ lap 22.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 22.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 23.00: No promising candidates, so no attempts. DELETE @ lap 23.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 23.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 24.00: No promising candidates, so no attempts. DELETE @ lap 24.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 24.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 DELETE @ lap 25.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. MERGE @ lap 25.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 25.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 26.00: No promising candidates, so no attempts. DELETE @ lap 26.00: 0/1 accepted. Ndiff 0.00. 26.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 27.00: No promising candidates, so no attempts. DELETE @ lap 27.00: 0/1 accepted. Ndiff 0.00. 27.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 28.00: No promising candidates, so no attempts. DELETE @ lap 28.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 28.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 29.00: No promising candidates, so no attempts. DELETE @ lap 29.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 29.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 DELETE @ lap 30.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. MERGE @ lap 30.00 : 0/1 accepted. Ndiff 0.00. 0 skipped. 30.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 MERGE @ lap 31.00: No promising candidates, so no attempts. DELETE @ lap 31.00: Empty plan. 0 UIDs eligible as delete target. 0 too busy with other moves. 0 too big. 2 have past failures. 31.000/100 after 1 sec. | 218.2 MiB | K 2 | loss 2.268076811e+00 | Ndiff 0.000 ... done. converged. .. GENERATED FROM PYTHON SOURCE LINES 156-159 Loss function trace plot ------------------------ .. GENERATED FROM PYTHON SOURCE LINES 160-177 .. code-block:: default pylab.figure() pylab.plot( diag1_info_dict['lap_history'][2:], diag1_info_dict['loss_history'][2:], 'r.-', label='diag_covar fixed') pylab.plot( diag_info_dict['lap_history'][2:], diag_info_dict['loss_history'][2:], 'k.-', label='diag_covar + moves') pylab.plot( full_info_dict['lap_history'][2:], full_info_dict['loss_history'][2:], 'b.-', label='full_covar + moves') pylab.legend(loc='upper right') pylab.xlabel('num. laps') pylab.ylabel('loss') pylab.tight_layout() .. image-sg:: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_005.png :alt: plot 02 demo=merge and delete model=dp mix+gauss :srcset: /examples/03_faithful/images/sphx_glr_plot-02-demo=merge_and_delete-model=dp_mix+gauss_005.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 5.473 seconds) .. _sphx_glr_download_examples_03_faithful_plot-02-demo=merge_and_delete-model=dp_mix+gauss.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot-02-demo=merge_and_delete-model=dp_mix+gauss.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot-02-demo=merge_and_delete-model=dp_mix+gauss.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_