Demonstration of Sparse Responsibilities for Mixtures of Gaussians

This demo illustrates “sparse responsibilities” for mixture models.

In this example, we show how BNPy makes it easy to vary per-example inference, so that for a 3-cluster model we can manually try each of the following:

  • each data point assigned to up to 1 cluster
  • each data point assigned to up to 2 clusters
  • each data point assigned to up to 3 clusters

In our variational approximation, when we fit a K-cluster mixture model to a specific data point, we learn a vector of posterior responsibilities – we often call these “resp” for short in the code – indicating how much of the unit probability mass for this example is explained by each of the K clusters.

For example, for a specific data point we might have a resp vector of:

[ 0. 0.9262 0.0738]

which we interpret as saying this point is 92% explained by the 2nd cluster, 7% by the 3rd cluster, and 0% by the first.

This short demo just shows how we can deliberately enforce sparsity in learned responsibilities, so that each example may have only L non-zero entries in its resp vector. For example, we’ll fit a K=3 model below, and show how we can enforce L=1 or L=2 sparsity.

For a technical introduction, see our ArXiv paper:

Michael C. Hughes and Erik B. Sudderth (2016) “Fast Learning of Clusters and Topics via Sparse Posteriors.” https://arxiv.org/abs/1609.07521
from __future__ import print_function

import bnpy
import numpy as np
import os

from matplotlib import pylab
import seaborn as sns

SMALL_FIG_SIZE = (2.5, 2.5)
FIG_SIZE = (5, 5)
pylab.rcParams['figure.figsize'] = FIG_SIZE

Load dataset from file

dataset_path = os.path.join(bnpy.DATASET_PATH, 'faithful')
dataset = bnpy.data.XData.read_csv(
    os.path.join(dataset_path, 'faithful.csv'))

# Identify "target" example ids to focus on
target_example_ids = np.asarray(
    [ 83,  46, 152, 173,  23, 164, 243, 239, 154, 100])
targeted_dataset = bnpy.data.XData(
    dataset.X[target_example_ids])

Make a simple plot of the raw data

pylab.plot(dataset.X[:, 0], dataset.X[:, 1], 'k.')
pylab.xlabel(dataset.column_names[0])
pylab.ylabel(dataset.column_names[1])
pylab.tight_layout()
pylab.title('Complete Dataset')
data_ax_h = pylab.gca()
../../_images/sphx_glr_plot-03-demo=sparse_resp_vb-model=mix_gauss_001.png

Setup: Func to print resp array nicely

def pprint_sparse_or_dense_resp(LP, target_example_ids=target_example_ids):
    np.set_printoptions(suppress=True, precision=4)
    if 'spR' in LP:
        # sparse case
        resp_NK = LP['spR'][target_example_ids].toarray()
    else:
        resp_NK = LP['resp'][target_example_ids]
    print(resp_NK)

VB with DP mixture model and diagonal Gaussian observations

Assumes exactly 3 clusters

Assumes diagonal covariances.

No sparsity assumptions during training

K = 3          # n clusters
gamma = 50.0   # DP concentration param
sF = 0.1       # scale of expected covariance

full_trained_model, full_info_dict = bnpy.run(
    dataset, 'DPMixtureModel', 'DiagGauss', 'VB',
    output_path='/tmp/faithful/demo_sparse_resp-K=3-lik=Gauss-ECovMat=5*eye/',
    nLap=1000, nTask=5, nBatch=1, convergeThr=0.0001,
    gamma0=gamma, sF=sF, ECovMat='eye',
    K=K, initname='randexamples',
    )

# Add this model into the current plot
bnpy.viz.PlotComps.plotCompsFromHModel(
    full_trained_model,
    )
../../_images/sphx_glr_plot-03-demo=sparse_resp_vb-model=mix_gauss_002.png

Out:

WARNING: Found unrecognized keyword args. These are ignored.
  --nBatch
Dataset Summary:
X Data
  num examples: 272
  num dims: 2
Allocation Model:  DP mixture with K=0. Concentration gamma0= 50.00
Obs. Data  Model:  Gaussian with diagonal covariance.
Obs. Data  Prior:  independent Gauss-Wishart prior on each dimension
  Wishart params
    nu = 4
  beta = [ 0.2  0.2]
  Expectations
  E[  mean[k]] =
  [ 0  0]
  E[ covar[k]] =
  [[0.1 0. ]
   [0.  0.1]]
Initialization:
  initname = randexamples
  K = 3 (number of clusters)
  seed = 1607680
  elapsed_time: 0.0 sec
Learn Alg: VB | task  1/5 | alg. seed: 1607680 | data order seed: 8541952
task_output_path: /tmp/faithful/demo_sparse_resp-K=3-lik=Gauss-ECovMat=5*eye/1
        1/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.024985920e+00 |
        2/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.963751576e+00 | Ndiff    2.846
        3/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.915936430e+00 | Ndiff   10.454
        4/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.841297326e+00 | Ndiff   18.204
        5/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.748858710e+00 | Ndiff   20.529
        6/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.657778518e+00 | Ndiff   15.677
        7/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.569542173e+00 | Ndiff   10.904
        8/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.534007328e+00 | Ndiff    4.528
        9/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.508961472e+00 | Ndiff    3.729
       10/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495535777e+00 | Ndiff    1.586
       11/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494290057e+00 | Ndiff    0.411
       12/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.493584130e+00 | Ndiff    0.696
       13/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.492995794e+00 | Ndiff    0.929
       14/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.492459487e+00 | Ndiff    1.135
       15/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.491945636e+00 | Ndiff    1.318
       16/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.491440653e+00 | Ndiff    1.481
       17/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.490939573e+00 | Ndiff    1.625
       18/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.490442396e+00 | Ndiff    1.749
       19/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.489951832e+00 | Ndiff    1.852
       20/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.489471711e+00 | Ndiff    1.933
       21/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.489005811e+00 | Ndiff    1.993
       22/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.488557093e+00 | Ndiff    2.033
       23/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.488127311e+00 | Ndiff    2.055
       24/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.487716959e+00 | Ndiff    2.063
       25/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.487325464e+00 | Ndiff    2.059
       26/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.486951484e+00 | Ndiff    2.047
       27/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.486593250e+00 | Ndiff    2.028
       28/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.486248844e+00 | Ndiff    2.006
       29/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485916412e+00 | Ndiff    1.981
       30/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485594294e+00 | Ndiff    1.954
       31/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485281079e+00 | Ndiff    1.927
       32/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484975632e+00 | Ndiff    1.900
       33/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484677073e+00 | Ndiff    1.873
       34/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484384755e+00 | Ndiff    1.845
       35/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484098229e+00 | Ndiff    1.817
       36/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483817216e+00 | Ndiff    1.789
       37/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483541575e+00 | Ndiff    1.760
       38/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483271282e+00 | Ndiff    1.731
       39/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483006407e+00 | Ndiff    1.701
       40/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482747096e+00 | Ndiff    1.669
       41/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482493559e+00 | Ndiff    1.637
       42/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482246056e+00 | Ndiff    1.604
       43/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482004886e+00 | Ndiff    1.569
       44/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481770375e+00 | Ndiff    1.533
       45/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481542863e+00 | Ndiff    1.496
       46/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481322699e+00 | Ndiff    1.458
       47/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481110224e+00 | Ndiff    1.419
       48/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480905765e+00 | Ndiff    1.379
       49/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480709619e+00 | Ndiff    1.338
       50/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480522049e+00 | Ndiff    1.296
       51/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480343269e+00 | Ndiff    1.254
       52/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480173442e+00 | Ndiff    1.211
       53/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480012666e+00 | Ndiff    1.167
       54/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479860980e+00 | Ndiff    1.124
       55/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479718351e+00 | Ndiff    1.081
       56/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479584685e+00 | Ndiff    1.038
       57/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479459820e+00 | Ndiff    0.995
       58/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479343538e+00 | Ndiff    0.953
       59/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479235566e+00 | Ndiff    0.912
       60/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479135586e+00 | Ndiff    0.871
       61/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479043241e+00 | Ndiff    0.832
       62/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478958148e+00 | Ndiff    0.793
       63/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478879901e+00 | Ndiff    0.756
       64/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478808084e+00 | Ndiff    0.720
       65/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478742273e+00 | Ndiff    0.686
       66/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478682051e+00 | Ndiff    0.653
       67/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478627005e+00 | Ndiff    0.621
       68/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478576736e+00 | Ndiff    0.591
       69/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478530861e+00 | Ndiff    0.562
       70/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478489016e+00 | Ndiff    0.534
       71/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478450858e+00 | Ndiff    0.508
       72/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478416065e+00 | Ndiff    0.483
       73/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478384339e+00 | Ndiff    0.460
       74/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478355403e+00 | Ndiff    0.438
       75/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478329003e+00 | Ndiff    0.417
       76/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478304903e+00 | Ndiff    0.397
       77/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478282892e+00 | Ndiff    0.378
       78/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478262773e+00 | Ndiff    0.361
       79/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478244370e+00 | Ndiff    0.344
       80/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478227522e+00 | Ndiff    0.328
       81/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478212083e+00 | Ndiff    0.313
       82/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478197921e+00 | Ndiff    0.299
       83/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478184918e+00 | Ndiff    0.286
       84/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478172966e+00 | Ndiff    0.274
       85/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478161968e+00 | Ndiff    0.262
       86/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478151836e+00 | Ndiff    0.251
       87/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478142491e+00 | Ndiff    0.241
       88/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478133862e+00 | Ndiff    0.231
       89/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478125885e+00 | Ndiff    0.221
       90/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478118502e+00 | Ndiff    0.213
       91/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478111660e+00 | Ndiff    0.204
       92/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478105312e+00 | Ndiff    0.196
       93/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478099416e+00 | Ndiff    0.189
       94/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478093933e+00 | Ndiff    0.182
       95/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478088827e+00 | Ndiff    0.175
       96/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478084067e+00 | Ndiff    0.169
       97/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478079625e+00 | Ndiff    0.163
       98/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478075475e+00 | Ndiff    0.157
       99/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478071593e+00 | Ndiff    0.152
      100/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478067957e+00 | Ndiff    0.147
      101/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478064548e+00 | Ndiff    0.142
      102/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478061348e+00 | Ndiff    0.137
      103/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478058342e+00 | Ndiff    0.133
      104/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478055514e+00 | Ndiff    0.129
      105/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478052851e+00 | Ndiff    0.125
      106/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478050340e+00 | Ndiff    0.121
      107/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478047971e+00 | Ndiff    0.118
      108/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478045734e+00 | Ndiff    0.114
      109/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478043618e+00 | Ndiff    0.111
      110/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478041615e+00 | Ndiff    0.108
      111/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478039717e+00 | Ndiff    0.105
      112/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478037917e+00 | Ndiff    0.102
      113/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478036209e+00 | Ndiff    0.099
      114/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478034585e+00 | Ndiff    0.097
      115/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478033041e+00 | Ndiff    0.094
      116/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478031571e+00 | Ndiff    0.092
      117/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478030171e+00 | Ndiff    0.089
      118/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478028835e+00 | Ndiff    0.087
      119/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478027560e+00 | Ndiff    0.085
      120/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478026342e+00 | Ndiff    0.083
      121/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478025178e+00 | Ndiff    0.081
      122/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478024063e+00 | Ndiff    0.079
      123/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478022996e+00 | Ndiff    0.078
      124/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478021972e+00 | Ndiff    0.076
      125/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478020991e+00 | Ndiff    0.074
      126/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478020048e+00 | Ndiff    0.073
      127/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478019142e+00 | Ndiff    0.071
      128/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478018271e+00 | Ndiff    0.070
      129/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478017433e+00 | Ndiff    0.069
      130/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478016626e+00 | Ndiff    0.067
      131/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478015848e+00 | Ndiff    0.066
      132/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478015097e+00 | Ndiff    0.065
      133/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478014373e+00 | Ndiff    0.064
      134/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478013673e+00 | Ndiff    0.062
      135/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478012996e+00 | Ndiff    0.061
      136/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478012342e+00 | Ndiff    0.060
      137/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.478011708e+00 | Ndiff    0.059
      138/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478011095e+00 | Ndiff    0.058
      139/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478010500e+00 | Ndiff    0.057
      140/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478009923e+00 | Ndiff    0.056
      141/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478009363e+00 | Ndiff    0.056
      142/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478008818e+00 | Ndiff    0.055
      143/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478008290e+00 | Ndiff    0.054
      144/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478007775e+00 | Ndiff    0.053
      145/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478007275e+00 | Ndiff    0.052
      146/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478006787e+00 | Ndiff    0.052
      147/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478006312e+00 | Ndiff    0.051
      148/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478005849e+00 | Ndiff    0.050
      149/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478005396e+00 | Ndiff    0.050
      150/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478004955e+00 | Ndiff    0.049
      151/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478004524e+00 | Ndiff    0.048
      152/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478004102e+00 | Ndiff    0.048
      153/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478003690e+00 | Ndiff    0.047
      154/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478003287e+00 | Ndiff    0.047
      155/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478002891e+00 | Ndiff    0.046
      156/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478002504e+00 | Ndiff    0.046
      157/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478002125e+00 | Ndiff    0.045
      158/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478001752e+00 | Ndiff    0.045
      159/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478001387e+00 | Ndiff    0.044
      160/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478001027e+00 | Ndiff    0.044
      161/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478000675e+00 | Ndiff    0.044
      162/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.478000328e+00 | Ndiff    0.043
      163/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477999986e+00 | Ndiff    0.043
      164/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477999650e+00 | Ndiff    0.042
      165/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477999319e+00 | Ndiff    0.042
      166/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477998993e+00 | Ndiff    0.042
      167/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477998671e+00 | Ndiff    0.042
      168/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477998353e+00 | Ndiff    0.041
      169/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477998040e+00 | Ndiff    0.041
      170/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477997731e+00 | Ndiff    0.041
      171/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477997425e+00 | Ndiff    0.040
      172/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477997122e+00 | Ndiff    0.040
      173/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477996823e+00 | Ndiff    0.040
      174/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477996526e+00 | Ndiff    0.040
      175/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477996233e+00 | Ndiff    0.039
      176/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477995942e+00 | Ndiff    0.039
      177/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477995654e+00 | Ndiff    0.039
      178/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477995368e+00 | Ndiff    0.039
      179/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477995084e+00 | Ndiff    0.039
      180/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477994802e+00 | Ndiff    0.039
      181/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477994521e+00 | Ndiff    0.038
      182/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477994243e+00 | Ndiff    0.038
      183/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477993966e+00 | Ndiff    0.038
      184/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477993690e+00 | Ndiff    0.038
      185/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477993415e+00 | Ndiff    0.038
      186/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477993141e+00 | Ndiff    0.038
      187/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477992868e+00 | Ndiff    0.038
      188/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477992596e+00 | Ndiff    0.038
      189/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477992324e+00 | Ndiff    0.038
      190/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477992053e+00 | Ndiff    0.038
      191/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477991782e+00 | Ndiff    0.038
      192/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477991511e+00 | Ndiff    0.038
      193/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477991239e+00 | Ndiff    0.038
      194/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477990968e+00 | Ndiff    0.038
      195/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477990696e+00 | Ndiff    0.038
      196/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477990424e+00 | Ndiff    0.038
      197/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477990151e+00 | Ndiff    0.038
      198/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477989878e+00 | Ndiff    0.038
      199/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477989603e+00 | Ndiff    0.038
      200/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477989327e+00 | Ndiff    0.038
      201/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477989050e+00 | Ndiff    0.038
      202/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477988772e+00 | Ndiff    0.038
      203/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477988492e+00 | Ndiff    0.038
      204/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477988210e+00 | Ndiff    0.038
      205/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477987926e+00 | Ndiff    0.038
      206/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477987640e+00 | Ndiff    0.038
      207/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477987351e+00 | Ndiff    0.039
      208/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477987061e+00 | Ndiff    0.039
      209/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477986767e+00 | Ndiff    0.039
      210/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477986470e+00 | Ndiff    0.039
      211/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477986171e+00 | Ndiff    0.039
      212/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477985868e+00 | Ndiff    0.039
      213/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477985561e+00 | Ndiff    0.040
      214/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477985251e+00 | Ndiff    0.040
      215/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477984936e+00 | Ndiff    0.040
      216/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477984618e+00 | Ndiff    0.040
      217/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477984294e+00 | Ndiff    0.041
      218/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477983966e+00 | Ndiff    0.041
      219/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477983633e+00 | Ndiff    0.041
      220/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477983294e+00 | Ndiff    0.041
      221/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477982950e+00 | Ndiff    0.042
      222/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477982600e+00 | Ndiff    0.042
      223/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477982243e+00 | Ndiff    0.042
      224/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477981879e+00 | Ndiff    0.043
      225/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477981508e+00 | Ndiff    0.043
      226/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477981130e+00 | Ndiff    0.044
      227/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477980744e+00 | Ndiff    0.044
      228/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477980349e+00 | Ndiff    0.044
      229/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477979945e+00 | Ndiff    0.045
      230/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477979532e+00 | Ndiff    0.045
      231/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477979109e+00 | Ndiff    0.046
      232/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477978676e+00 | Ndiff    0.046
      233/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477978231e+00 | Ndiff    0.047
      234/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477977775e+00 | Ndiff    0.048
      235/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477977306e+00 | Ndiff    0.048
      236/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477976824e+00 | Ndiff    0.049
      237/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477976329e+00 | Ndiff    0.050
      238/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477975818e+00 | Ndiff    0.050
      239/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477975292e+00 | Ndiff    0.051
      240/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477974749e+00 | Ndiff    0.052
      241/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477974189e+00 | Ndiff    0.052
      242/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477973610e+00 | Ndiff    0.053
      243/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477973011e+00 | Ndiff    0.054
      244/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477972391e+00 | Ndiff    0.055
      245/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477971749e+00 | Ndiff    0.056
      246/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477971083e+00 | Ndiff    0.057
      247/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477970391e+00 | Ndiff    0.058
      248/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477969672e+00 | Ndiff    0.059
      249/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477968924e+00 | Ndiff    0.060
      250/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477968146e+00 | Ndiff    0.061
      251/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477967334e+00 | Ndiff    0.063
      252/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477966486e+00 | Ndiff    0.064
      253/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477965601e+00 | Ndiff    0.065
      254/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477964675e+00 | Ndiff    0.067
      255/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477963705e+00 | Ndiff    0.068
      256/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477962687e+00 | Ndiff    0.070
      257/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477961619e+00 | Ndiff    0.071
      258/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477960497e+00 | Ndiff    0.073
      259/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477959315e+00 | Ndiff    0.075
      260/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477958069e+00 | Ndiff    0.077
      261/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477956753e+00 | Ndiff    0.079
      262/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477955363e+00 | Ndiff    0.081
      263/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477953891e+00 | Ndiff    0.083
      264/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477952330e+00 | Ndiff    0.085
      265/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477950671e+00 | Ndiff    0.088
      266/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477948907e+00 | Ndiff    0.090
      267/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477947028e+00 | Ndiff    0.093
      268/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477945021e+00 | Ndiff    0.096
      269/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477942874e+00 | Ndiff    0.099
      270/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477940573e+00 | Ndiff    0.102
      271/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477938103e+00 | Ndiff    0.106
      272/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477935445e+00 | Ndiff    0.110
      273/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477932578e+00 | Ndiff    0.114
      274/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477929479e+00 | Ndiff    0.118
      275/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477926122e+00 | Ndiff    0.122
      276/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477922476e+00 | Ndiff    0.127
      277/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477918505e+00 | Ndiff    0.132
      278/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477914169e+00 | Ndiff    0.138
      279/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477909421e+00 | Ndiff    0.144
      280/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477904205e+00 | Ndiff    0.150
      281/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477898458e+00 | Ndiff    0.157
      282/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477892103e+00 | Ndiff    0.164
      283/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477885051e+00 | Ndiff    0.172
      284/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477877196e+00 | Ndiff    0.181
      285/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477868411e+00 | Ndiff    0.191
      286/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477858544e+00 | Ndiff    0.201
      287/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477847411e+00 | Ndiff    0.212
      288/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477834786e+00 | Ndiff    0.225
      289/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477820395e+00 | Ndiff    0.238
      290/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477803898e+00 | Ndiff    0.253
      291/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477784870e+00 | Ndiff    0.270
      292/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477762778e+00 | Ndiff    0.288
      293/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477736945e+00 | Ndiff    0.308
      294/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477706505e+00 | Ndiff    0.331
      295/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477670330e+00 | Ndiff    0.356
      296/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477626947e+00 | Ndiff    0.384
      297/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477574395e+00 | Ndiff    0.416
      298/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477510040e+00 | Ndiff    0.452
      299/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477430283e+00 | Ndiff    0.492
      300/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477330143e+00 | Ndiff    0.537
      301/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477202613e+00 | Ndiff    0.588
      302/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.477037673e+00 | Ndiff    0.646
      303/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.476820794e+00 | Ndiff    0.710
      304/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.476530732e+00 | Ndiff    0.780
      305/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.476136866e+00 | Ndiff    0.855
      306/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.475598527e+00 | Ndiff    0.929
      307/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.474876012e+00 | Ndiff    0.989
      308/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.473971580e+00 | Ndiff    1.018
      309/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.472985078e+00 | Ndiff    0.999
      310/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.472086685e+00 | Ndiff    0.935
      311/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.471397849e+00 | Ndiff    0.841
      312/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.470920902e+00 | Ndiff    0.742
      313/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.470584489e+00 | Ndiff    0.653
      314/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.470324872e+00 | Ndiff    0.580
      315/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.470106989e+00 | Ndiff    0.523
      316/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.469911918e+00 | Ndiff    0.481
      317/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.469726780e+00 | Ndiff    0.453
      318/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.469539898e+00 | Ndiff    0.437
      319/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.469337523e+00 | Ndiff    0.434
      320/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.469099855e+00 | Ndiff    0.443
      321/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.468794891e+00 | Ndiff    0.463
      322/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.468371096e+00 | Ndiff    0.492
      323/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.467763504e+00 | Ndiff    0.513
      324/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.466960962e+00 | Ndiff    0.496
      325/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.466160474e+00 | Ndiff    0.400
      326/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465724445e+00 | Ndiff    0.232
      327/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465626565e+00 | Ndiff    0.094
      328/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465615098e+00 | Ndiff    0.031
      329/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465613516e+00 | Ndiff    0.014
      330/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465613125e+00 | Ndiff    0.008
      331/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612997e+00 | Ndiff    0.005
      332/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612952e+00 | Ndiff    0.003
      333/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612936e+00 | Ndiff    0.002
      334/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612930e+00 | Ndiff    0.001
      335/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612928e+00 | Ndiff    0.001
      336/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
      337/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
      338/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
      339/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  2/5 | alg. seed: 6454144 | data order seed: 7673856
task_output_path: /tmp/faithful/demo_sparse_resp-K=3-lik=Gauss-ECovMat=5*eye/2
        1/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.144280649e+00 |
        2/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.999427978e+00 | Ndiff   16.339
        3/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.889092617e+00 | Ndiff   29.328
        4/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.773372571e+00 | Ndiff   30.295
        5/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.670818147e+00 | Ndiff   19.860
        6/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.577419094e+00 | Ndiff   11.388
        7/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.531167799e+00 | Ndiff    4.469
        8/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495532492e+00 | Ndiff    3.497
        9/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485361099e+00 | Ndiff    1.346
       10/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484760839e+00 | Ndiff    1.534
       11/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484517089e+00 | Ndiff    1.442
       12/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484320192e+00 | Ndiff    1.384
       13/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484140099e+00 | Ndiff    1.351
       14/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483970718e+00 | Ndiff    1.326
       15/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483809360e+00 | Ndiff    1.305
       16/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483654299e+00 | Ndiff    1.286
       17/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483504339e+00 | Ndiff    1.267
       18/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483358650e+00 | Ndiff    1.248
       19/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483216669e+00 | Ndiff    1.230
       20/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483078037e+00 | Ndiff    1.211
       21/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482942536e+00 | Ndiff    1.192
       22/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482810056e+00 | Ndiff    1.173
       23/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482680562e+00 | Ndiff    1.153
       24/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482554074e+00 | Ndiff    1.133
       25/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482430645e+00 | Ndiff    1.113
       26/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482310357e+00 | Ndiff    1.092
       27/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482193304e+00 | Ndiff    1.070
       28/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482079589e+00 | Ndiff    1.048
       29/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481969318e+00 | Ndiff    1.025
       30/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481862594e+00 | Ndiff    1.002
       31/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481759514e+00 | Ndiff    0.978
       32/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481660166e+00 | Ndiff    0.954
       33/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481564628e+00 | Ndiff    0.929
       34/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481472961e+00 | Ndiff    0.904
       35/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481385213e+00 | Ndiff    0.879
       36/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481301412e+00 | Ndiff    0.853
       37/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481221570e+00 | Ndiff    0.828
       38/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481145680e+00 | Ndiff    0.802
       39/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481073718e+00 | Ndiff    0.777
       40/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481005638e+00 | Ndiff    0.751
       41/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480941380e+00 | Ndiff    0.726
       42/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480880866e+00 | Ndiff    0.700
       43/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480824003e+00 | Ndiff    0.675
       44/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480770686e+00 | Ndiff    0.651
       45/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480720795e+00 | Ndiff    0.626
       46/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480674203e+00 | Ndiff    0.602
       47/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480630775e+00 | Ndiff    0.579
       48/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480590367e+00 | Ndiff    0.556
       49/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480552834e+00 | Ndiff    0.533
       50/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480518027e+00 | Ndiff    0.512
       51/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480485796e+00 | Ndiff    0.490
       52/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480455992e+00 | Ndiff    0.470
       53/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480428469e+00 | Ndiff    0.450
       54/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480403082e+00 | Ndiff    0.431
       55/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480379690e+00 | Ndiff    0.412
       56/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480358160e+00 | Ndiff    0.394
       57/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480338360e+00 | Ndiff    0.377
       58/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480320167e+00 | Ndiff    0.360
       59/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480303462e+00 | Ndiff    0.344
       60/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480288134e+00 | Ndiff    0.329
       61/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480274078e+00 | Ndiff    0.314
       62/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480261194e+00 | Ndiff    0.300
       63/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480249391e+00 | Ndiff    0.287
       64/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480238582e+00 | Ndiff    0.274
       65/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480228686e+00 | Ndiff    0.261
       66/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480219629e+00 | Ndiff    0.250
       67/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480211341e+00 | Ndiff    0.238
       68/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480203759e+00 | Ndiff    0.227
       69/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480196823e+00 | Ndiff    0.217
       70/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480190480e+00 | Ndiff    0.207
       71/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480184678e+00 | Ndiff    0.198
       72/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480179372e+00 | Ndiff    0.189
       73/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480174520e+00 | Ndiff    0.181
       74/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480170083e+00 | Ndiff    0.172
       75/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480166025e+00 | Ndiff    0.165
       76/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480162313e+00 | Ndiff    0.157
       77/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480158918e+00 | Ndiff    0.150
       78/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480155813e+00 | Ndiff    0.144
       79/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480152973e+00 | Ndiff    0.137
       80/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480150374e+00 | Ndiff    0.131
       81/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480147996e+00 | Ndiff    0.125
       82/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480145820e+00 | Ndiff    0.120
       83/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480143828e+00 | Ndiff    0.114
       84/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480142005e+00 | Ndiff    0.109
       85/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480140336e+00 | Ndiff    0.105
       86/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480138808e+00 | Ndiff    0.100
       87/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480137409e+00 | Ndiff    0.096
       88/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480136127e+00 | Ndiff    0.091
       89/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480134953e+00 | Ndiff    0.087
       90/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480133878e+00 | Ndiff    0.084
       91/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480132893e+00 | Ndiff    0.080
       92/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480131990e+00 | Ndiff    0.077
       93/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480131162e+00 | Ndiff    0.073
       94/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480130403e+00 | Ndiff    0.070
       95/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480129708e+00 | Ndiff    0.067
       96/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480129070e+00 | Ndiff    0.064
       97/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480128485e+00 | Ndiff    0.061
       98/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480127949e+00 | Ndiff    0.059
       99/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480127457e+00 | Ndiff    0.056
      100/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480127005e+00 | Ndiff    0.054
      101/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480126591e+00 | Ndiff    0.052
      102/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480126211e+00 | Ndiff    0.049
      103/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480125862e+00 | Ndiff    0.047
      104/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480125542e+00 | Ndiff    0.045
      105/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480125248e+00 | Ndiff    0.043
      106/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480124978e+00 | Ndiff    0.042
      107/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480124731e+00 | Ndiff    0.040
      108/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480124503e+00 | Ndiff    0.038
      109/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480124294e+00 | Ndiff    0.037
      110/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480124102e+00 | Ndiff    0.035
      111/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123926e+00 | Ndiff    0.034
      112/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123764e+00 | Ndiff    0.032
      113/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123615e+00 | Ndiff    0.031
      114/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123478e+00 | Ndiff    0.030
      115/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123353e+00 | Ndiff    0.028
      116/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123237e+00 | Ndiff    0.027
      117/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123131e+00 | Ndiff    0.026
      118/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480123034e+00 | Ndiff    0.025
      119/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122944e+00 | Ndiff    0.024
      120/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122862e+00 | Ndiff    0.023
      121/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122786e+00 | Ndiff    0.022
      122/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122716e+00 | Ndiff    0.021
      123/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122652e+00 | Ndiff    0.020
      124/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122593e+00 | Ndiff    0.019
      125/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122539e+00 | Ndiff    0.019
      126/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122489e+00 | Ndiff    0.018
      127/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122444e+00 | Ndiff    0.017
      128/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122401e+00 | Ndiff    0.016
      129/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122363e+00 | Ndiff    0.016
      130/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122327e+00 | Ndiff    0.015
      131/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122294e+00 | Ndiff    0.014
      132/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122264e+00 | Ndiff    0.014
      133/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122236e+00 | Ndiff    0.013
      134/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122211e+00 | Ndiff    0.013
      135/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122187e+00 | Ndiff    0.012
      136/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122166e+00 | Ndiff    0.012
      137/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122146e+00 | Ndiff    0.011
      138/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480122128e+00 | Ndiff    0.011
      139/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122111e+00 | Ndiff    0.010
      140/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122095e+00 | Ndiff    0.010
      141/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122081e+00 | Ndiff    0.009
      142/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122068e+00 | Ndiff    0.009
      143/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122056e+00 | Ndiff    0.009
      144/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122045e+00 | Ndiff    0.008
      145/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122034e+00 | Ndiff    0.008
      146/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122025e+00 | Ndiff    0.008
      147/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122016e+00 | Ndiff    0.007
      148/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122008e+00 | Ndiff    0.007
      149/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480122001e+00 | Ndiff    0.007
      150/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121994e+00 | Ndiff    0.007
      151/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121988e+00 | Ndiff    0.006
      152/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121982e+00 | Ndiff    0.006
      153/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121977e+00 | Ndiff    0.006
      154/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121972e+00 | Ndiff    0.006
      155/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121968e+00 | Ndiff    0.005
      156/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121964e+00 | Ndiff    0.005
      157/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121960e+00 | Ndiff    0.005
      158/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121956e+00 | Ndiff    0.005
      159/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121953e+00 | Ndiff    0.004
      160/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121950e+00 | Ndiff    0.004
      161/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121948e+00 | Ndiff    0.004
      162/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121945e+00 | Ndiff    0.004
      163/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121943e+00 | Ndiff    0.004
      164/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121941e+00 | Ndiff    0.004
      165/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121939e+00 | Ndiff    0.004
      166/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121937e+00 | Ndiff    0.003
      167/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121935e+00 | Ndiff    0.003
      168/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121934e+00 | Ndiff    0.003
      169/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121932e+00 | Ndiff    0.003
      170/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121931e+00 | Ndiff    0.003
      171/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121930e+00 | Ndiff    0.003
      172/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121929e+00 | Ndiff    0.003
      173/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121928e+00 | Ndiff    0.003
      174/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121927e+00 | Ndiff    0.002
      175/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121926e+00 | Ndiff    0.002
      176/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121925e+00 | Ndiff    0.002
      177/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121924e+00 | Ndiff    0.002
      178/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121924e+00 | Ndiff    0.002
      179/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121923e+00 | Ndiff    0.002
      180/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121923e+00 | Ndiff    0.002
      181/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121922e+00 | Ndiff    0.002
      182/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121922e+00 | Ndiff    0.002
      183/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121921e+00 | Ndiff    0.002
      184/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121921e+00 | Ndiff    0.002
      185/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121920e+00 | Ndiff    0.002
      186/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121920e+00 | Ndiff    0.001
      187/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121920e+00 | Ndiff    0.001
      188/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
      189/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
      190/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
      191/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
      192/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
      193/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
      194/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
      195/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
      196/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
      197/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
      198/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      199/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      200/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      201/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      202/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      203/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      204/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      205/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      206/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      207/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      208/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      209/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
      210/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.001
      211/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.001
      212/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.001
      213/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      214/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      215/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      216/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      217/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      218/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      219/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      220/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      221/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      222/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      223/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      224/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      225/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      226/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      227/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      228/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      229/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      230/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      231/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      232/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      233/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      234/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      235/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      236/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      237/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      238/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      239/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      240/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      241/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      242/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      243/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      244/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      245/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      246/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      247/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      248/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      249/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      250/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      251/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
      252/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  3/5 | alg. seed: 6168832 | data order seed: 7360256
task_output_path: /tmp/faithful/demo_sparse_resp-K=3-lik=Gauss-ECovMat=5*eye/3
        1/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.597890645e+00 |
        2/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.500334722e+00 | Ndiff   12.103
        3/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481732540e+00 | Ndiff    6.728
        4/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.477128325e+00 | Ndiff    3.822
        5/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.475214582e+00 | Ndiff    2.612
        6/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.474025229e+00 | Ndiff    1.975
        7/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.473158433e+00 | Ndiff    1.625
        8/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.472487721e+00 | Ndiff    1.387
        9/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.471960617e+00 | Ndiff    1.189
       10/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.471546906e+00 | Ndiff    1.019
       11/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.471223928e+00 | Ndiff    0.873
       12/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470972980e+00 | Ndiff    0.748
       13/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470778682e+00 | Ndiff    0.642
       14/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470628726e+00 | Ndiff    0.551
       15/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470513450e+00 | Ndiff    0.473
       16/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470425312e+00 | Ndiff    0.405
       17/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470358389e+00 | Ndiff    0.347
       18/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470307985e+00 | Ndiff    0.297
       19/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470270346e+00 | Ndiff    0.253
       20/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470242474e+00 | Ndiff    0.215
       21/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470221992e+00 | Ndiff    0.183
       22/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470207040e+00 | Ndiff    0.155
       23/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470196186e+00 | Ndiff    0.131
       24/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470188343e+00 | Ndiff    0.111
       25/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470182696e+00 | Ndiff    0.094
       26/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470178643e+00 | Ndiff    0.079
       27/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470175740e+00 | Ndiff    0.067
       28/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470173665e+00 | Ndiff    0.056
       29/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470172185e+00 | Ndiff    0.047
       30/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470171130e+00 | Ndiff    0.040
       31/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470170380e+00 | Ndiff    0.033
       32/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470169846e+00 | Ndiff    0.028
       33/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470169466e+00 | Ndiff    0.024
       34/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470169197e+00 | Ndiff    0.020
       35/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470169005e+00 | Ndiff    0.017
       36/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168870e+00 | Ndiff    0.014
       37/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168773e+00 | Ndiff    0.012
       38/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168705e+00 | Ndiff    0.010
       39/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168657e+00 | Ndiff    0.008
       40/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168622e+00 | Ndiff    0.007
       41/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168598e+00 | Ndiff    0.006
       42/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168581e+00 | Ndiff    0.005
       43/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168569e+00 | Ndiff    0.004
       44/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168560e+00 | Ndiff    0.004
       45/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168554e+00 | Ndiff    0.003
       46/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168550e+00 | Ndiff    0.003
       47/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168547e+00 | Ndiff    0.002
       48/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168544e+00 | Ndiff    0.002
       49/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168543e+00 | Ndiff    0.001
       50/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168542e+00 | Ndiff    0.001
       51/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168541e+00 | Ndiff    0.001
       52/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
       53/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
       54/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
       55/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
       56/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       57/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       58/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       59/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       60/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       61/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       62/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       63/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       64/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
       65/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  4/5 | alg. seed: 8205184 | data order seed: 900864
task_output_path: /tmp/faithful/demo_sparse_resp-K=3-lik=Gauss-ECovMat=5*eye/4
        1/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.625818566e+00 |
        2/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.517079400e+00 | Ndiff    2.360
        3/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.500595953e+00 | Ndiff    2.568
        4/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.496654963e+00 | Ndiff    0.840
        5/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494337582e+00 | Ndiff    0.186
        6/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.492886037e+00 | Ndiff    0.280
        7/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.491894017e+00 | Ndiff    0.615
        8/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.491133552e+00 | Ndiff    0.875
        9/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.490496784e+00 | Ndiff    1.087
       10/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.489933426e+00 | Ndiff    1.260
       11/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.489419675e+00 | Ndiff    1.401
       12/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.488944406e+00 | Ndiff    1.512
       13/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.488502591e+00 | Ndiff    1.597
       14/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.488091868e+00 | Ndiff    1.656
       15/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.487710702e+00 | Ndiff    1.695
       16/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.487357459e+00 | Ndiff    1.716
       17/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.487030061e+00 | Ndiff    1.722
       18/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.486725982e+00 | Ndiff    1.718
       19/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.486442414e+00 | Ndiff    1.706
       20/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.486176489e+00 | Ndiff    1.689
       21/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485925474e+00 | Ndiff    1.668
       22/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485686912e+00 | Ndiff    1.646
       23/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485458687e+00 | Ndiff    1.623
       24/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485239049e+00 | Ndiff    1.600
       25/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485026593e+00 | Ndiff    1.576
       26/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484820224e+00 | Ndiff    1.554
       27/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484619104e+00 | Ndiff    1.532
       28/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484422613e+00 | Ndiff    1.510
       29/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484230302e+00 | Ndiff    1.488
       30/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484041861e+00 | Ndiff    1.467
       31/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483857084e+00 | Ndiff    1.445
       32/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483675854e+00 | Ndiff    1.423
       33/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483498117e+00 | Ndiff    1.402
       34/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483323872e+00 | Ndiff    1.379
       35/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.483153160e+00 | Ndiff    1.357
       36/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482986054e+00 | Ndiff    1.333
       37/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482822653e+00 | Ndiff    1.310
       38/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482663075e+00 | Ndiff    1.285
       39/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482507456e+00 | Ndiff    1.260
       40/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482355938e+00 | Ndiff    1.234
       41/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482208671e+00 | Ndiff    1.208
       42/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.482065808e+00 | Ndiff    1.181
       43/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481927496e+00 | Ndiff    1.153
       44/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481793878e+00 | Ndiff    1.125
       45/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481665086e+00 | Ndiff    1.096
       46/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481541238e+00 | Ndiff    1.067
       47/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481422435e+00 | Ndiff    1.037
       48/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481308757e+00 | Ndiff    1.007
       49/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481200261e+00 | Ndiff    0.977
       50/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.481096980e+00 | Ndiff    0.946
       51/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480998920e+00 | Ndiff    0.916
       52/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480906060e+00 | Ndiff    0.885
       53/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480818354e+00 | Ndiff    0.855
       54/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480735728e+00 | Ndiff    0.824
       55/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480658085e+00 | Ndiff    0.794
       56/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480585302e+00 | Ndiff    0.764
       57/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480517238e+00 | Ndiff    0.735
       58/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480453733e+00 | Ndiff    0.706
       59/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480394611e+00 | Ndiff    0.677
       60/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480339683e+00 | Ndiff    0.649
       61/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480288754e+00 | Ndiff    0.622
       62/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480241618e+00 | Ndiff    0.596
       63/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480198068e+00 | Ndiff    0.570
       64/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480157895e+00 | Ndiff    0.545
       65/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480120891e+00 | Ndiff    0.521
       66/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480086852e+00 | Ndiff    0.498
       67/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480055578e+00 | Ndiff    0.475
       68/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480026875e+00 | Ndiff    0.454
       69/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.480000557e+00 | Ndiff    0.433
       70/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479976447e+00 | Ndiff    0.413
       71/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479954376e+00 | Ndiff    0.394
       72/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479934185e+00 | Ndiff    0.376
       73/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479915723e+00 | Ndiff    0.358
       74/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479898850e+00 | Ndiff    0.342
       75/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479883435e+00 | Ndiff    0.326
       76/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479869356e+00 | Ndiff    0.311
       77/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479856501e+00 | Ndiff    0.296
       78/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479844764e+00 | Ndiff    0.282
       79/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479834050e+00 | Ndiff    0.269
       80/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479824270e+00 | Ndiff    0.257
       81/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479815342e+00 | Ndiff    0.245
       82/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479807191e+00 | Ndiff    0.233
       83/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479799751e+00 | Ndiff    0.223
       84/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479792957e+00 | Ndiff    0.212
       85/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479786752e+00 | Ndiff    0.203
       86/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479781086e+00 | Ndiff    0.193
       87/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479775909e+00 | Ndiff    0.185
       88/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479771178e+00 | Ndiff    0.176
       89/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479766855e+00 | Ndiff    0.168
       90/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479762903e+00 | Ndiff    0.161
       91/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479759289e+00 | Ndiff    0.153
       92/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479755983e+00 | Ndiff    0.147
       93/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479752958e+00 | Ndiff    0.140
       94/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479750190e+00 | Ndiff    0.134
       95/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479747656e+00 | Ndiff    0.128
       96/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479745336e+00 | Ndiff    0.122
       97/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479743210e+00 | Ndiff    0.117
       98/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479741263e+00 | Ndiff    0.112
       99/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479739478e+00 | Ndiff    0.107
      100/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479737841e+00 | Ndiff    0.102
      101/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479736341e+00 | Ndiff    0.098
      102/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479734964e+00 | Ndiff    0.094
      103/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479733701e+00 | Ndiff    0.090
      104/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479732542e+00 | Ndiff    0.086
      105/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479731477e+00 | Ndiff    0.082
      106/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479730500e+00 | Ndiff    0.079
      107/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479729602e+00 | Ndiff    0.075
      108/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479728777e+00 | Ndiff    0.072
      109/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479728019e+00 | Ndiff    0.069
      110/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479727321e+00 | Ndiff    0.066
      111/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479726681e+00 | Ndiff    0.064
      112/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479726091e+00 | Ndiff    0.061
      113/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479725549e+00 | Ndiff    0.058
      114/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479725050e+00 | Ndiff    0.056
      115/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479724590e+00 | Ndiff    0.054
      116/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479724168e+00 | Ndiff    0.051
      117/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479723778e+00 | Ndiff    0.049
      118/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479723420e+00 | Ndiff    0.047
      119/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479723090e+00 | Ndiff    0.045
      120/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479722785e+00 | Ndiff    0.044
      121/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479722505e+00 | Ndiff    0.042
      122/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479722247e+00 | Ndiff    0.040
      123/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479722009e+00 | Ndiff    0.039
      124/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479721789e+00 | Ndiff    0.037
      125/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479721587e+00 | Ndiff    0.036
      126/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479721400e+00 | Ndiff    0.034
      127/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479721228e+00 | Ndiff    0.033
      128/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479721069e+00 | Ndiff    0.031
      129/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720922e+00 | Ndiff    0.030
      130/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720787e+00 | Ndiff    0.029
      131/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720662e+00 | Ndiff    0.028
      132/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720547e+00 | Ndiff    0.027
      133/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720440e+00 | Ndiff    0.026
      134/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720342e+00 | Ndiff    0.025
      135/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720251e+00 | Ndiff    0.024
      136/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.479720168e+00 | Ndiff    0.023
      137/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479720090e+00 | Ndiff    0.022
      138/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479720019e+00 | Ndiff    0.021
      139/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719953e+00 | Ndiff    0.020
      140/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719892e+00 | Ndiff    0.019
      141/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719836e+00 | Ndiff    0.019
      142/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719783e+00 | Ndiff    0.018
      143/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719735e+00 | Ndiff    0.017
      144/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719691e+00 | Ndiff    0.017
      145/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719650e+00 | Ndiff    0.016
      146/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719612e+00 | Ndiff    0.015
      147/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719577e+00 | Ndiff    0.015
      148/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719544e+00 | Ndiff    0.014
      149/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719514e+00 | Ndiff    0.014
      150/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719486e+00 | Ndiff    0.013
      151/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719461e+00 | Ndiff    0.013
      152/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719437e+00 | Ndiff    0.012
      153/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719415e+00 | Ndiff    0.012
      154/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719395e+00 | Ndiff    0.011
      155/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719376e+00 | Ndiff    0.011
      156/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719359e+00 | Ndiff    0.010
      157/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719343e+00 | Ndiff    0.010
      158/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719328e+00 | Ndiff    0.010
      159/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719314e+00 | Ndiff    0.009
      160/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719301e+00 | Ndiff    0.009
      161/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719290e+00 | Ndiff    0.009
      162/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719279e+00 | Ndiff    0.008
      163/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719269e+00 | Ndiff    0.008
      164/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719259e+00 | Ndiff    0.008
      165/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719251e+00 | Ndiff    0.007
      166/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719243e+00 | Ndiff    0.007
      167/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719235e+00 | Ndiff    0.007
      168/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719228e+00 | Ndiff    0.006
      169/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719222e+00 | Ndiff    0.006
      170/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719216e+00 | Ndiff    0.006
      171/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719211e+00 | Ndiff    0.006
      172/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719206e+00 | Ndiff    0.006
      173/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719201e+00 | Ndiff    0.005
      174/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719197e+00 | Ndiff    0.005
      175/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719193e+00 | Ndiff    0.005
      176/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719189e+00 | Ndiff    0.005
      177/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719186e+00 | Ndiff    0.005
      178/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719183e+00 | Ndiff    0.004
      179/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719180e+00 | Ndiff    0.004
      180/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719177e+00 | Ndiff    0.004
      181/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719175e+00 | Ndiff    0.004
      182/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719172e+00 | Ndiff    0.004
      183/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719170e+00 | Ndiff    0.004
      184/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719168e+00 | Ndiff    0.003
      185/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719166e+00 | Ndiff    0.003
      186/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719165e+00 | Ndiff    0.003
      187/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719163e+00 | Ndiff    0.003
      188/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719161e+00 | Ndiff    0.003
      189/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719160e+00 | Ndiff    0.003
      190/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719159e+00 | Ndiff    0.003
      191/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719158e+00 | Ndiff    0.003
      192/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719157e+00 | Ndiff    0.003
      193/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719156e+00 | Ndiff    0.002
      194/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719155e+00 | Ndiff    0.002
      195/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719154e+00 | Ndiff    0.002
      196/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719153e+00 | Ndiff    0.002
      197/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719152e+00 | Ndiff    0.002
      198/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719152e+00 | Ndiff    0.002
      199/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719151e+00 | Ndiff    0.002
      200/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719150e+00 | Ndiff    0.002
      201/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719150e+00 | Ndiff    0.002
      202/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719149e+00 | Ndiff    0.002
      203/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719149e+00 | Ndiff    0.002
      204/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719149e+00 | Ndiff    0.002
      205/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719148e+00 | Ndiff    0.002
      206/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719148e+00 | Ndiff    0.001
      207/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
      208/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
      209/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
      210/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
      211/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
      212/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
      213/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
      214/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
      215/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
      216/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
      217/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
      218/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
      219/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
      220/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
      221/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
      222/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
      223/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      224/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      225/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      226/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      227/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      228/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      229/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      230/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      231/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      232/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      233/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      234/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
      235/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
      236/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
      237/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
      238/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
      239/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
      240/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
      241/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
      242/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      243/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      244/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      245/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      246/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      247/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      248/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      249/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      250/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      251/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      252/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      253/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      254/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      255/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      256/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      257/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      258/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      259/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      260/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      261/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      262/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      263/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      264/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      265/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      266/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      267/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      268/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      269/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      270/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      271/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      272/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      273/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      274/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      275/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      276/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
      277/1000 after      1 sec. |    194.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  5/5 | alg. seed: 2986368 | data order seed: 6479872
task_output_path: /tmp/faithful/demo_sparse_resp-K=3-lik=Gauss-ECovMat=5*eye/5
        1/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.172454094e+00 |
        2/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.141604650e+00 | Ndiff    2.137
        3/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.132916761e+00 | Ndiff    2.207
        4/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.116013242e+00 | Ndiff    5.951
        5/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.079969466e+00 | Ndiff   11.990
        6/1000 after      0 sec. |    194.2 MiB | K    3 | loss  3.017967278e+00 | Ndiff   19.686
        7/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.936658417e+00 | Ndiff   25.772
        8/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.854720770e+00 | Ndiff   26.020
        9/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.778771855e+00 | Ndiff   21.413
       10/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.688031379e+00 | Ndiff   17.921
       11/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.590482847e+00 | Ndiff   12.677
       12/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.538125017e+00 | Ndiff    5.219
       13/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.514221187e+00 | Ndiff    3.415
       14/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.497495257e+00 | Ndiff    2.846
       15/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495853387e+00 | Ndiff    1.084
       16/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495615725e+00 | Ndiff    0.686
       17/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495466026e+00 | Ndiff    0.569
       18/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495360868e+00 | Ndiff    0.492
       19/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495282138e+00 | Ndiff    0.438
       20/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495220122e+00 | Ndiff    0.397
       21/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495169130e+00 | Ndiff    0.366
       22/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495125609e+00 | Ndiff    0.342
       23/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495087248e+00 | Ndiff    0.322
       24/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495052510e+00 | Ndiff    0.305
       25/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.495020376e+00 | Ndiff    0.290
       26/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494990186e+00 | Ndiff    0.277
       27/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494961538e+00 | Ndiff    0.265
       28/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494934218e+00 | Ndiff    0.254
       29/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494908140e+00 | Ndiff    0.243
       30/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494883308e+00 | Ndiff    0.233
       31/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494859775e+00 | Ndiff    0.223
       32/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494837612e+00 | Ndiff    0.212
       33/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494816893e+00 | Ndiff    0.202
       34/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494797668e+00 | Ndiff    0.192
       35/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494779964e+00 | Ndiff    0.183
       36/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494763774e+00 | Ndiff    0.173
       37/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494749060e+00 | Ndiff    0.163
       38/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494735759e+00 | Ndiff    0.154
       39/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494723785e+00 | Ndiff    0.145
       40/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494713036e+00 | Ndiff    0.137
       41/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494703405e+00 | Ndiff    0.129
       42/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494694776e+00 | Ndiff    0.121
       43/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494687038e+00 | Ndiff    0.114
       44/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494680081e+00 | Ndiff    0.108
       45/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494673802e+00 | Ndiff    0.102
       46/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494668101e+00 | Ndiff    0.096
       47/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494662889e+00 | Ndiff    0.091
       48/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494658081e+00 | Ndiff    0.087
       49/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494653596e+00 | Ndiff    0.083
       50/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494649357e+00 | Ndiff    0.079
       51/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494645290e+00 | Ndiff    0.076
       52/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494641317e+00 | Ndiff    0.074
       53/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494637354e+00 | Ndiff    0.072
       54/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494633308e+00 | Ndiff    0.071
       55/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494629063e+00 | Ndiff    0.071
       56/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494624470e+00 | Ndiff    0.072
       57/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494619327e+00 | Ndiff    0.073
       58/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494613338e+00 | Ndiff    0.076
       59/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494606055e+00 | Ndiff    0.080
       60/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494596763e+00 | Ndiff    0.086
       61/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494584268e+00 | Ndiff    0.094
       62/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494566496e+00 | Ndiff    0.107
       63/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494539723e+00 | Ndiff    0.123
       64/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494497190e+00 | Ndiff    0.146
       65/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494427049e+00 | Ndiff    0.177
       66/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494311214e+00 | Ndiff    0.217
       67/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.494130236e+00 | Ndiff    0.262
       68/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.493873502e+00 | Ndiff    0.313
       69/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.493523728e+00 | Ndiff    0.371
       70/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.492991715e+00 | Ndiff    0.441
       71/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.492034383e+00 | Ndiff    0.514
       72/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.490416803e+00 | Ndiff    0.538
       73/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.489022467e+00 | Ndiff    0.469
       74/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.488517958e+00 | Ndiff    0.454
       75/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.488074913e+00 | Ndiff    0.435
       76/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.487289742e+00 | Ndiff    0.375
       77/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.486119349e+00 | Ndiff    0.236
       78/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.485305514e+00 | Ndiff    0.082
       79/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484928040e+00 | Ndiff    0.036
       80/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484776037e+00 | Ndiff    0.007
       81/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484732728e+00 | Ndiff    0.012
       82/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484723997e+00 | Ndiff    0.015
       83/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484722544e+00 | Ndiff    0.010
       84/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484722316e+00 | Ndiff    0.005
       85/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484722281e+00 | Ndiff    0.003
       86/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484722275e+00 | Ndiff    0.001
       87/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484722274e+00 | Ndiff    0.001
       88/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484722274e+00 | Ndiff    0.000
       89/1000 after      0 sec. |    194.2 MiB | K    3 | loss  2.484722274e+00 | Ndiff    0.000
... done. converged.

Do inference with L=1 sparsity

Kwarg to set is “nnzPerRowLP = 1”

nnzPerRowLP is read as “number of non-zero entries per row of local parameters”

This enforces that each row of resp array has ___ non zero entries

print('SPARSITY LEVEL: 1 of 3')
LP1 = full_trained_model.calc_local_params(dataset, nnzPerRowLP=1)
pprint_sparse_or_dense_resp(LP1)

Out:

SPARSITY LEVEL: 1 of 3
[[0. 0. 1.]
 [0. 0. 1.]
 [0. 1. 0.]
 [0. 0. 1.]
 [0. 0. 1.]
 [0. 0. 1.]
 [0. 0. 1.]
 [0. 1. 0.]
 [1. 0. 0.]
 [0. 1. 0.]]

Do inference with L=2 sparsity

print('SPARSITY LEVEL: 2 of 3')
LP2 = full_trained_model.calc_local_params(dataset, nnzPerRowLP=2)
pprint_sparse_or_dense_resp(LP2)

Out:

SPARSITY LEVEL: 2 of 3
[[0.     0.2424 0.7576]
 [0.2281 0.     0.7719]
 [0.     0.9262 0.0738]
 [0.0427 0.     0.9573]
 [0.0269 0.     0.9731]
 [0.0247 0.     0.9753]
 [0.     0.021  0.979 ]
 [0.     0.9866 0.0134]
 [0.9866 0.     0.0134]
 [0.     0.9905 0.0095]]

Do inference with L=3 sparsity

print('SPARSITY LEVEL: 3 of 3 (DENSE)')
LP3 = full_trained_model.calc_local_params(dataset, nnzPerRowLP=3)
pprint_sparse_or_dense_resp(LP3)


# Show the model
pylab.figure(2)
bnpy.viz.PlotComps.plotCompsFromHModel(
    full_trained_model,
    dataset=targeted_dataset,
    )
pylab.xlabel(dataset.column_names[0])
pylab.ylabel(dataset.column_names[1])
pylab.tight_layout()
pylab.title('Trained model with targeted examples')
../../_images/sphx_glr_plot-03-demo=sparse_resp_vb-model=mix_gauss_004.png

Out:

SPARSITY LEVEL: 3 of 3 (DENSE)
[[0.     0.2424 0.7575]
 [0.2281 0.     0.7719]
 [0.     0.9262 0.0738]
 [0.0427 0.     0.9573]
 [0.0269 0.0002 0.9729]
 [0.0247 0.     0.9753]
 [0.0002 0.021  0.9788]
 [0.     0.9866 0.0134]
 [0.9866 0.     0.0134]
 [0.     0.9905 0.0095]]

How to incorporate sparsity during training?

Just pass “nnzPerRowLP” kwarg to bnpy.run(…)

Total running time of the script: ( 0 minutes 4.058 seconds)

Gallery generated by Sphinx-Gallery