# Demonstration of Sparse Responsibilities for Mixtures of Gaussians¶

This demo illustrates “sparse responsibilities” for mixture models.

In this example, we show how BNPy makes it easy to vary per-example inference, so that for a 3-cluster model we can manually try each of the following:

• each data point assigned to up to 1 cluster
• each data point assigned to up to 2 clusters
• each data point assigned to up to 3 clusters

In our variational approximation, when we fit a K-cluster mixture model to a specific data point, we learn a vector of posterior responsibilities – we often call these “resp” for short in the code – indicating how much of the unit probability mass for this example is explained by each of the K clusters.

For example, for a specific data point we might have a resp vector of:

[ 0. 0.9262 0.0738]

which we interpret as saying this point is 92% explained by the 2nd cluster, 7% by the 3rd cluster, and 0% by the first.

This short demo just shows how we can deliberately enforce sparsity in learned responsibilities, so that each example may have only L non-zero entries in its resp vector. For example, we’ll fit a K=3 model below, and show how we can enforce L=1 or L=2 sparsity.

For a technical introduction, see our ArXiv paper:

Michael C. Hughes and Erik B. Sudderth (2016) “Fast Learning of Clusters and Topics via Sparse Posteriors.” https://arxiv.org/abs/1609.07521
from __future__ import print_function

import bnpy
import numpy as np
import os

from matplotlib import pylab
import seaborn as sns

SMALL_FIG_SIZE = (2.5, 2.5)
FIG_SIZE = (5, 5)
pylab.rcParams['figure.figsize'] = FIG_SIZE


dataset_path = os.path.join(bnpy.DATASET_PATH, 'faithful')
os.path.join(dataset_path, 'faithful.csv'))

# Identify "target" example ids to focus on
target_example_ids = np.asarray(
[ 83,  46, 152, 173,  23, 164, 243, 239, 154, 100])
targeted_dataset = bnpy.data.XData(
dataset.X[target_example_ids])


Make a simple plot of the raw data

pylab.plot(dataset.X[:, 0], dataset.X[:, 1], 'k.')
pylab.xlabel(dataset.column_names[0])
pylab.ylabel(dataset.column_names[1])
pylab.tight_layout()
pylab.title('Complete Dataset')
data_ax_h = pylab.gca()


## Setup: Func to print resp array nicely¶

def pprint_sparse_or_dense_resp(LP, target_example_ids=target_example_ids):
np.set_printoptions(suppress=True, precision=4)
if 'spR' in LP:
# sparse case
resp_NK = LP['spR'][target_example_ids].toarray()
else:
resp_NK = LP['resp'][target_example_ids]
print(resp_NK)


## VB with DP mixture model and diagonal Gaussian observations¶

Assumes exactly 3 clusters

Assumes diagonal covariances.

No sparsity assumptions during training

K = 3          # n clusters
gamma = 50.0   # DP concentration param
sF = 0.1       # scale of expected covariance

full_trained_model, full_info_dict = bnpy.run(
dataset, 'DPMixtureModel', 'DiagGauss', 'VB',
output_path='/tmp/faithful/demo_sparse_resp-K=3-lik=Gauss-ECovMat=5*eye/',
gamma0=gamma, sF=sF, ECovMat='eye',
K=K, initname='randexamples',
)

# Add this model into the current plot
bnpy.viz.PlotComps.plotCompsFromHModel(
full_trained_model,
)


Out:

WARNING: Found unrecognized keyword args. These are ignored.
--nBatch
Dataset Summary:
X Data
num examples: 272
num dims: 2
Allocation Model:  DP mixture with K=0. Concentration gamma0= 50.00
Obs. Data  Model:  Gaussian with diagonal covariance.
Obs. Data  Prior:  independent Gauss-Wishart prior on each dimension
Wishart params
nu = 4
beta = [ 0.2  0.2]
Expectations
E[  mean[k]] =
[ 0  0]
E[ covar[k]] =
[[0.1 0. ]
[0.  0.1]]
Initialization:
initname = randexamples
K = 3 (number of clusters)
seed = 1607680
elapsed_time: 0.0 sec
Learn Alg: VB | task  1/5 | alg. seed: 1607680 | data order seed: 8541952
1/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.024985920e+00 |
2/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.963751576e+00 | Ndiff    2.846
3/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.915936430e+00 | Ndiff   10.454
4/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.841297326e+00 | Ndiff   18.204
5/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.748858710e+00 | Ndiff   20.529
6/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.657778518e+00 | Ndiff   15.677
7/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.569542173e+00 | Ndiff   10.904
8/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.534007328e+00 | Ndiff    4.528
9/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.508961472e+00 | Ndiff    3.729
10/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495535777e+00 | Ndiff    1.586
11/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.494290057e+00 | Ndiff    0.411
12/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.493584130e+00 | Ndiff    0.696
13/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.492995794e+00 | Ndiff    0.929
14/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.492459487e+00 | Ndiff    1.135
15/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.491945636e+00 | Ndiff    1.318
16/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.491440653e+00 | Ndiff    1.481
17/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.490939573e+00 | Ndiff    1.625
18/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.490442396e+00 | Ndiff    1.749
19/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.489951832e+00 | Ndiff    1.852
20/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.489471711e+00 | Ndiff    1.933
21/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.489005811e+00 | Ndiff    1.993
22/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.488557093e+00 | Ndiff    2.033
23/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.488127311e+00 | Ndiff    2.055
24/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.487716959e+00 | Ndiff    2.063
25/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.487325464e+00 | Ndiff    2.059
26/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.486951484e+00 | Ndiff    2.047
27/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.486593250e+00 | Ndiff    2.028
28/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.486248844e+00 | Ndiff    2.006
29/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.485916412e+00 | Ndiff    1.981
30/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.485594294e+00 | Ndiff    1.954
31/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.485281079e+00 | Ndiff    1.927
32/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484975632e+00 | Ndiff    1.900
33/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484677073e+00 | Ndiff    1.873
34/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484384755e+00 | Ndiff    1.845
35/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484098229e+00 | Ndiff    1.817
36/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483817216e+00 | Ndiff    1.789
37/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483541575e+00 | Ndiff    1.760
38/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483271282e+00 | Ndiff    1.731
39/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483006407e+00 | Ndiff    1.701
40/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482747096e+00 | Ndiff    1.669
41/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482493559e+00 | Ndiff    1.637
42/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482246056e+00 | Ndiff    1.604
43/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482004886e+00 | Ndiff    1.569
44/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481770375e+00 | Ndiff    1.533
45/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481542863e+00 | Ndiff    1.496
46/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481322699e+00 | Ndiff    1.458
47/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481110224e+00 | Ndiff    1.419
48/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480905765e+00 | Ndiff    1.379
49/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480709619e+00 | Ndiff    1.338
50/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480522049e+00 | Ndiff    1.296
51/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480343269e+00 | Ndiff    1.254
52/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480173442e+00 | Ndiff    1.211
53/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480012666e+00 | Ndiff    1.167
54/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479860980e+00 | Ndiff    1.124
55/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479718351e+00 | Ndiff    1.081
56/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479584685e+00 | Ndiff    1.038
57/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479459820e+00 | Ndiff    0.995
58/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479343538e+00 | Ndiff    0.953
59/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479235566e+00 | Ndiff    0.912
60/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479135586e+00 | Ndiff    0.871
61/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.479043241e+00 | Ndiff    0.832
62/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478958148e+00 | Ndiff    0.793
63/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478879901e+00 | Ndiff    0.756
64/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478808084e+00 | Ndiff    0.720
65/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478742273e+00 | Ndiff    0.686
66/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478682051e+00 | Ndiff    0.653
67/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478627005e+00 | Ndiff    0.621
68/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478576736e+00 | Ndiff    0.591
69/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.478530861e+00 | Ndiff    0.562
70/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478489016e+00 | Ndiff    0.534
71/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478450858e+00 | Ndiff    0.508
72/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478416065e+00 | Ndiff    0.483
73/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478384339e+00 | Ndiff    0.460
74/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478355403e+00 | Ndiff    0.438
75/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478329003e+00 | Ndiff    0.417
76/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478304903e+00 | Ndiff    0.397
77/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478282892e+00 | Ndiff    0.378
78/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478262773e+00 | Ndiff    0.361
79/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478244370e+00 | Ndiff    0.344
80/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478227522e+00 | Ndiff    0.328
81/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478212083e+00 | Ndiff    0.313
82/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478197921e+00 | Ndiff    0.299
83/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478184918e+00 | Ndiff    0.286
84/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478172966e+00 | Ndiff    0.274
85/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478161968e+00 | Ndiff    0.262
86/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478151836e+00 | Ndiff    0.251
87/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478142491e+00 | Ndiff    0.241
88/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478133862e+00 | Ndiff    0.231
89/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478125885e+00 | Ndiff    0.221
90/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478118502e+00 | Ndiff    0.213
91/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478111660e+00 | Ndiff    0.204
92/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478105312e+00 | Ndiff    0.196
93/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478099416e+00 | Ndiff    0.189
94/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478093933e+00 | Ndiff    0.182
95/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478088827e+00 | Ndiff    0.175
96/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478084067e+00 | Ndiff    0.169
97/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478079625e+00 | Ndiff    0.163
98/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478075475e+00 | Ndiff    0.157
99/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478071593e+00 | Ndiff    0.152
100/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478067957e+00 | Ndiff    0.147
101/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478064548e+00 | Ndiff    0.142
102/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478061348e+00 | Ndiff    0.137
103/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478058342e+00 | Ndiff    0.133
104/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478055514e+00 | Ndiff    0.129
105/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478052851e+00 | Ndiff    0.125
106/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478050340e+00 | Ndiff    0.121
107/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478047971e+00 | Ndiff    0.118
108/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478045734e+00 | Ndiff    0.114
109/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478043618e+00 | Ndiff    0.111
110/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478041615e+00 | Ndiff    0.108
111/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478039717e+00 | Ndiff    0.105
112/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478037917e+00 | Ndiff    0.102
113/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478036209e+00 | Ndiff    0.099
114/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.478034585e+00 | Ndiff    0.097
115/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478033041e+00 | Ndiff    0.094
116/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478031571e+00 | Ndiff    0.092
117/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478030171e+00 | Ndiff    0.089
118/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478028835e+00 | Ndiff    0.087
119/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478027560e+00 | Ndiff    0.085
120/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478026342e+00 | Ndiff    0.083
121/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478025178e+00 | Ndiff    0.081
122/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478024063e+00 | Ndiff    0.079
123/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478022996e+00 | Ndiff    0.078
124/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478021972e+00 | Ndiff    0.076
125/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478020991e+00 | Ndiff    0.074
126/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478020048e+00 | Ndiff    0.073
127/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478019142e+00 | Ndiff    0.071
128/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478018271e+00 | Ndiff    0.070
129/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478017433e+00 | Ndiff    0.069
130/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478016626e+00 | Ndiff    0.067
131/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478015848e+00 | Ndiff    0.066
132/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478015097e+00 | Ndiff    0.065
133/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478014373e+00 | Ndiff    0.064
134/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478013673e+00 | Ndiff    0.062
135/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478012996e+00 | Ndiff    0.061
136/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478012342e+00 | Ndiff    0.060
137/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478011708e+00 | Ndiff    0.059
138/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478011095e+00 | Ndiff    0.058
139/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478010500e+00 | Ndiff    0.057
140/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478009923e+00 | Ndiff    0.056
141/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478009363e+00 | Ndiff    0.056
142/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478008818e+00 | Ndiff    0.055
143/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478008290e+00 | Ndiff    0.054
144/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478007775e+00 | Ndiff    0.053
145/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478007275e+00 | Ndiff    0.052
146/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478006787e+00 | Ndiff    0.052
147/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478006312e+00 | Ndiff    0.051
148/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478005849e+00 | Ndiff    0.050
149/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478005396e+00 | Ndiff    0.050
150/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478004955e+00 | Ndiff    0.049
151/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478004524e+00 | Ndiff    0.048
152/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478004102e+00 | Ndiff    0.048
153/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478003690e+00 | Ndiff    0.047
154/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478003287e+00 | Ndiff    0.047
155/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478002891e+00 | Ndiff    0.046
156/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478002504e+00 | Ndiff    0.046
157/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478002125e+00 | Ndiff    0.045
158/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478001752e+00 | Ndiff    0.045
159/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478001387e+00 | Ndiff    0.044
160/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.478001027e+00 | Ndiff    0.044
161/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.478000675e+00 | Ndiff    0.044
162/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.478000328e+00 | Ndiff    0.043
163/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477999986e+00 | Ndiff    0.043
164/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477999650e+00 | Ndiff    0.042
165/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477999319e+00 | Ndiff    0.042
166/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477998993e+00 | Ndiff    0.042
167/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477998671e+00 | Ndiff    0.042
168/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477998353e+00 | Ndiff    0.041
169/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477998040e+00 | Ndiff    0.041
170/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477997731e+00 | Ndiff    0.041
171/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477997425e+00 | Ndiff    0.040
172/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477997122e+00 | Ndiff    0.040
173/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477996823e+00 | Ndiff    0.040
174/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477996526e+00 | Ndiff    0.040
175/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477996233e+00 | Ndiff    0.039
176/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477995942e+00 | Ndiff    0.039
177/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477995654e+00 | Ndiff    0.039
178/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477995368e+00 | Ndiff    0.039
179/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477995084e+00 | Ndiff    0.039
180/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477994802e+00 | Ndiff    0.039
181/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477994521e+00 | Ndiff    0.038
182/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477994243e+00 | Ndiff    0.038
183/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477993966e+00 | Ndiff    0.038
184/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477993690e+00 | Ndiff    0.038
185/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477993415e+00 | Ndiff    0.038
186/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477993141e+00 | Ndiff    0.038
187/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477992868e+00 | Ndiff    0.038
188/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477992596e+00 | Ndiff    0.038
189/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477992324e+00 | Ndiff    0.038
190/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477992053e+00 | Ndiff    0.038
191/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477991782e+00 | Ndiff    0.038
192/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477991511e+00 | Ndiff    0.038
193/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477991239e+00 | Ndiff    0.038
194/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477990968e+00 | Ndiff    0.038
195/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477990696e+00 | Ndiff    0.038
196/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477990424e+00 | Ndiff    0.038
197/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477990151e+00 | Ndiff    0.038
198/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477989878e+00 | Ndiff    0.038
199/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477989603e+00 | Ndiff    0.038
200/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477989327e+00 | Ndiff    0.038
201/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477989050e+00 | Ndiff    0.038
202/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477988772e+00 | Ndiff    0.038
203/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477988492e+00 | Ndiff    0.038
204/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477988210e+00 | Ndiff    0.038
205/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477987926e+00 | Ndiff    0.038
206/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.477987640e+00 | Ndiff    0.038
207/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477987351e+00 | Ndiff    0.039
208/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477987061e+00 | Ndiff    0.039
209/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477986767e+00 | Ndiff    0.039
210/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477986470e+00 | Ndiff    0.039
211/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477986171e+00 | Ndiff    0.039
212/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477985868e+00 | Ndiff    0.039
213/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477985561e+00 | Ndiff    0.040
214/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477985251e+00 | Ndiff    0.040
215/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477984936e+00 | Ndiff    0.040
216/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477984618e+00 | Ndiff    0.040
217/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477984294e+00 | Ndiff    0.041
218/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477983966e+00 | Ndiff    0.041
219/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477983633e+00 | Ndiff    0.041
220/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477983294e+00 | Ndiff    0.041
221/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477982950e+00 | Ndiff    0.042
222/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477982600e+00 | Ndiff    0.042
223/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477982243e+00 | Ndiff    0.042
224/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477981879e+00 | Ndiff    0.043
225/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477981508e+00 | Ndiff    0.043
226/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477981130e+00 | Ndiff    0.044
227/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477980744e+00 | Ndiff    0.044
228/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477980349e+00 | Ndiff    0.044
229/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477979945e+00 | Ndiff    0.045
230/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477979532e+00 | Ndiff    0.045
231/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477979109e+00 | Ndiff    0.046
232/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477978676e+00 | Ndiff    0.046
233/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477978231e+00 | Ndiff    0.047
234/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477977775e+00 | Ndiff    0.048
235/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477977306e+00 | Ndiff    0.048
236/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477976824e+00 | Ndiff    0.049
237/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477976329e+00 | Ndiff    0.050
238/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477975818e+00 | Ndiff    0.050
239/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477975292e+00 | Ndiff    0.051
240/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477974749e+00 | Ndiff    0.052
241/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477974189e+00 | Ndiff    0.052
242/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477973610e+00 | Ndiff    0.053
243/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477973011e+00 | Ndiff    0.054
244/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477972391e+00 | Ndiff    0.055
245/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477971749e+00 | Ndiff    0.056
246/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477971083e+00 | Ndiff    0.057
247/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477970391e+00 | Ndiff    0.058
248/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477969672e+00 | Ndiff    0.059
249/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477968924e+00 | Ndiff    0.060
250/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477968146e+00 | Ndiff    0.061
251/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477967334e+00 | Ndiff    0.063
252/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.477966486e+00 | Ndiff    0.064
253/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477965601e+00 | Ndiff    0.065
254/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477964675e+00 | Ndiff    0.067
255/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477963705e+00 | Ndiff    0.068
256/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477962687e+00 | Ndiff    0.070
257/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477961619e+00 | Ndiff    0.071
258/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477960497e+00 | Ndiff    0.073
259/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477959315e+00 | Ndiff    0.075
260/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477958069e+00 | Ndiff    0.077
261/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477956753e+00 | Ndiff    0.079
262/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477955363e+00 | Ndiff    0.081
263/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477953891e+00 | Ndiff    0.083
264/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477952330e+00 | Ndiff    0.085
265/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477950671e+00 | Ndiff    0.088
266/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477948907e+00 | Ndiff    0.090
267/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477947028e+00 | Ndiff    0.093
268/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477945021e+00 | Ndiff    0.096
269/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477942874e+00 | Ndiff    0.099
270/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477940573e+00 | Ndiff    0.102
271/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477938103e+00 | Ndiff    0.106
272/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477935445e+00 | Ndiff    0.110
273/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477932578e+00 | Ndiff    0.114
274/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477929479e+00 | Ndiff    0.118
275/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477926122e+00 | Ndiff    0.122
276/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477922476e+00 | Ndiff    0.127
277/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477918505e+00 | Ndiff    0.132
278/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477914169e+00 | Ndiff    0.138
279/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477909421e+00 | Ndiff    0.144
280/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477904205e+00 | Ndiff    0.150
281/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477898458e+00 | Ndiff    0.157
282/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477892103e+00 | Ndiff    0.164
283/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477885051e+00 | Ndiff    0.172
284/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477877196e+00 | Ndiff    0.181
285/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477868411e+00 | Ndiff    0.191
286/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477858544e+00 | Ndiff    0.201
287/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477847411e+00 | Ndiff    0.212
288/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477834786e+00 | Ndiff    0.225
289/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477820395e+00 | Ndiff    0.238
290/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477803898e+00 | Ndiff    0.253
291/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477784870e+00 | Ndiff    0.270
292/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477762778e+00 | Ndiff    0.288
293/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477736945e+00 | Ndiff    0.308
294/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477706505e+00 | Ndiff    0.331
295/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477670330e+00 | Ndiff    0.356
296/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477626947e+00 | Ndiff    0.384
297/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477574395e+00 | Ndiff    0.416
298/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.477510040e+00 | Ndiff    0.452
299/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.477430283e+00 | Ndiff    0.492
300/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.477330143e+00 | Ndiff    0.537
301/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.477202613e+00 | Ndiff    0.588
302/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.477037673e+00 | Ndiff    0.646
303/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.476820794e+00 | Ndiff    0.710
304/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.476530732e+00 | Ndiff    0.780
305/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.476136866e+00 | Ndiff    0.855
306/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.475598527e+00 | Ndiff    0.929
307/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.474876012e+00 | Ndiff    0.989
308/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.473971580e+00 | Ndiff    1.018
309/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.472985078e+00 | Ndiff    0.999
310/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.472086685e+00 | Ndiff    0.935
311/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.471397849e+00 | Ndiff    0.841
312/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.470920902e+00 | Ndiff    0.742
313/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.470584489e+00 | Ndiff    0.653
314/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.470324872e+00 | Ndiff    0.580
315/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.470106989e+00 | Ndiff    0.523
316/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.469911918e+00 | Ndiff    0.481
317/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.469726780e+00 | Ndiff    0.453
318/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.469539898e+00 | Ndiff    0.437
319/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.469337523e+00 | Ndiff    0.434
320/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.469099855e+00 | Ndiff    0.443
321/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.468794891e+00 | Ndiff    0.463
322/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.468371096e+00 | Ndiff    0.492
323/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.467763504e+00 | Ndiff    0.513
324/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.466960962e+00 | Ndiff    0.496
325/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.466160474e+00 | Ndiff    0.400
326/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465724445e+00 | Ndiff    0.232
327/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465626565e+00 | Ndiff    0.094
328/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465615098e+00 | Ndiff    0.031
329/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465613516e+00 | Ndiff    0.014
330/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465613125e+00 | Ndiff    0.008
331/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612997e+00 | Ndiff    0.005
332/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612952e+00 | Ndiff    0.003
333/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612936e+00 | Ndiff    0.002
334/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612930e+00 | Ndiff    0.001
335/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612928e+00 | Ndiff    0.001
336/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
337/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
338/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
339/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.465612927e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  2/5 | alg. seed: 6454144 | data order seed: 7673856
1/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.144280649e+00 |
2/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.999427978e+00 | Ndiff   16.339
3/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.889092617e+00 | Ndiff   29.328
4/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.773372571e+00 | Ndiff   30.295
5/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.670818147e+00 | Ndiff   19.860
6/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.577419094e+00 | Ndiff   11.388
7/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.531167799e+00 | Ndiff    4.469
8/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495532492e+00 | Ndiff    3.497
9/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.485361099e+00 | Ndiff    1.346
10/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.484760839e+00 | Ndiff    1.534
11/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.484517089e+00 | Ndiff    1.442
12/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.484320192e+00 | Ndiff    1.384
13/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.484140099e+00 | Ndiff    1.351
14/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.483970718e+00 | Ndiff    1.326
15/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.483809360e+00 | Ndiff    1.305
16/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483654299e+00 | Ndiff    1.286
17/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483504339e+00 | Ndiff    1.267
18/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.483358650e+00 | Ndiff    1.248
19/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.483216669e+00 | Ndiff    1.230
20/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.483078037e+00 | Ndiff    1.211
21/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482942536e+00 | Ndiff    1.192
22/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482810056e+00 | Ndiff    1.173
23/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482680562e+00 | Ndiff    1.153
24/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482554074e+00 | Ndiff    1.133
25/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482430645e+00 | Ndiff    1.113
26/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482310357e+00 | Ndiff    1.092
27/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482193304e+00 | Ndiff    1.070
28/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.482079589e+00 | Ndiff    1.048
29/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481969318e+00 | Ndiff    1.025
30/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481862594e+00 | Ndiff    1.002
31/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481759514e+00 | Ndiff    0.978
32/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481660166e+00 | Ndiff    0.954
33/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481564628e+00 | Ndiff    0.929
34/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481472961e+00 | Ndiff    0.904
35/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481385213e+00 | Ndiff    0.879
36/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481301412e+00 | Ndiff    0.853
37/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481221570e+00 | Ndiff    0.828
38/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481145680e+00 | Ndiff    0.802
39/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481073718e+00 | Ndiff    0.777
40/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.481005638e+00 | Ndiff    0.751
41/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480941380e+00 | Ndiff    0.726
42/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480880866e+00 | Ndiff    0.700
43/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480824003e+00 | Ndiff    0.675
44/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480770686e+00 | Ndiff    0.651
45/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480720795e+00 | Ndiff    0.626
46/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480674203e+00 | Ndiff    0.602
47/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480630775e+00 | Ndiff    0.579
48/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480590367e+00 | Ndiff    0.556
49/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480552834e+00 | Ndiff    0.533
50/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480518027e+00 | Ndiff    0.512
51/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480485796e+00 | Ndiff    0.490
52/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480455992e+00 | Ndiff    0.470
53/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480428469e+00 | Ndiff    0.450
54/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480403082e+00 | Ndiff    0.431
55/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480379690e+00 | Ndiff    0.412
56/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480358160e+00 | Ndiff    0.394
57/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480338360e+00 | Ndiff    0.377
58/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480320167e+00 | Ndiff    0.360
59/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480303462e+00 | Ndiff    0.344
60/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480288134e+00 | Ndiff    0.329
61/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480274078e+00 | Ndiff    0.314
62/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480261194e+00 | Ndiff    0.300
63/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480249391e+00 | Ndiff    0.287
64/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480238582e+00 | Ndiff    0.274
65/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480228686e+00 | Ndiff    0.261
66/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480219629e+00 | Ndiff    0.250
67/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480211341e+00 | Ndiff    0.238
68/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480203759e+00 | Ndiff    0.227
69/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480196823e+00 | Ndiff    0.217
70/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480190480e+00 | Ndiff    0.207
71/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480184678e+00 | Ndiff    0.198
72/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480179372e+00 | Ndiff    0.189
73/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480174520e+00 | Ndiff    0.181
74/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480170083e+00 | Ndiff    0.172
75/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480166025e+00 | Ndiff    0.165
76/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480162313e+00 | Ndiff    0.157
77/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480158918e+00 | Ndiff    0.150
78/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480155813e+00 | Ndiff    0.144
79/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480152973e+00 | Ndiff    0.137
80/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480150374e+00 | Ndiff    0.131
81/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480147996e+00 | Ndiff    0.125
82/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480145820e+00 | Ndiff    0.120
83/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480143828e+00 | Ndiff    0.114
84/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480142005e+00 | Ndiff    0.109
85/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480140336e+00 | Ndiff    0.105
86/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480138808e+00 | Ndiff    0.100
87/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480137409e+00 | Ndiff    0.096
88/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480136127e+00 | Ndiff    0.091
89/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480134953e+00 | Ndiff    0.087
90/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480133878e+00 | Ndiff    0.084
91/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480132893e+00 | Ndiff    0.080
92/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480131990e+00 | Ndiff    0.077
93/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480131162e+00 | Ndiff    0.073
94/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480130403e+00 | Ndiff    0.070
95/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480129708e+00 | Ndiff    0.067
96/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480129070e+00 | Ndiff    0.064
97/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480128485e+00 | Ndiff    0.061
98/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480127949e+00 | Ndiff    0.059
99/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480127457e+00 | Ndiff    0.056
100/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480127005e+00 | Ndiff    0.054
101/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.480126591e+00 | Ndiff    0.052
102/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480126211e+00 | Ndiff    0.049
103/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480125862e+00 | Ndiff    0.047
104/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480125542e+00 | Ndiff    0.045
105/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480125248e+00 | Ndiff    0.043
106/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480124978e+00 | Ndiff    0.042
107/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480124731e+00 | Ndiff    0.040
108/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480124503e+00 | Ndiff    0.038
109/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480124294e+00 | Ndiff    0.037
110/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480124102e+00 | Ndiff    0.035
111/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123926e+00 | Ndiff    0.034
112/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123764e+00 | Ndiff    0.032
113/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123615e+00 | Ndiff    0.031
114/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123478e+00 | Ndiff    0.030
115/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123353e+00 | Ndiff    0.028
116/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123237e+00 | Ndiff    0.027
117/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123131e+00 | Ndiff    0.026
118/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480123034e+00 | Ndiff    0.025
119/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122944e+00 | Ndiff    0.024
120/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122862e+00 | Ndiff    0.023
121/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122786e+00 | Ndiff    0.022
122/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122716e+00 | Ndiff    0.021
123/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122652e+00 | Ndiff    0.020
124/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122593e+00 | Ndiff    0.019
125/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122539e+00 | Ndiff    0.019
126/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122489e+00 | Ndiff    0.018
127/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122444e+00 | Ndiff    0.017
128/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122401e+00 | Ndiff    0.016
129/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122363e+00 | Ndiff    0.016
130/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122327e+00 | Ndiff    0.015
131/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122294e+00 | Ndiff    0.014
132/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122264e+00 | Ndiff    0.014
133/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122236e+00 | Ndiff    0.013
134/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122211e+00 | Ndiff    0.013
135/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122187e+00 | Ndiff    0.012
136/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122166e+00 | Ndiff    0.012
137/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122146e+00 | Ndiff    0.011
138/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122128e+00 | Ndiff    0.011
139/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122111e+00 | Ndiff    0.010
140/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122095e+00 | Ndiff    0.010
141/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122081e+00 | Ndiff    0.009
142/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122068e+00 | Ndiff    0.009
143/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122056e+00 | Ndiff    0.009
144/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122045e+00 | Ndiff    0.008
145/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122034e+00 | Ndiff    0.008
146/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122025e+00 | Ndiff    0.008
147/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.480122016e+00 | Ndiff    0.007
148/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480122008e+00 | Ndiff    0.007
149/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480122001e+00 | Ndiff    0.007
150/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121994e+00 | Ndiff    0.007
151/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121988e+00 | Ndiff    0.006
152/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121982e+00 | Ndiff    0.006
153/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121977e+00 | Ndiff    0.006
154/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121972e+00 | Ndiff    0.006
155/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121968e+00 | Ndiff    0.005
156/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121964e+00 | Ndiff    0.005
157/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121960e+00 | Ndiff    0.005
158/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121956e+00 | Ndiff    0.005
159/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121953e+00 | Ndiff    0.004
160/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121950e+00 | Ndiff    0.004
161/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121948e+00 | Ndiff    0.004
162/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121945e+00 | Ndiff    0.004
163/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121943e+00 | Ndiff    0.004
164/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121941e+00 | Ndiff    0.004
165/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121939e+00 | Ndiff    0.004
166/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121937e+00 | Ndiff    0.003
167/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121935e+00 | Ndiff    0.003
168/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121934e+00 | Ndiff    0.003
169/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121932e+00 | Ndiff    0.003
170/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121931e+00 | Ndiff    0.003
171/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121930e+00 | Ndiff    0.003
172/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121929e+00 | Ndiff    0.003
173/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121928e+00 | Ndiff    0.003
174/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121927e+00 | Ndiff    0.002
175/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121926e+00 | Ndiff    0.002
176/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121925e+00 | Ndiff    0.002
177/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121924e+00 | Ndiff    0.002
178/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121924e+00 | Ndiff    0.002
179/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121923e+00 | Ndiff    0.002
180/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121923e+00 | Ndiff    0.002
181/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121922e+00 | Ndiff    0.002
182/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121922e+00 | Ndiff    0.002
183/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121921e+00 | Ndiff    0.002
184/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121921e+00 | Ndiff    0.002
185/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121920e+00 | Ndiff    0.002
186/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121920e+00 | Ndiff    0.001
187/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121920e+00 | Ndiff    0.001
188/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
189/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
190/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
191/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.480121919e+00 | Ndiff    0.001
192/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
193/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
194/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
195/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
196/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
197/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121918e+00 | Ndiff    0.001
198/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
199/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
200/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
201/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
202/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
203/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
204/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
205/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
206/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
207/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
208/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
209/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121917e+00 | Ndiff    0.001
210/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.001
211/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.001
212/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.001
213/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
214/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
215/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
216/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
217/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
218/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
219/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
220/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
221/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
222/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
223/1000 after      7 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
224/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
225/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
226/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
227/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
228/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
229/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
230/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
231/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
232/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
233/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
234/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
235/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
236/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
237/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
238/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
239/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
240/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
241/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
242/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
243/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
244/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
245/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
246/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
247/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
248/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
249/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
250/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
251/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
252/1000 after      8 sec. |    226.2 MiB | K    3 | loss  2.480121916e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  3/5 | alg. seed: 6168832 | data order seed: 7360256
1/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.597890645e+00 |
2/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.500334722e+00 | Ndiff   12.103
3/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.481732540e+00 | Ndiff    6.728
4/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.477128325e+00 | Ndiff    3.822
5/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.475214582e+00 | Ndiff    2.612
6/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.474025229e+00 | Ndiff    1.975
7/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.473158433e+00 | Ndiff    1.625
8/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.472487721e+00 | Ndiff    1.387
9/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.471960617e+00 | Ndiff    1.189
10/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.471546906e+00 | Ndiff    1.019
11/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.471223928e+00 | Ndiff    0.873
12/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470972980e+00 | Ndiff    0.748
13/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470778682e+00 | Ndiff    0.642
14/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470628726e+00 | Ndiff    0.551
15/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470513450e+00 | Ndiff    0.473
16/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470425312e+00 | Ndiff    0.405
17/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470358389e+00 | Ndiff    0.347
18/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470307985e+00 | Ndiff    0.297
19/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470270346e+00 | Ndiff    0.253
20/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470242474e+00 | Ndiff    0.215
21/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470221992e+00 | Ndiff    0.183
22/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.470207040e+00 | Ndiff    0.155
23/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470196186e+00 | Ndiff    0.131
24/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470188343e+00 | Ndiff    0.111
25/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470182696e+00 | Ndiff    0.094
26/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470178643e+00 | Ndiff    0.079
27/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470175740e+00 | Ndiff    0.067
28/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470173665e+00 | Ndiff    0.056
29/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470172185e+00 | Ndiff    0.047
30/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470171130e+00 | Ndiff    0.040
31/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470170380e+00 | Ndiff    0.033
32/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470169846e+00 | Ndiff    0.028
33/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470169466e+00 | Ndiff    0.024
34/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470169197e+00 | Ndiff    0.020
35/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470169005e+00 | Ndiff    0.017
36/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168870e+00 | Ndiff    0.014
37/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168773e+00 | Ndiff    0.012
38/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168705e+00 | Ndiff    0.010
39/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168657e+00 | Ndiff    0.008
40/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168622e+00 | Ndiff    0.007
41/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168598e+00 | Ndiff    0.006
42/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168581e+00 | Ndiff    0.005
43/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168569e+00 | Ndiff    0.004
44/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168560e+00 | Ndiff    0.004
45/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168554e+00 | Ndiff    0.003
46/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168550e+00 | Ndiff    0.003
47/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168547e+00 | Ndiff    0.002
48/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168544e+00 | Ndiff    0.002
49/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168543e+00 | Ndiff    0.001
50/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168542e+00 | Ndiff    0.001
51/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168541e+00 | Ndiff    0.001
52/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
53/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
54/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
55/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168540e+00 | Ndiff    0.001
56/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
57/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
58/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
59/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
60/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
61/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
62/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
63/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
64/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
65/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.470168539e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  4/5 | alg. seed: 8205184 | data order seed: 900864
1/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.625818566e+00 |
2/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.517079400e+00 | Ndiff    2.360
3/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.500595953e+00 | Ndiff    2.568
4/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.496654963e+00 | Ndiff    0.840
5/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.494337582e+00 | Ndiff    0.186
6/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.492886037e+00 | Ndiff    0.280
7/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.491894017e+00 | Ndiff    0.615
8/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.491133552e+00 | Ndiff    0.875
9/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.490496784e+00 | Ndiff    1.087
10/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.489933426e+00 | Ndiff    1.260
11/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.489419675e+00 | Ndiff    1.401
12/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.488944406e+00 | Ndiff    1.512
13/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.488502591e+00 | Ndiff    1.597
14/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.488091868e+00 | Ndiff    1.656
15/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.487710702e+00 | Ndiff    1.695
16/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.487357459e+00 | Ndiff    1.716
17/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.487030061e+00 | Ndiff    1.722
18/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.486725982e+00 | Ndiff    1.718
19/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.486442414e+00 | Ndiff    1.706
20/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.486176489e+00 | Ndiff    1.689
21/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.485925474e+00 | Ndiff    1.668
22/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.485686912e+00 | Ndiff    1.646
23/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.485458687e+00 | Ndiff    1.623
24/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.485239049e+00 | Ndiff    1.600
25/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.485026593e+00 | Ndiff    1.576
26/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484820224e+00 | Ndiff    1.554
27/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484619104e+00 | Ndiff    1.532
28/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484422613e+00 | Ndiff    1.510
29/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484230302e+00 | Ndiff    1.488
30/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.484041861e+00 | Ndiff    1.467
31/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483857084e+00 | Ndiff    1.445
32/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483675854e+00 | Ndiff    1.423
33/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483498117e+00 | Ndiff    1.402
34/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483323872e+00 | Ndiff    1.379
35/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.483153160e+00 | Ndiff    1.357
36/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482986054e+00 | Ndiff    1.333
37/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482822653e+00 | Ndiff    1.310
38/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482663075e+00 | Ndiff    1.285
39/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482507456e+00 | Ndiff    1.260
40/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482355938e+00 | Ndiff    1.234
41/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482208671e+00 | Ndiff    1.208
42/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.482065808e+00 | Ndiff    1.181
43/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481927496e+00 | Ndiff    1.153
44/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481793878e+00 | Ndiff    1.125
45/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481665086e+00 | Ndiff    1.096
46/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481541238e+00 | Ndiff    1.067
47/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481422435e+00 | Ndiff    1.037
48/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481308757e+00 | Ndiff    1.007
49/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481200261e+00 | Ndiff    0.977
50/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.481096980e+00 | Ndiff    0.946
51/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480998920e+00 | Ndiff    0.916
52/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480906060e+00 | Ndiff    0.885
53/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480818354e+00 | Ndiff    0.855
54/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480735728e+00 | Ndiff    0.824
55/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480658085e+00 | Ndiff    0.794
56/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480585302e+00 | Ndiff    0.764
57/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480517238e+00 | Ndiff    0.735
58/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480453733e+00 | Ndiff    0.706
59/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480394611e+00 | Ndiff    0.677
60/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480339683e+00 | Ndiff    0.649
61/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480288754e+00 | Ndiff    0.622
62/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480241618e+00 | Ndiff    0.596
63/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.480198068e+00 | Ndiff    0.570
64/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480157895e+00 | Ndiff    0.545
65/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480120891e+00 | Ndiff    0.521
66/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480086852e+00 | Ndiff    0.498
67/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480055578e+00 | Ndiff    0.475
68/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480026875e+00 | Ndiff    0.454
69/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.480000557e+00 | Ndiff    0.433
70/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479976447e+00 | Ndiff    0.413
71/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479954376e+00 | Ndiff    0.394
72/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479934185e+00 | Ndiff    0.376
73/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479915723e+00 | Ndiff    0.358
74/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479898850e+00 | Ndiff    0.342
75/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479883435e+00 | Ndiff    0.326
76/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479869356e+00 | Ndiff    0.311
77/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479856501e+00 | Ndiff    0.296
78/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479844764e+00 | Ndiff    0.282
79/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479834050e+00 | Ndiff    0.269
80/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479824270e+00 | Ndiff    0.257
81/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479815342e+00 | Ndiff    0.245
82/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479807191e+00 | Ndiff    0.233
83/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479799751e+00 | Ndiff    0.223
84/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479792957e+00 | Ndiff    0.212
85/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479786752e+00 | Ndiff    0.203
86/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479781086e+00 | Ndiff    0.193
87/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479775909e+00 | Ndiff    0.185
88/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479771178e+00 | Ndiff    0.176
89/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479766855e+00 | Ndiff    0.168
90/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479762903e+00 | Ndiff    0.161
91/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479759289e+00 | Ndiff    0.153
92/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479755983e+00 | Ndiff    0.147
93/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479752958e+00 | Ndiff    0.140
94/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479750190e+00 | Ndiff    0.134
95/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479747656e+00 | Ndiff    0.128
96/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479745336e+00 | Ndiff    0.122
97/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479743210e+00 | Ndiff    0.117
98/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479741263e+00 | Ndiff    0.112
99/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479739478e+00 | Ndiff    0.107
100/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479737841e+00 | Ndiff    0.102
101/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479736341e+00 | Ndiff    0.098
102/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479734964e+00 | Ndiff    0.094
103/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479733701e+00 | Ndiff    0.090
104/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479732542e+00 | Ndiff    0.086
105/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479731477e+00 | Ndiff    0.082
106/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479730500e+00 | Ndiff    0.079
107/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479729602e+00 | Ndiff    0.075
108/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.479728777e+00 | Ndiff    0.072
109/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479728019e+00 | Ndiff    0.069
110/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479727321e+00 | Ndiff    0.066
111/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479726681e+00 | Ndiff    0.064
112/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479726091e+00 | Ndiff    0.061
113/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479725549e+00 | Ndiff    0.058
114/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479725050e+00 | Ndiff    0.056
115/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479724590e+00 | Ndiff    0.054
116/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479724168e+00 | Ndiff    0.051
117/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479723778e+00 | Ndiff    0.049
118/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479723420e+00 | Ndiff    0.047
119/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479723090e+00 | Ndiff    0.045
120/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479722785e+00 | Ndiff    0.044
121/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479722505e+00 | Ndiff    0.042
122/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479722247e+00 | Ndiff    0.040
123/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479722009e+00 | Ndiff    0.039
124/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479721789e+00 | Ndiff    0.037
125/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479721587e+00 | Ndiff    0.036
126/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479721400e+00 | Ndiff    0.034
127/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479721228e+00 | Ndiff    0.033
128/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479721069e+00 | Ndiff    0.031
129/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720922e+00 | Ndiff    0.030
130/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720787e+00 | Ndiff    0.029
131/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720662e+00 | Ndiff    0.028
132/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720547e+00 | Ndiff    0.027
133/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720440e+00 | Ndiff    0.026
134/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720342e+00 | Ndiff    0.025
135/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720251e+00 | Ndiff    0.024
136/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720168e+00 | Ndiff    0.023
137/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720090e+00 | Ndiff    0.022
138/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479720019e+00 | Ndiff    0.021
139/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719953e+00 | Ndiff    0.020
140/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719892e+00 | Ndiff    0.019
141/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719836e+00 | Ndiff    0.019
142/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719783e+00 | Ndiff    0.018
143/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719735e+00 | Ndiff    0.017
144/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719691e+00 | Ndiff    0.017
145/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719650e+00 | Ndiff    0.016
146/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719612e+00 | Ndiff    0.015
147/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719577e+00 | Ndiff    0.015
148/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719544e+00 | Ndiff    0.014
149/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719514e+00 | Ndiff    0.014
150/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719486e+00 | Ndiff    0.013
151/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719461e+00 | Ndiff    0.013
152/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719437e+00 | Ndiff    0.012
153/1000 after      3 sec. |    226.2 MiB | K    3 | loss  2.479719415e+00 | Ndiff    0.012
154/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719395e+00 | Ndiff    0.011
155/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719376e+00 | Ndiff    0.011
156/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719359e+00 | Ndiff    0.010
157/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719343e+00 | Ndiff    0.010
158/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719328e+00 | Ndiff    0.010
159/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719314e+00 | Ndiff    0.009
160/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719301e+00 | Ndiff    0.009
161/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719290e+00 | Ndiff    0.009
162/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719279e+00 | Ndiff    0.008
163/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719269e+00 | Ndiff    0.008
164/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719259e+00 | Ndiff    0.008
165/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719251e+00 | Ndiff    0.007
166/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719243e+00 | Ndiff    0.007
167/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719235e+00 | Ndiff    0.007
168/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719228e+00 | Ndiff    0.006
169/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719222e+00 | Ndiff    0.006
170/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719216e+00 | Ndiff    0.006
171/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719211e+00 | Ndiff    0.006
172/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719206e+00 | Ndiff    0.006
173/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719201e+00 | Ndiff    0.005
174/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719197e+00 | Ndiff    0.005
175/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719193e+00 | Ndiff    0.005
176/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719189e+00 | Ndiff    0.005
177/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719186e+00 | Ndiff    0.005
178/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719183e+00 | Ndiff    0.004
179/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719180e+00 | Ndiff    0.004
180/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719177e+00 | Ndiff    0.004
181/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719175e+00 | Ndiff    0.004
182/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719172e+00 | Ndiff    0.004
183/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719170e+00 | Ndiff    0.004
184/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719168e+00 | Ndiff    0.003
185/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719166e+00 | Ndiff    0.003
186/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719165e+00 | Ndiff    0.003
187/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719163e+00 | Ndiff    0.003
188/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719161e+00 | Ndiff    0.003
189/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719160e+00 | Ndiff    0.003
190/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719159e+00 | Ndiff    0.003
191/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719158e+00 | Ndiff    0.003
192/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719157e+00 | Ndiff    0.003
193/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719156e+00 | Ndiff    0.002
194/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719155e+00 | Ndiff    0.002
195/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719154e+00 | Ndiff    0.002
196/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719153e+00 | Ndiff    0.002
197/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719152e+00 | Ndiff    0.002
198/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719152e+00 | Ndiff    0.002
199/1000 after      4 sec. |    226.2 MiB | K    3 | loss  2.479719151e+00 | Ndiff    0.002
200/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719150e+00 | Ndiff    0.002
201/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719150e+00 | Ndiff    0.002
202/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719149e+00 | Ndiff    0.002
203/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719149e+00 | Ndiff    0.002
204/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719149e+00 | Ndiff    0.002
205/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719148e+00 | Ndiff    0.002
206/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719148e+00 | Ndiff    0.001
207/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
208/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
209/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
210/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719147e+00 | Ndiff    0.001
211/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
212/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
213/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
214/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
215/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719146e+00 | Ndiff    0.001
216/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
217/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
218/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
219/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
220/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
221/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
222/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719145e+00 | Ndiff    0.001
223/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
224/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
225/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
226/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
227/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
228/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
229/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
230/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
231/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
232/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
233/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
234/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.001
235/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
236/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
237/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
238/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
239/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
240/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
241/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719144e+00 | Ndiff    0.000
242/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
243/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
244/1000 after      5 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
245/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
246/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
247/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
248/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
249/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
250/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
251/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
252/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
253/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
254/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
255/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
256/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
257/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
258/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
259/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
260/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
261/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
262/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
263/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
264/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
265/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
266/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
267/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
268/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
269/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
270/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
271/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
272/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
273/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
274/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
275/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
276/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
277/1000 after      6 sec. |    226.2 MiB | K    3 | loss  2.479719143e+00 | Ndiff    0.000
... done. converged.
Learn Alg: VB | task  5/5 | alg. seed: 2986368 | data order seed: 6479872
1/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.172454094e+00 |
2/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.141604650e+00 | Ndiff    2.137
3/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.132916761e+00 | Ndiff    2.207
4/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.116013242e+00 | Ndiff    5.951
5/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.079969466e+00 | Ndiff   11.990
6/1000 after      0 sec. |    226.2 MiB | K    3 | loss  3.017967278e+00 | Ndiff   19.686
7/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.936658417e+00 | Ndiff   25.772
8/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.854720770e+00 | Ndiff   26.020
9/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.778771855e+00 | Ndiff   21.413
10/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.688031379e+00 | Ndiff   17.921
11/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.590482847e+00 | Ndiff   12.677
12/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.538125017e+00 | Ndiff    5.219
13/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.514221187e+00 | Ndiff    3.415
14/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.497495257e+00 | Ndiff    2.846
15/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495853387e+00 | Ndiff    1.084
16/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495615725e+00 | Ndiff    0.686
17/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495466026e+00 | Ndiff    0.569
18/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495360868e+00 | Ndiff    0.492
19/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495282138e+00 | Ndiff    0.438
20/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495220122e+00 | Ndiff    0.397
21/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495169130e+00 | Ndiff    0.366
22/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495125609e+00 | Ndiff    0.342
23/1000 after      0 sec. |    226.2 MiB | K    3 | loss  2.495087248e+00 | Ndiff    0.322
24/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.495052510e+00 | Ndiff    0.305
25/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.495020376e+00 | Ndiff    0.290
26/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494990186e+00 | Ndiff    0.277
27/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494961538e+00 | Ndiff    0.265
28/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494934218e+00 | Ndiff    0.254
29/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494908140e+00 | Ndiff    0.243
30/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494883308e+00 | Ndiff    0.233
31/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494859775e+00 | Ndiff    0.223
32/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494837612e+00 | Ndiff    0.212
33/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494816893e+00 | Ndiff    0.202
34/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494797668e+00 | Ndiff    0.192
35/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494779964e+00 | Ndiff    0.183
36/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494763774e+00 | Ndiff    0.173
37/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494749060e+00 | Ndiff    0.163
38/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494735759e+00 | Ndiff    0.154
39/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494723785e+00 | Ndiff    0.145
40/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494713036e+00 | Ndiff    0.137
41/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494703405e+00 | Ndiff    0.129
42/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494694776e+00 | Ndiff    0.121
43/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494687038e+00 | Ndiff    0.114
44/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494680081e+00 | Ndiff    0.108
45/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494673802e+00 | Ndiff    0.102
46/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494668101e+00 | Ndiff    0.096
47/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494662889e+00 | Ndiff    0.091
48/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494658081e+00 | Ndiff    0.087
49/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494653596e+00 | Ndiff    0.083
50/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494649357e+00 | Ndiff    0.079
51/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494645290e+00 | Ndiff    0.076
52/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494641317e+00 | Ndiff    0.074
53/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494637354e+00 | Ndiff    0.072
54/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494633308e+00 | Ndiff    0.071
55/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494629063e+00 | Ndiff    0.071
56/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494624470e+00 | Ndiff    0.072
57/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494619327e+00 | Ndiff    0.073
58/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494613338e+00 | Ndiff    0.076
59/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494606055e+00 | Ndiff    0.080
60/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494596763e+00 | Ndiff    0.086
61/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494584268e+00 | Ndiff    0.094
62/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494566496e+00 | Ndiff    0.107
63/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494539723e+00 | Ndiff    0.123
64/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494497190e+00 | Ndiff    0.146
65/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494427049e+00 | Ndiff    0.177
66/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494311214e+00 | Ndiff    0.217
67/1000 after      1 sec. |    226.2 MiB | K    3 | loss  2.494130236e+00 | Ndiff    0.262
68/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.493873502e+00 | Ndiff    0.313
69/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.493523728e+00 | Ndiff    0.371
70/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.492991715e+00 | Ndiff    0.441
71/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.492034383e+00 | Ndiff    0.514
72/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.490416803e+00 | Ndiff    0.538
73/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.489022467e+00 | Ndiff    0.469
74/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.488517958e+00 | Ndiff    0.454
75/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.488074913e+00 | Ndiff    0.435
76/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.487289742e+00 | Ndiff    0.375
77/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.486119349e+00 | Ndiff    0.236
78/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.485305514e+00 | Ndiff    0.082
79/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484928040e+00 | Ndiff    0.036
80/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484776037e+00 | Ndiff    0.007
81/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484732728e+00 | Ndiff    0.012
82/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484723997e+00 | Ndiff    0.015
83/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484722544e+00 | Ndiff    0.010
84/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484722316e+00 | Ndiff    0.005
85/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484722281e+00 | Ndiff    0.003
86/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484722275e+00 | Ndiff    0.001
87/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484722274e+00 | Ndiff    0.001
88/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484722274e+00 | Ndiff    0.000
89/1000 after      2 sec. |    226.2 MiB | K    3 | loss  2.484722274e+00 | Ndiff    0.000
... done. converged.


## Do inference with L=1 sparsity¶

Kwarg to set is “nnzPerRowLP = 1”

nnzPerRowLP is read as “number of non-zero entries per row of local parameters”

This enforces that each row of resp array has ___ non zero entries

print('SPARSITY LEVEL: 1 of 3')
LP1 = full_trained_model.calc_local_params(dataset, nnzPerRowLP=1)
pprint_sparse_or_dense_resp(LP1)


Out:

SPARSITY LEVEL: 1 of 3
[[0. 0. 1.]
[0. 0. 1.]
[0. 1. 0.]
[0. 0. 1.]
[0. 0. 1.]
[0. 0. 1.]
[0. 0. 1.]
[0. 1. 0.]
[1. 0. 0.]
[0. 1. 0.]]


## Do inference with L=2 sparsity¶

print('SPARSITY LEVEL: 2 of 3')
LP2 = full_trained_model.calc_local_params(dataset, nnzPerRowLP=2)
pprint_sparse_or_dense_resp(LP2)


Out:

SPARSITY LEVEL: 2 of 3
[[0.     0.2424 0.7576]
[0.2281 0.     0.7719]
[0.     0.9262 0.0738]
[0.0427 0.     0.9573]
[0.0269 0.     0.9731]
[0.0247 0.     0.9753]
[0.     0.021  0.979 ]
[0.     0.9866 0.0134]
[0.9866 0.     0.0134]
[0.     0.9905 0.0095]]


## Do inference with L=3 sparsity¶

print('SPARSITY LEVEL: 3 of 3 (DENSE)')
LP3 = full_trained_model.calc_local_params(dataset, nnzPerRowLP=3)
pprint_sparse_or_dense_resp(LP3)

# Show the model
pylab.figure(2)
bnpy.viz.PlotComps.plotCompsFromHModel(
full_trained_model,
dataset=targeted_dataset,
)
pylab.xlabel(dataset.column_names[0])
pylab.ylabel(dataset.column_names[1])
pylab.tight_layout()
pylab.title('Trained model with targeted examples')


Out:

SPARSITY LEVEL: 3 of 3 (DENSE)
[[0.     0.2424 0.7575]
[0.2281 0.     0.7719]
[0.     0.9262 0.0738]
[0.0427 0.     0.9573]
[0.0269 0.0002 0.9729]
[0.0247 0.     0.9753]
[0.0002 0.021  0.9788]
[0.     0.9866 0.0134]
[0.9866 0.     0.0134]
[0.     0.9905 0.0095]]


## How to incorporate sparsity during training?¶

Just pass “nnzPerRowLP” kwarg to bnpy.run(…)

Total running time of the script: ( 0 minutes 25.920 seconds)

Gallery generated by Sphinx-Gallery