Execution Time6.99s

Test: tutorial-tmva-TMVAMulticlass (Passed)
Build: master-x86_64-fedora31-gcc9 (root-fedora-31-1.cern.ch) on 2019-11-14 00:48:30

Test Timing: Passed
Processors1

Show Command Line
Display graphs:

Test output
Processing /build/night/LABEL/ROOT-fedora31/SPEC/cxx17/V/master/root/tutorials/tmva/TMVAMulticlass.C...

==> Start TMVAMulticlass
create data set info dataset
--- TMVAMulticlass   : Accessing ./tmva_example_multiple_background.root
DataSetInfo              : [dataset] : Added class "Signal"
                         : Add Tree TreeS of type Signal with 200 events
DataSetInfo              : [dataset] : Added class "bg0"
                         : Add Tree TreeB0 of type bg0 with 200 events
DataSetInfo              : [dataset] : Added class "bg1"
                         : Add Tree TreeB1 of type bg1 with 200 events
DataSetInfo              : [dataset] : Added class "bg2"
                         : Add Tree TreeB2 of type bg2 with 200 events
                         : Dataset[dataset] : Class index : 0  name : Signal
                         : Dataset[dataset] : Class index : 1  name : bg0
                         : Dataset[dataset] : Class index : 2  name : bg1
                         : Dataset[dataset] : Class index : 3  name : bg2
Factory                  : Booking method: [NON-XML-CHAR-0x1B][1mBDTG[NON-XML-CHAR-0x1B][0m
                         : 
                         : the option NegWeightTreatment=InverseBoostNegWeights does not exist for BoostType=Grad
                         : --> change to new default NegWeightTreatment=Pray
                         : Building event vectors for type 2 Signal
                         : Dataset[dataset] :  create input formulas for tree TreeS
                         : Building event vectors for type 2 bg0
                         : Dataset[dataset] :  create input formulas for tree TreeB0
                         : Building event vectors for type 2 bg1
                         : Dataset[dataset] :  create input formulas for tree TreeB1
                         : Building event vectors for type 2 bg2
                         : Dataset[dataset] :  create input formulas for tree TreeB2
DataSetFactory           : [dataset] : Number of events in input trees
                         : 
                         : 
                         : 
                         : 
                         : Number of training and testing events
                         : ---------------------------------------------------------------------------
                         : Signal -- training events            : 100
                         : Signal -- testing events             : 100
                         : Signal -- training and testing events: 200
                         : bg0    -- training events            : 100
                         : bg0    -- testing events             : 100
                         : bg0    -- training and testing events: 200
                         : bg1    -- training events            : 100
                         : bg1    -- testing events             : 100
                         : bg1    -- training and testing events: 200
                         : bg2    -- training events            : 100
                         : bg2    -- testing events             : 100
                         : bg2    -- training and testing events: 200
                         : 
DataSetInfo              : Correlation matrix (Signal):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  +0.427  +0.620  +0.834
                         :    var2:  +0.427  +1.000  +0.756  +0.779
                         :    var3:  +0.620  +0.756  +1.000  +0.854
                         :    var4:  +0.834  +0.779  +0.854  +1.000
                         : ----------------------------------------
DataSetInfo              : Correlation matrix (bg0):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  +0.390  +0.543  +0.801
                         :    var2:  +0.390  +1.000  +0.787  +0.768
                         :    var3:  +0.543  +0.787  +1.000  +0.837
                         :    var4:  +0.801  +0.768  +0.837  +1.000
                         : ----------------------------------------
DataSetInfo              : Correlation matrix (bg1):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  +0.409  +0.627  +0.844
                         :    var2:  +0.409  +1.000  +0.729  +0.754
                         :    var3:  +0.627  +0.729  +1.000  +0.850
                         :    var4:  +0.844  +0.754  +0.850  +1.000
                         : ----------------------------------------
DataSetInfo              : Correlation matrix (bg2):
                         : ----------------------------------------
                         :             var1    var2    var3    var4
                         :    var1:  +1.000  -0.638  +0.131  +0.235
                         :    var2:  -0.638  +1.000  -0.074  -0.116
                         :    var3:  +0.131  -0.074  +1.000  -0.006
                         :    var4:  +0.235  -0.116  -0.006  +1.000
                         : ----------------------------------------
DataSetFactory           : [dataset] :  
                         : 
Factory                  : Booking method: [NON-XML-CHAR-0x1B][1mMLP[NON-XML-CHAR-0x1B][0m
                         : 
MLP                      : Building Network. 
                         : Initializing weights
Factory                  : [NON-XML-CHAR-0x1B][1mTrain all methods[NON-XML-CHAR-0x1B][0m
Factory                  : [dataset] : Create Transformation "I" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "D" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "P" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "G" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "D" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'var1' <---> Output : variable 'var1'
                         : Input : variable 'var2' <---> Output : variable 'var2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.055064    0.98047   [    -3.1150     2.2852 ]
                         :     var2:    0.36366     1.0875   [    -3.6952     3.1113 ]
                         :     var3:    0.17759     1.1645   [    -3.3587     3.9796 ]
                         :     var4:   0.037620     1.2554   [    -3.7913     4.1179 ]
                         : -----------------------------------------------------------
                         : Preparing the Decorrelation transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.055452     1.0000   [    -2.9725     2.9280 ]
                         :     var2:    0.36352     1.0000   [    -2.5888     2.6502 ]
                         :     var3:   0.098247     1.0000   [    -2.3882     2.6768 ]
                         :     var4:   -0.11022     1.0000   [    -2.7533     2.5119 ]
                         : -----------------------------------------------------------
                         : Preparing the Principle Component (PCA) transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1: 1.2858e-09     1.8198   [    -7.0149     6.1518 ]
                         :     var2: 1.0256e-09    0.90985   [    -2.6803     2.5579 ]
                         :     var3: 2.0399e-09    0.78177   [    -1.8331     2.4843 ]
                         :     var4:-2.9686e-10    0.57053   [    -1.9659     1.5575 ]
                         : -----------------------------------------------------------
                         : Preparing the Gaussian transformation...
                         : Preparing the Decorrelation transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:   0.098573     1.0000   [    -1.6751     6.2421 ]
                         :     var2:   0.085657     1.0000   [    -1.8847     5.8820 ]
                         :     var3:   0.062179     1.0000   [    -2.2131     5.5785 ]
                         :     var4:   0.047258     1.0000   [    -2.5264     5.2132 ]
                         : -----------------------------------------------------------
                         : Ranking input variables (method unspecific)...
Factory                  : Train method: BDTG for Multiclass classification
                         : 
                         : Training 1000 Decision Trees ... patience please
0%, time left: unknown
6%, time left: 0 sec
12%, time left: 0 sec
18%, time left: 0 sec
25%, time left: 0 sec
31%, time left: 0 sec
37%, time left: 0 sec
43%, time left: 0 sec
50%, time left: 0 sec
56%, time left: 0 sec
62%, time left: 0 sec
68%, time left: 0 sec
75%, time left: 0 sec
81%, time left: 0 sec
87%, time left: 0 sec
93%, time left: 0 sec
                         : Elapsed time for training with 400 events: 0.888 sec         
                         : Dataset[dataset] : Create results for training
                         : Dataset[dataset] : Multiclass evaluation of BDTG on training sample
0%, time left: unknown
6%, time left: 0 sec
12%, time left: 0 sec
19%, time left: 0 sec
25%, time left: 0 sec
31%, time left: 0 sec
37%, time left: 0 sec
44%, time left: 0 sec
50%, time left: 0 sec
56%, time left: 0 sec
62%, time left: 0 sec
69%, time left: 0 sec
75%, time left: 0 sec
81%, time left: 0 sec
87%, time left: 0 sec
94%, time left: 0 sec
                         : Dataset[dataset] : Elapsed time for evaluation of 400 events: 0.261 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating xml weight file: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAMulticlass_BDTG.weights.xml[NON-XML-CHAR-0x1B][0m
                         : Creating standalone class: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAMulticlass_BDTG.class.C[NON-XML-CHAR-0x1B][0m
                         : TMVAMulticlass.root:/dataset/Method_BDT/BDTG
Factory                  : Training finished
                         : 
Factory                  : Train method: MLP for Multiclass classification
                         : 
                         : Training Network
                         : 
0%, time left: unknown
6%, time left: 4 sec
12%, time left: 4 sec
18%, time left: 3 sec
25%, time left: 2 sec
31%, time left: 2 sec
37%, time left: 2 sec
43%, time left: 1 sec
50%, time left: 1 sec
56%, time left: 1 sec
62%, time left: 1 sec
68%, time left: 0 sec
75%, time left: 0 sec
81%, time left: 0 sec
87%, time left: 0 sec
93%, time left: 0 sec
                         : Elapsed time for training with 400 events: 2.78 sec         
                         : Dataset[dataset] : Create results for training
                         : Dataset[dataset] : Multiclass evaluation of MLP on training sample
0%, time left: unknown
6%, time left: 0 sec
12%, time left: 0 sec
19%, time left: 0 sec
25%, time left: 0 sec
31%, time left: 0 sec
37%, time left: 0 sec
44%, time left: 0 sec
50%, time left: 0 sec
56%, time left: 0 sec
62%, time left: 0 sec
69%, time left: 0 sec
75%, time left: 0 sec
81%, time left: 0 sec
87%, time left: 0 sec
94%, time left: 0 sec
                         : Dataset[dataset] : Elapsed time for evaluation of 400 events: 0.00247 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating xml weight file: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAMulticlass_MLP.weights.xml[NON-XML-CHAR-0x1B][0m
                         : Creating standalone class: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAMulticlass_MLP.class.C[NON-XML-CHAR-0x1B][0m
                         : Write special histos to file: TMVAMulticlass.root:/dataset/Method_MLP/MLP
Factory                  : Training finished
                         : 
                         : Ranking input variables (method specific)...
BDTG                     : Ranking result (top variable is best ranked)
                         : --------------------------------------
                         : Rank : Variable  : Variable Importance
                         : --------------------------------------
                         :    1 : var2      : 2.709e-01
                         :    2 : var4      : 2.683e-01
                         :    3 : var1      : 2.426e-01
                         :    4 : var3      : 2.181e-01
                         : --------------------------------------
MLP                      : Ranking result (top variable is best ranked)
                         : -----------------------------
                         : Rank : Variable  : Importance
                         : -----------------------------
                         :    1 : var4      : 3.424e+01
                         :    2 : var2      : 3.358e+01
                         :    3 : var1      : 3.061e+01
                         :    4 : var3      : 2.602e+01
                         : -----------------------------
Factory                  : === Destroy and recreate all methods via weight files for testing ===
                         : 
                         : Reading weight file: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAMulticlass_BDTG.weights.xml[NON-XML-CHAR-0x1B][0m
                         : Reading weight file: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAMulticlass_MLP.weights.xml[NON-XML-CHAR-0x1B][0m
MLP                      : Building Network. 
                         : Initializing weights
Factory                  : [NON-XML-CHAR-0x1B][1mTest all methods[NON-XML-CHAR-0x1B][0m
Factory                  : Test method: BDTG for Multiclass classification performance
                         : 
                         : Dataset[dataset] : Create results for testing
                         : Dataset[dataset] : Multiclass evaluation of BDTG on testing sample
0%, time left: unknown
6%, time left: 0 sec
12%, time left: 0 sec
19%, time left: 0 sec
25%, time left: 0 sec
31%, time left: 0 sec
37%, time left: 0 sec
44%, time left: 0 sec
50%, time left: 0 sec
56%, time left: 0 sec
62%, time left: 0 sec
69%, time left: 0 sec
75%, time left: 0 sec
81%, time left: 0 sec
87%, time left: 0 sec
94%, time left: 0 sec
                         : Dataset[dataset] : Elapsed time for evaluation of 400 events: 0.115 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
Factory                  : Test method: MLP for Multiclass classification performance
                         : 
                         : Dataset[dataset] : Create results for testing
                         : Dataset[dataset] : Multiclass evaluation of MLP on testing sample
0%, time left: unknown
6%, time left: 0 sec
12%, time left: 0 sec
19%, time left: 0 sec
25%, time left: 0 sec
31%, time left: 0 sec
37%, time left: 0 sec
44%, time left: 0 sec
50%, time left: 0 sec
56%, time left: 0 sec
62%, time left: 0 sec
69%, time left: 0 sec
75%, time left: 0 sec
81%, time left: 0 sec
87%, time left: 0 sec
94%, time left: 0 sec
                         : Dataset[dataset] : Elapsed time for evaluation of 400 events: 0.00241 sec       
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
Factory                  : [NON-XML-CHAR-0x1B][1mEvaluate all methods[NON-XML-CHAR-0x1B][0m
                         : Evaluate multiclass classification method: BDTG
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
TFHandler_BDTG           : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:    0.12222    0.94478   [    -2.3727     2.9998 ]
                         :     var2:    0.36466    0.91063   [    -1.9349     2.3468 ]
                         :     var3:    0.24555     1.0146   [    -2.4774     3.9796 ]
                         :     var4:   0.046576     1.1949   [    -2.9030     3.3317 ]
                         : -----------------------------------------------------------
                         : Evaluate multiclass classification method: MLP
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
                         : Creating multiclass response histograms...
                         : Creating multiclass performance histograms...
TFHandler_MLP            : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :     var1:    0.12222    0.94478   [    -2.3727     2.9998 ]
                         :     var2:    0.36466    0.91063   [    -1.9349     2.3468 ]
                         :     var3:    0.24555     1.0146   [    -2.4774     3.9796 ]
                         :     var4:   0.046576     1.1949   [    -2.9030     3.3317 ]
                         : -----------------------------------------------------------
                         : 
                         : 1-vs-rest performance metrics per class
                         : -------------------------------------------------------------------------------------------------------
                         : 
                         : Considers the listed class as signal and the other classes
                         : as background, reporting the resulting binary performance.
                         : A score of 0.820 (0.850) means 0.820 was acheived on the
                         : test set and 0.850 on the training set.
                         : 
                         : Dataset        MVA Method     ROC AUC        Sig eff@B=0.01 Sig eff@B=0.10 Sig eff@B=0.30 
                         : Name:          / Class:       test  (train)  test  (train)  test  (train)  test  (train)  
                         : 
                         : dataset        BDTG           
                         : ------------------------------
                         :                Signal         0.960 (1.000)  0.390 (1.000)  0.890 (1.000)  1.000 (1.000)  
                         :                bg0            0.866 (0.983)  0.210 (0.770)  0.590 (0.950)  0.830 (1.000)  
                         :                bg1            0.923 (0.982)  0.190 (0.710)  0.780 (0.950)  0.910 (1.000)  
                         :                bg2            0.963 (0.999)  0.690 (0.980)  0.890 (1.000)  0.970 (1.000)  
                         : 
                         : dataset        MLP            
                         : ------------------------------
                         :                Signal         0.966 (0.985)  0.370 (0.580)  0.950 (0.970)  0.990 (1.000)  
                         :                bg0            0.867 (0.979)  0.180 (0.580)  0.670 (0.970)  0.880 (1.000)  
                         :                bg1            0.924 (0.988)  0.210 (0.820)  0.700 (0.990)  0.980 (0.990)  
                         :                bg2            0.912 (0.989)  0.540 (0.810)  0.800 (0.980)  0.890 (0.990)  
                         : 
                         : -------------------------------------------------------------------------------------------------------
                         : 
                         : 
                         : Confusion matrices for all methods
                         : -------------------------------------------------------------------------------------------------------
                         : 
                         : Does a binary comparison between the two classes given by a 
                         : particular row-column combination. In each case, the class 
                         : given by the row is considered signal while the class given 
                         : by the column index is considered background.
                         : 
                         : === Showing confusion matrix for method : BDTG           
                         : (Signal Efficiency for Background Efficiency 0.01%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              1.000 (0.360)  1.000 (0.430)  1.000 (0.360) 
                         :  bg0            0.830 (0.290)  -              0.470 (0.070)  0.940 (0.570) 
                         :  bg1            0.990 (0.550)  0.390 (0.150)  -              0.810 (0.330) 
                         :  bg2            1.000 (0.690)  0.980 (0.540)  0.930 (0.700)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.10%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              1.000 (0.840)  1.000 (0.990)  1.000 (0.800) 
                         :  bg0            1.000 (0.540)  -              0.830 (0.490)  1.000 (0.830) 
                         :  bg1            1.000 (0.900)  0.820 (0.650)  -              0.980 (0.780) 
                         :  bg2            1.000 (0.910)  1.000 (0.900)  1.000 (0.860)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.30%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              1.000 (0.990)  1.000 (1.000)  1.000 (1.000) 
                         :  bg0            1.000 (0.840)  -              0.960 (0.790)  1.000 (0.960) 
                         :  bg1            1.000 (0.990)  0.960 (0.880)  -              1.000 (0.910) 
                         :  bg2            1.000 (1.000)  1.000 (0.970)  1.000 (0.970)  -             
                         : 
                         : === Showing confusion matrix for method : MLP            
                         : (Signal Efficiency for Background Efficiency 0.01%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.330 (0.370)  1.000 (0.850)  0.580 (0.090) 
                         :  bg0            0.410 (0.050)  -              0.070 (0.170)  0.940 (0.320) 
                         :  bg1            0.960 (0.670)  0.580 (0.060)  -              0.830 (0.280) 
                         :  bg2            0.720 (0.680)  0.950 (0.180)  0.880 (0.540)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.10%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              0.970 (0.950)  1.000 (0.980)  0.890 (0.780) 
                         :  bg0            0.970 (0.690)  -              0.900 (0.460)  1.000 (0.830) 
                         :  bg1            0.990 (1.000)  0.950 (0.570)  -              0.990 (0.660) 
                         :  bg2            0.960 (0.820)  0.990 (0.820)  0.990 (0.750)  -             
                         : 
                         : (Signal Efficiency for Background Efficiency 0.30%)
                         : ---------------------------------------------------
                         :                 Signal         bg0            bg1            bg2           
                         :                  test (train)   test (train)   test (train)   test (train) 
                         :  Signal         -              1.000 (0.970)  1.000 (1.000)  1.000 (0.990) 
                         :  bg0            1.000 (0.880)  -              0.970 (0.780)  1.000 (0.920) 
                         :  bg1            1.000 (1.000)  0.990 (0.800)  -              0.990 (0.960) 
                         :  bg2            0.990 (0.850)  1.000 (0.910)  0.990 (0.920)  -             
                         : 
                         : -------------------------------------------------------------------------------------------------------
                         : 
Dataset:dataset          : Created tree 'TestTree' with 400 events
                         : 
Dataset:dataset          : Created tree 'TrainTree' with 400 events
                         : 
Factory                  : [NON-XML-CHAR-0x1B][1mThank you for using TMVA![NON-XML-CHAR-0x1B][0m
                         : [NON-XML-CHAR-0x1B][1mFor citation information, please visit: http://tmva.sf.net/citeTMVA.html[NON-XML-CHAR-0x1B][0m
==> Wrote root file: TMVAMulticlass.root
==> TMVAMulticlass is done!