Execution Time18.58s

Test: TMVA-DNN-MethodDL-Adam-Optimization-Cpu (Passed)
Build: master-x86_64-centos7-gcc48-dbg (olhswep22.cern.ch) on 2019-11-14 11:42:18

Test Timing: Passed
Processors1

Show Command Line
Display graphs:

Test output
Testing Method DL with Adam Optimizer for CPU backend: 
Info in <TFile::OpenFromCache>: using local cache copy of http://root.cern.ch/files/tmva_class_example.root [./files/tmva_class_example.root]
create data set info dataset
DataSetInfo              : [dataset] : Added class "Signal"
                         : Add Tree TreeS of type Signal with 6000 events
DataSetInfo              : [dataset] : Added class "Background"
                         : Add Tree TreeB of type Background with 6000 events
Factory                  : Booking method: [NON-XML-CHAR-0x1B][1mDL_ADAM_CPU[NON-XML-CHAR-0x1B][0m
                         : 
                         : Parsing option string: 
                         : ... "!H:V:ErrorStrategy=SUMOFSQUARES:WeightInitialization=XAVIERUNIFORM:InputLayout=1|1|4:BatchLayout=256|1|4:Layout=RESHAPE|1|1|4|FLAT,DENSE|128|TANH,DENSE|128|TANH,DENSE|128|TANH,DENSE|1|LINEAR:TrainingStrategy=LearningRate=1e-2,Optimizer=ADAM,Momentum=0.9,Repetitions=1,ConvergenceSteps=20,BatchSize=256,TestRepetitions=10,WeightDecay=1e-4,Regularization=L2,DropConfig=0.0+0.5+0.5+0.5,MaxEpochs=100:Architecture=CPU"
                         : The following options are set:
                         : - By User:
                         :     <none>
                         : - Default:
                         :     Boost_num: "0" [Number of times the classifier will be boosted]
                         : Parsing option string: 
                         : ... "!H:V:ErrorStrategy=SUMOFSQUARES:WeightInitialization=XAVIERUNIFORM:InputLayout=1|1|4:BatchLayout=256|1|4:Layout=RESHAPE|1|1|4|FLAT,DENSE|128|TANH,DENSE|128|TANH,DENSE|128|TANH,DENSE|1|LINEAR:TrainingStrategy=LearningRate=1e-2,Optimizer=ADAM,Momentum=0.9,Repetitions=1,ConvergenceSteps=20,BatchSize=256,TestRepetitions=10,WeightDecay=1e-4,Regularization=L2,DropConfig=0.0+0.5+0.5+0.5,MaxEpochs=100:Architecture=CPU"
                         : The following options are set:
                         : - By User:
                         :     V: "True" [Verbose output (short form of "VerbosityLevel" below - overrides the latter one)]
                         :     H: "False" [Print method-specific help message]
                         :     InputLayout: "1|1|4" [The Layout of the input]
                         :     BatchLayout: "256|1|4" [The Layout of the batch]
                         :     Layout: "RESHAPE|1|1|4|FLAT,DENSE|128|TANH,DENSE|128|TANH,DENSE|128|TANH,DENSE|1|LINEAR" [Layout of the network.]
                         :     ErrorStrategy: "SUMOFSQUARES" [Loss function: Mean squared error (regression) or cross entropy (binary classification).]
                         :     WeightInitialization: "XAVIERUNIFORM" [Weight initialization strategy]
                         :     Architecture: "CPU" [Which architecture to perform the training on.]
                         :     TrainingStrategy: "LearningRate=1e-2,Optimizer=ADAM,Momentum=0.9,Repetitions=1,ConvergenceSteps=20,BatchSize=256,TestRepetitions=10,WeightDecay=1e-4,Regularization=L2,DropConfig=0.0+0.5+0.5+0.5,MaxEpochs=100" [Defines the training strategies.]
                         : - Default:
                         :     VerbosityLevel: "Default" [Verbosity level]
                         :     VarTransform: "None" [List of variable transformations performed before training, e.g., "D_Background,P_Signal,G,N_AllClasses" for: "Decorrelation, PCA-transformation, Gaussianisation, Normalisation, each for the given class of events ('AllClasses' denotes all events of all classes, if no class indication is given, 'All' is assumed)"]
                         :     CreateMVAPdfs: "False" [Create PDFs for classifier outputs (signal and background)]
                         :     IgnoreNegWeightsInTraining: "False" [Events with negative weights are ignored in the training (but are included for testing and performance evaluation)]
                         :     RandomSeed: "0" [Random seed used for weight initialization and batch shuffling]
                         :     ValidationSize: "20%" [Part of the training data to use for validation. Specify as 0.2 or 20% to use a fifth of the data set as validation set. Specify as 100 to use exactly 100 events. (Default: 20%)]
                         : Will use now the CPU architecture !
Factory                  : [NON-XML-CHAR-0x1B][1mTrain all methods[NON-XML-CHAR-0x1B][0m
                         : Building event vectors for type 2 Signal
                         : Dataset[dataset] :  create input formulas for tree TreeS
                         : Building event vectors for type 2 Background
                         : Dataset[dataset] :  create input formulas for tree TreeB
DataSetFactory           : [dataset] : Number of events in input trees
                         : 
                         : 
                         : Number of training and testing events
                         : ---------------------------------------------------------------------------
                         : Signal     -- training events            : 1000
                         : Signal     -- testing events             : 5000
                         : Signal     -- training and testing events: 6000
                         : Background -- training events            : 1000
                         : Background -- testing events             : 5000
                         : Background -- training and testing events: 6000
                         : 
DataSetInfo              : Correlation matrix (Signal):
                         : ----------------------------------------
                         :           myvar1  myvar2    var3    var4
                         :  myvar1:  +1.000  -0.039  +0.778  +0.931
                         :  myvar2:  -0.039  +1.000  -0.111  +0.033
                         :    var3:  +0.778  -0.111  +1.000  +0.860
                         :    var4:  +0.931  +0.033  +0.860  +1.000
                         : ----------------------------------------
DataSetInfo              : Correlation matrix (Background):
                         : ----------------------------------------
                         :           myvar1  myvar2    var3    var4
                         :  myvar1:  +1.000  +0.033  +0.784  +0.931
                         :  myvar2:  +0.033  +1.000  -0.014  +0.112
                         :    var3:  +0.784  -0.014  +1.000  +0.863
                         :    var4:  +0.931  +0.112  +0.863  +1.000
                         : ----------------------------------------
DataSetFactory           : [dataset] :  
                         : 
Factory                  : [dataset] : Create Transformation "I" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'myvar1' <---> Output : variable 'myvar1'
                         : Input : variable 'myvar2' <---> Output : variable 'myvar2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "D" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'myvar1' <---> Output : variable 'myvar1'
                         : Input : variable 'myvar2' <---> Output : variable 'myvar2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "P" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'myvar1' <---> Output : variable 'myvar1'
                         : Input : variable 'myvar2' <---> Output : variable 'myvar2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "G" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'myvar1' <---> Output : variable 'myvar1'
                         : Input : variable 'myvar2' <---> Output : variable 'myvar2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
Factory                  : [dataset] : Create Transformation "D" with events from all classes.
                         : 
                         : Transformation, Variable selection : 
                         : Input : variable 'myvar1' <---> Output : variable 'myvar1'
                         : Input : variable 'myvar2' <---> Output : variable 'myvar2'
                         : Input : variable 'var3' <---> Output : variable 'var3'
                         : Input : variable 'var4' <---> Output : variable 'var4'
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :   myvar1:   0.015795     1.7576   [    -8.1442     7.2697 ]
                         :   myvar2:  -0.032371     1.1075   [    -3.9664     4.0259 ]
                         :     var3:  0.0079046     1.0833   [    -5.0373     4.2785 ]
                         :     var4:    0.15209     1.2738   [    -5.9505     4.6404 ]
                         : -----------------------------------------------------------
                         : Preparing the Decorrelation transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :   myvar1:   -0.10632     1.0000   [    -4.3319     4.1392 ]
                         :   myvar2:  -0.048116     1.0000   [    -3.6771     3.6585 ]
                         :     var3:   -0.11234     1.0000   [    -3.8237     3.8767 ]
                         :     var4:    0.31322     1.0000   [    -4.0755     3.2827 ]
                         : -----------------------------------------------------------
                         : Preparing the Principle Component (PCA) transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :   myvar1:   -0.23164     2.3251   [    -11.197     8.9872 ]
                         :   myvar2:   0.020728     1.1139   [    -4.0534     3.9437 ]
                         :     var3: -0.0058717    0.59139   [    -2.0118     1.9675 ]
                         :     var4:  0.0015685    0.33921   [   -0.99780     1.0229 ]
                         : -----------------------------------------------------------
                         : Preparing the Gaussian transformation...
                         : Preparing the Decorrelation transformation...
TFHandler_Factory        : Variable        Mean        RMS   [        Min        Max ]
                         : -----------------------------------------------------------
                         :   myvar1:  -0.068457     1.0000   [    -3.7008     8.2137 ]
                         :   myvar2:  -0.015438     1.0000   [    -3.2309     5.6408 ]
                         :     var3:  -0.036609     1.0000   [    -3.3991     8.0121 ]
                         :     var4:    0.10026     1.0000   [    -3.6179     5.5580 ]
                         : -----------------------------------------------------------
                         : Ranking input variables (method unspecific)...
IdTransformation         : Ranking result (top variable is best ranked)
                         : -------------------------------------
                         : Rank : Variable     : Separation
                         : -------------------------------------
                         :    1 : Variable 4   : 2.788e-01
                         :    2 : Variable 3   : 1.457e-01
                         :    3 : myvar1       : 8.442e-02
                         :    4 : Expression 2 : 2.854e-02
                         : -------------------------------------
Factory                  : Train method: DL_ADAM_CPU for Classification
                         : 
                         : Start of deep neural network training on CPU using (for ROOT-IMT) nthreads = 1
                         : 
                         : *****   Deep Learning Network *****
DEEP NEURAL NETWORK:   Depth = 5  Input = ( 1, 1, 4 )  Batch size = 256  Loss function = R
	Layer 0	 RESHAPE Layer 	 Input = ( 1 , 1 , 4 ) 	Output = ( 1 , 256 , 4 ) 
	Layer 1	 DENSE Layer: 	 ( Input =     4 , Width =   128 ) 	Output = (  1 ,   256 ,   128 ) 	 Activation Function = Tanh	 Dropout prob. = 0.5
	Layer 2	 DENSE Layer: 	 ( Input =   128 , Width =   128 ) 	Output = (  1 ,   256 ,   128 ) 	 Activation Function = Tanh	 Dropout prob. = 0.5
	Layer 3	 DENSE Layer: 	 ( Input =   128 , Width =   128 ) 	Output = (  1 ,   256 ,   128 ) 	 Activation Function = Tanh	 Dropout prob. = 0.5
	Layer 4	 DENSE Layer: 	 ( Input =   128 , Width =     1 ) 	Output = (  1 ,   256 ,     1 ) 	 Activation Function = Identity
                         : Using 1600 events for training and 400 for testing
                         : Training phase 1 of 1:  Optimizer ADAM Learning rate = 0.01 regularization 2 minimum error = 0.495944
                         : --------------------------------------------------------------
                         :      Epoch |   Train Err.   Val. Err.  t(s)/epoch   t(s)/Loss   nEvents/s Conv. Steps
                         : --------------------------------------------------------------
                         :         10 Minimum Test error found - save the configuration 
                         :         10 |     0.223844    0.214203    0.134875    0.101216     12312.3           0
                         :         20 Minimum Test error found - save the configuration 
                         :         20 |     0.219737    0.212965    0.148226    0.101661     11125.6           0
                         :         30 Minimum Test error found - save the configuration 
                         :         30 |     0.214995    0.206654    0.190264    0.157062     8799.38           0
                         :         40 Minimum Test error found - save the configuration 
                         :         40 |     0.206983    0.202224    0.173371    0.105652     9434.57           0
                         :         50 Minimum Test error found - save the configuration 
                         :         50 |     0.198318    0.192919    0.155917     0.14874     10890.3           0
                         :         60 Minimum Test error found - save the configuration 
                         :         60 |     0.198778    0.192068    0.157665    0.160483     10846.2           0
                         :         70 Minimum Test error found - save the configuration 
                         :         70 |     0.178527    0.175218    0.158645    0.101682     10345.1           0
                         :         80 Minimum Test error found - save the configuration 
                         :         80 |     0.176187    0.169624    0.147414    0.102652     11199.5           0
                         :         90 |     0.192622    0.182218     0.13711   0.0966069     12051.9          10
                         :        100 Minimum Test error found - save the configuration 
                         :        100 |     0.163912    0.159687    0.134044    0.101599     12398.7           0
                         : 
                         : Elapsed time for training with 2000 events: 15.4 sec         
                         : Evaluate deep neural network on CPU using batches with size = 256
                         : 
DL_ADAM_CPU              : [dataset] : Evaluation of DL_ADAM_CPU on training sample (2000 events)
                         : Elapsed time for evaluation of 2000 events: 0.127 sec       
                         : Creating xml weight file: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAClassification_DL_ADAM_CPU.weights.xml[NON-XML-CHAR-0x1B][0m
                         : Creating standalone class: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAClassification_DL_ADAM_CPU.class.C[NON-XML-CHAR-0x1B][0m
Factory                  : Training finished
                         : 
                         : Ranking input variables (method specific)...
                         : No variable ranking supplied by classifier: DL_ADAM_CPU
Factory                  : === Destroy and recreate all methods via weight files for testing ===
                         : 
                         : Reading weight file: [NON-XML-CHAR-0x1B][0;36mdataset/weights/TMVAClassification_DL_ADAM_CPU.weights.xml[NON-XML-CHAR-0x1B][0m
==> Wrote root file: TMVA_DNN_ADAM_CPU.root
==> TMVAClassification is done!