It is known that He initialization works better with ReLU since Xavier kill half of the input. So, in layers.py
self.vars['weights'] = tf.get_variable('weights', shape=(input_dim, output_dim),
dtype=tf.float32,
initializer=tf.contrib.layers.variance_scaling_initializer(),
regularizer=tf.contrib.layers.l2_regularizer(FLAGS.weight_decay))
JFR.