python保存log日志,实现用log日志画图

Python  /  管理员 发布于 7年前   291

在神经网络训练中，我们常常需要画出loss function的变化图，log日志里会显示每一次迭代的loss function的值，于是我们先把log日志保存为log.txt文档，再利用这个文档来画图。 　　

1，先来产生一个log日志。

import mxnet as mximport numpy as npimport osimport logginglogging.getLogger().setLevel(logging.DEBUG)# Training datalogging.basicConfig(filename = os.path.join(os.getcwd(), 'log.txt'), level = logging.DEBUG) # 把log日志保存为log.txttrain_data = np.random.uniform(0, 1, [100, 2])train_label = np.array([train_data[i][0] + 2 * train_data[i][1] for i in range(100)])batch_size = 1num_epoch=5# Evaluation Dataeval_data = np.array([[7,2],[6,10],[12,2]])eval_label = np.array([11,26,16])train_iter = mx.io.NDArrayIter(train_data,train_label, batch_size, shuffle=True,label_name='lin_reg_label')eval_iter = mx.io.NDArrayIter(eval_data, eval_label, batch_size, shuffle=False)X = mx.sym.Variable('data')Y = mx.sym.Variable('lin_reg_label')fully_connected_layer = mx.sym.FullyConnected(data=X, name='fc1', num_hidden = 1)lro = mx.sym.LinearRegressionOutput(data=fully_connected_layer, label=Y, name="lro")model = mx.mod.Module(  symbol = lro ,  data_names=['data'],  label_names = ['lin_reg_label'] # network structure)model.fit(train_iter, eval_iter,      optimizer_params={'learning_rate':0.005, 'momentum': 0.9},      num_epoch=20,      eval_metric='mse',)model.predict(eval_iter).asnumpy()metric = mx.metric.MSE()model.score(eval_iter, metric)

上面的代码中logging.basicConfig(filename = os.path.join(os.getcwd(), 'log.txt'), level = logging.DEBUG) # 把log日志保存为log.txt 就是把log日志保存为log.txt文件。 　　

2，log.txt文档如下。

INFO:root:Epoch[0] Train-mse=0.470638INFO:root:Epoch[0] Time cost=0.047INFO:root:Epoch[0] Validation-mse=73.642301INFO:root:Epoch[1] Train-mse=0.082987INFO:root:Epoch[1] Time cost=0.047INFO:root:Epoch[1] Validation-mse=41.625072INFO:root:Epoch[2] Train-mse=0.044817INFO:root:Epoch[2] Time cost=0.063INFO:root:Epoch[2] Validation-mse=23.743375INFO:root:Epoch[3] Train-mse=0.024459INFO:root:Epoch[3] Time cost=0.063INFO:root:Epoch[3] Validation-mse=13.511120INFO:root:Epoch[4] Train-mse=0.013431INFO:root:Epoch[4] Time cost=0.063INFO:root:Epoch[4] Validation-mse=7.670062INFO:root:Epoch[5] Train-mse=0.007408INFO:root:Epoch[5] Time cost=0.063INFO:root:Epoch[5] Validation-mse=4.344374INFO:root:Epoch[6] Train-mse=0.004099INFO:root:Epoch[6] Time cost=0.063INFO:root:Epoch[6] Validation-mse=2.455608INFO:root:Epoch[7] Train-mse=0.002274INFO:root:Epoch[7] Time cost=0.062INFO:root:Epoch[7] Validation-mse=1.385449INFO:root:Epoch[8] Train-mse=0.001263INFO:root:Epoch[8] Time cost=0.063INFO:root:Epoch[8] Validation-mse=0.780387INFO:root:Epoch[9] Train-mse=0.000703INFO:root:Epoch[9] Time cost=0.063INFO:root:Epoch[9] Validation-mse=0.438943INFO:root:Epoch[10] Train-mse=0.000391INFO:root:Epoch[10] Time cost=0.125INFO:root:Epoch[10] Validation-mse=0.246581INFO:root:Epoch[11] Train-mse=0.000218INFO:root:Epoch[11] Time cost=0.047INFO:root:Epoch[11] Validation-mse=0.138368INFO:root:Epoch[12] Train-mse=0.000121INFO:root:Epoch[12] Time cost=0.047INFO:root:Epoch[12] Validation-mse=0.077573INFO:root:Epoch[13] Train-mse=0.000068INFO:root:Epoch[13] Time cost=0.063INFO:root:Epoch[13] Validation-mse=0.043454INFO:root:Epoch[14] Train-mse=0.000038INFO:root:Epoch[14] Time cost=0.063INFO:root:Epoch[14] Validation-mse=0.024325INFO:root:Epoch[15] Train-mse=0.000021INFO:root:Epoch[15] Time cost=0.063INFO:root:Epoch[15] Validation-mse=0.013609INFO:root:Epoch[16] Train-mse=0.000012INFO:root:Epoch[16] Time cost=0.063INFO:root:Epoch[16] Validation-mse=0.007610INFO:root:Epoch[17] Train-mse=0.000007INFO:root:Epoch[17] Time cost=0.063INFO:root:Epoch[17] Validation-mse=0.004253INFO:root:Epoch[18] Train-mse=0.000004INFO:root:Epoch[18] Time cost=0.063INFO:root:Epoch[18] Validation-mse=0.002376INFO:root:Epoch[19] Train-mse=0.000002INFO:root:Epoch[19] Time cost=0.063INFO:root:Epoch[19] Validation-mse=0.001327

3，利用log.txt文件来画图。

import reimport matplotlib.pyplot as pltimport numpy as npdef main():  file = open('log.txt','r')  list = []  # search the line including accuracy  for line in file:    m=re.search('Train-mse', line)    if m:      n=re.search('[0]\.[0-9]+', line) # 正则表达式      if n is not None:        list.append(n.group()) # 提取精度数字  file.close()  plt.plot(list, 'go')  plt.plot(list, 'r')  plt.xlabel('count')  plt.ylabel('accuracy')  plt.title('Accuracy')  plt.show()if __name__ == '__main__':  main()

以上这篇python保存log日志,实现用log日志来画图就是小编分享给大家的全部内容了，希望能给大家一个参考，也希望大家多多支持。