2,086 views

この記事は最終更新から 970日 が経過しています。

cuda-convnet2上で (15) cuda-convnetでMNIST自動認識(その2) とほぼ同じネットワーク構成で学習させてみた。

「まったく同じ」 ではなく 「ほぼ同じ」 なのは、同じに出来ない点があったため…

cuda-convnet2 では、convolution層のフィルター数 filters の値が 32の倍数でないとエラーになる。
cuda-convnet のときと同じ filters=16 を指定したら、以下のようなエラーが出て止まった。

1.1 (0.00%)...python: src/img_acts.cu:1204: void _imgActs(NVMatrix&, NVMatrix&, NVMatrix&,
int, int, int, int, int, int, int, float, float, bool):
Assertion `numFilters % (32*numGroups) == 0' failed.

1. 準備

(1) 入力データは(14) cuda-convnetでMNIST自動認識(その1) で作成したものをそのまま使用
(2) convnet.py に MNIST用data provider を追加

from convdata import ImageDataProvider, CIFARDataProvider, DummyConvNetLogRegDataProvider, MNISTDataProvider
 :
DataProvider.register_data_provider('MNIST', 'MNIST data provider', MNISTDataProvider)

(3) convdata.py に MNIST用data provider を追加実装

class MNISTDataProvider(LabeledDataProvider):
 :

(4) ネットワーク定義ファイルを作成(CIFAR10用のものをコピーして編集)
layers-MNIST.cfg

[data]
type=data
dataIdx=0

[labels]
type=data
dataIdx=1

[conv1]
type=conv
inputs=data
channels=1
filters=32
padding=0
stride=1
filterSize=5
neuron=tanh[1,1]
initW=0.0001
sumWidth=4
sharedBiases=1
gpu=0

[pool1]
type=pool
pool=max
inputs=conv1
start=0
sizeX=2
stride=2
outputsX=0
channels=32

[conv2]
type=conv
inputs=pool1
filters=32
padding=0
stride=1
filterSize=5
channels=32
neuron=tanh[1,1]
initW=0.01
sumWidth=2
sharedBiases=1

[pool2]
type=pool
pool=avg
inputs=conv2
start=0
sizeX=2
stride=2
outputsX=0
channels=32

[fcOut]
type=fc
outputs=10
inputs=pool2
initW=0.01
initB=0.1

[probs]
type=softmax
inputs=fcOut

[logprob]
type=cost.logreg
inputs=labels,probs
gpu=0

layer-params-MNIST.cfg

[conv1]
epsW=0.01
epsB=0.01
momW=0.9
momB=0.9
wc=0.0001

[conv2]
epsW=0.01
epsB=0.01
momW=0.9
momB=0.9
wc=0.0001

[fcOut]
epsW=0.01
epsB=0.01
momW=0.9
momB=0.9
wc=0.0001

[logprob]
coeff=1

2. 実行結果

10epochs で 97.0% の正解率だった。

=========================
Running on CUDA device(s) 0
Current time: Thu Sep 11 23:26:07 2014
Saving checkpoints to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
=========================
1.1 (0.00%)... logprob:  1.172366, 0.338300, 0.338300 (0.408 sec)
1.2 (1.67%)... logprob:  0.644550, 0.195300, 0.195300 (0.283 sec)
1.3 (3.33%)... logprob:  0.534813, 0.158900, 0.158900 (0.261 sec)
1.4 (5.00%)... logprob:  0.490603, 0.137000, 0.137000 (0.266 sec)
1.5 (6.67%)... logprob:  0.414907, 0.118000, 0.118000 (0.270 sec)
1.6 (8.33%)... logprob:  0.323441, 0.090700, 0.090700
======================Test output======================
logprob:  0.331363, 0.096700, 0.096700
----------------------Averages-------------------------
logprob:  0.331363, 0.096700, 0.096700
-------------------------------------------------------
Layer 'conv1' weights[0]: 5.603852e-02 [4.257733e-04] [7.597867e-03]
Layer 'conv1' biases: 4.830383e-03 [1.214175e-05]
Layer 'conv2' weights[0]: 1.724966e-02 [6.070063e-05] [3.518947e-03]
Layer 'conv2' biases: 1.526958e-02 [5.254558e-05]
Layer 'fcOut' weights[0]: 5.033513e-02 [1.485135e-04] [2.950494e-03]
Layer 'fcOut' biases: 1.051513e-01 [2.423674e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.386 sec)
2.1 (10.00%)... logprob:  0.318453, 0.086400, 0.086400 (0.262 sec)
2.2 (11.67%)... logprob:  0.310761, 0.089500, 0.089500 (0.272 sec)
2.3 (13.33%)... logprob:  0.286774, 0.080500, 0.080500 (0.277 sec)
2.4 (15.00%)... logprob:  0.278840, 0.075800, 0.075800 (0.268 sec)
2.5 (16.67%)... logprob:  0.254464, 0.074000, 0.074000 (0.260 sec)
2.6 (18.33%)... logprob:  0.205112, 0.055300, 0.055300
======================Test output======================
logprob:  0.212422, 0.060000, 0.060000
----------------------Averages-------------------------
logprob:  0.212422, 0.060000, 0.060000
-------------------------------------------------------
Layer 'conv1' weights[0]: 7.594936e-02 [4.058031e-04] [5.343074e-03]
Layer 'conv1' biases: 6.179797e-03 [1.451384e-05]
Layer 'conv2' weights[0]: 2.133178e-02 [6.337569e-05] [2.970952e-03]
Layer 'conv2' biases: 2.470282e-02 [9.729735e-05]
Layer 'fcOut' weights[0]: 6.420852e-02 [1.101083e-04] [1.714855e-03]
Layer 'fcOut' biases: 1.128721e-01 [1.821996e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.381 sec)
3.1 (20.00%)... logprob:  0.218257, 0.060000, 0.060000 (0.261 sec)
3.2 (21.67%)... logprob:  0.221481, 0.064700, 0.064700 (0.260 sec)
3.3 (23.33%)... logprob:  0.210569, 0.059100, 0.059100 (0.269 sec)
3.4 (25.00%)... logprob:  0.210470, 0.059600, 0.059600 (0.280 sec)
3.5 (26.67%)... logprob:  0.197011, 0.057800, 0.057800 (0.276 sec)
3.6 (28.33%)... logprob:  0.163356, 0.045600, 0.045600
======================Test output======================
logprob:  0.174262, 0.049500, 0.049500
----------------------Averages-------------------------
logprob:  0.174262, 0.049500, 0.049500
-------------------------------------------------------
Layer 'conv1' weights[0]: 8.960275e-02 [4.249032e-04] [4.742078e-03]
Layer 'conv1' biases: 7.505222e-03 [1.565793e-05]
Layer 'conv2' weights[0]: 2.407989e-02 [6.371664e-05] [2.646052e-03]
Layer 'conv2' biases: 3.122083e-02 [1.076589e-04]
Layer 'fcOut' weights[0]: 7.343277e-02 [1.054850e-04] [1.436484e-03]
Layer 'fcOut' biases: 1.184271e-01 [2.027215e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.390 sec)
4.1 (30.00%)... logprob:  0.177056, 0.049900, 0.049900 (0.267 sec)
4.2 (31.67%)... logprob:  0.177898, 0.052100, 0.052100 (0.260 sec)
4.3 (33.33%)... logprob:  0.175247, 0.050700, 0.050700 (0.268 sec)
4.4 (35.00%)... logprob:  0.171205, 0.047300, 0.047300 (0.276 sec)
4.5 (36.67%)... logprob:  0.165775, 0.048300, 0.048300 (0.277 sec)
4.6 (38.33%)... logprob:  0.138243, 0.037900, 0.037900
======================Test output======================
logprob:  0.145980, 0.043900, 0.043900
----------------------Averages-------------------------
logprob:  0.145980, 0.043900, 0.043900
-------------------------------------------------------
Layer 'conv1' weights[0]: 1.005846e-01 [2.952142e-04] [2.934983e-03]
Layer 'conv1' biases: 8.529577e-03 [8.077021e-06]
Layer 'conv2' weights[0]: 2.600699e-02 [4.605853e-05] [1.771006e-03]
Layer 'conv2' biases: 3.634325e-02 [5.782335e-05]
Layer 'fcOut' weights[0]: 8.008937e-02 [8.605960e-05] [1.074545e-03]
Layer 'fcOut' biases: 1.239006e-01 [1.301457e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.394 sec)
5.1 (40.00%)... logprob:  0.154051, 0.043600, 0.043600 (0.265 sec)
5.2 (41.67%)... logprob:  0.152657, 0.042800, 0.042800 (0.260 sec)
5.3 (43.33%)... logprob:  0.157166, 0.045600, 0.045600 (0.269 sec)
5.4 (45.00%)... logprob:  0.156089, 0.043000, 0.043000 (0.275 sec)
5.5 (46.67%)... logprob:  0.151101, 0.044100, 0.044100 (0.279 sec)
5.6 (48.33%)... logprob:  0.127177, 0.034000, 0.034000
======================Test output======================
logprob:  0.134478, 0.040300, 0.040300
----------------------Averages-------------------------
logprob:  0.134478, 0.040300, 0.040300
-------------------------------------------------------
Layer 'conv1' weights[0]: 1.092488e-01 [3.290545e-04] [3.011972e-03]
Layer 'conv1' biases: 9.419627e-03 [9.917402e-06]
Layer 'conv2' weights[0]: 2.765511e-02 [5.169003e-05] [1.869095e-03]
Layer 'conv2' biases: 4.023369e-02 [7.040862e-05]
Layer 'fcOut' weights[0]: 8.545814e-02 [8.551481e-05] [1.000663e-03]
Layer 'fcOut' biases: 1.289145e-01 [1.510616e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.390 sec)
6.1 (50.00%)... logprob:  0.138790, 0.039100, 0.039100 (0.256 sec)
6.2 (51.67%)... logprob:  0.136038, 0.040000, 0.040000 (0.267 sec)
6.3 (53.33%)... logprob:  0.140736, 0.043300, 0.043300 (0.271 sec)
6.4 (55.00%)... logprob:  0.140494, 0.040500, 0.040500 (0.257 sec)
6.5 (56.67%)... logprob:  0.135578, 0.042900, 0.042900 (0.260 sec)
6.6 (58.33%)... logprob:  0.117202, 0.034200, 0.034200
======================Test output======================
logprob:  0.125607, 0.037000, 0.037000
----------------------Averages-------------------------
logprob:  0.125607, 0.037000, 0.037000
-------------------------------------------------------
Layer 'conv1' weights[0]: 1.185013e-01 [3.215680e-04] [2.713624e-03]
Layer 'conv1' biases: 1.045737e-02 [8.683801e-06]
Layer 'conv2' weights[0]: 2.906979e-02 [5.124793e-05] [1.762928e-03]
Layer 'conv2' biases: 4.338235e-02 [6.073256e-05]
Layer 'fcOut' weights[0]: 8.998768e-02 [8.469566e-05] [9.411917e-04]
Layer 'fcOut' biases: 1.330563e-01 [1.375708e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.389 sec)
7.1 (60.00%)... logprob:  0.127772, 0.036500, 0.036500 (0.264 sec)
7.2 (61.67%)... logprob:  0.129048, 0.035300, 0.035300 (0.264 sec)
7.3 (63.33%)... logprob:  0.126389, 0.036900, 0.036900 (0.257 sec)
7.4 (65.00%)... logprob:  0.126924, 0.036500, 0.036500 (0.258 sec)
7.5 (66.67%)... logprob:  0.124740, 0.039200, 0.039200 (0.262 sec)
7.6 (68.33%)... logprob:  0.107535, 0.031300, 0.031300
======================Test output======================
logprob:  0.117275, 0.036500, 0.036500
----------------------Averages-------------------------
logprob:  0.117275, 0.036500, 0.036500
-------------------------------------------------------
Layer 'conv1' weights[0]: 1.253645e-01 [3.182444e-04] [2.538553e-03]
Layer 'conv1' biases: 1.130659e-02 [9.655061e-06]
Layer 'conv2' weights[0]: 3.027309e-02 [5.097299e-05] [1.683772e-03]
Layer 'conv2' biases: 4.638661e-02 [6.874600e-05]
Layer 'fcOut' weights[0]: 9.390774e-02 [8.618318e-05] [9.177431e-04]
Layer 'fcOut' biases: 1.369651e-01 [1.598123e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.395 sec)
8.1 (70.00%)... logprob:  0.118632, 0.034100, 0.034100 (0.279 sec)
8.2 (71.67%)... logprob:  0.116539, 0.033800, 0.033800 (0.284 sec)
8.3 (73.33%)... logprob:  0.119332, 0.033300, 0.033300 (0.263 sec)
8.4 (75.00%)... logprob:  0.117164, 0.035000, 0.035000 (0.259 sec)
8.5 (76.67%)... logprob:  0.115454, 0.034100, 0.034100 (0.265 sec)
8.6 (78.33%)... logprob:  0.098626, 0.027800, 0.027800
======================Test output======================
logprob:  0.111595, 0.035600, 0.035600
----------------------Averages-------------------------
logprob:  0.111595, 0.035600, 0.035600
-------------------------------------------------------
Layer 'conv1' weights[0]: 1.320663e-01 [2.742416e-04] [2.076545e-03]
Layer 'conv1' biases: 1.205624e-02 [9.303420e-06]
Layer 'conv2' weights[0]: 3.131152e-02 [4.480714e-05] [1.431011e-03]
Layer 'conv2' biases: 4.911382e-02 [6.194661e-05]
Layer 'fcOut' weights[0]: 9.736039e-02 [8.148113e-05] [8.369022e-04]
Layer 'fcOut' biases: 1.395094e-01 [1.539289e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.375 sec)
9.1 (80.00%)... logprob:  0.109055, 0.030600, 0.030600 (0.263 sec)
9.2 (81.67%)... logprob:  0.108679, 0.031500, 0.031500 (0.276 sec)
9.3 (83.33%)... logprob:  0.114341, 0.034500, 0.034500 (0.270 sec)
9.4 (85.00%)... logprob:  0.113470, 0.034200, 0.034200 (0.264 sec)
9.5 (86.67%)... logprob:  0.113821, 0.034900, 0.034900 (0.260 sec)
9.6 (88.33%)... logprob:  0.097121, 0.027200, 0.027200
======================Test output======================
logprob:  0.109638, 0.036300, 0.036300
----------------------Averages-------------------------
logprob:  0.109638, 0.036300, 0.036300
-------------------------------------------------------
Layer 'conv1' weights[0]: 1.373190e-01 [3.058636e-04] [2.227395e-03]
Layer 'conv1' biases: 1.302156e-02 [9.149435e-06]
Layer 'conv2' weights[0]: 3.226066e-02 [4.938124e-05] [1.530695e-03]
Layer 'conv2' biases: 5.098233e-02 [6.026183e-05]
Layer 'fcOut' weights[0]: 1.004117e-01 [8.105130e-05] [8.071900e-04]
Layer 'fcOut' biases: 1.419481e-01 [1.483235e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.374 sec)
10.1 (90.00%)... logprob:  0.109652, 0.031600, 0.031600 (0.265 sec)
10.2 (91.67%)... logprob:  0.106537, 0.031300, 0.031300 (0.274 sec)
10.3 (93.33%)... logprob:  0.108742, 0.031500, 0.031500 (0.277 sec)
10.4 (95.00%)... logprob:  0.107402, 0.031800, 0.031800 (0.281 sec)
10.5 (96.67%)... logprob:  0.108577, 0.033200, 0.033200 (0.265 sec)
10.6 (98.33%)... logprob:  0.089522, 0.025200, 0.025200
======================Test output======================
logprob:  0.103471, 0.032400, 0.032400
----------------------Averages-------------------------
logprob:  0.103471, 0.032400, 0.032400
-------------------------------------------------------
Layer 'conv1' weights[0]: 1.426484e-01 [3.130984e-04] [2.194896e-03]
Layer 'conv1' biases: 1.376614e-02 [7.678883e-06]
Layer 'conv2' weights[0]: 3.318742e-02 [4.514968e-05] [1.360446e-03]
Layer 'conv2' biases: 5.307174e-02 [5.835572e-05]
Layer 'fcOut' weights[0]: 1.032455e-01 [6.837120e-05] [6.622197e-04]
Layer 'fcOut' biases: 1.431666e-01 [1.101904e-04]
-------------------------------------------------------
Saved checkpoint to /home/user/cuda/cuda-convnet2/save/MNIST/ConvNet__2014-09-11_23.26.06
======================================================= (0.386 sec)
11.1 (100.00%)... logprob:  0.102062, 0.029500, 0.029500 (0.265 sec)