Where the model pays attention to

https://jacobgil.github.io/deeplearning/class-activation-maps

Regularization technique

https://www.analyticsvidhya.com/blog/2018/04/fundamentals-deep-learning-regularization-techniques/

On batch size

https://stats.stackexchange.com/questions/140811/how-large-should-the-batch-size-be-for-stochastic-gradient-descent


https://datascience.stackexchange.com/questions/10204/should-i-take-random-elements-for-mini-batch-gradient-descent

https://machinelearningmastery.com/gentle-introduction-mini-batch-gradient-descent-configure-batch-size/

On fine-tuning of neural network

• 
  https://github.com/BVLC/caffe/issues/967
• 
  https://flyyufelix.github.io/2016/10/03/fine-tuning-in-keras-part1.html
• 
  

On creating lmdb file for caffe

1. This article was helpful for me:
   http://deepdish.io/2015/04/28/creating-lmdb-in-python/
2. Caffe's datum, being blob, stores data as a binary block and its dimensions are recorded as its properties(datum.height and datum.width).
   
   I was not aware of the fact that OpenCV's Mat instance can be created in a similar manner; give a data block and its dimensions.

   

   https://docs.opencv.org/2.4.9/modules/core/doc/basic_structures.html

   
   
3. "does fstream read/write move file pointer"
   

   

   https://stackoverflow.com/questions/14148238/does-fstream-read-write-move-file-pointer

4. Get a value of a specific key?
   https://github.com/drycpp/lmdbxx/issues/1
   https://blog.csdn.net/dcrmg/article/details/79144507
5. tips to use lmdb
6. https://blogs.kolabnow.com/2018/06/07/a-short-guide-to-lmdb
7. Caffeを通してCNNを理解する #3
   http://mugichoko.hatenablog.com/entry/2017/11/30/230019
8. multi-label
   https://github.com/BVLC/caffe/issues/1698
   http://deepdish.io/2015/04/28/creating-lmdb-in-python/
   http://blog.kostyaev.me/blog/Multilabel-Dataset
   https://stackoverflow.com/questions/31774953/test-labels-for-regression-caffe-float-not-allowed/31808324#31808324
   https://stackoverflow.com/questions/33627888/generating-lmdb-for-caffe
9. merging multiple lmdb fiels:
   https://stackoverflow.com/questions/33443020/merge-two-lmdb-databases-for-feeding-to-the-network-caffe
10. Though not being from an answer, the code posted by OP was helpful in that it teaches me we can divide writing to database.
    https://stackoverflow.com/questions/31649216/writing-data-to-lmdb-with-python-very-slow
    Note that, .begin resets transaction
    https://lmdb.readthedocs.io/en/release/#lmdb.Transaction
    In caffe's convert_imageset, transaction is committed and reset.
    https://github.com/BVLC/caffe/blob/master/tools/convert_imageset.cpp

#// Repeat calls to abort() have no effect after a previously successful commit() or abort(), or after the associated Environment has been closed.

# Transaction commits automatically:
with env.begin(write=True) as txn:
    txn.put('a', 'b')

1. 
   

Open training and test dataset for machine learning

https://www.quora.com/Where-can-I-find-a-data-set-of-images-to-train-and-test-my-CNN-on-classification-and-object-localization

https://medium.com/playment/top-10-open-image-datasets-for-machine-learning-research-93ab9c18bed1

https://skymind.ai/wiki/open-datasets

legend and color in matplotlib's scatter plot

As in the example you mentioned, call plt.scatter for each group:

import numpy as np
from matplotlib import pyplot as plt

scatter_x = np.array([1,2,3,4,5])
scatter_y = np.array([5,4,3,2,1])
group = np.array([1,3,2,1,3])
cdict = {1: 'red', 2: 'blue', 3: 'green'}

fig, ax = plt.subplots()
for g in np.unique(group):
    ix = np.where(group == g)
    ax.scatter(scatter_x[ix], scatter_y[ix], c = cdict[g], label = g, s = 100)
ax.legend()
plt.show()

(from https://stackoverflow.com/questions/47006268/matplotlib-scatter-plot-with-color-label-and-legend-specified-by-c-option)

I got stuck
A solution was found in SO(https://stackoverflow.com/questions/3551242/numpy-index-slice-without-losing-dimension-information):

X[[10],:]

or

I = array([10])
X[I,:]

The dimensionality of an array is preserved when indexing is performed by a list (or an array) of indexes. This is nice because it leaves you with the choice between keeping the dimension and squeezing.

• This did what I wanted: to generate arbitrary number of colors automatically.

https://stackoverflow.com/questions/12236566/setting-different-color-for-each-series-in-scatter-plot-on-matplotlib

Some commands to work with lmdb

• Get the number of records in lmdb: stat() returns a dictionary object:
  (https://stackoverflow.com/questions/34420375/count-number-of-records-in-lmdb-databse-with-python)
  

• lmdb_env = lmdb.open(lmdb_file_name, readonly=True)
  print lmdb_env.stat()['entries']
  

• 
  
• s

Reading mnist data

https://groups.google.com/forum/#!topic/digits-users/CzHG1aHizsw

http://deepdish.io/2015/04/28/creating-lmdb-in-python/

https://stackoverflow.com/questions/33117607/caffe-reading-lmdb-from-python

Some notes on Caffe

1. To deploy a model to run feeding a single input file, use the following:
   
   cd caffe ./build/examples/cpp_classification/classification.bin path/to/deploy.prototxt path/to/caffemodel path/to/mean.binaryproto path/to/labels.txt path/to/image.jpg
   
   https://groups.google.com/forum/#!topic/caffe-users/zMVHKdX1vj8 
2. Difference between caffe.Net and caffe.classifier'
   

   

   https://github.com/BVLC/caffe/blob/master/python/caffe/classifier.py
   (commit id = 0050c20)

3. Deploy
   

   

   https://github.com/BVLC/caffe/wiki/Using-a-Trained-Network:-Deploy

   
   
4. In docker's caffe, importing pyplot causes an error, demanding python-tk package's install.
   This is due to that pyplot's backend is prepared for interactive mode and can be circumvented by executing matplotlib.use('Agg').

   

   https://matplotlib.org/faq/howto_faq.html

   
   Note that it seems that 'import caffe' implicitly imports
   
5. We can feed input data in an array format directly to Caffe's network.

   

   http://christopher5106.github.io/deep/learning/2015/09/04/Deep-learning-tutorial-on-Caffe-Technology.html

   
   
6. There is a log parser inside Caffe

   

   https://stackoverflow.com/questions/31978186/monitor-training-validation-process-in-caffe

   
   
7. The order of "mean" parameters are BGR for reference_Caffe_model; The following answer 

   

   https://groups.google.com/forum/#!topic/caffe-users/9opH6AW3Irw

   
   
8. To feed data to caffe's network, apart from db files such as lmdb or hdf5, one can use normal image format files by reading them via "ImageData" layer.
9. To copy weights from layer to layer, just simply copy its blob data, a technique called "Net Surgery"(https://github.com/BVLC/caffe/blob/master/examples/net_surgery.ipynb).

   

   https://stackoverflow.com/questions/46420040/can-you-copy-the-weights-from-just-the-first-3-layers-of-a-network-not-exactly

10. https://github.com/BVLC/caffe/issues/2152

• "Blob is more general, and the real currency for data in Caffe, so life could be simpler with Blob alone."
• "A Blob<uint8> could be used for storing data at lower precision then casting to Blob<float> for computation. This takes care of the use case of Datum for int / less-than-float precision data storage."
• "The one responsibility of Datum that is distinct from Blob is holding encoded image data. For this purpose, the current Datum dimensions and implementation are fine."(https://github.com/BVLC/caffe/issues/2152)

2. d

Basics in Docker

• Major difference between docker run and docker start is that the former first creates the container and then starts it.
• To install some application you may need to run the container by root(you may also need to apt-get update to update package list).
  
  from https://stackoverflow.com/questions/30853247/how-do-i-edit-a-file-after-i-shell-to-a-docker-container
• You can exec a container not only by container id but also by its name.
• To  "Copy files/folders between a container and the local filesystem"
  docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH
  docker cp [OPTIONS] SRC_PATH CONTAINER:DEST_PATH
  For example,
  docker cp myFile.txt container_name:/home/work
• To exit from container without stopping one, ctrl+p and ctrl+q.
• Run multiple terminals in a single running container:
  docker exec -it <container> bash
• If you fail to run or start a container with some directory in the host machine bound, doubt the shared drives' setting.
  https://www.v2ex.com/t/527183
• We can mount multiple volumes or disks by adding --mount options

https://docs.docker.com/storage/bind-mounts/

•