Monday, March 18, 2019

On creating lmdb file for caffe


  1. This article was helpful for me:
    http://deepdish.io/2015/04/28/creating-lmdb-in-python/
  2. Caffe's datum, being blob, stores data as a binary block and its dimensions are recorded as its properties(datum.height and datum.width).

    I was not aware of the fact that OpenCV's Mat instance can be created in a similar manner; give a data block and its dimensions.
    https://docs.opencv.org/2.4.9/modules/core/doc/basic_structures.html

  3. "does fstream read/write move file pointer"
    https://stackoverflow.com/questions/14148238/does-fstream-read-write-move-file-pointer
  4. Get a value of a specific key?
    https://github.com/drycpp/lmdbxx/issues/1
    https://blog.csdn.net/dcrmg/article/details/79144507
  5. tips to use lmdb
  6. https://blogs.kolabnow.com/2018/06/07/a-short-guide-to-lmdb
  7. Caffeを通してCNNを理解する #3
    http://mugichoko.hatenablog.com/entry/2017/11/30/230019
  8. multi-label
    https://github.com/BVLC/caffe/issues/1698
    http://deepdish.io/2015/04/28/creating-lmdb-in-python/
    http://blog.kostyaev.me/blog/Multilabel-Dataset
    https://stackoverflow.com/questions/31774953/test-labels-for-regression-caffe-float-not-allowed/31808324#31808324
    https://stackoverflow.com/questions/33627888/generating-lmdb-for-caffe
  9. merging multiple lmdb fiels:
    https://stackoverflow.com/questions/33443020/merge-two-lmdb-databases-for-feeding-to-the-network-caffe
  10. Though not being from an answer, the code posted by OP was helpful in that it teaches me we can divide writing to database.
    https://stackoverflow.com/questions/31649216/writing-data-to-lmdb-with-python-very-slow
    Note that, .begin resets transaction
    https://lmdb.readthedocs.io/en/release/#lmdb.Transaction
    In caffe's convert_imageset, transaction is committed and reset.
    https://github.com/BVLC/caffe/blob/master/tools/convert_imageset.cpp
#// Repeat calls to abort() have no effect after a previously successful commit() or abort(), or after the associated Environment has been closed.

# Transaction commits automatically:
with env.begin(write=True) as txn:
    txn.put('a', 'b')




No comments:

Post a Comment

Writing a post in Markdown in Blogger

Following articles are helpful. * [【令和最新版】Blogger で Markdown を用いて記事を書く方法](https://qiita.com/hoge_fu-ga/items/267ea5ad4f8c120eaeb8) * [Bl...