longling¶

Overview¶

The project contains several modules for different purposes:

lib serves as the basic toolkit that can be used in any place without extra dependencies.
ML provides many interfaces to quickly build machine learning tools.

Quick scripts¶

The project provide several cli scripts to help construct different architecture.

Neural Network¶

glue for mxnet-gluon

CLI¶

Provide several general tools, consistently invoked by:

longling $subcommand $parameters1 $parameters2

To see the help information:

longling -- --help
longling $subcommand --help

Take a glance on all available cli.

The cli tools is constructed based on fire. Refer to the documentation for detailed usage.

Demo¶

Split dataset¶

target: split a dataset into train/valid/test

longling train_valid_test $filename1 $filename2 -- --train_ratio 0.8 --valid_ratio 0.1 --test_ratio 0.1

Similar commands:

train_test

longling train_test $filename1 -- --train_ratio 0.8 --test_ratio 0.2

train_valid

longling train_valid $filename1 -- --train_ratio 0.8 --valid_ratio 0.2

Cross Validation kfold

longling kfold $filename1 $filename2 -- --n_splits 5

Display the tree of content¶

longling toc .

such as

/
├── __init__.py
├── __pycache__/
│   ├── __init__.cpython-36.pyc
│   └── toc.cpython-36.pyc
└── toc.py

Quickly construct a project¶

longling arch

or you can also directly copy the template files

longling arch-cli

To be noticed that, you need to check $VARIABLE in the template files.

Result Analysis¶

The cli tools for result analysis is specially designed for json result format:

longling  max $filename $key1 $key2 $key3
longling  amax $key1 $key2 $key3 --src $filename

For the composite key like {'prf':{'avg': {'f1': 0.77}}}, the key should be presented as prf:avg:f1. Thus, all the key used in the result file should not contain :.