1. Introduction¶
1.1. Overview¶
At the most fundamental level, an “execution” or a “run” of any data-processing can be thought like that:
.--------------. _____________ .-------------.
; DataTree ; | | ; DataTree ;
;--------------; ==> | <cfunc_1> | ==> ;--------------;
; /some/data ; | <cfunc_2> | ; /some/data ;
; /some/other ; | ... | ; /some/other ;
; /foo/bar ; |_____________| ; /foo/bar ;
'--------------' '--------------.
The data-tree might come from json, hdf5, excel-workbooks, or plain dictionaries and lists. Its values are strings and numbers, numpy-lists, pandas or xray-datasets, etc.
The component-functions must abide to the following simple signature:
cfunc_do_something(pandelone, datatree)
and must not return any value, just read and write into the data-tree.
Here is a simple component-function:
def cfunc_standardize(pandelone, datatree): pin, pon = pandelone.paths(), df = datatree.get(pin.A) df[pon.A.B_std] = df[pin.A.B] / df[pin.A.B].std()
Notice the use of the relocatable-paths marked specifically as input or output.
TODO: continue rough example in tutorial…
1.2. Quick-start¶
The program runs on Python-3.5+ and requires numpy, pandas and (optionally) win32 libraries along with their native backends. .. code-block:: bash
pip install pandalone ## Use--pre
if version-string has a build-suffix.
Or in case you need the very latest from master
branch :
pip install git+https://github.com/pandalone/pandalone.git
Or in to install in develop mode, with all dependencies needed for development, and with pre-commit hook for auto-formatting python-code with black, clone locally this project from the remote repo, and run:
pip install -e <pandalone-dr>[dev]
pre-commit install
1.3. Project files and folders¶
The files and folders of the project are listed below:
+--pandalone/ ## (package) Python-code
+--tests/ ## (package) Test-cases
+--doc/ ## Documentation folder
+--setup.py ## (script) The entry point for `setuptools`, installing, testing, etc
+--requirements/ ## (txt-files) Various pip and conda dependencies.
+--README.rst
+--CHANGES.rst
+--AUTHORS.rst
+--CONTRIBUTING.rst
+--LICENSE.txt