Process data-trees with reconfigurable-paths in Python¶
pandalone is a collection of utilities for working with hierarchical-data using relocatable-paths.
Release: | 0.5.0 |
---|---|
Date: | 2020-05-14 23:59 |
Documentation: | https://pandalone.readthedocs.org/ |
Source: | https://github.com/pandalone/pandalone |
PyPI repo: | https://pypi.python.org/pypi/pandalone |
Keywords: | calculation, data, dependencies, engineering, excel, library, numpy, pandas, processing, python, resolution, scientific, simulink, tree, utility |
Copyright: | 2015 European Commission (JRC-IET) |
License: | EUPL 1.1+ |
Currently only 2 portions of the envisioned functionality are ready for use:
pandalone.xleash
: A mini-language for “throwing the rope” around rectangular areas of Excel-sheets.pandalone.mappings
: Hierarchical string-like objects that may be used for indexing, facilitating renaming keys and column-names at a later stage.
Our goal is to facilitate the composition of engineering-models from loosely-coupled components. Initially envisioned as an indirection-framework around pandas coupled with a dependency-resolver, every such model should auto-adapt and process only values available, and allow remapping of the paths accessing them, to run on renamed/relocated value-trees without component-code modifications.
It is an open source library written and tested on Python-3.6+ , Windows and Linux.
Note
The project, as of May-2015, is considered at an alpha-stage, without any released version in pypi yet.
- 1. Introduction
- 2. Usage
- 3. Getting Involved
- 4. API reference
- 4.1. Module:
pandalone.xleash
- 4.1.1. About
- 4.1.2. Overview
- 4.1.3. Definitions
- 4.1.4. Details
- 4.1.5. Plugin Extensions
- 4.1.6. API
- 4.1.7. Submodule:
pandalone.xleash._parse
- 4.1.8. Submodule:
pandalone.xleash.io
- 4.1.9. Submodule:
pandalone.xleash.io.backend
- 4.1.10. Submodule:
pandalone.xleash.io._xlrd
- 4.1.11. Submodule:
pandalone.xleash._capture
- 4.1.12. Submodule:
pandalone.xleash._filter
- 4.1.13. Submodule:
pandalone.xleash._lasso
- 4.2. Module:
pandalone.mappings
- 4.3. Module:
pandalone.components
- 4.4. Module:
pandalone.pandata
- 4.1. Module:
- 5. Changes
- 5.1. Known deficiencies
- 5.2. Changelog
- 5.2.1. v0.5.0 (14-May-2020): fix null-check np-arrays
- 5.2.2. v0.4.0 (5-Apr-2020): Auto-default Pandel
- 5.2.3. v0.3.4 (28-July-2019): xleash for pandas 0.25+
- 5.2.4. v0.3.3 (21-June-2019, UNRELEASED): jsonschema multi-dim numpy-arrays
- 5.2.5. v0.3.2 (19-June-2019, UNRELEASED): relax asteval lower-bound
- 5.2.6. v0.2.8 (7-Sept-2019): BACKPORT fix for pandas 0.25+ (19-July 2019)
- 5.2.7. v0.2.7 (15-July-2019): stray sources
- 5.2.8. v0.3.3 (21-June-2019): accept multi-dim numpy-arrays as jsonschema-arrays
- 5.2.9. v0.3.2 (19-June-2019): relax asteval lower-bound
- 5.2.10. v0.3.1 (18-June-2019): DROP PY2 & bump jsonschema 2.x–>3.x
- 5.2.11. v0.2.6 (5-June-2019): deps cleanup
- 5.2.12. v0.2.5 (29-May-2018)
- 5.2.13. v0.2.4.post1 (23-Aug-2018)
- 5.2.14. v0.2.4 (21-Mar-2017)
- 5.2.15. v0.2.3 (25-Feb-2017)
- 5.2.16. v0.2.2 (7-Feb-2017): “Telos” release
- 5.2.17. v0.2.1 (2-Dec-2016): “Stop” release
- 5.2.18. v0.2.0 (2-Nov-2016): “Really?” release
- 5.2.19. v0.1.13 (1-Nov-2016):
- 5.2.20. v0.1.12 (July-2016): “Stegh” release
- 5.2.21. v0.1.11 (Apr-2016):
- 5.2.22. v0.1.10 (Apr-2016):
- 6. Indices
Glossary¶
- data-tree
The container of data consumed and produced by a :term`model`, which may contain also the model. Its values are accessed using path s. It is implemented by
pandalone.pandata.Pandel
as a mergeable stack of JSON-schema abiding trees of strings and numbers, formed with:- sequences,
- dictionaries,
pandas
instances, and- URI-references.
- value-tree
- That part of the data-tree that relates only to the I/O data processed.
- model
- A collection of component s and accompanying mappings.
- component
- Encapsulates a data-transformation function, using path to refer to its inputs/outputs within the value-tree.
- path
- A
/file/like
string functioning as the id of data-values in the data-tree. It is composed of step, and it follows the syntax of the JSON-pointer. - step
- pstep
- path-step
- pstep
- The parts between between two conjecutive slashes(
/
) within a path. ThePstep
facilitates their manipulation. - pmod
- pmods
- pmods-hierarchy
- mapping
- mappings
- pmods
Specifies a transformation of an “origin” path to a “destination” one (also called as “from” and “to” paths). The mapping always transforms the final path-step, and it can either rename or relocate that step, like that:
ORIGIN DESTINATION RESULT_PATH ------ ----------- ----------- /rename/path foo --> /rename/foo ## renaming /relocate/path foo/bar --> /relocate/foo/bar ## relocation /root a/b/c --> /a/b/c ## Relocates all /root sub-paths.
The hierarchy is formed by
Pmod
instances, which are build when parsing the mappings list, above.- JSON-schema
- The JSON schema is an IETF draft that provides a contract for what JSON-data is required for a given application and how to interact with it. JSON Schema is intended to define validation, documentation, hyperlink navigation, and interaction control of JSON data. You can learn more about it from this excellent guide, and experiment with this on-line validator.
- JSON-pointer
- JSON Pointer(RFC 6901) defines a string syntax for identifying a specific value within a JavaScript Object Notation (JSON) document. It aims to serve the same purpose as XPath from the XML world, but it is much simpler.