Arrow support (data representations of jagged arrays) | Voters

TileDB

Create

Roadmap

TileDB Core

Arrow support (data representations of jagged arrays)

drahnreb

Working on nested arrays is a crucial task in most scientific fields. I think TileDB could perfectly leverage its strengths to support the community working in that field: https://youtu.be/jvt4v2LTGK0?t=1366
Working with Data Management in TileDB and Data Wrangling in awkward-array (https://github.com/scikit-hep/awkward-1.0) or other libraries with arrow support would be extremely beneficial workhorse.
Any updates on when Arrow will be supported?

August 15, 2020

Isaiah Norton

Hi @drahnreb, we will add Arrow support in the 2.1 release, which we are planning for the end of August. TileDB supports variable-length subarrays, so for jagged arrays of scalars (at the very least), there should be a simple mapping. Awkward looks very interesting, thanks - I will take a look.