Commits
Wes McKinney authored 08ca13f83f3
ARROW-4782: [C++] Prototype array and scalar expression types to help with building an deferred compute graph Basic ideas * `Expr` is the base class for the "edges" of the graph, i.e. data dependencies * `Operation` is the base for nodes in the graph. An Operation takes input Expr dependencies, plus any static / non-Expr arguments, and produces an output Expr. During this output resolution, type checking and validation is performed This patch does not get into various other necessary expression types, particularly table level operations like projections, filters, aggregations, and joins. I'll look at those in follow up patches. I also added the `arrow::compute::LogicalType` idea which provides the idea of an "unbound" / non-concrete type. The idea of this type is to permit instances of "type classes" like "number" or "integer" in addition to concrete data types like "int32". I think we need to have type objects that are decoupled from the metadata used for the Arrow columnar format, even though in some cases there is a 1-to-1 mapping. There are some other things to contemplate in future patches like "unbound" names in nested fields. We may not always know the field names when building an expression, and the "binding" to physical data may need to occur later. The general approach here is inspired by a pure Python expression algebra system I designed in 2015 called Ibis https://github.com/ibis-project/ibis/tree/master/ibis/expr With this system I was able to accurately model a superset of SQL and, with the help of some other open source developers, create compilers from the expression algebra to SQL or other backend execution. Author: Wes McKinney <wesm+git@apache.org> Author: Krisztián Szűcs <szucs.krisztian@gmail.com> Closes #3820 from wesm/compute-expr-prototyping and squashes the following commits: 3a837351 <Wes McKinney> Code review comments, add some type convenience aliases 1571b3b8 <Krisztián Szűcs> virtual destructor for LogicalType 0142cfeb <Wes McKinney> Fix issues on Windows 7a6b1ed8 <Wes McKinney> Smoke tests for example ops 86283288 <Wes McKinney> More expr boxing scaffolding and logical type tests c00b6299 <Wes McKinney> Add some basic expression type factories a6651678 <Wes McKinney> Get some very basic unit tests passing 03a3ed83 <Wes McKinney> Remove superfluous file a56b81cb <Wes McKinney> More scaffolding, a bit cleaner API, factory methods for expr types 8eaadd93 <Wes McKinney> More boilerplate 9859982e <Wes McKinney> Prototyping 65dbdcbf <Wes McKinney> Prototyping 77303f0d <Wes McKinney> Prototype 86ec1521 <Wes McKinney> More prototyping / scaffolding 8d424aae <Wes McKinney> Prototyping 510bb7d5 <Wes McKinney> Prototyping c73f420b <Wes McKinney> Prototyping 1fff4b81 <Wes McKinney> seed