Commits


Jorge C. Leitao authored and Andy Grove committed 735c870cbaa
ARROW-9809: [Rust][DataFusion] Fixed type coercion, supertypes and type checking. This commit makes all type coercion happen on the physical plane instead of logical plane and fixes the supertype function. This makes field names to not change due to coercion rules, better control of how the coercion supports physical calculations, and others. This commit also makes it more clear how we enforce type checking during planning. the Logical plan now knows how to derive its schema directly from binary expressions, even before the coercion is applied. The rational for this change is that coercions are simplifications to a physical computation (it is easier to sum two numbers of the same type at the hardware level). This partially solves ARROW-9809 (for binary expressions, not for udfs), an issue on which the physical schema could be modified by coercion rules, causing the RecordBatch's schema to be different from the logical batch. This also addresses some inconsistencies in how we coerced certain types for binary operators, causing such inconsistencies to error during planning instead of execution. This also introduces a significant number of tests into the overall consistency of binary operators: it is now explicit what types they expect and how coercion happens to each operator. It also adds tests to different parts of the physical execution, to ensure schema consistency for binary operators, including negative tests (when it should error). This also makes `like` and `nlike` generally available, and added some tests to it. This closes ARROW-4957. @andygrove and @alamb, I am really sorry for this long commit, but I was unable to split this in smaller parts with passing tests. There was a strong coupling between the `get_supertype` and the physical expressions that made it hard to work this through. Closes #8024 from jorgecarleitao/fix_types Authored-by: Jorge C. Leitao <jorgecarleitao@gmail.com> Signed-off-by: Andy Grove <andygrove73@gmail.com>