Commits


Marco Neumann authored and Korn, Uwe committed 1ba7d51b577
ARROW-2222: handle untrusted inputs This is a proof of concept of handling untrusted inputs. The error messages aren't filled yet and tests aren't there, but this is everything the fuzzer found after running over an hour. It basically includes the following parts: ## Flatbuffer validation This may be the only thing that could be optional and where we should test the performance impact. It's a single, simple change in `Message::ReadFrom`. ## nullptr validation It seems, that even when we validate the flatbuffer, nullptrs are still legit in many places and we just ignore them. I've added a bunch of checks. We may want to introduce a simple macro for that, that includes error message generation and the return w/ an `IOError`. ## Integer handling in `BufferReader` The `nbytes` argument can be negative since it's a signed integer. I've seen at least one case where the fuzzer tried to produce a negative body size of a message that led to a negative `nbytes` argument, a subsequent negative `position_` and a out of bounds read. I think we should catch the negative body length somewhere else and produce a proper error message, but I'm not sure if that's the only way of provoking "negative reads". For robustness, the checks should be there. If someone is to worried about performance, we should rather implement a template-based, header-based, function-inlined interface that reads exactly the number of bytes that are required for certain types. Author: Marco Neumann <marco.neumann@blue-yonder.com> Closes #1763 from crepererum/ARROW-2222 and squashes the following commits: 826dcc2c <Marco Neumann> increase flatbuffers verifier max depth to 128 9bd80fd3 <Marco Neumann> catch nullptr in flatbuffer-encoded data d9be6b1b <Marco Neumann> validate flatbuffer message before reading 9a2faf83 <Marco Neumann> catch negative reads in BufferReader::Read