Menachem Kluft
Menachem Kluft - Senior Backend Developer in Mobileye.
Uri Mogilevsky - Technical lead in Mobileye.
We're part of a group creating a data processing infrastructure. This system serves different groups, addresses various needs, and tackles challenging data processing tasks across the entire company.
Sessions
Deep learning algorithms often require the use of multiple large datasets, with the parquet format being one of the most prevalent formats to store big data. While datasets are typically consumed by batch processing where several tasks process small slices of the data, big data query engines like Trino and Spark are more efficient when working on large chunks of data. In this talk we will present a solution based on the Row Groups feature that allows both small and large amounts of data to be consumed quickly.