11-04, 13:00–13:30 (Asia/Jerusalem), Red Track
Pandas and SQL are both great tools for data manipulation and analysis, yet they each come with their unique challenges. Pandas is handy and easy to use, especially for iterative research, but can slow down or crash with anything larger than a few gigabytes. SQL is efficient with huge datasets, but can be tricky and rigid when your analysis becomes more complex.
In our talk, we'll explore how these tools, instead of working against each other, can complement each other in your data science work. We'll show you how ibis can bring the python experience with the power of SQL environment, helping you get the best of both worlds.
Pandas and SQL are both great tools for data manipulation and analysis, yet they each come with their unique challenges. Pandas is handy and easy to use, especially for iterative research, but can slow down or crash with anything larger than a few gigabytes. SQL is efficient with huge datasets, but can be tricky and rigid when your analysis becomes more complex.
In our talk, we'll explore how these tools, instead of working against each other, can complement each other in your data science work. We'll show you how ibis can bring the python experience with the power of SQL environment, helping you get the best of both worlds.
We'll share our journey from complex distributed data science pipelines to jobs that can be run efficiently from a notebook. and share useful tips, common patterns and how to avoid frequent mistakes when working with ibis.
Join us as we show you how pandas and SQL can work together, making your data science tasks easier on any scale.
Omri is a Data Hacker, Maker, and LEGO builder. Currently, He is a Prinicipal Engineer at Walmart Global Tech.