refactor(bigframes): Make join nullity optimizations more robust#16541
refactor(bigframes): Make join nullity optimizations more robust#16541TrevorBergeron wants to merge 3 commits intomainfrom
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a nulls_equal attribute to JoinNode to explicitly handle pandas-style null equality in joins. It includes a new rewrite rule, simplify_join, which optimizes SQL generation by removing unnecessary nullity checks when join columns are non-nullable. The feedback suggests refining the simplify_join logic to be less restrictive by using an and condition for nullability checks, ensuring the optimization is applied more broadly. Additionally, it is recommended to reorder the compiler steps to ensure all join nodes are optimized and to remove an unused import in the new rewrite module.
packages/bigframes/bigframes/core/compile/ibis_compiler/ibis_compiler.py
Outdated
Show resolved
Hide resolved
TrevorBergeron
force-pushed
the
tbergeron_null_prop_rewriter
branch
from
84eccc4 to
7fd26f9
Compare
TrevorBergeron
force-pushed
the
tbergeron_null_prop_rewriter
branch
from
7fd26f9 to
7f635df
Compare
| right_child: BigFrameNode | ||
| left_col: ex.DerefOp | ||
| indicator_col: identifiers.ColumnId | ||
| nulls_equal: bool = True |
There was a problem hiding this comment.
Could you add a comment on what this field means? It's not very obvious to me
There was a problem hiding this comment.
added comment
2 participants
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕