Skip to content

Improve the remove layout in handling loop op. - #7634

Merged
chengjunlu merged 5 commits into
mainfrom
chengjun/improve_remove_layout
Aug 12, 2026
Merged

Improve the remove layout in handling loop op.#7634
chengjunlu merged 5 commits into
mainfrom
chengjun/improve_remove_layout

Conversation

@chengjunlu

@chengjunlu chengjunlu commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This PR improves the Intel GPU RemoveLayoutConversions pass to better handle loop-related values during backward/forward rematerialization, aiming to avoid duplicated loop bodies and improve kernel performance (referenced in issue #7604).

chunk_gated_delta_rule_fwd_kernel_h_blockdim64 kernel time 207us vs 235us in #7604

@chengjunlu
chengjunlu marked this pull request as draft August 4, 2026 05:58
@chengjunlu
chengjunlu force-pushed the chengjun/improve_remove_layout branch from c471fca to 4d0d68a Compare August 6, 2026 06:13
@chengjunlu
chengjunlu marked this pull request as ready for review August 6, 2026 06:14
@chengjunlu
chengjunlu requested review from etiotto and whitneywhtsang and a lite review from Copilot August 6, 2026 06:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the Intel GPU RemoveLayoutConversions pass to better handle loop-related values during backward/forward rematerialization, aiming to avoid duplicated loop bodies and improve kernel performance (referenced in issue #7604).

Changes:

  • Extend forward propagation to traverse/control loop ops and handle tt.descriptor_store source rewrites.
  • Improve forward-propagation candidate collection to account for loop iter-args / loop results (to prevent duplicated loop bodies after rematerialization).
  • Add an MLIR regression test reproducing the loop-related inefficiency.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
third_party/intel/lib/TritonIntelGPUTransforms/RemoveLayoutConversions.cpp Updates rematerialization forward-propagation logic and loop-related candidate selection; adds extra debug logging.
test/TritonIntelGPU/RemoveLayoutConversions/remove_layout_conversions_7625.mlir Adds a regression test covering loop handling in remove-layout-conversions.

Comment thread test/TritonIntelGPU/RemoveLayoutConversions/remove_layout_conversions_7625.mlir Outdated
@chengjunlu
chengjunlu force-pushed the chengjun/improve_remove_layout branch from 4d0d68a to d046f06 Compare August 7, 2026 00:50
Comment thread third_party/intel/lib/Dialect/TritonIntelGPU/IR/Dialect.cpp
Comment thread test/TritonIntelGPU/RemoveLayoutConversions/remove_layout_conversions_7625.mlir Outdated
@chengjunlu
chengjunlu force-pushed the chengjun/improve_remove_layout branch from e9c8adc to 1c3b5f1 Compare August 10, 2026 01:43
@chengjunlu
chengjunlu requested a review from etiotto August 10, 2026 06:47
@@ -1478,10 +1481,6 @@

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The removal of the scf::ForOp/scf::IfOp skip is too broad. The needRewrite path below calls rewriteOp(&op, ...) which is not designed for structured control flow ops — if a ForOp result happens to be in valuesToPropagate, this will attempt to rewrite the loop itself.

Consider keeping the skip for the needRewrite path while still entering regions for the DescriptorStoreOp case.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add back the guard of the rewriteOp. The value in the valuesToPropagate is chosen conservitively of the element-wise operations. But anyway we have added the protection.

@chengjunlu
chengjunlu force-pushed the chengjun/improve_remove_layout branch from 1c3b5f1 to 257c021 Compare August 11, 2026 05:06
chengjunlu and others added 4 commits August 11, 2026 12:55
Signed-off-by: Lu,Chengjun <chengjun.lu@intel.com>
Co-authored-by: Ettore Tiotto <ettore.tiotto@intel.com>
Signed-off-by: Lu,Chengjun <chengjun.lu@intel.com>
Signed-off-by: Lu,Chengjun <chengjun.lu@intel.com>

@etiotto etiotto left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

One optional non-blocker: the assert(newOperand) in the DescriptorStoreOp path (RemoveLayoutConversions.cpp) compiles out in release builds — an if (!newOperand) continue; would be safer. (The dead sliceOps is noted inline.) Neither blocks merge.

Comment thread third_party/intel/lib/TritonIntelGPUTransforms/RemoveLayoutConversions.cpp Outdated
@chengjunlu
chengjunlu enabled auto-merge (squash) August 12, 2026 01:32
@chengjunlu
chengjunlu merged commit 7a464a6 into main Aug 12, 2026
13 checks passed
@chengjunlu
chengjunlu deleted the chengjun/improve_remove_layout branch August 12, 2026 03:10
Signed-off-by: Lu,Chengjun <chengjun.lu@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[vLLM][Kimi-K3] Duplicated expressions are generated with different layout in RemoveLayout optimization.

4 participants