Skip to content

Activation Ordering #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 84 commits into from
Aug 26, 2024
Merged

Activation Ordering #97

merged 84 commits into from
Aug 26, 2024

Conversation

horheynm
Copy link
Contributor

@horheynm horheynm commented Jun 24, 2024 •
edited by kylesayrs
Loading

Add helper functions for vllm-project/llm-compressor#94 and support for out-of-order group observation

  • extract and write g_idx to/from layer.weight_g_idx
  • pass g_idx to weight observations
  • small refactor of get_qparams to support out-of-order observation
  • Add activation order option to QuantArgs

PR Dependencies

Move safe_permute, update layer helper arguments (#137)
Group Index Quantization Support (#134)

bfineran
bfineran previously approved these changes Jul 1, 2024
View reviewed changes
Copy link
Contributor

@bfineran bfineran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for serialization only. will need to immediately follow up support for inference by updating the observers/base.py and lifecycle/forward.py to support g_idx when set.

bfineran
bfineran approved these changes Jul 9, 2024
View reviewed changes
bfineran
bfineran suggested changes Jul 9, 2024
View reviewed changes
Copy link
Contributor

@bfineran bfineran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

meant to request

bfineran
bfineran previously approved these changes Jul 22, 2024
View reviewed changes
@kylesayrs kylesayrs changed the title act order Activation Ordering Aug 14, 2024
@kylesayrs kylesayrs changed the title Activation Ordering GPTQ Activation Ordering Aug 14, 2024
horheynm and others added 22 commits August 14, 2024 18:57
@horheynm @kylesayrs
propagate g_idx to scale, zp
cfbe5e9
@horheynm @kylesayrs
intialize empty g_idx
27ee8d9
@horheynm @kylesayrs
get rid of g_idx in forward
4918140
@horheynm @kylesayrs
g_idx in forward
88908da
@horheynm @kylesayrs
g_idx in _process_quantization
c884a3e
@horheynm @kylesayrs
remove unnec comments
460dfbd
@horheynm @kylesayrs
initialize param to -1, g_idx in forward call
a19788a
@horheynm @kylesayrs
comments
9607bdb
@kylesayrs
add ability to disable forward pass override during quantization
00b8772
@kylesayrs
add pathway for updating weight quant params from curent weight
3ae6698
@horheynm @kylesayrs
add g_idx to observer
d472287
@horheynm @kylesayrs
fix on load
6b13bd2
@horheynm @kylesayrs
fix prepopulated observer params
c1eb3b4
@horheynm @kylesayrs
draft
ae2c158
@horheynm @kylesayrs
draft
8069335
@horheynm @kylesayrs
draft
0b2e9ab
@kylesayrs
two bug fixes: pass g_idx during compress and decompress, deep copy w…
93749ef 
…eight
@kylesayrs
remove perm from , add and remove comments
bd92a10
@kylesayrs
implement refresh_layer_weight_quant_params
d51a8b6
@kylesayrs
update_layer_weight_quant_params reuse
5025fb6
@kylesayrs
simplify grouped forward logic
bb35198
@kylesayrs
rename group_id to group_index
ddc015a
@kylesayrs kylesayrs marked this pull request as ready for review August 21, 2024 21:56
@kylesayrs kylesayrs changed the base branch from main to kylesayrs/quant_args_typo August 22, 2024 05:25
@kylesayrs kylesayrs changed the base branch from kylesayrs/quant_args_typo to main August 22, 2024 05:25
Satrat
Satrat suggested changes Aug 22, 2024
View reviewed changes
@kylesayrs kylesayrs mentioned this pull request Aug 22, 2024
Merged
horheynm
horheynm commented Aug 22, 2024
View reviewed changes
@kylesayrs kylesayrs requested review from Satrat and bfineran August 23, 2024 19:52
@kylesayrs kylesayrs merged commit 4bbb6a8 into main Aug 26, 2024
1 check passed
@kylesayrs kylesayrs deleted the act-order branch August 26, 2024 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kylesayrs kylesayrs kylesayrs left review comments

@Satrat Satrat Satrat approved these changes

@bfineran bfineran bfineran approved these changes

Assignees

@kylesayrs kylesayrs

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@horheynm @Satrat @bfineran @kylesayrs