Skip to content

Directly save meta files to disk on model save#1954

Merged
Qubitium merged 4 commits into
mainfrom
meta-dedup
Sep 30, 2025
Merged

Directly save meta files to disk on model save#1954
Qubitium merged 4 commits into
mainfrom
meta-dedup

Conversation

@Qubitium

@Qubitium Qubitium commented Sep 30, 2025

Copy link
Copy Markdown
Collaborator

zoom zoom

Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
Signed-off-by: Qubitium <Qubitium@modelcloud.ai>
@Qubitium Qubitium marked this pull request as ready for review September 30, 2025 09:52
@Qubitium

Copy link
Copy Markdown
Collaborator Author

@codex Check this PR for bugs

@Qubitium Qubitium merged commit 10f3d1f into main Sep 30, 2025
5 checks passed
@Qubitium Qubitium deleted the meta-dedup branch September 30, 2025 09:54
@avtc

avtc commented Oct 1, 2025

Copy link
Copy Markdown
Contributor

@Qubitium Please check error during save, can be related to this PR:
main branch hash: 3da0344

INFO  Format: Converting GPTQ v2 to v1                                                                                     
Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-air-gptqmodel-clean.py", line 63, in <module>
    model.save(OUTPUT_DIR)
    ~~~~~~~~~~^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/base.py", line 832, in save
    self.save_quantized(
    ~~~~~~~~~~~~~~~~~~~^
        save_dir=save_dir,
        ^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        meta_quantizer=meta_quantizer,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        eora_path=eora_path)
        ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/writer.py", line 231, in save_quantized
    model = convert_gptq_v2_to_v1_format(
        model, quantize_config=quantize_config, qlinear_kernel=self.qlinear_kernel
    )
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 669, in convert_gptq_v2_to_v1_format
    convert_gptq_v2_to_v1_format_module(module=submodule, quantize_config=quantize_config)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 635, in convert_gptq_v2_to_v1_format_module
    module.qzeros.data[:, range(0, module.qzeros.data.shape[1], 3)] -= (
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See https://github.com/pytorch/rfcs/pull/17 for more details.

@Qubitium

Qubitium commented Oct 1, 2025

Copy link
Copy Markdown
Collaborator Author

@Qubitium Please check error during save, can be related to this PR: main branch hash: 3da0344

INFO  Format: Converting GPTQ v2 to v1                                                                                     
Traceback (most recent call last):
  File "/home/ubuntu/Documents/Quantize/quantize-glm4.5-air-gptqmodel-clean.py", line 63, in <module>
    model.save(OUTPUT_DIR)
    ~~~~~~~~~~^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/base.py", line 832, in save
    self.save_quantized(
    ~~~~~~~~~~~~~~~~~~~^
        save_dir=save_dir,
        ^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        meta_quantizer=meta_quantizer,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        eora_path=eora_path)
        ^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/models/writer.py", line 231, in save_quantized
    model = convert_gptq_v2_to_v1_format(
        model, quantize_config=quantize_config, qlinear_kernel=self.qlinear_kernel
    )
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 669, in convert_gptq_v2_to_v1_format
    convert_gptq_v2_to_v1_format_module(module=submodule, quantize_config=quantize_config)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/venvs/gptqmodelt/lib/python3.13t/site-packages/gptqmodel/utils/model.py", line 635, in convert_gptq_v2_to_v1_format_module
    module.qzeros.data[:, range(0, module.qzeros.data.shape[1], 3)] -= (
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See https://github.com/pytorch/rfcs/pull/17 for more details.

Ok. this look to be an easy fix. Looks unrelated to the PR but new code caused different timing so you are hitting more/different thread state bugs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants