Skip to content

cap str/bytes node expansion in container and dict.fromkeys inference#3127

Open
kali834x wants to merge 2 commits into
pylint-dev:mainfrom
kali834x:container-str-node-expansion
Open

cap str/bytes node expansion in container and dict.fromkeys inference#3127
kali834x wants to merge 2 commits into
pylint-dev:mainfrom
kali834x:container-str-node-expansion

Conversation

@kali834x

@kali834x kali834x commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Type of Changes

Type
🐛 Bug fix

Description

_container_generic_transform (for list/set/tuple/frozenset) and infer_dict_fromkeys build one Const node per character when their argument infers to a str/bytes constant, with no size cap. the multiply guard in const_infer_binary_op bounds "x" * n, but string concatenation ("a" + "b") is not size-bounded and neither is a large literal, so list(("a" * 10 ** 8) + "b") walks the whole string and materializes ~100 million Const nodes (tens of GB) during inference of otherwise untrusted source. cap both sites the same way _multiply_seq_by_int already caps sequence repetition: fall back to the default inference once the string exceeds 1e8 characters, before the nodes are built. small calls (list("abc"), dict.fromkeys("ab")) keep inferring their exact contents.

@codspeed-hq

codspeed-hq Bot commented Jul 1, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 3 untouched benchmarks
⏩ 1 skipped benchmark1


Comparing kali834x:container-str-node-expansion (198e6aa) with main (faf2884)

Open in CodSpeed

Footnotes

  1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports.

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.62%. Comparing base (faf2884) to head (198e6aa).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #3127   +/-   ##
=======================================
  Coverage   93.62%   93.62%           
=======================================
  Files          92       92           
  Lines       11381    11385    +4     
=======================================
+ Hits        10655    10659    +4     
  Misses        726      726           
Flag Coverage Δ
linux 93.49% <100.00%> (+<0.01%) ⬆️
pypy 93.62% <100.00%> (+<0.01%) ⬆️
windows 93.59% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
astroid/brain/brain_builtin_inference.py 93.36% <100.00%> (+0.05%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@DanielNoord DanielNoord left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a test for this :)

@kali834x

kali834x commented Jul 2, 2026

Copy link
Copy Markdown
Contributor Author

Added tests in tests/brain/test_brain.py covering list/set/tuple/frozenset and dict.fromkeys: the oversized concat case declines (Instance / empty Dict) and the small-string case still infers its exact contents.

@DanielNoord

Copy link
Copy Markdown
Collaborator

Can you fix merge conflicts and the pre-commit check?

@kali834x kali834x force-pushed the container-str-node-expansion branch from 0830bae to 198e6aa Compare July 7, 2026 06:12
@kali834x

kali834x commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main, the conflict was just the ChangeLog entry. pre-commit passes locally with the updated hooks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants