Skip to content

Memory leak in Python in version 3.15.0+ AND in 4.21.1 #10088

@Atheuz

Description

@Atheuz

What version of protobuf and what language are you using?
Version: 3.14.0/v3.15.0/v4.21.1
Language: Python

What operating system (Linux, Windows, ...) and version?
Ubuntu Linux

What runtime / compiler are you using (e.g., python version or gcc version)
Python v3.9.5
Built protobuf schema with protoc 3.14.0
Built protobuf schema with protoc 3.21.1 too.

What did you do?
Steps to reproduce the behavior:

  1. Go to https://github.com/Atheuz/proto_leak and clone it with: git clone https://github.com/Atheuz/proto_leak.git.
  2. Go to the repository with cd proto_leak.
  3. Create a virtual environment with virtualenv .venv --python=3.9 and activate it with source .venv/bin/activate.
  4. Run: pip install -r requirements.txt. (installs protobuf==3.14.0)
  5. Run python leak.py
  6. Observe:
Memory consumed at the beginning 14.50390625
Memory consumed after parsed 735.421875
Memory consumed after deallocating 16.15625
  1. Run: pip install protobuf==3.15.0
  2. Run python leak.py
  3. Observe:
Memory consumed at the beginning 17.1953125
Memory consumed after parsed 636.6875
Memory consumed after deallocating 536.734375
  1. Run: pip install protobuf==4.21.1
  2. Replace from schema_pb2 import value_test_topic with from schema_4_21_1_pb2 import value_test_topic on line 4 in leak.py
  3. Run python leak.py
  4. Observe:
Memory consumed at the beginning 14.54296875
Memory consumed after parsed 1118.91015625
Memory consumed after deallocating 1002.75390625

What did you expect to see
In 3.15.0, I expected it to free basically all memory used for the list of pb items.
In 4.21.1, I expected it to free basically all memory used for the list of pb items.

What did you see instead?
In 3.15.0, it retains 500+ MB of memory that is not retained in 3.14.0.
In 4.21.1, it retains 1000+ MB of memory that is not retained in 3.14.0.

Previously reported in #9917, was closed as to be fixed in 4.21.1, but doesn't seem like it actually was.

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions