Skip to content

Commit fbd2500

Browse files
committed
apacheGH-50313: [C++][Docs] Add guidance about memory bombs
1 parent b19c476 commit fbd2500

3 files changed

Lines changed: 33 additions & 0 deletions

File tree

docs/source/cpp/security.rst

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,22 @@ variants, but these typically fall into two categories:
120120
arguments (this is typical of the :ref:`array builders <cpp-api-array-builders>`
121121
and :ref:`buffer builders <cpp-api-buffer-builders>`).
122122

123+
Controlling and restricting memory allocation
124+
---------------------------------------------
125+
126+
By construction, many Arrow C++ APIs can allocate large amounts of memory, depending
127+
on their input parameters. Arrow C++ allows customizing the memory allocator for
128+
such large data areads through the :ref:`MemoryPool <cpp_memory_pool>` interface.
129+
130+
You can therefore implement a MemoryPool class enforcing the restrictions
131+
of your choise (for example to limit the total number of allocated bytes), and pass
132+
it to any Arrow C++ APIs you use.
133+
134+
.. note::
135+
Unlike memory used for Arrow data, smaller metadata structures (such as field
136+
names, etc.) instead rely on the C++ standard library allocators for convenience.
137+
They will therefore be invisible to the MemoryPool memory accounting.
138+
123139
Ingesting untrusted data
124140
========================
125141

@@ -144,6 +160,13 @@ from an untrusted source), you **must** follow these steps:
144160
2. If the API returned successfully, validate the returned Arrow data in full
145161
(see "Full validity" above)
146162

163+
Furthermore, both the IPC and the Parquet format allow for powerful forms of
164+
compression, and can therefore exhibit large expansion factors when reading.
165+
If you need to guard against potential denial-of-service attacks that would
166+
exhaust available memory, we recommend you enforce memory allocation limits
167+
using a dedicated MemoryPool implementation (see "Controlling and restricting
168+
memory allocation" above).
169+
147170
CSV reader
148171
----------
149172

docs/source/format/Columnar.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1391,6 +1391,8 @@ have two entries in each RecordBatch. For a RecordBatch of this schema with
13911391
buffer 12: col2 data
13921392
buffer 13: col2 data
13931393

1394+
.. _buffer-compression:
1395+
13941396
Compression
13951397
-----------
13961398

docs/source/format/Security.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -180,6 +180,11 @@ their own risks. For example, buffer offsets and sizes encoded in IPC messages
180180
may be out of bounds for the IPC stream; Flatbuffers-encoded metadata payloads
181181
may carry incorrect offsets pointing outside of the designated metadata area.
182182

183+
Besides, the IPC format provides optional :ref:`buffer compression <buffer-compression>`
184+
using general-purpose compression algorithms. It is therefore possible to craft an IPC
185+
stream or file that acts as a decompression bomb by consuming all available memory,
186+
opening a potential channel for denial-of-service attacks.
187+
183188
Advice for users
184189
----------------
185190

@@ -194,6 +199,9 @@ It is **extremely recommended** to run dedicated validation checks when decoding
194199
the IPC format, to make sure that the decoding can not induce unwanted behavior.
195200
Failing those checks should return a well-known error to the caller, not crash.
196201

202+
It is **recommended** to provide facilities for users to control the memory
203+
allocation behavior when reading an IPC file or stream (for example by making
204+
the allocator customizable).
197205

198206
Extension Types
199207
===============

0 commit comments

Comments
 (0)