[Bug] BE Crash with SIGABRT on Load queries and throws bit_stream_utils.h:187] Check failed: buffer != nullptr

[nsivarajan]

Search before asking

• I had searched in the issues and found no similar issues.

Version

Doris - 4.1.2

What's Wrong?

start BE in cloud mode, cloud_unique_id: 1:65674569:r1nl60nc, meta_service_endpoint: 57.102.48.85:5000,57.102.48.100:5000,57.102.48.115:5000
F20260804 01:08:22.742511 1267671 bit_stream_utils.h:187] Check failed: buffer != nullptr
*** Check failure stack trace: ***
@ 0x5b7a6182760f google::LogMessage::SendToLog()
@ 0x5b7a6181dc20 google::LogMessage::Flush()
@ 0x5b7a61821319 google::LogMessageFatal::~LogMessageFatal()
@ 0x5b7a5e641b3e doris::BatchedBitReader::Reset()
@ 0x5b7a5e7022b2 doris::vectorized::BoolPlainDecoder::set_data()
@ 0x5b7a5e638d04 doris::vectorized::ColumnChunkReader<>::load_page_data()
@ 0x5b7a5e6252f0 doris::vectorized::ColumnChunkReader<>::load_page_data_idempotent()
@ 0x5b7a5e61191a doris::vectorized::ScalarColumnReader<>::read_column_data()
@ 0x5b7a5e5f75a1 doris::vectorized::RowGroupReader::_read_column_data()
@ 0x5b7a5e5f1c8d doris::vectorized::RowGroupReader::next_batch()
@ 0x5b7a5e5c8be4 doris::vectorized::ParquetReader::get_next_block()
@ 0x5b7a5e762820 doris::vectorized::FileScanner::_get_block_wrapped()
@ 0x5b7a5e75c3b0 doris::vectorized::FileScanner::_get_block_impl()
@ 0x5b7a5e803e92 doris::vectorized::Scanner::get_block()
@ 0x5b7a5e803a4a doris::vectorized::Scanner::get_block_after_projects()
@ 0x5b7a5e80966a doris::vectorized::ScannerScheduler::_scanner_scan()
@ 0x5b7a5e80856d _ZZZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS0_14ScannerContextEES2_INS0_8ScanTaskEEENK3$_0clEvENKUlvE_clEvENKUlvE_clEv
@ 0x5b7a5e808387 _ZZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS0_14ScannerContextEES2_INS0_8ScanTaskEEENK3$_0clEvENKUlvE_clEv
@ 0x5b7a5e808345 _ZSt13__invoke_implIbRZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS1_14ScannerContextEES3_INS1_8ScanTaskEEENK3$0clEvEUlvE_JEET_St14__invoke_otherOT0_DpOT1
@ 0x5b7a5e808305 _ZSt10__invoke_rIbRZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS1_14ScannerContextEES3_INS1_8ScanTaskEEENK3$0clEvEUlvE_JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESC_E4typeEOSD_DpOSE
@ 0x5b7a5e8081bd _ZNSt17_Function_handlerIFbvEZZN5doris10vectorized16ScannerScheduler6submitESt10shared_ptrINS2_14ScannerContextEES4_INS2_8ScanTaskEEENK3$_0clEvEUlvE_E9_M_invokeERKSt9_Any_data
@ 0x5b7a57d1edfe std::function<>::operator()()
@ 0x5b7a5e807762 doris::vectorized::ScannerSplitRunner::process_for()
@ 0x5b7a5e863ce8 doris::vectorized::PrioritizedSplitRunner::process()
@ 0x5b7a5e84a020 doris::vectorized::TimeSharingTaskExecutor::_dispatch_thread()
@ 0x5b7a5e858692 std::__invoke_impl<>()
@ 0x5b7a5e8585dd std::__invoke<>()
@ 0x5b7a5e8585ad _ZNSt5_BindIFMN5doris10vectorized23TimeSharingTaskExecutorEFvvEPS2_EE6__callIvJEJLm0EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
@ 0x5b7a5e858566 std::_Bind<>::operator()<>()
@ 0x5b7a5e858535 std::__invoke_impl<>()
@ 0x5b7a5e8584f5 ZSt10__invoke_rIvRSt5_BindIFMN5doris10vectorized23TimeSharingTaskExecutorEFvvEPS3_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESB_E4typeEOSC_DpOSD
@ 0x5b7a5e85835d std::_Function_handler<>::_M_invoke()
*** Query id: c22121b9ab894ef4-acd8857bdff4d757 ***
*** is nereids: 1 ***
*** tablet id: 0 ***
*** Aborted at 1785820103 (unix time) try "date -d @1785820103" if you are using GNU date ***
*** Current BE git commitID: d5cfef0 ***
*** SIGABRT unknown detail explain (@0x3424100100796) received by PID 1050518 (TID 1267671 OR 0x783b240e6700) from PID 1050518; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /ngs/app/doris/doris/be/src/common/signal_handler.h:420
1# 0x0000784C42A8CCF0 in /lib64/libpthread.so.0
2# __GI_raise in /lib64/libc.so.6
3# __GI_abort in /lib64/libc.so.6
4# 0x00005B7A6182C645 in /ngs/app/doris/doris-current/be/lib/doris_be
5# 0x00005B7A6181DEFA in /ngs/app/doris/doris-current/be/lib/doris_be
6# google::LogMessage::SendToLog() in /ngs/app/doris/doris-current/be/lib/doris_be
7# google::LogMessage::Flush() in /ngs/app/doris/doris-current/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in /ngs/app/doris/doris-current/be/lib/doris_be
9# doris::BatchedBitReader::Reset(unsigned char const*, long) in /ngs/app/doris/doris-current/be/lib/doris_be
10# doris::vectorized::BoolPlainDecoder::set_data(doris::Slice*) at /ngs/app/doris/doris/be/src/vec/exec/format/parquet/bool_plain_decoder.h:50
11# doris::vectorized::ColumnChunkReader<false, false>::load_page_data() at /ngs/app/doris/doris/be/src/vec/exec/format/parquet/vparquet_column_chunk_reader.cpp:248
12# doris::vectorized::ColumnChunkReader<false, false>::load_page_data_idempotent() in /ngs/app/doris/doris-current/be/lib/doris_be
13# doris::vectorized::ScalarColumnReader<false, false>::read_column_data(doris::COWdoris::vectorized::IColumn::immutable_ptrdoris::vectorized::IColumn&, std::shared_ptr<doris::vectorized::IDataType const>&, std::shared_ptrdoris::vectorized::TableSchemaChangeHelper::Node const&, doris::vectorized::FilterMap&, unsigned long, unsigned long*, bool*, bool, long) at /ngs/app/doris/doris/be/src/vec/exec/format/parquet/vparquet_column_reader.cpp:594
14# doris::vectorized::RowGroupReader::_read_column_data(doris::vectorized::Block*, std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&, unsigned long, unsigned long*, bool*, doris::vectorized::FilterMap&) at /ngs/app/doris/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:452
15# doris::vectorized::RowGroupReader::next_batch(doris::vectorized::Block*, unsigned long, unsigned long*, bool*) at /ngs/app/doris/doris/be/src/vec/exec/format/parquet/vparquet_group_reader.cpp:324
16# doris::vectorized::ParquetReader::get_next_block(doris::vectorized::Block*, unsigned long*, bool*) at /ngs/app/doris/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:616
17# doris::vectorized::FileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /ngs/app/doris/doris/be/src/vec/exec/scan/file_scanner.cpp:468
18# doris::vectorized::FileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /ngs/app/doris/doris/be/src/vec/exec/scan/file_scanner.cpp:407
19# doris::vectorized::Scanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) at /ngs/app/doris/doris/be/src/vec/exec/scan/scanner.cpp:110
20# doris::vectorized::Scanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) in /ngs/app/doris/doris-current/be/lib/doris_be
21# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask) at /ngs/app/doris/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:186
22# doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const at /ngs/app/doris/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:76
23# doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}::operator()() const at /ngs/app/doris/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:75
24# bool std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63
25# std::enable_if<is_invocable_r_v<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}&>, bool>::type std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116
26# std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptrdoris::vectorized::ScannerContext, std::shared_ptrdoris::vectorized::ScanTask)::$_0::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292
27# std::function<bool ()>::operator()() const at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593
28# doris::vectorized::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) at /ngs/app/doris/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:427
29# doris::vectorized::PrioritizedSplitRunner::process() at /ngs/app/doris/doris/be/src/vec/exec/executor/time_sharing/prioritized_split_runner.cpp:104
30# doris::vectorized::TimeSharingTaskExecutor::_dispatch_thread() at /ngs/app/doris/doris/be/src/vec/exec/executor/time_sharing/time_sharing_task_executor.cpp:566
31# void std::__invoke_impl<void, void (doris::vectorized::TimeSharingTaskExecutor::&)(), doris::vectorized::TimeSharingTaskExecutor&>(std::__invoke_memfun_deref, void (doris::vectorized::TimeSharingTaskExecutor::&)(), doris::vectorized::TimeSharingTaskExecutor&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:76
32# std::__invoke_result<void (doris::vectorized::TimeSharingTaskExecutor::&)(), doris::vectorized::TimeSharingTaskExecutor&>::type std::__invoke<void (doris::vectorized::TimeSharingTaskExecutor::&)(), doris::vectorized::TimeSharingTaskExecutor&>(void (doris::vectorized::TimeSharingTaskExecutor::&)(), doris::vectorized::TimeSharingTaskExecutor&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:98
33# void std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/functional:515
34# void std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()>::operator()<, void>() at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/functional:600
35# void std::__invoke_impl<void, std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()>&>(std::__invoke_other, std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()>&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63
36# std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()>&>(std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()>&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:119
37# std::_Function_handler<void (), std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::(doris::vectorized::TimeSharingTaskExecutor))()> >::_M_invoke(std::_Any_data const&) at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292
38# std::function<void ()>::operator()() const at /ngs/app/doris/master-build/ldb_toolchain/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593
39# doris::Thread::supervise_thread(void*) at /ngs/app/doris/doris/be/src/util/thread.cpp:460
40# start_thread in /lib64/libpthread.so.0
41# __clone in /lib64/libc.so.6

What You Expected?

BE should not crash based on Query or data/parquet corrupted file

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Doris-Breakwater]

Breakwater-GitHub-Analysis-Slot: slot_7876f7977bf5

Initial triage

This is a BE crash in the legacy Parquet reader used by a load, not a failure in the cloud meta-service startup path. The cloud configuration is useful context, but no cloud metadata component appears in the failing stack.

Judgment: there is a verified process-fatal handling problem in the reader, and a compressed Parquet DataPageV2 containing definition levels but no physical BOOLEAN values is the most likely trigger. That page can be valid when every logical value is NULL; a malformed page can reach a similar state, so file corruption is not established from this stack alone.

Verified facts

• At the reported commit, ColumnChunkReader::load_page_data() subtracts the DataPageV2 repetition/definition-level lengths from uncompressed_page_size. It then creates _page_data from the decompression buffer and unconditionally calls the page decoder's set_data().
• When the remaining value-section size is zero, _reserve_decompress_buf(0) does not allocate a buffer, so _page_data can be Slice(nullptr, 0).
• BoolPlainDecoder::set_data() forwards that pointer to BatchedBitReader::Reset(), whose DCHECK(buffer != nullptr) aborts the BE. This matches every relevant frame and source line in the supplied stack.
• The Parquet format specification explicitly permits an all-NULL nullable page to have definition levels and no encoded values.
• The newer native Parquet reader already models this case with an EmptyValueSectionDecoder: it accepts a level-only all-NULL page and returns Corruption if definition levels require values that are absent. Its tests cover both outcomes, including BOOLEAN. This is useful implementation precedent, but it does not prove that this issue is fixed: loads currently remain on the legacy scanner path, and the legacy reader still has the unconditional set_data() behavior.
• The reported build identity needs clarification. Full SHA d5cfef083307be7f2dd5c3601b770e88fba40501 is based directly on the public 4.0.5 tag commit plus one OSS SDK commit; it is not the public 4.1.2 tag (aec169d20256a788de448f46737b5fb53ee3b4e3). The same risky legacy-reader pattern is present in the public 4.1.2 source, but reproduction against an official 4.1.2 build has not been provided.

Missing information needed to confirm the trigger

1. The exact load mechanism and minimal SQL: CREATE TABLE plus LOAD, COPY INTO, or INSERT ... SELECT, including relevant properties with credentials removed.
2. The failing Parquet file, or a sanitized/minimized reproducer preserving its schema and page boundaries. If the job reads many files, please first isolate the file that crashes query c22121b9ab894ef4-acd8857bdff4d757.
3. Page-level metadata for the suspected BOOLEAN column: writer library/version, compression codec, DataPage version, encoding, num_values, num_nulls, compressed/uncompressed page sizes, and repetition/definition-level byte lengths. Please also indicate whether the column/page is all NULL. A parquet-tools/PyArrow metadata dump and whether another conforming reader can read the file would help.
4. The complete BE INFO/WARNING log around the query ID, the corresponding FE audit/load log, and the query/load profile if one was produced. These should identify the load statement, scan range, and source file.
5. The exact package/build provenance (doris_be --version or equivalent full build information) and why it reports 4.1.2 while embedding the SHA above. Please also retry with an official 4.1.2 binary if possible.

Recommended next steps

• Add a focused legacy-reader reproducer for compressed DataPageV2, nullable BOOLEAN, PLAIN encoding, and an all-NULL/level-only page. Also cover DataPageV1/V2, compressed/uncompressed pages, and an invalid level-only page whose definition levels require a non-NULL value.
• Handle an empty physical value section in ColumnChunkReader, before invoking an encoding-specific decoder. Accept it only when decoded definition levels require zero physical values; otherwise return a bounded Status::Corruption. Do not merely relax BatchedBitReader::Reset() and do not leave a reused decoder with stale state.
• Ensure malformed input fails the load/query without terminating the BE.

Suggested triage labels: kind/fix, area/load, and investigation needed until the file/reproducer confirms the trigger.

[acarofpigs]

I would like to investigate this issue and submit a PR.

[nsivarajan]

@acarofpigs

Thanks! please investigate.
A potential fix was suggested by @Doris-Breakwater: the formatv2 implementation uses EmptyValueSectionDecoder. It may be worth following the same pattern here, as it appears to handle this case correctly.

[acarofpigs]

@nsivarajan Thanks for the suggestion!

I followed the EmptyValueSectionDecoder approach and opened #66556.

Would appreciate a review when you have time. Thanks!