dinosaur

一次nacos变更引发的死锁

Sat, 23 May 2026 00:00:00 GMT

背景

生产环境有很多xxljob的定时任务，大概每1-2周会触发一次xxljob的任务失败。触发时机也很不固定，有时是早上，有时是下午，而且表现是对应的后续的任务全部都没有执行，需要重启整个jvm实例。

问题排查

问题怀疑：

一开始怀疑是网络不稳定，但是其他服务没有这个现状，只有这个服务对应的实例会卡住，所以可能性很低。
后续怀疑是xxljob执行器线程满了，但是查看代码，每次xxljob回调执行器，执行器会new一个线程池，不会存在线程池满导致排队的情况
第三点是怀疑是http请求超时，但是超时怎么会导致其他任务也卡住在某个点
第四个点，有个公共的业务上的锁，锁住了，但是翻看业务代码，都不存在

问题定位

在一次偶然的机会，在改动nacos变更之后，发现几十分钟之后一堆xxljob任务都卡住了。刚好用命令jstak 把线程dump出来

定位到问题：所有xxljob线程阻塞在一个地方,就是例子中的B和C之前，然后还有一个nacos线程，他们都锁住了同一个对象。

代码类似

@Service
@RefreshScope
class A{

    @Xxljob("b")
    public B(){

    }

    @Xxljob("c")
    public C(){

    }
    ...
}

所以原因是xxljob有个耗时一个多小时的任务刚好在改nacos之前执行，然后RefreshScope导致需要刷新整个类，但是由于重新加载bean需要获取读写锁，但是正在执行的任务还有一个多小时才能执行，所以任务一直获取读锁没有释放，nacos变更之后需要获取写锁需要等这个任务执行完。那为什么后续任务都卡住呢？因为读写锁为了防止写锁饥饿，所以写锁插队，但是写锁是需要等这个正在执行的读锁释放。最后形成了：

正在执行的任务：还有一个多小时才能执行完
nacos更新线程：需要更新bean（因为更新nacos配置刷新bean），但是写锁需要等正在执行的读锁释放
其他调动线程：由于读写锁，为了防止饥饿，其他任务需要持有读锁，但是读锁需要让位写锁，所以在队列等待写锁执行。

最后形成了看到的场景：所有任务的卡住了

说到底：锁粒度太大，如果任务不用执行一个多小时，那可能没什么问题，因为也不是死锁

lsmtree

Sun, 01 Feb 2026 00:00:00 GMT

背景

学习lsm tree

Doris in 太长

Sat, 03 Jan 2026 00:00:00 GMT

背景

生产环境遇到这个错误： Exceeded the maximum children of an expression tree (10000).

原因: doris in的最大长度是10000 ，改成later view 不适应in就行

Doris 写入

Sat, 03 Jan 2026 00:00:00 GMT

Doris 存储层次

doris 首先有内存的memtable-->rowset->segment->column writer

rowset：rowset 是描述一次写入
segment：一个rowset可能由多个segment组成
tablet：tablet 是包括多个segment，一个segment包括多个rowset
column一行数据有多个column

导入代码

整个流程第一步：

Table: user_orders
├── Partition: p202401 (范围分区)
│   ├── Tablet 167 (Hash分桶)
│   │   ├── Rowset v1-v5 (基础版本)
│   │   │   ├── Segment 0 (256MB, 200万行)
│   │   │   │   ├── Page 0-4095: user_id 列
│   │   │   │   ├── Page 4096-8191: amount 列
│   │   │   │   └── Page 8192-12287: order_time 列
│   │   │   └── Segment 1 (256MB, 200万行)
│   │   ├── Rowset v6-v9 (增量版本)
│   │   └── Rowset v10 (最新版本)
│   ├── Tablet 168
│   └── ...
└── Partition: p202402

逻辑层：       表 (Table)
                │
                ▼
物理层：       Tablet (数据分片)
                │
                ▼
版本层：       Rowset (行集，版本管理单元)
                │
                ▼
文件层：       Segment (段，物理文件)
                │
                ▼
存储层：       Page (页，数据块)
                │
                ▼
数据层：       Column (列，存储格式)

// 位置: doris/be/src/vec/exprs/vexpr.cpp
// 表达式计算得到列数据

ColumnPtr VExpr::get_const_col(const Block& block, size_t row_idx) {
    // 计算这一行这个列的值
    // 例如：对于常量值 'Alice'，直接返回包含'Alice'的列
}

// 位置: doris/be/src/vec/columns/column.h
// 实际的列数据插入

template <typename T>
void ColumnVector<T>::insert_range_from(const IColumn& src, size_t start, size_t length) {
    const auto& src_vec = static_cast<const ColumnVector<T>&>(src);
    
    // 将源列的数据追加到当前列
    size_t old_size = data.size();
    data.resize(old_size + length);
    
    // 关键：这里发生了数据复制，从源列到目标列
    memcpy(&data[old_size], &src_vec.data[start], length * sizeof(T));
}

    void insert_data(const char* pos, size_t length) override {
        const size_t old_size = chars.size();
        const size_t new_size = old_size + length;

        if (length) {
            check_chars_length(new_size, offsets.size() + 1);
            chars.resize(new_size);
            memcpy(chars.data() + old_size, pos, length);
        }
        offsets.push_back(new_size);
        sanity_check_simple();
    }

以streamload 为例

streamload 使用生产者和消费者模型

生产者

下面是生产者的堆栈

* thread #431, name = 'rs_normal [work', stop reason = breakpoint 6.1
  * frame #0: 0x0000556a4b123c4b doris_be`doris::vectorized::NewJsonReader::_simdjson_set_column_value(this=0x00007f9433b9a180, value=0x00007f96db7721f8, block=0x00007f97c81a8ce8, slot_descs=size=4, valid=0x00007f96db7721ef) at new_json_reader.cpp:935:13
    frame #1: 0x0000556a4b1234c4 doris_be`doris::vectorized::NewJsonReader::_simdjson_handle_simple_json_write_columns(this=0x00007f9433b9a180, block=0x00007f97c81a8ce8, slot_descs=size=4, is_empty_row=0x00007f96db7723ef, eof=0x00007f9433b9a388) at new_json_reader.cpp:705:17
    frame #2: 0x0000556a4b11ffea doris_be`doris::vectorized::NewJsonReader::_simdjson_handle_simple_json(this=0x00007f9433b9a180, (null)=0x00007f953b52ba00, block=0x00007f97c81a8ce8, slot_descs=size=4, is_empty_row=0x00007f96db7723ef, eof=0x00007f9433b9a388) at new_json_reader.cpp:668:9
    frame #3: 0x0000556a4b11e737 doris_be`doris::vectorized::NewJsonReader::_read_json_column(this=0x00007f9433b9a180, state=0x00007f953b52ba00, block=0x00007f97c81a8ce8, slot_descs=size=4, is_empty_row=0x00007f96db7723ef, eof=0x00007f9433b9a388) at new_json_reader.cpp:502:12
    frame #4: 0x0000556a4b11bcee doris_be`doris::vectorized::NewJsonReader::get_next_block(this=0x00007f9433b9a180, block=0x00007f97c81a8ce8, read_rows=0x00007f96db7724b8, eof=0x00007f97c81a8a30) at new_json_reader.cpp:217:9
    frame #5: 0x0000556a4b60e080 doris_be`doris::vectorized::FileScanner::_get_block_wrapped(this=0x00007f97c81a8000, state=0x00007f953b52ba00, block=0x00007f978f75cfe0, eof=0x00007f96db7730d7) at file_scanner.cpp:465:13
    frame #6: 0x0000556a4b607e20 doris_be`doris::vectorized::FileScanner::_get_block_impl(this=0x00007f97c81a8000, state=0x00007f953b52ba00, block=0x00007f978f75cfe0, eof=0x00007f96db7730d7) at file_scanner.cpp:402:17
    frame #7: 0x0000556a4b6eeaab doris_be`doris::vectorized::Scanner::get_block(this=0x00007f97c81a8000, state=0x00007f953b52ba00, block=0x00007f978f75cfe0, eof=0x00007f96db7730d7) at scanner.cpp:143:17
    frame #8: 0x0000556a4b6ee67e doris_be`doris::vectorized::Scanner::get_block_after_projects(this=0x00007f97c81a8000, state=0x00007f953b52ba00, block=0x00007f978f75cfe0, eos=0x00007f96db7730d7) at scanner.cpp:119:16
    frame #9: 0x0000556a4b6f67a3 doris_be`doris::vectorized::ScannerScheduler::_scanner_scan(ctx=std::__shared_ptr<doris::vectorized::ScannerContext, __gnu_cxx::_S_atomic>::element_type @ 0x00007f953b0f8210, scan_task=std::__shared_ptr<doris::vectorized::ScanTask, __gnu_cxx::_S_atomic>::element_type @ 0x00007f953b542290) at scanner_scheduler.cpp:177:5
    frame #10: 0x0000556a4b6f569d doris_be`doris::vectorized::ScannerScheduler::submit(this=0x00007f96db773620)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const at scanner_scheduler.cpp:75:17
    frame #11: 0x0000556a4b6f54b7 doris_be`doris::vectorized::ScannerScheduler::submit(this=0x00007f97cafd0300)::$_0::operator()() const::'lambda'()::operator()() const at scanner_scheduler.cpp:74:27
    frame #12: 0x0000556a4b6f5475 doris_be`bool std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>((null)=__invoke_other @ 0x00007f96db77366f, __f=0x00007f97cafd0300) at invoke.h:61:14
    frame #13: 0x0000556a4b6f5435 doris_be`std::enable_if<is_invocable_r_v<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>(__fn=0x00007f97cafd0300) at invoke.h:114:9
    frame #14: 0x0000556a4b6f52ed doris_be`std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(__functor=0x00007f97c8daab38) at std_function.h:290:9
    frame #15: 0x0000556a43737e1e doris_be`std::function<bool ()>::operator()(this=0x00007f97c8daab38) const at std_function.h:591:9
    frame #16: 0x0000556a4b6f4782 doris_be`doris::vectorized::ScannerSplitRunner::process_for(this=0x00007f97c8daab10, (null)=(__r = 1000000000)) at scanner_scheduler.cpp:414:25
    frame #17: 0x0000556a4b75987c doris_be`doris::vectorized::PrioritizedSplitRunner::process(this=0x00007f97ce996b90) at prioritized_split_runner.cpp:103:35
    frame #18: 0x0000556a4b73fac0 doris_be`doris::vectorized::TimeSharingTaskExecutor::_dispatch_thread(this=0x00007f97651fe810) at time_sharing_task_executor.cpp:566:77
    frame #19: 0x0000556a4b74e5c2 doris_be`void std::__invoke_impl<void, void (doris::vectorized::TimeSharingTaskExecutor::*&)(), doris::vectorized::TimeSharingTaskExecutor*&>((null)=__invoke_memfun_deref @ 0x00007f96db7746cf, __f=0x00007f978f80af80, __t=0x00007f978f80af90) at invoke.h:74:14
    frame #20: 0x0000556a4b74e50d doris_be`std::__invoke_result<void (doris::vectorized::TimeSharingTaskExecutor::*&)(), doris::vectorized::TimeSharingTaskExecutor*&>::type std::__invoke<void (doris::vectorized::TimeSharingTaskExecutor::*&)(), doris::vectorized::TimeSharingTaskExecutor*&>(__fn=0x00007f978f80af80, __args=0x00007f978f80af90) at invoke.h:96:14
    frame #21: 0x0000556a4b74e4dd doris_be`void std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::* (doris::vectorized::TimeSharingTaskExecutor*))()>::__call<void, 0ul>(this=0x00007f978f80af80, __args=0x00007f96db774767, (null)=_Index_tuple<0UL> @ 0x00007f96db77473f) at functional:513:11
    frame #22: 0x0000556a4b74e496 doris_be`void std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::* (doris::vectorized::TimeSharingTaskExecutor*))()>::operator()<void>(this=0x00007f978f80af80) at functional:598:17
    frame #23: 0x0000556a4b74e465 doris_be`void std::__invoke_impl<void, std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::* (doris::vectorized::TimeSharingTaskExecutor*))()>&>((null)=__invoke_other @ 0x00007f96db77478f, __f=0x00007f978f80af80) at invoke.h:61:14
    frame #24: 0x0000556a4b74e425 doris_be`std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::* (doris::vectorized::TimeSharingTaskExecutor*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::* (doris::vectorized::TimeSharingTaskExecutor*))()>&>(__fn=0x00007f978f80af80) at invoke.h:111:2
    frame #25: 0x0000556a4b74e28d doris_be`std::_Function_handler<void (), std::_Bind<void (doris::vectorized::TimeSharingTaskExecutor::* (doris::vectorized::TimeSharingTaskExecutor*))()>>::_M_invoke(__functor=0x00007f978f821620) at std_function.h:290:9
    frame #26: 0x0000556a436da65e doris_be`std::function<void ()>::operator()(this=0x00007f978f821620) const at std_function.h:591:9
    frame #27: 0x0000556a4587d962 doris_be`doris::Thread::supervise_thread(arg=0x00007f978f821610) at thread.cpp:460:5
    frame #28: 0x00007f9898d66b7b libc.so.6`___lldb_unnamed_symbol3696 + 667
    frame #29: 0x00007f9898de47b8 libc.so.6`___lldb_unnamed_symbol4129 + 7

消费者

下面是消费者的堆栈

* thread #519, name = 'brpc_heavy', stop reason = breakpoint 4.1
  * frame #0: 0x000055ad89c6a1e6 doris_be`doris::MemTable::insert(this=0x00007f62caea3600, input_block=0x00007f64cb71fe98, row_idxs=size=1) at memtable.cpp:199:5
    frame #1: 0x000055ad89cacc15 doris_be`doris::MemTableWriter::write(this=0x00007f6579f88000, block=0x00007f64cb71fe98, row_idxs=size=1) at memtable_writer.cpp:118:27
    frame #2: 0x000055ad8a76e50b doris_be`doris::DeltaWriter::write(this=0x00007f6579f08e00, block=0x00007f64cb71fe98, row_idxs=size=1) at delta_writer.cpp:160:30
    frame #3: 0x000055ad8aad6744 doris_be`doris::BaseTabletsChannel::_write_block_data(doris::PTabletWriterAddBlockRequest const&, long, std::unordered_map<long, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>>>>&, doris::PTabletWriterAddBlockResult*)::$_0::operator()(this=0x00007f64cb71fd60, writer=0x00007f6579f08e00) const at tablets_channel.cpp:619:9
    frame #4: 0x000055ad8aad66fb doris_be`doris::Status std::__invoke_impl<doris::Status, doris::BaseTabletsChannel::_write_block_data(doris::PTabletWriterAddBlockRequest const&, long, std::unordered_map<long, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>>>>&, doris::PTabletWriterAddBlockResult*)::$_0&, doris::BaseDeltaWriter*>((null)=__invoke_other @ 0x00007f64cb71f927, __f=0x00007f64cb71fd60, __args=0x00007f64cb71f9d8) at invoke.h:61:14
    frame #5: 0x000055ad8aad6688 doris_be`std::enable_if<is_invocable_r_v<doris::Status, doris::BaseTabletsChannel::_write_block_data(doris::PTabletWriterAddBlockRequest const&, long, std::unordered_map<long, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>>>>&, doris::PTabletWriterAddBlockResult*)::$_0&, doris::BaseDeltaWriter*>, doris::Status>::type std::__invoke_r<doris::Status, doris::BaseTabletsChannel::_write_block_data(doris::PTabletWriterAddBlockRequest const&, long, std::unordered_map<long, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>>>>&, doris::PTabletWriterAddBlockResult*)::$_0&, doris::BaseDeltaWriter*>(__fn=0x00007f64cb71fd60, __args=0x00007f64cb71f9d8) at invoke.h:114:9
    frame #6: 0x000055ad8aad6588 doris_be`std::_Function_handler<doris::Status (doris::BaseDeltaWriter*), doris::BaseTabletsChannel::_write_block_data(doris::PTabletWriterAddBlockRequest const&, long, std::unordered_map<long, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>>>>&, doris::PTabletWriterAddBlockResult*)::$_0>::_M_invoke(__functor=0x00007f64cb71fd60, __args=0x00007f64cb71f9d8) at std_function.h:290:9
    frame #7: 0x000055ad8aae0b59 doris_be`std::function<doris::Status (doris::BaseDeltaWriter*)>::operator()(this=0x00007f64cb71fd60, __args=0x00007f6579f08e00) const at std_function.h:591:9
    frame #8: 0x000055ad8aad5ca5 doris_be`doris::BaseTabletsChannel::_write_block_data(doris::PTabletWriterAddBlockRequest const&, long, std::unordered_map<long, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>, std::hash<long>, std::equal_to<long>, std::allocator<std::pair<long const, std::vector<unsigned int, doris::CustomStdAllocator<unsigned int, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, true>>>>>>&, doris::PTabletWriterAddBlockResult*)::$_2::operator()(this=0x00007f64cb71fde0, tablet_id=1772371123463, write_func=function<doris::Status (doris::BaseDeltaWriter *)> @ 0x00007f64cb71fd60) const at tablets_channel.cpp:599:21
    frame #9: 0x000055ad8aad5855 doris_be`doris::BaseTabletsChannel::_write_block_data(this=0x00007f6579d54b10, request=0x00007f631fa65d00, cur_seq=0, tablet_to_rowidxs=size=1, response=0x00007f61bef4f9c0) at tablets_channel.cpp:619:9
    frame #10: 0x000055ad8aad05ad doris_be`doris::TabletsChannel::add_batch(this=0x00007f6579d54b10, request=0x00007f631fa65d00, response=0x00007f61bef4f9c0) at tablets_channel.cpp:657:12
    frame #11: 0x000055ad8a919e7c doris_be`doris::LoadChannel::add_batch(this=0x00007f6579c1a600, request=0x00007f631fa65d00, response=0x00007f61bef4f9c0) at load_channel.cpp:195:9
    frame #12: 0x000055ad8a90e6c1 doris_be`doris::LoadChannelMgr::add_batch(this=0x00007f65e4f0d280, request=0x00007f631fa65d00, response=0x00007f61bef4f9c0) at load_channel_mgr.cpp:178:26
    frame #13: 0x000055ad8ac5dc47 doris_be`doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0::operator()(this=0x00007f65e2726640) const at internal_service.cpp:502:54
    frame #14: 0x000055ad8ac5db85 doris_be`void std::__invoke_impl<void, doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0&>((null)=__invoke_other @ 0x00007f64cb72057f, __f=0x00007f65e2726640) at invoke.h:61:14
    frame #15: 0x000055ad8ac5db45 doris_be`std::enable_if<is_invocable_r_v<void, doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0&>, void>::type std::__invoke_r<void, doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0&>(__fn=0x00007f65e2726640) at invoke.h:111:2
    frame #16: 0x000055ad8ac5da2d doris_be`std::_Function_handler<void (), doris::PInternalService::tablet_writer_add_block(google::protobuf::RpcController*, doris::PTabletWriterAddBlockRequest const*, doris::PTabletWriterAddBlockResult*, google::protobuf::Closure*)::$_0>::_M_invoke(__functor=0x00007f64cb7206d0) at std_function.h:290:9
    frame #17: 0x000055ad88d2265e doris_be`std::function<void ()>::operator()(this=0x00007f64cb7206d0) const at std_function.h:591:9
    frame #18: 0x000055ad8ac7e6e2 doris_be`doris::WorkThreadPool<false>::work_thread(this=0x00007f658ae2ac10, thread_id=1) at work_thread_pool.hpp:159:17
    frame #19: 0x000055ad8ac7f1cc doris_be`void std::__invoke_impl<void, void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>((null)=__invoke_memfun_deref @ 0x00007f64cb7207df, __f=0x00007f658108b2a8, __t=0x00007f658108b2c0, __args=0x00007f658108b2b8) at invoke.h:74:14
    frame #20: 0x000055ad8ac7f155 doris_be`std::__invoke_result<void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>::type std::__invoke<void (doris::WorkThreadPool<false>::* const&)(int), doris::WorkThreadPool<false>*&, int&>(__fn=0x00007f658108b2a8, __args=0x00007f658108b2c0, __args=0x00007f658108b2b8) at invoke.h:96:14
    frame #21: 0x000055ad8ac7f125 doris_be`decltype(std::__invoke((*this)._M_pmf, std::forward<doris::WorkThreadPool<false>*&>(fp), std::forward<int&>(fp))) std::_Mem_fn_base<void (doris::WorkThreadPool<false>::*)(int), true>::operator()<doris::WorkThreadPool<false>*&, int&>(this=0x00007f658108b2a8, __args=0x00007f658108b2c0, __args=0x00007f658108b2b8) const at functional:177:11
    frame #22: 0x000055ad8ac7f0f5 doris_be`void std::__invoke_impl<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>((null)=__invoke_other @ 0x00007f64cb72086f, __f=0x00007f658108b2a8, __args=0x00007f658108b2c0, __args=0x00007f658108b2b8) at invoke.h:61:14
    frame #23: 0x000055ad8ac7f065 doris_be`std::enable_if<is_invocable_r_v<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>, void>::type std::__invoke_r<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)>&, doris::WorkThreadPool<false>*&, int&>(__fn=0x00007f658108b2a8, __args=0x00007f658108b2c0, __args=0x00007f658108b2b8) at invoke.h:111:2
    frame #24: 0x000055ad8ac7f032 doris_be`void std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>::__call<void, 0ul, 1ul>(this=0x00007f658108b2a8, __args=0x00007f64cb720907, (null)=_Index_tuple<0UL, 1UL> @ 0x00007f64cb7208df) at functional:661:11
    frame #25: 0x000055ad8ac7efc6 doris_be`void std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>::operator()<>(this=0x00007f658108b2a8) at functional:720:17
    frame #26: 0x000055ad8ac7ef95 doris_be`void std::__invoke_impl<void, std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>((null)=__invoke_other @ 0x00007f64cb72092f, __f=0x00007f658108b2a8) at invoke.h:61:14
    frame #27: 0x000055ad8ac7ef55 doris_be`std::__invoke_result<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>::type std::__invoke<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>(__fn=0x00007f658108b2a8) at invoke.h:96:14
    frame #28: 0x000055ad8ac7ef2d doris_be`void std::thread::_Invoker<std::tuple<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>>::_M_invoke<0ul>(this=0x00007f658108b2a8, (null)=_Index_tuple<0UL> @ 0x00007f64cb72096f) at std_thread.h:301:13
    frame #29: 0x000055ad8ac7ef05 doris_be`std::thread::_Invoker<std::tuple<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>>::operator()(this=0x00007f658108b2a8) at std_thread.h:308:11
    frame #30: 0x000055ad8ac7ee49 doris_be`std::thread::_State_impl<std::thread::_Invoker<std::tuple<std::_Bind_result<void, std::_Mem_fn<void (doris::WorkThreadPool<false>::*)(int)> (doris::WorkThreadPool<false>*, int)>>>>::_M_run(this=0x00007f658108b2a0) at std_thread.h:253:13
    frame #31: 0x00007f66b50e1224 libstdc++.so.6`___lldb_unnamed_symbol8036 + 20
    frame #32: 0x00007f66b4d66b7b libc.so.6`___lldb_unnamed_symbol3696 + 667
    frame #33: 0x00007f66b4de47b8 libc.so.6`___lldb_unnamed_symbol4129 + 7

如果memtable满了之后会推到线程池然后

void FlushToken::_flush_memtable(std::shared_ptr<MemTable> memtable_ptr, int32_t segment_id,
                                 int64_t submit_task_time) {
    signal::set_signal_task_id(_rowset_writer->load_id());
    signal::tablet_id = memtable_ptr->tablet_id();
    Defer defer {[&]() {
        std::lock_guard<std::mutex> lock(_mutex);
        _stats.flush_submit_count--;
        if (_stats.flush_submit_count == 0) {
            _submit_task_finish_cond.notify_one();
        }
        _stats.flush_running_count--;
        if (_stats.flush_running_count == 0) {
            _running_task_finish_cond.notify_one();
        }
    }};
    DBUG_EXECUTE_IF("FlushToken.flush_memtable.wait_before_first_shutdown",
                    { std::this_thread::sleep_for(std::chrono::milliseconds(10 * 1000)); });
    if (_is_shutdown()) {
        return;
    }
    DBUG_EXECUTE_IF("FlushToken.flush_memtable.wait_after_first_shutdown",
                    { std::this_thread::sleep_for(std::chrono::milliseconds(10 * 1000)); });
    _stats.flush_running_count++;
    // double check if shutdown to avoid wait running task finish count not accurate
    if (_is_shutdown()) {
        return;
    }
    DBUG_EXECUTE_IF("FlushToken.flush_memtable.wait_after_second_shutdown",
                    { std::this_thread::sleep_for(std::chrono::milliseconds(10 * 1000)); });
    uint64_t flush_wait_time_ns = MonotonicNanos() - submit_task_time;
    _stats.flush_wait_time_ns += flush_wait_time_ns;
    // If previous flush has failed, return directly
    {
        std::shared_lock rdlk(_flush_status_lock);
        if (!_flush_status.ok()) {
            return;
        }
    }

    MonotonicStopWatch timer;
    timer.start();
    size_t memory_usage = memtable_ptr->memory_usage();

    int64_t flush_size;
    Status s = _do_flush_memtable(memtable_ptr.get(), segment_id, &flush_size);

    {
        std::shared_lock rdlk(_flush_status_lock);
        if (!_flush_status.ok()) {
            return;
        }
    }
    if (!s.ok()) {
        std::lock_guard wrlk(_flush_status_lock);
        LOG(WARNING) << "Flush memtable failed with res = " << s
                     << ", load_id: " << print_id(_rowset_writer->load_id());
        _flush_status = s;
        return;
    }

    VLOG_CRITICAL << "flush memtable wait time: "
                  << PrettyPrinter::print(flush_wait_time_ns, TUnit::TIME_NS)
                  << ", flush memtable cost: "
                  << PrettyPrinter::print(timer.elapsed_time(), TUnit::TIME_NS)
                  << ", submit count: " << _stats.flush_submit_count
                  << ", running count: " << _stats.flush_running_count
                  << ", finish count: " << _stats.flush_finish_count
                  << ", mem size: " << PrettyPrinter::print_bytes(memory_usage)
                  << ", disk size: " << PrettyPrinter::print_bytes(flush_size);
    _stats.flush_time_ns += timer.elapsed_time();
    _stats.flush_finish_count++;
    _stats.flush_size_bytes += memtable_ptr->memory_usage();
    _stats.flush_disk_size_bytes += flush_size;
}

线程池的task最后执行的是

class MemtableFlushTask final : public Runnable {
    ENABLE_FACTORY_CREATOR(MemtableFlushTask);

public:
    MemtableFlushTask(std::shared_ptr<FlushToken> flush_token, std::shared_ptr<MemTable> memtable,
                      int32_t segment_id, int64_t submit_task_time)
            : _flush_token(flush_token),
              _memtable(memtable),
              _segment_id(segment_id),
              _submit_task_time(submit_task_time) {
        g_flush_task_num << 1;
    }

    ~MemtableFlushTask() override { g_flush_task_num << -1; }

    void run() override {
        auto token = _flush_token.lock();
        if (token) {
            token->_flush_memtable(_memtable, _segment_id, _submit_task_time); // 核心代码
        } else {
            LOG(WARNING) << "flush token is deconstructed, ignore the flush task";
        }
    }

private:
    std::weak_ptr<FlushToken> _flush_token;
    std::shared_ptr<MemTable> _memtable;
    int32_t _segment_id;
    int64_t _submit_task_time;
};

执行的是FlushToken::_do_flush_memtable

void FlushToken::_flush_memtable(std::shared_ptr<MemTable> memtable_ptr, int32_t segment_id,
                                 int64_t submit_task_time) {
    signal::set_signal_task_id(_rowset_writer->load_id());
    signal::tablet_id = memtable_ptr->tablet_id();
    Defer defer {[&]() {
        std::lock_guard<std::mutex> lock(_mutex);
        _stats.flush_submit_count--;
        if (_stats.flush_submit_count == 0) {
            _submit_task_finish_cond.notify_one();
        }
        _stats.flush_running_count--;
        if (_stats.flush_running_count == 0) {
            _running_task_finish_cond.notify_one();
        }
    }};
    DBUG_EXECUTE_IF("FlushToken.flush_memtable.wait_before_first_shutdown",
                    { std::this_thread::sleep_for(std::chrono::milliseconds(10 * 1000)); });
    if (_is_shutdown()) {
        return;
    }
    DBUG_EXECUTE_IF("FlushToken.flush_memtable.wait_after_first_shutdown",
                    { std::this_thread::sleep_for(std::chrono::milliseconds(10 * 1000)); });
    _stats.flush_running_count++;
    // double check if shutdown to avoid wait running task finish count not accurate
    if (_is_shutdown()) {
        return;
    }
    DBUG_EXECUTE_IF("FlushToken.flush_memtable.wait_after_second_shutdown",
                    { std::this_thread::sleep_for(std::chrono::milliseconds(10 * 1000)); });
    uint64_t flush_wait_time_ns = MonotonicNanos() - submit_task_time;
    _stats.flush_wait_time_ns += flush_wait_time_ns;
    // If previous flush has failed, return directly
    {
        std::shared_lock rdlk(_flush_status_lock);
        if (!_flush_status.ok()) {
            return;
        }
    }

    MonotonicStopWatch timer;
    timer.start();
    size_t memory_usage = memtable_ptr->memory_usage();

    int64_t flush_size;
    Status s = _do_flush_memtable(memtable_ptr.get(), segment_id, &flush_size);

    {
        std::shared_lock rdlk(_flush_status_lock);
        if (!_flush_status.ok()) {
            return;
        }
    }
    if (!s.ok()) {
        std::lock_guard wrlk(_flush_status_lock);
        LOG(WARNING) << "Flush memtable failed with res = " << s
                     << ", load_id: " << print_id(_rowset_writer->load_id());
        _flush_status = s;
        return;
    }

    VLOG_CRITICAL << "flush memtable wait time: "
                  << PrettyPrinter::print(flush_wait_time_ns, TUnit::TIME_NS)
                  << ", flush memtable cost: "
                  << PrettyPrinter::print(timer.elapsed_time(), TUnit::TIME_NS)
                  << ", submit count: " << _stats.flush_submit_count
                  << ", running count: " << _stats.flush_running_count
                  << ", finish count: " << _stats.flush_finish_count
                  << ", mem size: " << PrettyPrinter::print_bytes(memory_usage)
                  << ", disk size: " << PrettyPrinter::print_bytes(flush_size);
    _stats.flush_time_ns += timer.elapsed_time();
    _stats.flush_finish_count++;
    _stats.flush_size_bytes += memtable_ptr->memory_usage();
    _stats.flush_disk_size_bytes += flush_size;
}

最后执行的是FlushToken::_do_flush_memtable

Status FlushToken::_do_flush_memtable(MemTable* memtable, int32_t segment_id, int64_t* flush_size) {
    VLOG_CRITICAL << "begin to flush memtable for tablet: " << memtable->tablet_id()
                  << ", memsize: " << PrettyPrinter::print_bytes(memtable->memory_usage())
                  << ", rows: " << memtable->stat().raw_rows;
    memtable->update_mem_type(MemType::FLUSH);
    int64_t duration_ns = 0;
    {
        SCOPED_RAW_TIMER(&duration_ns);
        SCOPED_ATTACH_TASK(memtable->resource_ctx());
        SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER(
                memtable->resource_ctx()->memory_context()->mem_tracker()->write_tracker());
        SCOPED_CONSUME_MEM_TRACKER(memtable->mem_tracker());

        // DEFER_RELEASE_RESERVED();

        // auto reserve_size = memtable->get_flush_reserve_memory_size();
        // if (memtable->resource_ctx()->task_controller()->is_enable_reserve_memory() &&
        //     reserve_size > 0) {
        //     RETURN_IF_ERROR(_try_reserve_memory(memtable->resource_ctx(), reserve_size));
        // }

        // Defer defer {[&]() {
        //     ExecEnv::GetInstance()->storage_engine().memtable_flush_executor()->dec_flushing_task();
        // }};
        std::unique_ptr<Block> block;
        RETURN_IF_ERROR(memtable->to_block(&block));
        RETURN_IF_ERROR(_rowset_writer->flush_memtable(block.get(), segment_id, flush_size));
        memtable->set_flush_success();
    }
    _memtable_stat += memtable->stat();
    DorisMetrics::instance()->memtable_flush_total->increment(1);
    DorisMetrics::instance()->memtable_flush_duration_us->increment(duration_ns / 1000);
    VLOG_CRITICAL << "after flush memtable for tablet: " << memtable->tablet_id()
                  << ", flushsize: " << PrettyPrinter::print_bytes(*flush_size);
    return Status::OK();
}

Status FlushToken::_do_flush_memtable(MemTable* memtable, int32_t segment_id, int64_t* flush_size) {
    VLOG_CRITICAL << "begin to flush memtable for tablet: " << memtable->tablet_id()
                  << ", memsize: " << PrettyPrinter::print_bytes(memtable->memory_usage())
                  << ", rows: " << memtable->stat().raw_rows;
    memtable->update_mem_type(MemType::FLUSH);
    int64_t duration_ns = 0;
    {
        ... 
        RETURN_IF_ERROR(_rowset_writer->flush_memtable(block.get(), segment_id, flush_size));
        ...
    }

我们看看实现

Status BetaRowsetWriterV2::flush_memtable(vectorized::Block* block, int32_t segment_id,
                                          int64_t* flush_size) {
    if (block->rows() == 0) {
        return Status::OK();
    }

    {
        SCOPED_RAW_TIMER(&_segment_writer_ns);
        RETURN_IF_ERROR(_segment_creator.flush_single_block(block, segment_id, flush_size));
    }
    // delete bitmap and seg compaction are done on the destination BE.
    return Status::OK();
}

void Tablet::add_rowsets(const std::vector<RowsetSharedPtr>& to_add) {
    if (to_add.empty()) {
        return;
    }
    std::vector<RowsetMetaSharedPtr> rs_metas;
    rs_metas.reserve(to_add.size());
    for (auto& rs : to_add) {
        _rs_version_map.emplace(rs->version(), rs);
        _timestamped_version_tracker.add_version(rs->version());
        rs_metas.push_back(rs->rowset_meta());
    }
    _tablet_meta->modify_rs_metas(rs_metas, {});
}

Status BetaRowsetWriterV2::flush_memtable(vectorized::Block* block, int32_t segment_id,
                                          int64_t* flush_size) {
    if (block->rows() == 0) {
        return Status::OK();
    }

    {
        SCOPED_RAW_TIMER(&_segment_writer_ns);
        RETURN_IF_ERROR(_segment_creator.flush_single_block(block, segment_id, flush_size));
    }
    // delete bitmap and seg compaction are done on the destination BE.
    return Status::OK();
}

调用：

Status SegmentFlusher::flush_single_block(const vectorized::Block* block, int32_t segment_id,
                                          int64_t* flush_size) {
    if (block->rows() == 0) {
        return Status::OK();
    }
    vectorized::Block flush_block(*block);
    if (_context.write_type != DataWriteType::TYPE_COMPACTION &&
        _context.tablet_schema->num_variant_columns() > 0) {
        RETURN_IF_ERROR(_parse_variant_columns(flush_block));
    }
    bool no_compression = flush_block.bytes() <= config::segment_compression_threshold_kb * 1024;
    if (config::enable_vertical_segment_writer) {
        std::unique_ptr<segment_v2::VerticalSegmentWriter> writer;
        RETURN_IF_ERROR(_create_segment_writer(writer, segment_id, no_compression));
        RETURN_IF_ERROR_OR_CATCH_EXCEPTION(_add_rows(writer, &flush_block, 0, flush_block.rows()));
        RETURN_IF_ERROR(_flush_segment_writer(writer, flush_size));
    } else {
        std::unique_ptr<segment_v2::SegmentWriter> writer;
        RETURN_IF_ERROR(_create_segment_writer(writer, segment_id, no_compression));
        RETURN_IF_ERROR_OR_CATCH_EXCEPTION(_add_rows(writer, &flush_block, 0, flush_block.rows()));
        RETURN_IF_ERROR(_flush_segment_writer(writer, flush_size));
    }
    return Status::OK();
}

后面调用

Status SegmentWriter::append_block(const Block* block, size_t row_pos, size_t num_rows) {
    if (_opts.rowset_ctx->partial_update_info &&
        _opts.rowset_ctx->partial_update_info->is_partial_update() &&
        _opts.write_type == DataWriteType::TYPE_DIRECT &&
        !_opts.rowset_ctx->is_transient_rowset_writer) {
        if (_opts.rowset_ctx->partial_update_info->is_fixed_partial_update()) {
            RETURN_IF_ERROR(append_block_with_partial_content(block, row_pos, num_rows));
        } else {
            return Status::NotSupported<false>(
                    "SegmentWriter doesn't support flexible partial update, please set "
                    "enable_vertical_segment_writer=true in be.conf on all BEs to use "
                    "VerticalSegmentWriter.");
        }
        return Status::OK();
    }
    if (block->columns() < _column_writers.size()) {
        return Status::InternalError(
                "block->columns() < _column_writers.size(), block->columns()=" +
                std::to_string(block->columns()) +
                ", _column_writers.size()=" + std::to_string(_column_writers.size()) +
                ", _tablet_schema->dump_structure()=" + _tablet_schema->dump_structure());
    }
    CHECK(block->columns() >= _column_writers.size())
            << ", block->columns()=" << block->columns()
            << ", _column_writers.size()=" << _column_writers.size()
            << ", _tablet_schema->dump_structure()=" << _tablet_schema->dump_structure();
    // Row column should be filled here when it's a directly write from memtable
    // or it's schema change write(since column data type maybe changed, so we should reubild)
    if (_opts.write_type == DataWriteType::TYPE_DIRECT ||
        _opts.write_type == DataWriteType::TYPE_SCHEMA_CHANGE) {
        _serialize_block_to_row_column(*block);
    }

    if (_opts.rowset_ctx->write_type != DataWriteType::TYPE_COMPACTION &&
        _tablet_schema->num_variant_columns() > 0) {
        RETURN_IF_ERROR(variant_util::parse_and_materialize_variant_columns(
                const_cast<Block&>(*block), *_tablet_schema, _column_ids));
    }

    _olap_data_convertor->set_source_content(block, row_pos, num_rows);

    // find all row pos for short key indexes
    std::vector<size_t> short_key_pos;
    if (_has_key) {
        // We build a short key index every `_opts.num_rows_per_block` rows. Specifically, we
        // build a short key index using 1st rows for first block and `_short_key_row_pos - _row_count`
        // for next blocks.
        // Ensure we build a short key index using 1st rows only for the first block (ISSUE-9766).
        if (UNLIKELY(_short_key_row_pos == 0 && _num_rows_written == 0)) {
            short_key_pos.push_back(0);
        }
        while (_short_key_row_pos + _opts.num_rows_per_block < _num_rows_written + num_rows) {
            _short_key_row_pos += _opts.num_rows_per_block;
            short_key_pos.push_back(_short_key_row_pos - _num_rows_written);
        }
    }

    // convert column data from engine format to storage layer format
    std::vector<IOlapColumnDataAccessor*> key_columns;
    IOlapColumnDataAccessor* seq_column = nullptr;
    for (size_t id = 0; id < _column_writers.size(); ++id) {
        // olap data convertor alway start from id = 0
        auto converted_result = _olap_data_convertor->convert_column_data(id);
        if (!converted_result.first.ok()) {
            return converted_result.first;
        }
        auto cid = _column_ids[id];
        if (_has_key && cid < _tablet_schema->num_key_columns()) {
            key_columns.push_back(converted_result.second);
        } else if (_has_key && _tablet_schema->has_sequence_col() &&
                   cid == _tablet_schema->sequence_col_idx()) {
            seq_column = converted_result.second;
        }
        RETURN_IF_ERROR(_column_writers[id]->append(converted_result.second->get_nullmap(),
                                                    converted_result.second->get_data(), num_rows));
    }
    if (_opts.write_type == DataWriteType::TYPE_COMPACTION) {
        RETURN_IF_ERROR(
                _variant_stats_calculator->calculate_variant_stats(block, row_pos, num_rows));
    }
    if (_has_key) {
        if (_is_mow_with_cluster_key()) {
            // for now we don't need to query short key index for CLUSTER BY feature,
            // but we still write the index for future usage.
            // 1. generate primary key index, the key_columns is primary_key_columns
            RETURN_IF_ERROR(_generate_primary_key_index(_primary_key_coders, key_columns,
                                                        seq_column, num_rows, true));
            // 2. generate short key index (use cluster key)
            key_columns.clear();
            for (const auto& cid : _tablet_schema->cluster_key_uids()) {
                // find cluster key index in tablet schema
                auto cluster_key_index = _tablet_schema->field_index(cid);
                if (cluster_key_index == -1) {
                    return Status::InternalError(
                            "could not find cluster key column with unique_id=" +
                            std::to_string(cid) + " in tablet schema");
                }
                bool found = false;
                for (auto i = 0; i < _column_ids.size(); ++i) {
                    if (_column_ids[i] == cluster_key_index) {
                        auto converted_result = _olap_data_convertor->convert_column_data(i);
                        if (!converted_result.first.ok()) {
                            return converted_result.first;
                        }
                        key_columns.push_back(converted_result.second);
                        found = true;
                        break;
                    }
                }
                if (!found) {
                    return Status::InternalError(
                            "could not found cluster key column with unique_id=" +
                            std::to_string(cid) +
                            ", tablet schema index=" + std::to_string(cluster_key_index));
                }
            }
            RETURN_IF_ERROR(_generate_short_key_index(key_columns, num_rows, short_key_pos));
        } else if (_is_mow()) {
            RETURN_IF_ERROR(_generate_primary_key_index(_key_coders, key_columns, seq_column,
                                                        num_rows, false));
        } else {
            RETURN_IF_ERROR(_generate_short_key_index(key_columns, num_rows, short_key_pos));
        }
    }

    _num_rows_written += num_rows;
    _olap_data_convertor->clear_source_content();
    return Status::OK();
}

核心转换逻辑，将行转换成列：

std::pair<Status, IOlapColumnDataAccessor*> OlapBlockDataConvertor::convert_column_data(
        size_t cid) {
    assert(cid < _convertors.size());
    auto convert_func = [&]() -> Status {
        RETURN_IF_ERROR_OR_CATCH_EXCEPTION(_convertors[cid]->convert_to_olap());
        return Status::OK();
    };
    auto status = convert_func();
    return {status, _convertors[cid].get()};
}

最后segment 写入数据

Status SegmentWriter::finalize(uint64_t* segment_file_size, uint64_t* index_size) {
    MonotonicStopWatch timer;
    timer.start();
    // check disk capacity
    if (_data_dir != nullptr && _data_dir->reach_capacity_limit((int64_t)estimate_segment_size())) {
        return Status::Error<DISK_REACH_CAPACITY_LIMIT>("disk {} exceed capacity limit, path: {}",
                                                        _data_dir->path_hash(), _data_dir->path());
    }
    // write data
    RETURN_IF_ERROR(finalize_columns_data());
    // Get the index start before finalize_footer since this function would write new data.
    uint64_t index_start = _file_writer->bytes_appended();
    // write index
    RETURN_IF_ERROR(finalize_columns_index(index_size));
    // write footer
    RETURN_IF_ERROR(finalize_footer(segment_file_size));

    if (timer.elapsed_time() > 5000000000l) {
        LOG(INFO) << "segment flush consumes a lot time_ns " << timer.elapsed_time()
                  << ", segmemt_size " << *segment_file_size;
    }
    // When the cache type is not ttl(expiration time == 0), the data should be split into normal cache queue
    // and index cache queue
    if (auto* cache_builder = _file_writer->cache_builder(); cache_builder != nullptr &&
                                                             cache_builder->_expiration_time == 0 &&
                                                             config::is_cloud_mode()) {
        auto size = *index_size + *segment_file_size;
        auto holder = cache_builder->allocate_cache_holder(index_start, size, _tablet->tablet_id());
        for (auto& segment : holder->file_blocks) {
            static_cast<void>(segment->change_cache_type(io::FileCacheType::INDEX));
        }
    }
    return Status::OK();
}

实际的堆栈：

* thread #439, name = 'mf_normal [work', stop reason = breakpoint 2.1
  * frame #0: 0x000055ddfe4eaabc doris_be`doris::SegmentFlusher::flush_single_block(this=0x00007fbbbce0e450, block=0x00007fbc60901420, segment_id=0, flush_size=0x00007fbe639f4b50) at segment_creator.cpp:64:9
    frame #1: 0x000055ddfe4ef61a doris_be`doris::SegmentCreator::flush_single_block(this=0x00007fbbbce0e448, block=0x00007fbc60901420, segment_id=0, flush_size=0x00007fbe639f4b50) at segment_creator.cpp:404:5
    frame #2: 0x000055ddfe46a3c5 doris_be`doris::BaseBetaRowsetWriter::flush_memtable(this=0x00007fbbbce0e000, block=0x00007fbc60901420, segment_id=0, flush_size=0x00007fbe639f4b50) at beta_rowset_writer.cpp:817:9
    frame #3: 0x000055ddfe311da3 doris_be`doris::FlushToken::_do_flush_memtable(this=0x00007fbbbce22010, memtable=0x00007fbbbcce9200, segment_id=0, flush_size=0x00007fbe639f4b50) at memtable_flush_executor.cpp:210:9
    frame #4: 0x000055ddfe30fed4 doris_be`doris::FlushToken::_flush_memtable(this=0x00007fbbbce22010, memtable_ptr=std::__shared_ptr<doris::MemTable, __gnu_cxx::_S_atomic>::element_type @ 0x00007fbbbcce9200, segment_id=0, submit_task_time=120805632954059) at memtable_flush_executor.cpp:265:16
    frame #5: 0x000055ddfe3177b2 doris_be`doris::MemtableFlushTask::run(this=0x00007fbbbcdd70f0) at memtable_flush_executor.cpp:63:20
    frame #6: 0x000055ddff559fc8 doris_be`doris::ThreadPool::dispatch_thread(this=0x00007fbef9d2e980) at threadpool.cpp:616:24
    frame #7: 0x000055ddff568d42 doris_be`void std::__invoke_impl<void, void (doris::ThreadPool::*&)(), doris::ThreadPool*&>((null)=__invoke_memfun_deref @ 0x00007fbe639f56cf, __f=0x00007fbef9e46b20, __t=0x00007fbef9e46b30) at invoke.h:74:14
    frame #8: 0x000055ddff568c8d doris_be`std::__invoke_result<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(), doris::ThreadPool*&>(__fn=0x00007fbef9e46b20, __args=0x00007fbef9e46b30) at invoke.h:96:14
    frame #9: 0x000055ddff568c5d doris_be`void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::__call<void, 0ul>(this=0x00007fbef9e46b20, __args=0x00007fbe639f5767, (null)=_Index_tuple<0UL> @ 0x00007fbe639f573f) at functional:513:11
    frame #10: 0x000055ddff568c16 doris_be`void std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>::operator()<void>(this=0x00007fbef9e46b20) at functional:598:17
    frame #11: 0x000055ddff568be5 doris_be`void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>((null)=__invoke_other @ 0x00007fbe639f578f, __f=0x00007fbef9e46b20) at invoke.h:61:14
    frame #12: 0x000055ddff568ba5 doris_be`std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>&>(__fn=0x00007fbef9e46b20) at invoke.h:111:2
    frame #13: 0x000055ddff568a0d doris_be`std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::* (doris::ThreadPool*))()>>::_M_invoke(__functor=0x00007fbef9e4b520) at std_function.h:290:9
    frame #14: 0x000055ddfd3a265e doris_be`std::function<void ()>::operator()(this=0x00007fbef9e4b520) const at std_function.h:591:9
    frame #15: 0x000055ddff545962 doris_be`doris::Thread::supervise_thread(arg=0x00007fbef9e4b510) at thread.cpp:460:5
    frame #16: 0x00007fc02369db7b libc.so.6`___lldb_unnamed_symbol3696 + 667
    frame #17: 0x00007fc02371b7b8 libc.so.6`___lldb_unnamed_symbol4129 + 7

rowset/tablelet/segment/sst/row 关系

rowset/tablelet/segment和sst是什么关系呢？

tablet 包含多个rowset
rowset是一次写入，一次写入包括一行或者多行
一个rowset 包含一个或者多个segment
segment 包括一行或者多行

Markdown Extended Features

Wed, 01 May 2024 00:00:00 GMT

GitHub Repository Cards

You can add dynamic cards that link to GitHub repositories, on page load, the repository information is pulled from the GitHub API.

::github{repo="Fabrizz/MMM-OnSpotify"}

Create a GitHub repository card with the code ::github{repo="<owner>/<repo>"}.

::github{repo="saicaca/fuwari"}

Admonitions

Following types of admonitions are supported: note tip important warning caution

:::note Highlights information that users should take into account, even when skimming. :::

:::tip Optional information to help a user be more successful. :::

:::important Crucial information necessary for users to succeed. :::

:::warning Critical content demanding immediate user attention due to potential risks. :::

:::caution Negative potential consequences of an action. :::

Basic Syntax

:::note
Highlights information that users should take into account, even when skimming.
:::

:::tip
Optional information to help a user be more successful.
:::

Custom Titles

The title of the admonition can be customized.

:::note[MY CUSTOM TITLE] This is a note with a custom title. :::

:::note[MY CUSTOM TITLE]
This is a note with a custom title.
:::

GitHub Syntax

[!TIP] The GitHub syntax is also supported.

> [!NOTE]
> The GitHub syntax is also supported.

> [!TIP]
> The GitHub syntax is also supported.

Spoiler

You can add spoilers to your text. The text also supports Markdown syntax.

The content :spoiler[is hidden ayyy]!

The content :spoiler[is hidden **ayyy**]!

Expressive Code Example

Wed, 10 Apr 2024 00:00:00 GMT

Here, we'll explore how code blocks look using Expressive Code. The provided examples are based on the official documentation, which you can refer to for further details.

Expressive Code

Syntax Highlighting

Regular syntax highlighting

console.log('This code is syntax highlighted!')

Rendering ANSI escape sequences

ANSI colors:
- Regular: [31mRed[0m [32mGreen[0m [33mYellow[0m [34mBlue[0m [35mMagenta[0m [36mCyan[0m
- Bold:    [1;31mRed[0m [1;32mGreen[0m [1;33mYellow[0m [1;34mBlue[0m [1;35mMagenta[0m [1;36mCyan[0m
- Dimmed:  [2;31mRed[0m [2;32mGreen[0m [2;33mYellow[0m [2;34mBlue[0m [2;35mMagenta[0m [2;36mCyan[0m

256 colors (showing colors 160-177):
[38;5;160m160 [38;5;161m161 [38;5;162m162 [38;5;163m163 [38;5;164m164 [38;5;165m165[0m
[38;5;166m166 [38;5;167m167 [38;5;168m168 [38;5;169m169 [38;5;170m170 [38;5;171m171[0m
[38;5;172m172 [38;5;173m173 [38;5;174m174 [38;5;175m175 [38;5;176m176 [38;5;177m177[0m

Full RGB colors:
[38;2;34;139;34mForestGreen - RGB(34, 139, 34)[0m

Text formatting: [1mBold[0m [2mDimmed[0m [3mItalic[0m [4mUnderline[0m

Editor & Terminal Frames

Code editor frames

console.log('Title attribute example')

<!-- src/content/index.html -->
<div>File name comment example</div>

Terminal frames

echo "This terminal frame has no title"

Write-Output "This one has a title!"

Overriding frame types

echo "Look ma, no frame!"

# Without overriding, this would be a terminal frame
function Watch-Tail { Get-Content -Tail 20 -Wait $args }
New-Alias tail Watch-Tail

Text & Line Markers

Marking full lines & line ranges

// Line 1 - targeted by line number
// Line 2
// Line 3
// Line 4 - targeted by line number
// Line 5
// Line 6
// Line 7 - targeted by range "7-8"
// Line 8 - targeted by range "7-8"

Selecting line marker types (mark, ins, del)

function demo() {
  console.log('this line is marked as deleted')
  // This line and the next one are marked as inserted
  console.log('this is the second inserted line')

  return 'this line uses the neutral default marker type'
}

Adding labels to line markers

// labeled-line-markers.jsx
<button
  role="button"
  {...props}
  value={value}
  className={buttonClassName}
  disabled={disabled}
  active={active}
>
  {children &&
    !active &&
    (typeof children === 'string' ? <span>{children}</span> : children)}
</button>

Adding long labels on their own lines

// labeled-line-markers.jsx
<button
  role="button"
  {...props}

  value={value}
  className={buttonClassName}

  disabled={disabled}
  active={active}
>

  {children &&
    !active &&
    (typeof children === 'string' ? <span>{children}</span> : children)}
</button>

Using diff-like syntax

+this line will be marked as inserted
-this line will be marked as deleted
this is a regular line

--- a/README.md
+++ b/README.md
@@ -1,3 +1,4 @@
+this is an actual diff file
-all contents will remain unmodified
 no whitespace will be removed either

Combining syntax highlighting with diff-like syntax

  function thisIsJavaScript() {
    // This entire block gets highlighted as JavaScript,
    // and we can still add diff markers to it!
-   console.log('Old code to be removed')
+   console.log('New and shiny code!')
  }

Marking individual text inside lines

function demo() {
  // Mark any given text inside lines
  return 'Multiple matches of the given text are supported';
}

Regular expressions

console.log('The words yes and yep will be marked.')

Escaping forward slashes

echo "Test" > /home/test.txt

Selecting inline marker types (mark, ins, del)

function demo() {
  console.log('These are inserted and deleted marker types');
  // The return statement uses the default marker type
  return true;
}

Word Wrap

Configuring word wrap per block

// Example with wrap
function getLongString() {
  return 'This is a very long string that will most probably not fit into the available space unless the container is extremely wide'
}

// Example with wrap=false
function getLongString() {
  return 'This is a very long string that will most probably not fit into the available space unless the container is extremely wide'
}

Configuring indentation of wrapped lines

// Example with preserveIndent (enabled by default)
function getLongString() {
  return 'This is a very long string that will most probably not fit into the available space unless the container is extremely wide'
}

// Example with preserveIndent=false
function getLongString() {
  return 'This is a very long string that will most probably not fit into the available space unless the container is extremely wide'
}

Collapsible Sections

// All this boilerplate setup code will be collapsed
import { someBoilerplateEngine } from '@example/some-boilerplate'
import { evenMoreBoilerplate } from '@example/even-more-boilerplate'

const engine = someBoilerplateEngine(evenMoreBoilerplate())

// This part of the code will be visible by default
engine.doSomething(1, 2, 3, calcFn)

function calcFn() {
  // You can have multiple collapsed sections
  const a = 1
  const b = 2
  const c = a + b

  // This will remain visible
  console.log(`Calculation result: ${a} + ${b} = ${c}`)
  return c
}

// All this code until the end of the block will be collapsed again
engine.closeConnection()
engine.freeMemory()
engine.shutdown({ reason: 'End of example boilerplate code' })

Line Numbers

Displaying line numbers per block

// This code block will show line numbers
console.log('Greetings from line 2!')
console.log('I am on line 3')

// Line numbers are disabled for this block
console.log('Hello?')
console.log('Sorry, do you know what line I am on?')

Changing the starting line number

console.log('Greetings from line 5!')
console.log('I am on line 6')

Simple Guides for Fuwari

Mon, 01 Apr 2024 00:00:00 GMT

Cover image source: Source

This blog template is built with Astro. For the things that are not mentioned in this guide, you may find the answers in the Astro Docs.

Front-matter of Posts

---
title: My First Blog Post
published: 2023-09-09
description: This is the first post of my new Astro blog.
image: ./cover.jpg
tags: [Foo, Bar]
category: Front-end
draft: false
---

Attribute	Description
`title`	The title of the post.
`published`	The date the post was published.
`description`	A short description of the post. Displayed on index page.
`image`	The cover image path of the post.<br/>1. Start with `http://` or `https://`: Use web image<br/>2. Start with `/`: For image in `public` dir<br/>3. With none of the prefixes: Relative to the markdown file
`tags`	The tags of the post.
`category`	The category of the post.
`draft`	If this post is still a draft, which won't be displayed.

Where to Place the Post Files

Your post files should be placed in src/content/posts/ directory. You can also create sub-directories to better organize your posts and assets.

src/content/posts/
├── post-1.md
└── post-2/
    ├── cover.png
    └── index.md

Markdown Example

Sun, 01 Oct 2023 00:00:00 GMT

An h1 header

Paragraphs are separated by a blank line.

2nd paragraph. Italic, bold, and monospace. Itemized lists look like:

this one
that one
the other one

Note that --- not considering the asterisk --- the actual text content starts at 4-columns in.

Block quotes are written like so.

They can span multiple paragraphs, if you like.

Use 3 dashes for an em-dash. Use 2 dashes for ranges (ex., "it's all in chapters 12--14"). Three dots ... will be converted to an ellipsis. Unicode is supported. ☺

An h2 header

Here's a numbered list:

first item
second item
third item

Note again how the actual text starts at 4 columns in (4 characters from the left side). Here's a code sample:

# Let me re-iterate ...
for i in 1 .. 10 { do-something(i) }

As you probably guessed, indented 4 spaces. By the way, instead of indenting the block, you can use delimited blocks, if you like:

define foobar() {
    print "Welcome to flavor country!";
}

(which makes copying & pasting easier). You can optionally mark the delimited block for Pandoc to syntax highlight it:

import time
# Quick, count to ten!
for i in range(10):
    # (but not *too* quick)
    time.sleep(0.5)
    print i

An h3 header

Now a nested list:

First, get these ingredients:
- carrots
- celery
- lentils
Boil some water.

Dump everything in the pot and follow this algorithm:

 find wooden spoon
 uncover pot
 stir
 cover pot
 balance wooden spoon precariously on pot handle
 wait 10 minutes
 goto first step (or shut off burner when done)

Do not bump wooden spoon or it will fall.

Notice again how text always lines up on 4-space indents (including that last line which continues item 3 above).

Here's a link to a website, to a local doc, and to a section heading in the current doc. Here's a footnote [^1].

[^1]: Footnote text goes here.

Tables can look like this:

size material color

9 leather brown 10 hemp canvas natural 11 glass transparent

Table: Shoes, their sizes, and what they're made of

(The above is the caption for the table.) Pandoc also supports multi-line tables:

keyword text

red Sunsets, apples, and other red or reddish things.

green Leaves, grass, frogs and other things it's not easy being.

A horizontal rule follows.

Here's a definition list:

apples : Good for making applesauce. oranges : Citrus! tomatoes : There's no "e" in tomatoe.

Again, text is indented 4 spaces. (Put a blank line between each term/definition pair to spread things out more.)

Here's a "line block":

| Line one | Line too | Line tree

and images can be specified like so:

Inline math equations go in like so: $\omega = d\phi / dt$. Display math should get its own line and be put in in double-dollarsigns:

$$I = \int \rho R^{2} dV$$

$$ \begin{equation*} \pi =3.1415926535 ;8979323846;2643383279;5028841971;6939937510;5820974944 ;5923078164;0628620899;8628034825;3421170679;\ldots \end{equation*} $$

And note that you can backslash-escape any punctuation characters which you wish to be displayed literally, ex.: `foo`, *bar*, etc.

Include Video in the Posts

Tue, 01 Aug 2023 00:00:00 GMT

Just copy the embed code from YouTube or other platforms, and paste it in the markdown file.

---
title: Include Video in the Post
published: 2023-10-19
// ...
---

<iframe width="100%" height="468" src="https://www.youtube.com/embed/5gIf0_xpFPI?si=N1WTorLKL0uwLsU_" title="YouTube video player" frameborder="0" allowfullscreen></iframe>

YouTube

Bilibili