TiKV 源码解析系列文章（十七）raftstore 概览| PingCAP

相关文章推荐

热心肠的冲锋衣 · flink shuffle机制-掘金· 2 年前 ·

热心肠的冲锋衣 · 产品| PingCAP· 2 年前 ·

热心肠的冲锋衣 · TiKV 简介| PingCAP 文档中心· 2 年前 ·

热心肠的冲锋衣 · TiDB 整体架构| PingCAP 文档中心· 2 年前 ·

</noscript><nav class="navbar index-module--header--+tJQO" role="navigation" aria-label="main navigation"><div class="navbar-brand index-module--mobileNavBar--shD-Y"><span><a class="navbar-item index-module--logo--nlzXC" href="/"><img src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCA2MTcuMzYgMTYwIj48ZGVmcz48c3R5bGU+LmNscy0xe2ZpbGw6I2ZmZjt9LmNscy0ye2ZpbGw6bm9uZTt9LmNscy0ze2ZpbGw6IzE3MmQ3Mjt9LmNscy00e2ZpbGw6IzAwYWVlZjt9LmNscy01e2ZpbGw6IzNkM2ZlYTt9PC9zdHlsZT48L2RlZnM+PGcgaWQ9IuWbvuWxgl8yIiBkYXRhLW5hbWU9IuWbvuWxgiAyIj48ZyBpZD0iTGF5ZXJfMSIgZGF0YS1uYW1lPSJMYXllciAxIj48cGF0aCBjbGFzcz0iY2xzLTEiIGQ9Ik00NzEuOCw5OC4zM2gwbC0uMTcuMjdhMzAuMjYsMzAuMjYsMCwwLDEtMTAuMzEsOS44NCwyNy42MSwyNy42MSwwLDAsMS0xNC4xLDQuMDhjLTguMjIsMC0xNS40MS0zLjI4LTIxLjM1LTkuNzRhMzIuOSwzMi45LDAsMCwxLTguNTQtMjMuMjcsMzIuMzYsMzIuMzYsMCwwLDEsOC44Mi0yMi4wN2M1LjczLTYuMTQsMTIuNjItOS4yNSwyMC41MS05LjI1LDEwLjc4LS4wOSwxOC45Miw0LjM2LDI0Ljg3LDEzLjYybC4xOC4yN2guMTZsNy00LjM0YTM5LDM5LDAsMCwwLTExLjU5LTEyLjMxLDM2LjEyLDM2LjEyLDAsMCwwLTIwLjA4LTYuMDgsMzUuMjIsMzUuMjIsMCwwLDAtMjYuOCwxMS40MmMtNy40Nyw3LjY5LTExLjI1LDE3LjQ5LTExLjI1LDI5LjEzYTQyLjEsNDIuMSwwLDAsMCwxMS4xNCwyOS4yMWM3LjM3LDgsMTYuMjksMTIsMjYuNTQsMTJoLjM4YzEzLjQ2LS4yOCwyNC41Ni02LjgsMzEuNjMtMTguNDdaIi8+PHBvbHlnb24gY2xhc3M9ImNscy0xIiBwb2ludHM9IjUyNC4xOCA0MC4zNyA1MTQuNjYgNDAuMzcgNDc5LjI0IDExOS42MyA0OTAuNTUgMTE5LjYzIDUwMS40OSA5My4yOSA1MDEuNTEgOTMuMjkgNTA1LjM3IDg1LjA1IDUwNS4zNiA4NS4wNSA1MTkuNjEgNTEuNiA1MzMuNTggODUuMDUgNTEzLjUzIDg1LjA1IDUwOS44NyA5My4yOSA1MzYuODQgOTMuMjkgNTQ3Ljg4IDExOS42MyA1NTcuODUgMTE5LjYzIDUyNC4xOCA0MC4zNyIvPjxwYXRoIGNsYXNzPSJjbHMtMSIgZD0iTTYwOS42NSw0Ny42MmMtNS4yNi00Ljc1LTExLjg4LTcuMTUtMTkuNjctNy4xNUg1NjQuMzN2NzkuMTZoOS4zNXYtMzFoMFY4MC4yOWgwVjQ5aDE2LjcxYTE4LjU0LDE4LjU0LDAsMCwxLDEyLjE4LDQuMjMsMTMuNjQsMTMuNjQsMCwwLDEsNS41NCwxMC40OSwxMy45MiwxMy45MiwwLDAsMS01LDEyLjA3LDE3LjU0LDE3LjU0LDAsMCwxLTEyLjI1LDQuNWgtOS42OHY4LjM0aDExLjJBMjUuMDUsMjUuMDUsMCwwLDAsNjA5Ljg1LDgyYTIxLjc0LDIxLjc0LDAsMCwwLDcuNS0xNi40NkEyMi40MSwyMi40MSwwLDAsMCw2MDkuNjUsNDcuNjJaIi8+PHJlY3QgY2xhc3M9ImNscy0xIiB4PSIyNjEuNjEiIHk9IjQxLjkzIiB3aWR0aD0iOS4zNCIgaGVpZ2h0PSI5LjM0Ii8+PHJlY3QgY2xhc3M9ImNscy0xIiB4PSIyNjEuNjEiIHk9IjYyLjg2IiB3aWR0aD0iOS4zNCIgaGVpZ2h0PSI1Ni44OSIvPjxwYXRoIGNsYXNzPSJjbHMtMSIgZD0iTTQwMi4zNSwxMTQuMDZWODkuNjNhMjksMjksMCwwLDAtOS4yLTIxLjkyLDI5LjYyLDI5LjYyLDAsMCwwLTQyLjk0LjQxLDMwLjE2LDMwLjE2LDAsMCwwLTguNDgsMjEuNDEsMzAuNzQsMzAuNzQsMCwwLDAsOS4yLDIyLjIyLDI4LjU4LDI4LjU4LDAsMCwwLDIxLjk0LDguOCwyOS4xNiwyOS4xNiwwLDAsMCwyMC42NC05LjI2djMuODdtMCwwYTE3LjgxLDE3LjgxLDAsMCwxLTYuMDksMTMuNDQsMjAuMzgsMjAuMzgsMCwwLDEtMTQuMjcsNS44LDIwLjg2LDIwLjg2LDAsMCwxLTEyLjcxLTQuMjQsMTkuNDEsMTkuNDEsMCwwLDEtNi43My03LjkxbC03LjE2LDQuMTNhMjcuNTIsMjcuNTIsMCwwLDAsNy40Nyw5LDMwLjcxLDMwLjcxLDAsMCwwLDE4LjIsNi40MWMuNDQsMCwuODksMCwxLjMzLDBhMzAuNywzMC43LDAsMCwwLDE5LjU1LTYuOTVjNi4xNC00Ljg5LDkuMjUtMTEuMTYsOS4yNS0xOC42MnYtMi4yMm0tOS0yMy44NEEyMS45MywyMS45MywwLDAsMSwzODcsMTA1LjQxYTE5LjMsMTkuMywwLDAsMS0xNC42NSw2LjMsMjAuNSwyMC41LDAsMCwxLTE1LjE2LTYuMzEsMjEuMTksMjEuMTksMCwwLDEtNi41MS0xNS4yOCwyMS41MywyMS41MywwLDAsMSw1LjkyLTE1LjQ4LDIwLjQ0LDIwLjQ0LDAsMCwxLDE0LjY3LTcuMDljLjM4LDAsLjc1LDAsMS4xMywwQTE5LjE3LDE5LjE3LDAsMCwxLDM4Nyw3NCwyMS40NCwyMS40NCwwLDAsMSwzOTMuMzEsOTAuMjJaIi8+PHBhdGggY2xhc3M9ImNscy0xIiBkPSJNMjQ1LDQ3LjYyYy01LjI2LTQuNzUtMTEuODgtNy4xNS0xOS42Ny03LjE1SDE5OS42NnY3OS4xNkgyMDl2LTMxaDBWODAuMjloMFY0OWgxNi43MWExOC41NCwxOC41NCwwLDAsMSwxMi4xOCw0LjIzLDEzLjY0LDEzLjY0LDAsMCwxLDUuNTQsMTAuNDksMTMuOTIsMTMuOTIsMCwwLDEtNSwxMi4wNywxNy41NCwxNy41NCwwLDAsMS0xMi4yNSw0LjVoLTkuNjh2OC4zNGgxMS4yQTI1LDI1LDAsMCwwLDI0NS4xOCw4MmEyMS43NCwyMS43NCwwLDAsMCw3LjUtMTYuNDZBMjIuNDEsMjIuNDEsMCwwLDAsMjQ1LDQ3LjYyWiIvPjxwYXRoIGNsYXNzPSJjbHMtMSIgZD0iTTMwNy42NSw1OS4zOEEyNS4yNywyNS4yNywwLDAsMCwyODguOTIsNjdhMjYuMjQsMjYuMjQsMCwwLDAtNy40NSwxOXYzMy40NWg5Vjg1LjI2YTE2LjMzLDE2LjMzLDAsMCwxLDUuMDgtMTIuMDksMTYuNTQsMTYuNTQsMCwwLDEsMTIuMDktNS4wNywxNi4xLDE2LjEsMCwwLDEsMTEuODgsNS4wNywxNi40MiwxNi40MiwwLDAsMSw1LjE4LDEyLjA5djM0LjI2aDlWODYuMDdhMjYuMjQsMjYuMjQsMCwwLDAtNy40NS0xOUEyNS4xNSwyNS4xNSwwLDAsMCwzMDcuNjUsNTkuMzhaIi8+PHJlY3QgY2xhc3M9ImNscy0xIiB3aWR0aD0iMTYwIiBoZWlnaHQ9IjE2MCIgcng9IjE2LjY3Ii8+PHJlY3QgY2xhc3M9ImNscy0yIiB4PSIyMCIgeT0iMjAiIHdpZHRoPSIxMjAiIGhlaWdodD0iMTIwIi8+PHBhdGggY2xhc3M9ImNscy0zIiBkPSJNOTIsNDgsMzkuMjYsMzMuMTRhNS43Miw1LjcyLDAsMCwwLTUsLjkyQTUuNTYsNS41NiwwLDAsMCwzMiwzOC41NHY5NS44NGE1LjU1LDUuNTUsMCwwLDAsMi4xNyw0LjQxQTUuNyw1LjcsMCwwLDAsMzcuNjgsMTQwYTUuNSw1LjUsMCwwLDAsMS4zMy0uMTZsNTIuNzUtMTIuMzloMEE1LjYxLDUuNjEsMCwwLDAsOTYuMTMsMTIyVjUzLjM3QTUuNjYsNS42NiwwLDAsMCw5Miw0OFoiLz48cGF0aCBjbGFzcz0iY2xzLTQiIGQ9Ik0xMjcuOTIsNDQuNTdhNS42NCw1LjY0LDAsMCwwLTQuMTItNS4zOWwtNjcuMTMtMTlhNS43Miw1LjcyLDAsMCwwLTUsLjkxLDUuNTUsNS41NSwwLDAsMC0yLjI2LDQuNDlWOTguMzRhNS41NSw1LjU1LDAsMCwwLDIuMTcsNC40MUE1LjcsNS43LDAsMCwwLDU1LjA5LDEwNGE1LjU4LDUuNTgsMCwwLDAsMS4zNC0uMTZsNjcuMjEtMTZBNS42MSw1LjYxLDAsMCwwLDEyOCw4Mi4zOFoiLz48cGF0aCBjbGFzcz0iY2xzLTUiIGQ9Ik05Miw0OCw0OS40MSwzNlY5OC4zNGE1LjU1LDUuNTUsMCwwLDAsMi4xNyw0LjQxQTUuNyw1LjcsMCwwLDAsNTUuMDksMTA0YTUuNTgsNS41OCwwLDAsMCwxLjM0LS4xNmwzOS43LTkuNDJ2LTQxQTUuNjYsNS42NiwwLDAsMCw5Miw0OFoiLz48L2c+PC9nPjwvc3ZnPg==" style="width:100px" alt="PingCAP logo"/></a></span><span class="index-module--mobileNavBarEnd--wJBf2"><a class="navbar-burger index-module--burger--JHKki" href="https://github.com/pingcap" target="_blank"><i class="iconfont icon-github index-module--icon--HqAmx"/></a><a role="button" class="navbar-burger index-module--burger--JHKki" aria-label="menu" aria-expanded="false"><i class="iconfont icon-menu index-module--icon--HqAmx"/></a></span></div><div class="navbar-menu index-module--desktopMenu--FioJ8"><div class="navbar-start"><div class="navbar-item px-5 index-module--menu--N9jNb"><a>产品</a></div><div class="navbar-item px-5 index-module--menu--N9jNb"><a>解决方案</a></div><div class="navbar-item px-5 index-module--menu--N9jNb"><a>文档</a></div><div class="navbar-item px-5 index-module--menu--N9jNb"><a>客户案例</a></div><div class="navbar-item px-5 index-module--menu--N9jNb"><a>合作伙伴</a></div><div class="navbar-item px-5 index-module--menu--N9jNb"><a>服务与支持</a></div><div class="navbar-item px-5 index-module--menu--N9jNb"><a>学习与社区</a></div></div><div class="navbar-end index-module--navbarEndContent--HqBKU"><div class="index-module--navbarEndItem--rYRre index-module--searchWrapper--JvFXI"><span class="index-module--icon--HqAmx index-module--navItem--ou3c9"><i class="iconfont icon-search"/><span class="index-module--iconLabel--GRGwr">搜索</span></span><div class="index-module--searchPanel--yTlLW"/></div><div class="navbar-item index-module--navItem--ou3c9 index-module--navbarEndItem--rYRre"><div class="dropdown is-center index-module--langSelector--mPyrj"><div class="dropdown-trigger"><div aria-haspopup="true" aria-controls="lang-menu" class="index-module--trigger--8EEgs"><i class="iconfont icon-language index-module--icon--HqAmx"/><span class="index-module--iconLabel--GRGwr">语言</span></div></div><div class="dropdown-menu index-module--menu--QgjOl" id="lang-menu" role="menu"><div class="dropdown-content index-module--dropdownContent--l+MWk"><a target="_blank" rel="noopener" href="https://www.pingcap.com/?from=en" class="dropdown-item index-module--item--Q3TbR">English</a><a target="_blank" rel="noopener" href="https://pingcap.co.jp/" class="dropdown-item index-module--item--Q3TbR">日本語</a></div></div></div></div><div class="navbar-item index-module--navbarEndItem--rYRre"><a class="index-module--icon--HqAmx index-module--navItem--ou3c9" href="https://github.com/pingcap" target="_blank"><i class="iconfont icon-github"/></a></div><div class="navbar-item"><a class="button is-white index-module--useFree--DEvR+" href="/product#SelectProduct">免费试用</a></div></div></div></nav><div class="index-module--layout--pDdDW"><main><div class="blog-header-module--container--EsMbA"><div class="container px-4 blog-header-module--fixedContainer--7ndgE"><div class="blog-detail-module--wrapper--esMKp"><h1 class="blog-detail-module--h1--Q3T+9">TiKV 源码解析系列文章（十七）raftstore 概览</h1><a class="index-module--blogReturnBtn--igugB"><i class="iconfont icon-arrow-right"/><label>返回全部</label></a></div></div></div><div class="blog-detail-module--container--88yCh"><div class="container px-4 "><div class="blog-detail-module--row--BOEUq"><span>李建俊</span><span><a href="/blog/categories/产品技术解读">产品技术解读</a></span><span>2020-01-15</span></div><div class="columns"><div class="column is-8 px-4"><div class="index-module--markdown--RI+u5"><p>TiKV 作为一个分布式 KV 数据库，使用 Raft 算法来提供强一致性。Raft 算法提供了单一 group 的一致性，但是单一 group 无法扩展和均衡。因此，TiKV 采用了 MultiRaft 的方式基于 Raft 算法提供能兼顾一致性、扩展均衡的 KV 储存。下文以 3.0 版本代码为例，讲述 raftstore 源码中的关键定义和设计。</p> <h2 id="multiraft">MultiRaft</h2><p>MultiRaft 顾名思义就是多个 Raft group。数据组织上，TiKV 将数据按范围划分成多个分片，这些分片称之为 region。每个 region 由一个 Raft group 来管理。Raft group 和 region 是一对一的关系。由下方示意图可以看到一个 Raft group 管理的多个副本分别落在不同的机器上，一个机器的数据包含了多个不同 region 的副本。通过这种组织方式，我们让 Raft group 并行起来，从而实现扩展和均衡。</p> <h2 id="batch-system">Batch System</h2><p>Batch System 是 raftstore 处理的基石，是一套用来并发驱动状态机的机制。状态机的核心定义如下：</p> <pre><code><span class="hljs-keyword">pub</span> <span class="hljs-keyword">trait</span> <span class="hljs-title class_">Fsm</span> { <span class="hljs-keyword">type</span> <span class="hljs-title class_">Message</span>: <span class="hljs-built_in">Send</span>; <span class="hljs-keyword">fn</span> <span class="hljs-title function_">is_stopped</span>(&amp;<span class="hljs-keyword">self</span>) <span class="hljs-punctuation">-&gt;</span> <span class="hljs-type">bool</span>; <p>状态机通过 PollHandler 来驱动，其定义如下：</p> <pre><code><span class="hljs-keyword">pub</span> <span class="hljs-keyword">trait</span> <span class="hljs-title class_">PollHandler</span>&lt;N, C&gt; { <span class="hljs-keyword">fn</span> <span class="hljs-title function_">begin</span>(&amp;<span class="hljs-keyword">mut</span> <span class="hljs-keyword">self</span>, batch_size: <span class="hljs-type">usize</span>); <span class="hljs-keyword">fn</span> <span class="hljs-title function_">handle_control</span>(&amp;<span class="hljs-keyword">mut</span> <span class="hljs-keyword">self</span>, control: &amp;<span class="hljs-keyword">mut</span> C) <span class="hljs-punctuation">-&gt;</span> <span class="hljs-type">Option</span>&lt;<span class="hljs-type">usize</span>&gt;; <span class="hljs-keyword">fn</span> <span class="hljs-title function_">handle_normal</span>(&amp;<span class="hljs-keyword">mut</span> <span class="hljs-keyword">self</span>, normal: &amp;<span class="hljs-keyword">mut</span> N) <span class="hljs-punctuation">-&gt;</span> <span class="hljs-type">Option</span>&lt;<span class="hljs-type">usize</span>&gt;; <span class="hljs-keyword">fn</span> <span class="hljs-title function_">end</span>(&amp;<span class="hljs-keyword">mut</span> <span class="hljs-keyword">self</span>, batch: &amp;<span class="hljs-keyword">mut</span> [<span class="hljs-type">Box</span>&lt;N&gt;]); <span class="hljs-keyword">fn</span> <span class="hljs-title function_">pause</span>(&amp;<span class="hljs-keyword">mut</span> <span class="hljs-keyword">self</span>) {} <p>大体来看，状态机分成两种，normal 和 control。对于每一个 Batch System，只有一个 control 状态机，负责管理和处理一些需要全局视野的任务。其他 normal 状态机负责处理其自身相关的任务。每个状态机都有其绑定的消息和消息队列。PollHandler 负责驱动状态机，处理自身队列中的消息。Batch System 的职责就是检测哪些状态机需要驱动，然后调用 PollHandler 去消费消息。消费消息会产生副作用，而这些副作用或要落盘，或要网络交互。PollHandler 在一个批次中可以处理多个 normal 状态机，这些状态机在作为参数传入 end 方法时被命名为 batch，意思就是副作用会被聚合，一批批地处理。具体实现细节，后面的源码阅读章节会提到，这里就先不展开。</p> <h2 id="raftbatchsystem-和-applybatchsystem">RaftBatchSystem 和 ApplyBatchSystem</h2><p>在 raftstore 里，一共有两个 Batch System。分别是 RaftBatchSystem 和 ApplyBatchSystem。RaftBatchSystem 用于驱动 Raft 状态机，包括日志的分发、落盘、状态跃迁等。当日志被提交以后会发往 ApplyBatchSystem 进行处理。ApplyBatchSystem 将日志解析并应用到底层 KV 数据库中，执行回调函数。所有的写操作都遵循着这个流程。当客户端发起一个请求时，RaftBatchSystem 会将其序列化成日志，并提交给 raft。一个简化的流程如下：</p> <p>从代码来看，序列化发生在 propose 过程中。</p> <pre><code>fn propose<span class="hljs-constructor">_normal(&amp;<span class="hljs-params">mut</span> <span class="hljs-params">self</span>, <span class="hljs-params">req</span>: RaftCmdRequest)</span> -&gt; Result&lt;<span class="hljs-literal">()</span>&gt; { <span class="hljs-keyword">let</span> ctx = <span class="hljs-keyword">match</span> self.pre<span class="hljs-constructor">_propose(<span class="hljs-params">poll_ctx</span>, &amp;<span class="hljs-params">mut</span> <span class="hljs-params">req</span>)</span>?; <span class="hljs-keyword">let</span> data = req.write<span class="hljs-constructor">_to_bytes()</span>?; self.raft_group.propose(ctx.<span class="hljs-keyword">to</span><span class="hljs-constructor">_vec()</span>, data)?; <span class="hljs-constructor">Ok(()</span>) <p>而 propose 的副作用后续会通过 Raft 库的 Ready 机制搜集，batch 处理。</p> <pre><code><span class="hljs-keyword">if</span> !self.raft_group.has<span class="hljs-constructor">_ready_since(Some(<span class="hljs-params">self</span>.<span class="hljs-params">last_applying_idx</span>)</span>) { return None; <span class="hljs-keyword">let</span> mut ready = self.raft_group.ready<span class="hljs-constructor">_since(<span class="hljs-params">self</span>.<span class="hljs-params">last_applying_idx</span>)</span>; self.mut<span class="hljs-constructor">_store()</span>.handle<span class="hljs-constructor">_raft_ready(<span class="hljs-params">ctx</span>, &amp;<span class="hljs-params">ready</span>)</span>; <span class="hljs-keyword">if</span> !self.is<span class="hljs-constructor">_applying_snapshot()</span><span class="hljs-operator"> &amp;&amp; </span>!ready.committed_entries.is<span class="hljs-constructor">_empty()</span> { <span class="hljs-keyword">let</span> apply = Apply::<span class="hljs-keyword">new</span>(self.region_id, self.term<span class="hljs-literal">()</span>, mem::replace(&amp;mut ready.committed_entries, vec!<span class="hljs-literal">[]</span>)); apply_router .schedule<span class="hljs-constructor">_task(<span class="hljs-params">self</span>.<span class="hljs-params">region_id</span>, ApplyTask::<span class="hljs-params">apply</span>(<span class="hljs-params">apply</span>)</span>); self.raft_group.advance<span class="hljs-constructor">_append(<span class="hljs-params">ready</span>)</span>; <p>在 PeerStorage 的 handle_raft_ready 方法中，会将收集到 Ready 中的 Raft 日志收集到一个 WriteBatch 中，最终在 RaftPoller 的 end 方法中批量写入磁盘。而 Ready 中收集到的确认过的 Raft 日志，会被 <code>apply_router</code> 发送到 apply 线程中，由 ApplyBatchSystem 来处理。关于一个写入在 Raftstore 模块中从提交到确认的整条链路，将在后续的章节中更详细地探讨，这里就不作展开了。</p> <h2 id="split-和-merge">Split 和 Merge</h2><p>TiKV 的每一个 Raft group 都是一个 Region 的冗余复制集，而 Region 数据不断增减时，它的大小也会不断发生变化，因此必须支持 Region 的分裂和合并，才能确保 TiKV 能够长时间稳定运行。Region Split 会将一段包含大量数据的 range 切割成多个小段，并创建新的 Raft Group 进行管理，如将 [a, z) 切割成 [a, h), [h, x) 和 [x, z)，并产生两个新的 Raft group。Region Merge 则会将 2 个相邻的 Raft group 合并成一个，如 [a, h) 和 [h, x) 合并成 [a, x）。这些逻辑也在 Raftstore 模块中实现。这些特殊管理操作也作为一个特殊的写命令走一遍上节所述的 Raft propose/commit/apply 流程。为了保证 split/merge 前后的写命令不会落在错误的范围，我们给 region 加了一个版本的概念。每 split 一次，版本加一。假设 region A 合并到 region B，则 B 的版本为 max(versionB, versionA + 1) + 1。更多的细节实现包括各种 corner case 的处理在后续文章中展开。</p> <h2 id="localreader">LocalReader</h2><p>对于读操作，如果和写操作混在一起，会带来不必要的延迟和抖动。所以 TiKV 实现了一个单独的组件来处理。Raft group 的 leader 会维护一个 lease 机制，对于在 lease 内收到的请求，会立刻进行读操作；lease 外的请求，会触发 lease 续期。续期是通过心跳完成的。也就是读操作不会触发写盘行为。Lease 定义如下：</p> <pre><code>pub struct <span class="hljs-title class_">RemoteLease</span> <span class="hljs-punctuation">{</span> <span class="hljs-symbol"> expired_time:</span> Arc<span class="hljs-params">&lt;AtomicU64&gt;</span>, <span class="hljs-symbol"> term:</span> u64, pub struct <span class="hljs-title class_">Lease</span> <span class="hljs-punctuation">{</span> <span class="hljs-comment">// A suspect timestamp is in the Either::Left(_),</span> <span class="hljs-comment">// a valid timestamp is in the Either::Right(_).</span> <span class="hljs-symbol"> bound:</span> Option<span class="hljs-params">&lt;Either&lt;Timespec, Timespec&gt;</span>&gt;, <span class="hljs-symbol"> max_lease:</span> Duration, max_drift:</span> Duration, <span class="hljs-symbol"> last_update:</span> Timespec, <span class="hljs-symbol"> remote:</span> Option<span class="hljs-params">&lt;RemoteLease&gt;</span>, <p>RemoteLease 是读行为发生线程里所持有的 lease，它的状态由 Lease 来维护。Lease 自身由 RaftBatchSystem 来实际维护。bound 记录的是 lease 的失效时间，max_drift 表示允许精度误差。</p> <h2 id="coprocessor">Coprocessor</h2><p>虽然读写已经包含了绝大多数 KV 操作，但是我们仍然需要一些特殊机制来自定义行为。比如为了保证事务正确，region 分裂不应该将同一个 key 的 MVCC 数据拆分到不同的 region 里。这些行为由 Coprocessor 来实现。TiKV 中一共有两种 Coprocessor。之前 <a href="https://pingcap.com/blog-cn/tikv-source-code-reading-14/"> </a>介绍的 SQL 下推逻辑属于 Endpoint，这里主要涉及的是 Observer。Observer 的作用是监听 KV 处理过程中的各种事件，并在事件发生时执行自定义逻辑。Coprocessor 的定义如下：</p> <pre><code>pub <span class="hljs-class"><span class="hljs-keyword">trait</span> <span class="hljs-title">Coprocessor</span> </span>{ <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">start</span>(<span class="hljs-params">&amp;<span class="hljs-built_in">self</span></span>) </span>{} <span class="hljs-function"><span class="hljs-keyword">fn</span> <span class="hljs-title">stop</span>(<span class="hljs-params">&amp;<span class="hljs-built_in">self</span></span>) </span>{} <p>目前已经定义的 Coprocessor 包括 AdminObserver、QueryObserver、SplitCheckObserver、RoleObserver、RegionChangeObserver。拿 QueryObserver 举个例子：</p> <pre><code>pub trait QueryObserver: Coprocessor { <span class="hljs-comment">/// Hook to call before proposing write request.</span> <span class="hljs-comment">/// We don't propose read request, hence there is no hook for it yet.</span> fn pre<span class="hljs-constructor">_propose_query(&amp;<span class="hljs-params">self</span>, <span class="hljs-params">_</span>: &amp;<span class="hljs-params">mut</span> ObserverContext&lt;'<span class="hljs-params">_</span>&gt;, <span class="hljs-params">_</span>: &amp;<span class="hljs-params">mut</span> Vec&lt;Request&gt;)</span> -&gt; Result&lt;<span class="hljs-literal">()</span>&gt; { <span class="hljs-constructor">Ok(()</span>) <span class="hljs-comment">/// Hook to call before applying write request.</span> fn pre<span class="hljs-constructor">_apply_query(&amp;<span class="hljs-params">self</span>, <span class="hljs-params">_</span>: &amp;<span class="hljs-params">mut</span> ObserverContext&lt;'<span class="hljs-params">_</span>&gt;, <span class="hljs-params">_</span>: &amp;[Request])</span> {} <span class="hljs-comment">/// Hook to call after applying write request.</span> fn post<span class="hljs-constructor">_apply_query(&amp;<span class="hljs-params">self</span>, <span class="hljs-params">_</span>: &amp;<span class="hljs-params">mut</span> ObserverContext&lt;'<span class="hljs-params">_</span>&gt;, <span class="hljs-params">_</span>: &amp;<span class="hljs-params">mut</span> Vec&lt;Response&gt;)</span> {} <p>此 Observer 监听了 pre_propose、pre_apply 和 post_apply 三个事件。ObserverContext 里面包含了 region 信息以及是否继续处理的标记。具体实现细节由后续文章介绍，这里不展开了。</p> <h2 id="小结">小结</h2><p>这篇文章主要针对 TiKV 项目中 src/raftstore 里源码涉及的概念和原理做了一个大概的介绍。深入解读请留意后续的系列文章，欢迎大家关注。</p>