面向非易失性内存缓冲区的SQLite-CC

胡耀艺 胡卉芪 周烜 周傲英

摘要：近年来非易失性存储（Non Volatile Memory，NVM）飞速发展，它具有的持久化、大容量、低延迟、按字节寻址、高密度和低能耗等优越特性，强烈冲击着目前的数据库系统架构.SQLite是一款轻量级关系型数据库，实现了无服务器、零配置、事务性的SQL数据库引擎.其为每个连接维护一个缓冲区，有着空间开销大和数据一致性检测的问题，同时由于采用了相对简单的串行化单写事务执行方式和按页记录日志等方案，带来了回滚日志模式性能低和写入放大以及WAL模式存储空间要求等问题.为了解决上述挑战，构建了一种新的基于非易失性内存的SQLite缓冲区的方案SQLite-CC（Copy Cache），充分考虑了非易失性内存的硬件特性，引入CC-manager用于维护事务原子性，加入修改页面索引来保证数据库文件与缓存的一致性.实验表明，其达到了和SQLite-WAL模式相当的并发性能，相比于回滚日志模式事务吞吐量提升了3倍，读写延迟降低了40%且有效解决了磁盘中的写放大问题.

关键词：NVM;SQLite;缓冲区优化;日志方案

中图分类号：TP392文献标志码：ADOI：10.3969/j.issn.l000-5641.2021.06.013

SQLite-CC based on non-volatile memory cache

HU Yaoyi，HU Huiqi，ZHOU Xuan，ZHOU Aoying

（School of Data Science and Engineering. East China Normal University，Shanghai 200062，China）

Abstract：In recent years，non-volatile memory （NVM）has developed rapidly. Its advantages，among others，include：persistence，large capacity，low latency，byte addressing，high density，and low energy consumption - all of which have impacted current database system architecture. SQLite is a lightweight relational database widely used in embedded fields such as mobile platforms. It operates as a serverless，zero-configuration，transactional SQL database engine. It maintains a cache for each connection，which results in problems with large space overhead and data consistency detection. At the same time，it adopts a relatively simple serialized single-write transaction execution method and page-based logging，which offers low performance and write amplification in the journal mode and a storage space requirement in the WAL mode during execution. In order to address the above challenges，a new scheme of SQLite Cache based on non­volatile memory，SQLite-CC （Copy Cache），is constructed，which fully considers the hardware characteristics of non-volatile memory and ensures the atomicity of transactions using a CC- manager and by adding an updated page index to ensure the consistency of database files and cache. Benchmarking tests show that it can achieve the same concurrency performance as SQLite- WAL mode. Compared with the rollback mode，it improves the execution performance of transactions by 3 times，reduces latency by 40%，and effectively solves the issue of writeamplification on disks.

Keywords：NVM;SQLite;cache optimization;logging solution

0引言

一直以來，关系型数据库系统都是面向磁盘的，建立了由HDD/SSD和DRAM构成的两层存储架构：高延迟、持久化、大容量的磁盘搭配低延迟、易失性、容量小的内存.然而，近年来随着以Intel's optane DIMM为代表的非易失性内存（Non Volatile Memory，NVM）的出现和飞速发展，凭借其持久化、大容量、低延迟、按字节寻址等优越特性，被认为是具有“革命性的”下一代存储设备.这种新型硬件强烈地冲击了文件系统，缓存……