diff --git a/README.md b/README.md index 754a8b3..2464222 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ Crate types: `rlib`, `staticlib`, `cdylib` (`libkvspace_durable.so`). C ABI exported from the cdylib (`src/ffi.rs`): - lifecycle: `kvspaceConnect`, `kvspaceClose` -- KV read/write (zero-copy borrow model, no free): `kvspaceGet` (borrow), `kvspaceWriteInPlace`, `kvspaceWriteNewPlace` +- KV read/write: `kvspaceGet` (borrow), synchronous `kvspaceSetValue`, `kvspaceWriteInPlace`, `kvspaceWriteNewPlace` - enumerate / delete / copy: `kvspaceListLen`, `kvspaceListAt`, `kvspaceDel`, `kvspaceDelTree`, `kvspaceCp`, `kvspaceCpTree` - directories / extindex: `kvspaceMkindex`, `kvspaceMkindexExt`, `kvspaceRmindexExt` - watch / clear: `kvspaceWatch`, `kvspaceClear` @@ -35,17 +35,14 @@ The same ABI is implemented by `kvspace-c` (`shm://`), so a consumer (e.g. the k ## XValue -kindexpr TLV head, byte-identical to `kvspace-c`: +The headlenpow wire format is byte-identical across this backend, `kvspace-c`, and the `kvspace` frontend. Its 18-byte prefix is `[headlenpow:u8][flags:u8][a:u64 LE][b:u64 LE]`. The UTF-8 langtype occupies the rest of the `2^headlenpow` head, followed by the body. `flags` uses its low two bits for the storage class and bit 2 for a pointer. -``` -[1B kindexprlen][kindexpr + 0x00 pad][1B ro][4B vid LE][4B raw_len LE][raw] -``` - -- kindexpr first byte: `*` = soft link (raw = target path), `@` = ext handle, otherwise inline. -- `[d0,d1]kind` carries ndim+dims; bare `kind` is a scalar; `char/*` is always a 1-D sequence (`[n]`). -- `None` is encoded as NULL / length 0. +- Class 0 stores short fixed values, `None`, maps, structs, and directory markers. `None` has an empty langtype; map members and directory children are enumerated from physical key prefixes. +- Class 1 stores a byte length and reserved capacity (`a`, `b`) for strings and code slots. UTF-8 records the code point count in its langtype and the byte count in `a`. +- Class 2 stores tensor element count and width. Its langtype carries the dimensions. +- Class 3 stores an external locator, with the target's actual langtype. Pointer values use class 1 with the pointer bit set and a target key in the body. -Kinds: `bool`, `int8..int64`, `uint8..uint64`, `float32/64`, `char/utf32|utf8|ascii`, `stringkeymap`, `index`, `extindex`, `rwir`, `rwfunc`, `defrwir`, `scope`, `time`, `duration`. +`ro` and `vid` live at `/.kvspace-meta/` as a separate XValue. Writes and copies preserve the sidecar; its absence means `ro=false, vid=0`. ## Tutorial @@ -53,4 +50,4 @@ Kinds: `bool`, `int8..int64`, `uint8..uint64`, `float32/64`, `char/utf32|utf8|as python3 tutorial/test.py ``` -Fourteen shell cases (`01-basic.sh` … `14-head-perm.sh`) covering link, extindex, unlink, dir/value coexistence, type variety, edge cases, bulk ops, and head permission. The harness also cross-validates that `kvspace-c` and `kvspace-durable` produce byte-identical head encoding (ro/vid). +The shell cases cover links, extensions, directory/value coexistence, type variety, and bulk operations. The codec tests compare the wire format across implementations. diff --git a/docs/strkeymapindex-ndarray.md b/docs/strkeymapindex-ndarray.md index 110e599..21ec5c6 100644 --- a/docs/strkeymapindex-ndarray.md +++ b/docs/strkeymapindex-ndarray.md @@ -1,115 +1,7 @@ -# strkeymapindex：散 key ndarray 的坐标 key 布局 +# Map members and sparse coordinates -> 本文是 `strkeymapindex` 这一 XValue kind 的权威定义：它的语义、它的元素 key 形态、以及各后端与上层（kvlang layout/runtime、json 扩展）必须遵守的契约。 -> 本文取代并作废旧文档《XValue 类型系统与 JSON 全类型 key 布局》中关于散 key 数组 key 形态的全部约定。 +A map's main key stores a class-0 XValue with an empty body. Its langtype is the full `key-type·value-type` expression. For example, `/m` can hold `[int64,int64]·float32`. There is no separate index matrix or stored logical shape. -## 1. 两种 ndarray +Each member is an independent physical key under `/m·`. The member `/m·[1,2]` is a scalar `float32` XValue; its coordinates are in the key, not in the parent's body. Nested maps use the same rule recursively, such as `/m·[1,2]·field`. Directory children use `/` as the separator. Listing derives direct members from physical prefixes, sorts coordinate names numerically, and suppresses duplicate directory names. -kvspace 有且只有两种 ndarray，二者的类型表达式同为 `[d0,d1,...]kind`，`ndim`/`dims` 同在 XValueHead 的 kindexpr 中： - -| | compact ndarray | 散 key ndarray | -|---|---|---| -| kind | 元素 kind（`float32`/`int64`/…） | `strkeymapindex` | -| 存储 | 单 XValue，元素连续打包在 body | 目录标记 XValue + 每元素一个独立 key | -| 元素约束 | 必须定宽（bool/int/uint/float/char） | 任意，可变长、可异构、可嵌套 | -| 变长 | 否，dims 固定 | 是，可增删元素 | -| 元素寻址 | body 内 row-major 偏移 | 元素 key | - -两者的 `xv.numel` / `xv.dim` / `xv.shape` / `xv.at` / `xv.set` 语义完全一致，差别只在存储形态。`array.scatter` / `array.compact` 是两者之间的转换。 - -## 2. 元素 key = 坐标段 - -设 `m` 的 kindexpr 为 `[d0,d1,...,d(n-1)]strkeymapindex`，则 `n = ndim ≥ 1`，其元素 key 为： - -``` -m·[s0,s1,...,s(n-1)] -``` - -- 坐标段整体是**父目录 `m·` 下的一个成员名**，不是多级路径。成员分隔符为 `·`（U+00B7）。 -- 坐标个数**恒等于 ndim**，不足或超出即非法。 -- 坐标是十进制非负整数或小数（如 `12.24`），`,` 分隔，**无空格**，`[` `]` 包围。 - -``` -一维 [3]strkeymapindex m·[0] m·[1] m·[2] -二维 [2,3]strkeymapindex m·[0,0] m·[0,1] … m·[1,2] -三维 [2,2,2]strkeymapindex m·[0,0,0] m·[0,0,1] … -小数一维 m·[12.24] m·[0.5] … -``` - -一维**没有例外**：是 `m·[0]`，不是 `m·0`。 - -## 3. ndim ≥ 1 是硬约束 - -`strkeymapindex` 恒有维度。`ndim == 0` 的 `strkeymapindex` **不合法**，编解码两侧均须拒绝——无维度的字符串键容器是 `objindex`，不是本 kind。 - -由此两个 kind 的分工无重叠： - -| kind | dims | 元素 key | 用途 | -|---|---|---|---| -| `objindex` | 无（ndim=0） | `m·` | 命名成员字典，键是任意字符串 | -| `strkeymapindex` | ndim ≥ 1 | `m·[s0,...]` | 散 key ndarray，键是整数/小数坐标 | - -## 4. key 形态字面自明 - -坐标段以 `[` 开头，而 `objindex` 成员名禁止 `[`（§6），因此： - -``` -m·[0] 数组第 0 元素 -m·0 名为 "0" 的命名成员 -``` - -二者字面可分。反向映射（如 json.to）**不需要**读父节点 kind 才能判断子键是元素还是成员。旧布局中 `m·0` 两义、必须靠父节点 kind 消歧的问题不再存在。 - -## 5. 嵌套：规整走 dims，不规整走元素 - -- **规整多维**一律扁平为单层坐标段：`[2,3]strkeymapindex` 落 6 个 key `m·[0,0]`…`m·[1,2]`，**不产生任何中间目录节点**。 -- **不规整（ragged）**结构靠「元素本身又是 strkeymapindex」表达：`m·[0]` 是一个一维 map 目录，其元素为 `m·[0]·[0]`、`m·[0]·[1]`。 - -扁平化消除了旧布局的中间层记账节点。旧布局中 `m·0·0` 要求 `m·0·` 是一个目录 XValue，而该中间层若无人显式写入，`backend.rs` 会兜底建成 `objindex`——二维数组的中间层 kind 因此是错的。扁平后中间层不存在，该缺陷随之消失。 - -### JSON 数组的映射 - -`json.from` 产生的数组**一律是一维 map**，嵌套 JSON 数组对应「元素本身是 map」，逐层各自一维。不对 JSON 数组做规整性探测、不自动折叠成多维。理由：JSON 数组天然 ragged，规整性探测会让 `[[1,2],[3,4]]` 与 `[[1,2],[3]]` 落成两种结构，往返不稳定。 - -多维扁平 dims 供 kvlang 显式声明的 ndarray（tensor 场景）使用。 - -## 6. 成员名字符约束 - -**objindex 成员名**禁止：`/` `·` `[` `]` `\n` `\r` `\0`、`‥`(U+2025)、`…`(U+2026)、ASCII 控制字符（<0x20）、空串。禁 `[` 保证了 §4 的字面可分。`.` 已放开——小数/含点字符串可作 key。 - -**strkeymapindex 坐标段**必须完全匹配 `\[([0-9]+(\.[0-9]+)?)(,([0-9]+(\.[0-9]+)?))*\]`（非负整数或小数），其余一律非法。 - -违反者由写入方报错拒绝，不做转义、不静默丢弃。 - -## 7. dims 与成员的关系 - -- `dims` 是**逻辑形状**，存于目录标记 XValue 的 kindexpr。 -- 成员名列表（body `[4B count LE][name\n...]`）是**实际存在的元素**。 -- 二者可不一致：坐标可缺席（元素被删、或从未写入），此时该坐标读为 `None`。`count` 是实际成员数，不是 `prod(dims)`。 -- 一维 map 追加元素时 `dims[0]` 随之增长；多维 map 的 dims 由创建者给定。 - -## 8. list 顺序 - -`strkeymapindex` 的 `list` 按坐标 **row-major 数值升序**返回：先比 s0，相等再比 s1，依此类推；数值比较而非字节序（`[10]` 排在 `[2]` 之后）。 - -`objindex` / `index` 不保证顺序。 - -## 9. 各后端 - -| 后端 | 元素落盘 | 坐标段可行性 | -|---|---|---| -| redis | key 为完整路径字符串 | `scan_keys` 已不用 Redis glob，主动规避 `[` `]` 元字符误匹配 | -| fs | `m·` 目录下的文件名 `[0,1]` | Linux 文件名允许 `[` `]` `,` | -| shm (kvspace-c) | ART 树字节串 | 无字符限制 | - -fs 后端当前对 `·` 结尾目录一律返回 `objindex`（`fs/kvspace.rs` `dir_value`），`strkeymapindex` 的 kind 与 dims 在该后端存不住，需修。扁平化后 map 只有一层目录，修复点收敛为一处。 - -## 10. 破坏性 - -不向后兼容，不留别名，不做兼容读取。以下形态一律废弃： - -- `m[0]`（kvlang runtime 旧散 key 形态，无点号） -- `m.0`（json 扩展旧形态、旧文档规定形态，`.` 成员分隔符时代） -- `m.0.0`（旧多维嵌套形态） - -旧形态落盘数据不做迁移。当前唯一合法形态为 `m·[s0,s1,...]`。 +The parent value must exist before writing a map member. A missing coordinate has no key and reads as `None`. Deleting or copying a tree acts on its main key and member prefix together. diff --git a/src/backend.rs b/src/backend.rs index f047357..7d7b713 100644 --- a/src/backend.rs +++ b/src/backend.rs @@ -5,17 +5,12 @@ use std::time::Duration; use crate::kvspace::{KVPair, KVSpace}; use crate::kvspace_common::{ - join_path, mk_index_recursive, sep_path, split_index, strip_dir_suf, validate_ptr, watch_value, + is_descendant, is_frame_operand_key, join_path, sep_path, split_index, strip_dir_suf, + validate_ptr, watch_value, }; use crate::r#const::*; use crate::store::KVStore; -use crate::xvalue::{ - decode_xvalue, decode_xvalue_head, encode_head, is_none, is_ptr, ptr_target, XValue, -}; -use crate::xvalue_index::{ - encode_ext_index_grow, encode_index_grow, grow_cap, matrix_cap, matrix_width, new_ext_index, - new_index, -}; +use crate::xvalue::{decode_xvalue, decode_xvalue_head, is_ptr, ptr_target, XValue, REF_EXT}; pub struct Backend { store: S, @@ -26,6 +21,25 @@ impl Backend { Backend { store } } + fn sync_metadata(&mut self, key: &str, raw: &[u8]) -> Result<(), String> { + let head = decode_xvalue_head(raw); + if !raw.is_empty() && head.headlen == 0 { + return Err(format!("invalid XValue at {key}")); + } + self.set_metadata(key, head.ro, head.vid) + } + + fn metadata_at(&self, key: &str) -> Result<(bool, u32), String> { + let meta = + crate::metadata::key_for(key).ok_or_else(|| format!("reserved metadata key: {key}"))?; + match self.store.get(&meta) { + None => Ok((false, 0)), + Some(data) => { + crate::metadata::decode(&data).ok_or_else(|| format!("invalid metadata at {key}")) + } + } + } + // ── 目录与路径工具 ────────────────────────────────────────────── fn is_dir(path: &str) -> bool { @@ -38,30 +52,6 @@ impl Backend { } } - fn parent_name(path: &str) -> (String, String) { - let clean = if Self::is_dir(path) && path != PATH_SEP { - strip_dir_suf(path) - } else { - path - }; - let (parent, name, _) = split_index(clean); - (parent, name) - } - - /// 确保父目录存在：成员目录（尾 ·）建 index，层级目录（尾 /）递归建。 - fn ensure_parent_dir(&mut self, dir: &str) { - if dir == PATH_SEP { - return; - } - if dir.ends_with(OBJ_SEP) { - if self.store.get(dir).is_none() { - self.store.set(dir, &new_index(&[]).encode()); - } - } else { - mk_index_recursive(self, dir); - } - } - // ── link 解析 ─────────────────────────────────────────────────── fn resolve_path(&self, path: &str) -> String { @@ -85,7 +75,11 @@ impl Backend { let resolved = self.resolve_path(&parent); let mut result = join_path(&resolved, &last); if dir_suf { - result.push_str(DIR_INDEX_SUF); + result.push_str(if path.ends_with(OBJ_SEP) { + OBJ_SEP + } else { + DIR_INDEX_SUF + }); } result } @@ -125,162 +119,48 @@ impl Backend { (path.to_string(), false) } - // ── 子树枚举（沿目录索引递归）──────────────────────────────────── - - /// 列出 `base` 子树下的全部物理 key，等价于原来 `store.scan_keys(base)` 的结果集。 - /// - /// **为什么不用 `scan_keys`**：那是让后端做「前缀扫描」，而 KVSpace 要的是 - /// **词边界**匹配（`k[base.len()..]` 以 `/` 或 `·` 开头）。S3 的 `ListObjects` - /// 只能做裸字节前缀，`/proto` 会把 `/protocol` 一并吐出来，还得在客户端再筛一遍。 - /// 而 KVSpace 本来就在每个目录 key 上维护着子项列表——直接用索引递归， - /// 既不必依赖后端有没有前缀扫描能力，也不会多拿无关数据。 - /// - /// **前提：目录索引必须完整。** 任何绕过 `add_child` 的写入都会在这里漏掉。 - /// 这条由 cp / map_coord 两套语义测试在 redis 与 s3 上共同守着。 - /// - /// 返回顺序与原来不同（原来是后端的字典序），调用方都不依赖顺序。 - fn collect_subtree(&self, base: &str) -> Vec { - let mut out = Vec::new(); - self.collect_into(base, &mut out); - out - } - - fn collect_into(&self, key: &str, out: &mut Vec) { - if self.store.get(key).is_some() { - out.push(key.to_string()); - } - // 一个节点最多带两个索引：成员索引（尾 `·`）与层级索引（尾 `/`）。 - // 根是特例 —— `split_index` 对根不追加 `/`，所以根的层级索引就是 `/` 自身。 - let idxes: Vec = if key == PATH_SEP { - vec![PATH_SEP.to_string()] - } else { - vec![ - format!("{}{}", key, OBJ_SEP), - format!("{}{}", key, DIR_INDEX_SUF), - ] - }; - for idx in idxes { - let names = self.index_names(&idx); - if names.is_empty() { - continue; - } - if self.store.get(&idx).is_some() { - out.push(idx.clone()); - } - for n in names { - self.collect_into(&format!("{}{}", idx, n), out); - } - } - } - - /// 读索引的子项名。**与 `read_dir_index` 的区别：不是索引就返回空，不 panic。** - /// 递归时必然会碰到普通叶子值（`/proto` 自己可能就是 int64），不能炸。 - fn index_names(&self, idx: &str) -> Vec { - match self.store.get(idx) { - None => Vec::new(), - Some(data) => match decode_xvalue(&data) { - XValue::Index(c) => normalize_children(c), - XValue::ExtIndex(e) => e.childs, - _ => Vec::new(), - }, - } - } - // ── 目录 index 读写 ───────────────────────────────────────────── fn read_dir_index(&self, dir: &str) -> Vec { - match self.store.get(dir) { - None => Vec::new(), - Some(data) => { - let v = decode_xvalue(&data); - if is_none(&v) { - return Vec::new(); - } - match v { - XValue::Index(c) => normalize_children(c), - XValue::ExtIndex(e) => e.childs, - other => panic!("read_dir_index: unexpected kind {}", other.kind()), - } - } - } - } - - fn add_child(&self, parent: &str, name: &str) { - match self.store.get(parent) { - None => { - if parent.ends_with(OBJ_SEP) { - let v = new_index_for_member(name); - self.store.set(parent, &v.encode()); - } else { - let v = new_index(&[name.to_string()]); - self.store.set(parent, &v.encode()); - } + let mut names = std::collections::HashMap::::new(); + let scan_prefix = dir + .strip_suffix('/') + .or_else(|| dir.strip_suffix('·')) + .unwrap_or(dir); + for key in self.store.scan_keys(scan_prefix) { + let Some(rest) = key.strip_prefix(dir) else { + continue; + }; + if rest.is_empty() { + continue; } - Some(data) => { - let dims = decode_xvalue_head(&data).dims(); - let old_cap = matrix_cap(&dims); - let old_m = matrix_width(&dims); - let v = decode_xvalue(&data); - match v { - XValue::Index(nodes) => { - let mut nodes = normalize_children(nodes); - if nodes.iter().any(|n| n == name) { - return; - } - nodes.push(name.to_string()); - let cap = grow_cap(old_cap, nodes.len()); - let (d, b) = encode_index_grow(&nodes, cap, old_m); - self.store.set(parent, &encode_head(KIND_INDEX, 0, &d, &b)); - } - XValue::ExtIndex(e) => { - if e.childs.iter().any(|c| c == name) { - return; - } - let mut childs = e.childs.clone(); - childs.push(name.to_string()); - let cap = grow_cap(old_cap, childs.len()); - let (d, b) = encode_ext_index_grow(&e.ext_path, &childs, cap, old_m); - self.store - .set(parent, &encode_head(KIND_EXT_INDEX, 0, &d, &b)); - } - other => panic!("add_child: unexpected kind {}", other.kind()), + let end = rest.find(['/', '·']).unwrap_or(rest.len()); + if end > 0 { + let name = &rest[..end]; + if dir != PATH_SEP || name != META_ROOT_NAME { + let direct_dir = + rest.as_bytes().get(end) == Some(&b'/') && rest.len() == end + 1; + names + .entry(name.to_string()) + .and_modify(|v| *v |= direct_dir) + .or_insert(direct_dir); } } } - } - - fn remove_child(&self, parent: &str, names: &[String]) { - let is_removed = |n: &str| { - names - .iter() - .any(|name| n == name || n == format!("{}{}", name, DIR_INDEX_SUF)) - }; - match self.store.get(parent) { - None => {} - Some(data) => { - let dims = decode_xvalue_head(&data).dims(); - let old_cap = matrix_cap(&dims); // 删除不缩 cap - let old_m = matrix_width(&dims); - let v = decode_xvalue(&data); - match v { - XValue::Index(nodes) => { - let nodes = normalize_children(nodes); - let filtered: Vec = - nodes.into_iter().filter(|n| !is_removed(n)).collect(); - let (d, b) = encode_index_grow(&filtered, old_cap, old_m); - self.store.set(parent, &encode_head(KIND_INDEX, 0, &d, &b)); - } - XValue::ExtIndex(e) => { - let filtered: Vec = - e.childs.into_iter().filter(|n| !is_removed(n)).collect(); - let (d, b) = encode_ext_index_grow(&e.ext_path, &filtered, old_cap, old_m); - self.store - .set(parent, &encode_head(KIND_EXT_INDEX, 0, &d, &b)); + let mut result: Vec = names + .into_iter() + .map( + |(name, direct_dir)| { + if direct_dir { + format!("{name}/") + } else { + name } - other => panic!("remove_child: unexpected kind {}", other.kind()), - } - } - } + }, + ) + .collect(); + result.sort_by(|a, b| crate::coord::cmp_coord(a, b)); + result } // ── Get 内部 ──────────────────────────────────────────────────── @@ -295,49 +175,27 @@ impl Backend { fn prefix_ext(&self, prefix: &str) -> String { if let Some(data) = self.store.get(prefix) { let head = decode_xvalue_head(&data); - if head.kind() == KIND_EXT_INDEX { - let body = head.body(&data); - return crate::xvalue_index::decode_ext_index(body, &head.dims()).ext_path; + if head.r#ref == REF_EXT { + return String::from_utf8_lossy(head.body(&data)).into_owned(); } } String::new() } +} - /// listlen/listat O(1) 快路径的取值口：仅纯 index memindex 返 (dims=[N,M], body=N×M 矩阵)。 - /// ext_index（body 头部含 ext_path）与非目录/非 index 返 None，交回退全量 list()。 - fn index_head_body(&mut self, prefix: &str, resolve: bool) -> Option<(Vec, Vec)> { - let resolved = if resolve { - self.resolve_path(prefix) - } else { - prefix.to_string() - }; - if !Self::is_dir(&resolved) { - return None; - } - let data = self.store.get(&resolved)?; - let head = decode_xvalue_head(&data); - if head.kind() == KIND_INDEX { - Some((head.dims(), head.body(&data).to_vec())) +impl KVSpace for Backend { + fn set_metadata(&mut self, key: &str, ro: bool, vid: u32) -> Result<(), String> { + let meta = + crate::metadata::key_for(key).ok_or_else(|| format!("reserved metadata key: {key}"))?; + if ro || vid != 0 { + let value = crate::metadata::encode(ro, vid) + .ok_or_else(|| format!("cannot encode metadata at {key}"))?; + self.store.set(&meta, &value); } else { - None + self.store.del(&[&meta]); } + Ok(()) } -} - -/// 成员目录（memindex，`·` 结尾）新建时注册首个成员；成员顺序/kind 由容器值 object/stringkeymap 决定。 -fn new_index_for_member(name: &str) -> XValue { - new_index(&[name.to_string()]) -} - -fn normalize_children(children: Vec) -> Vec { - if children.len() == 1 && children[0].is_empty() { - Vec::new() - } else { - children - } -} - -impl KVSpace for Backend { fn get(&mut self, prefix: &str, keys: &[String], resolve: bool) -> Vec { Self::assert_dir(prefix); let prefix = if resolve { @@ -349,6 +207,10 @@ impl KVSpace for Backend { let mut full_keys: Vec<(usize, String)> = Vec::new(); for (i, k) in keys.iter().enumerate() { let full = join_path(&prefix, k); + if crate::metadata::is_reserved(&full) { + results[i] = Some(XValue::None); + continue; + } if Self::is_dir(&full) { results[i] = Some(self.get_dir(&full)); } else { @@ -398,12 +260,18 @@ impl KVSpace for Backend { } fn get_raw(&mut self, key: &str) -> Vec { + if crate::metadata::is_reserved(key) { + return Vec::new(); + } let (mut p, l) = sep_path(key); if p != PATH_SEP { p.push_str(DIR_INDEX_SUF); } let p = self.resolve_path(&p); let full = join_path(&p, &l); + if crate::metadata::is_reserved(&full) { + return Vec::new(); + } if let Some(data) = self.store.get(&full) { return data; } @@ -417,13 +285,24 @@ impl KVSpace for Backend { Vec::new() } + fn get_metadata(&mut self, key: &str) -> Result<(bool, u32), String> { + let resolved = self.resolve_path(key); + self.metadata_at(&resolved) + } + fn get_part(&mut self, key: &str, off: u32, len: u32) -> Vec { + if crate::metadata::is_reserved(key) { + return Vec::new(); + } let (mut p, l) = sep_path(key); if p != PATH_SEP { p.push_str(DIR_INDEX_SUF); } let p = self.resolve_path(&p); let full = join_path(&p, &l); + if crate::metadata::is_reserved(&full) { + return Vec::new(); + } if self.store.exists(&full) { return self.store.get_part(&full, off, len).unwrap_or_default(); } @@ -438,12 +317,18 @@ impl KVSpace for Backend { } fn set_part(&mut self, key: &str, off: u32, buf: &[u8]) -> Result<(), String> { + if crate::metadata::is_reserved(key) { + return Err(format!("reserved metadata key: {key}")); + } let (mut p, l) = sep_path(key); if p != PATH_SEP { p.push_str(DIR_INDEX_SUF); } let p = self.resolve_path(&p); let full = join_path(&p, &l); + if crate::metadata::is_reserved(&full) { + return Err(format!("reserved metadata key: {full}")); + } if self.store.exists(&full) { self.store.set_part(&full, off, buf); return Ok(()); @@ -460,183 +345,52 @@ impl KVSpace for Backend { } fn set(&mut self, pairs: &[KVPair]) -> Result<(), String> { - let mut children: Vec<(String, String)> = Vec::new(); - - for p in pairs { - // 只解析父路径，整键不穿透：写 Ptr 变量/帧槽时写的是指针本体（槽本身）， - // 解引用由 runtime 显式 `*` 掌控（对齐 kvspace-c WriteNewPlace/Del）。 - let resolved = self.resolve_parent(&p.key); - if resolved.contains("//") { - return Err(format!("Set: double-slash in key {:?}", resolved)); + for pair in pairs { + if crate::metadata::is_reserved(&pair.key) { + return Err(format!("reserved metadata key: {}", pair.key)); } - match &p.val { - XValue::Index(_) | XValue::ExtIndex(_) => { - if !Self::is_dir(&resolved) { - return Err(format!( - "Set: directory-kind value at non-directory key {:?}", - resolved - )); - } - } - _ => {} + let key = self.resolve_parent(&pair.key); + if key.contains("//") || crate::metadata::is_reserved(&key) { + return Err(format!("invalid key: {key}")); } - if let XValue::Ptr(ptr) = &p.val { - validate_ptr(self, &ptr.target, &ptr.target_kindexpr)?; + let raw = pair.raw.clone().unwrap_or_else(|| pair.val.encode()); + let head = crate::headlenpow::decode(&raw) + .ok_or_else(|| format!("invalid XValue at {key}"))?; + if head.total != raw.len() { + return Err(format!("invalid XValue at {key}")); } - - // 容器值（object/stringkeymap）：值存 p（无后缀），memindex 存 p·（空 index，成员后续写入维护）。 - if let XValue::Map(_) = &p.val { - let base = if resolved == PATH_SEP { - resolved.clone() - } else { - strip_dir_suf(&resolved).to_string() - }; - let bytes = p.raw.clone().unwrap_or_else(|| p.val.encode()); - let mem = format!("{}{}", base, OBJ_SEP); - self.store.set(&base, &bytes); - self.store.set(&mem, &new_index(&[]).encode()); - let (parent, name) = Self::parent_name(&base); - // 成员目录与成员名都要注册进各自 index：redis 后端的目录成员是显式 index - // XValue，不像 fs 靠真实目录自动可见，故这趟链必须走完（只去掉「自动建容器」）。 - let mut dir = parent.clone(); - let mut child = name.clone(); - loop { - self.ensure_parent_dir(&dir); - children.push((dir.clone(), child.clone())); - let (dp, dn) = Self::parent_name(&dir); - if dp.ends_with(OBJ_SEP) { - dir = dp; - child = dn; - continue; - } - children.push((dp, dn)); - break; - } - continue; - } - - if Self::is_dir(&resolved) { - let (parent, name) = Self::parent_name(&resolved); - self.ensure_parent_dir(&parent); - let bytes = p.raw.clone().unwrap_or_else(|| p.val.encode()); - self.store.set(&resolved, &bytes); - // 成员目录（尾 ·）注册裸 name（memindex 与容器值同名）；层级目录（尾 /）注册 name/。 - let child = if resolved.ends_with(OBJ_SEP) { - name - } else { - format!("{}{}", name, DIR_INDEX_SUF) - }; - children.push((parent, child)); - continue; + if let XValue::Ptr(ptr) = decode_xvalue(&raw) { + validate_ptr(self, &ptr.target, &ptr.target_kindexpr)?; } - - let (parent, name, _) = split_index(&resolved); + let (parent, _, _) = split_index(&key); if parent.ends_with(OBJ_SEP) { - // 写成员前 memhead 必须已存在：容器值不在即**拒绝**，绝不兜底自动建。 - // 「memhead 不存在则禁止写 memitem」是 kvspace 层的拦截（与 runtime 的 - // kvlangBuiltinCheckMemhead 同一规则的两侧），自动建会把拼错的名字悄悄变成一个 - // 新容器，且两个后端行为分叉（fs 无此兜底）。 - // 同 fs：`/lib` 下的 `·` 是包·函数命名分隔符，不是 memindex 标记，豁免。 - let base = strip_dir_suf(&parent).to_string(); - if !base.starts_with("/lib") && self.store.get(&base).is_none() { + let base = strip_dir_suf(&parent); + if !base.starts_with("/lib") && self.store.get(base).is_none() { return Err(format!( "{}: memhead {} does not exist — declare the container first", ERR_MEMHEAD_MISSING, base )); } - // 成员目录与成员名都要注册进各自 index：redis 后端的目录成员是显式 index - // XValue，不像 fs 靠真实目录自动可见，故这趟链必须走完（只去掉「自动建容器」）。 - let mut dir = parent.clone(); - let mut child = name.clone(); - loop { - self.ensure_parent_dir(&dir); - children.push((dir.clone(), child.clone())); - let (dp, dn) = Self::parent_name(&dir); - if dp.ends_with(OBJ_SEP) { - dir = dp; - child = dn; - continue; - } - children.push((dp, dn)); - break; - } - } else { - mk_index_recursive(self, &parent); } - - // extindex 写保护：只读扩展层上的同名节点禁止写入。 - if let Some(data) = self.store.get(&parent) { - let head = decode_xvalue_head(&data); - if head.kind() == KIND_EXT_INDEX { - let body = head.body(&data); - let ext_t = crate::xvalue_index::decode_ext_index(body, &head.dims()).ext_path; - let local_nodes = self.read_dir_index(&parent); - let local_exists = local_nodes.iter().any(|n| n == &name); - if !local_exists { - let ext_nodes = self.read_dir_index(&ext_t); - if ext_nodes.iter().any(|n| n == &name) { - return Err(format!("{}: {}", ERR_EXT_WRITE, resolved)); - } - } + let ext_target = self.prefix_ext(&parent); + if !ext_target.is_empty() && self.store.get(&key).is_none() { + let suffix = key.strip_prefix(&parent).unwrap_or(""); + if self.store.get(&format!("{ext_target}{suffix}")).is_some() + && !(head.flags & 4 != 0 && is_frame_operand_key(&key)) + { + return Err(format!("{}: {}", ERR_EXT_WRITE, key)); } } - - let bytes = p.raw.clone().unwrap_or_else(|| p.val.encode()); - self.store.set(&resolved, &bytes); - children.push((parent, name)); - } - - // 按 parent 分组，去重合并 children 进父目录 index。 - let mut parent_children: std::collections::HashMap> = - std::collections::HashMap::new(); - for (parent, name) in children { - parent_children.entry(parent).or_default().push(name); - } - for (parent, names) in parent_children { - let mut nodes: Vec = Vec::new(); - let mut ext_path = String::new(); - let mut is_ext = false; - let mut old_cap = 0usize; - let mut old_m = 0usize; - - if let Some(data) = self.store.get(&parent) { - let dims = decode_xvalue_head(&data).dims(); - old_cap = matrix_cap(&dims); - old_m = matrix_width(&dims); - let v = decode_xvalue(&data); - match v { - XValue::Index(c) => nodes = normalize_children(c), - XValue::ExtIndex(e) => { - nodes = e.childs; - ext_path = e.ext_path; - is_ext = true; - } - other => panic!("Set parentChildren: unexpected kind {}", other.kind()), - } - } - - let mut seen: std::collections::HashSet = nodes.iter().cloned().collect(); - for n in &names { - if seen.insert(n.clone()) { - nodes.push(n.clone()); - } - } - - let cap = grow_cap(old_cap, nodes.len()); - if is_ext { - let (d, b) = encode_ext_index_grow(&ext_path, &nodes, cap, old_m); - self.store - .set(&parent, &encode_head(KIND_EXT_INDEX, 0, &d, &b)); - } else { - let (d, b) = encode_index_grow(&nodes, cap, old_m); - self.store.set(&parent, &encode_head(KIND_INDEX, 0, &d, &b)); - } + self.sync_metadata(&key, &raw)?; + self.store.set(&key, &raw); } - Ok(()) } fn list(&mut self, prefix: &str, expand_ext: bool, resolve: bool) -> Vec { + if crate::metadata::is_reserved(prefix) { + return Vec::new(); + } Self::assert_dir(prefix); let resolved = if resolve { self.resolve_path(prefix) @@ -646,8 +400,10 @@ impl KVSpace for Backend { if !Self::is_dir(&resolved) { return Vec::new(); } + if crate::metadata::is_reserved(&resolved) { + return Vec::new(); + } - // map 成员在 add_child 时已按坐标 row-major 有序存储，list/listat 一律信任存储序，无读时排序。 let members = self.read_dir_index(&resolved); let mut ext_members: Vec = Vec::new(); @@ -670,20 +426,14 @@ impl KVSpace for Backend { } result.push(m); } + result.sort_by(|a, b| crate::coord::cmp_coord(a, b)); result } - /// O(1) 覆写：无 ext 展开的 index memindex 直接读 head dims[0]=N；ext/其余回退全量。 fn list_len(&mut self, prefix: &str, expand_ext: bool, resolve: bool) -> i32 { - if !expand_ext { - if let Some((dims, _)) = self.index_head_body(prefix, resolve) { - return crate::xvalue_index::matrix_count(&dims) as i32; - } - } self.list(prefix, expand_ext, resolve).len() as i32 } - /// O(1) 覆写：无 ext 展开的 index memindex 取矩阵第 idx 行；ext/其余回退全量。 fn list_at( &mut self, prefix: &str, @@ -694,18 +444,6 @@ impl KVSpace for Backend { if idx < 0 { return None; } - if !expand_ext { - if let Some((dims, body)) = self.index_head_body(prefix, resolve) { - if idx as usize >= crate::xvalue_index::matrix_count(&dims) { - return None; - } - return crate::xvalue_index::matrix_at( - &body, - crate::xvalue_index::matrix_width(&dims), - idx as usize, - ); - } - } self.list(prefix, expand_ext, resolve) .into_iter() .nth(idx as usize) @@ -713,38 +451,39 @@ impl KVSpace for Backend { fn del(&mut self, keys: &[String]) -> Result<(), String> { for key in keys { + if crate::metadata::is_reserved(key) { + return Err(format!("reserved metadata key: {key}")); + } let resolved = self.resolve_parent(key); - let (parent, name) = Self::parent_name(&resolved); - - // extindex 删除保护：只读扩展层上的同名节点禁止删除。 - if let Some(data) = self.store.get(&parent) { - let head = decode_xvalue_head(&data); - if head.kind() == KIND_EXT_INDEX { - let body = head.body(&data); - let ext_t = crate::xvalue_index::decode_ext_index(body, &head.dims()).ext_path; - let local_nodes = self.read_dir_index(&parent); - let local_exists = local_nodes.iter().any(|n| n == &name); - if !local_exists { - let ext_nodes = self.read_dir_index(&ext_t); - if ext_nodes.iter().any(|n| n == &name) { - return Err(format!("{}: {}", ERR_EXT_DEL, resolved)); - } - } + if crate::metadata::is_reserved(&resolved) { + return Err(format!("reserved metadata key: {resolved}")); + } + let (parent, _, _) = split_index(&resolved); + let ext_target = self.prefix_ext(&parent); + if !ext_target.is_empty() && self.store.get(&resolved).is_none() { + let suffix = resolved.strip_prefix(&parent).unwrap_or(""); + if self.store.get(&format!("{ext_target}{suffix}")).is_some() { + return Err(format!("{}: {}", ERR_EXT_DEL, resolved)); } } if Self::is_dir(&resolved) { let link_key = strip_dir_suf(&resolved); self.store.del(&[link_key, &resolved]); + self.sync_metadata(link_key, &[])?; + self.sync_metadata(&resolved, &[])?; } else { self.store.del(&[&resolved]); + self.sync_metadata(&resolved, &[])?; } - self.remove_child(&parent, &[name]); } Ok(()) } fn del_tree(&mut self, prefix: &str) -> Result<(), String> { + if crate::metadata::is_reserved(prefix) { + return Err(format!("reserved metadata key: {prefix}")); + } let mut link_key = prefix; if Self::is_dir(link_key) && link_key != PATH_SEP { link_key = strip_dir_suf(prefix); @@ -757,44 +496,60 @@ impl KVSpace for Backend { } let resolved = self.resolve_path(prefix); - // 枚举前缀用去尾斜杠/点的形式：索引里的子项名是相对当前节点的裸名， - // 带上尾斜杠会变成 `/proto/` + `/` 那种重复层级。见 `collect_subtree`。 + if crate::metadata::is_reserved(&resolved) { + return Err(format!("reserved metadata key: {resolved}")); + } let mut scan = resolved.clone(); if Self::is_dir(&scan) && scan != PATH_SEP { scan.pop(); } - let keys = self.collect_subtree(&scan); + let keys: Vec<_> = self + .store + .scan_keys(&scan) + .into_iter() + .filter(|key| !crate::metadata::is_reserved(key)) + .collect(); self.store.del(&[&resolved]); + self.sync_metadata(&resolved, &[])?; for k in &keys { self.store.del(&[k]); + self.sync_metadata(k, &[])?; } - let (parent, name) = Self::parent_name(&resolved); - let names = vec![name.clone(), format!("{}{}", name, OBJ_SEP)]; - self.remove_child(&parent, &names); Ok(()) } /// 单 key 拷贝：src 处 XValue（head+body 原样）写到 dst，并注册进 dst 父 index；不触碰 src·/成员。 fn cp(&mut self, src: &str, dst: &str) -> Result<(), String> { + if crate::metadata::is_reserved(src) || crate::metadata::is_reserved(dst) { + return Err("reserved metadata key".into()); + } let raw = self.get_raw(src); if raw.is_empty() { return Err(format!("Cp: source not found: {}", src)); } + let (ro, vid) = self.metadata_at(src)?; let v = decode_xvalue(&raw); self.set(&[KVPair { key: dst.to_string(), val: v, raw: Some(raw), - }]) + }])?; + self.set_metadata(dst, ro, vid) } /// 递归子树拷贝：以 src 为根，把整棵物理子树（base + 所有 ·/ 后代 key）字节级重映射到 dst。 /// extindex 成员的 marker（含 ext_path）原样复制 → 在 dst 侧生成指向同一只读扩展的新 extindex。 fn cp_tree(&mut self, src: &str, dst: &str) -> Result<(), String> { + if crate::metadata::is_reserved(src) || crate::metadata::is_reserved(dst) { + return Err("reserved metadata key".into()); + } let src_res = self.resolve_path(src); let dst_res = self.resolve_path(dst); + if crate::metadata::is_reserved(&src_res) || crate::metadata::is_reserved(&dst_res) { + return Err("reserved metadata key".into()); + } let mut src_scan = src_res.clone(); if Self::is_dir(&src_scan) && src_scan != PATH_SEP { src_scan.pop(); @@ -806,81 +561,80 @@ impl KVSpace for Backend { if src_scan == dst_base { return Ok(()); } - let keys = self.collect_subtree(&src_scan); + if is_descendant(&src_scan, &dst_base) { + return Err("CpTree: destination is inside source".into()); + } + let keys: Vec<_> = self + .store + .scan_keys(&src_scan) + .into_iter() + .filter(|key| !crate::metadata::is_reserved(key)) + .collect(); if keys.is_empty() { return Err(format!("CpTree: source not found: {}", src)); } - // 覆盖语义确定：先清 dst 既有子树（也从父 index 摘除，随后重新登记）。 + // Replace the destination subtree. let _ = self.del_tree(&dst_base); for k in &keys { let suffix = &k[src_scan.len()..]; let new_key = format!("{}{}", dst_base, suffix); if let Some(data) = self.store.get(k) { + let (ro, vid) = self.metadata_at(k)?; self.store.set(&new_key, &data); + self.set_metadata(&new_key, ro, vid)?; } } - // 按 dst 根物理形态登记进父 index（base 值 / 层级目录 / 成员目录）。 - let (parent, name) = Self::parent_name(&dst_base); - self.ensure_parent_dir(&parent); - if self.store.get(&dst_base).is_some() { - self.add_child(&parent, &name); - } else if self - .store - .get(&format!("{}{}", dst_base, DIR_INDEX_SUF)) - .is_some() - { - self.add_child(&parent, &format!("{}{}", name, DIR_INDEX_SUF)); - } else if self - .store - .get(&format!("{}{}", dst_base, OBJ_SEP)) - .is_some() - { - self.add_child(&parent, &format!("{}{}", name, OBJ_SEP)); - } Ok(()) } /// 浅拷贝：base 值 + 一层 · 成员（不递归成员子树、不遍历 / 子节点）。用于单 struct/扁平容器。 fn cp_list(&mut self, src: &str, dst: &str) -> Result<(), String> { + if crate::metadata::is_reserved(src) || crate::metadata::is_reserved(dst) { + return Err("reserved metadata key".into()); + } let mut src_base = self.resolve_path(src); if Self::is_dir(&src_base) && src_base != PATH_SEP { src_base.pop(); } let mut dst_base = self.resolve_path(dst); + if crate::metadata::is_reserved(&src_base) || crate::metadata::is_reserved(&dst_base) { + return Err("reserved metadata key".into()); + } if Self::is_dir(&dst_base) && dst_base != PATH_SEP { dst_base.pop(); } if src_base == dst_base { return Ok(()); } - let keys = self.collect_subtree(&src_base); + let keys: Vec<_> = self + .store + .scan_keys(&src_base) + .into_iter() + .filter(|key| !crate::metadata::is_reserved(key)) + .collect(); if keys.is_empty() { return Err(format!("CpList: source not found: {}", src)); } - let dst_mem = format!("{}{}", dst_base, OBJ_SEP); let _ = self.del_tree(&dst_base); for k in &keys { let suffix = &k[src_base.len()..]; - // 一层：base 自身、memindex 标记、无更深分隔的直接 · 成员；跳过 / 子节点与更深后代。 + // Copy the base and direct members, including ext directories. let one_level = suffix.is_empty() || (suffix.starts_with(OBJ_SEP) && { let rest = &suffix[OBJ_SEP.len()..]; - !rest.contains(OBJ_SEP) && !rest.contains(PATH_SEP) + let member = rest.strip_suffix(PATH_SEP).unwrap_or(rest); + !member.is_empty() && !member.contains(OBJ_SEP) && !member.contains(PATH_SEP) }); if !one_level { continue; } if let Some(data) = self.store.get(k) { - self.store.set(&format!("{}{}", dst_base, suffix), &data); + let new_key = format!("{}{}", dst_base, suffix); + let (ro, vid) = self.metadata_at(k)?; + self.store.set(&new_key, &data); + self.set_metadata(&new_key, ro, vid)?; } } - let (parent, name) = Self::parent_name(&dst_base); - self.ensure_parent_dir(&parent); - if self.store.get(&dst_base).is_some() { - self.add_child(&parent, &name); - } else if self.store.get(&dst_mem).is_some() { - self.add_child(&parent, &format!("{}{}", name, OBJ_SEP)); - } Ok(()) } @@ -892,33 +646,12 @@ impl KVSpace for Backend { if !Self::is_dir(path) { return Err(format!("{}: Mkindex {}", ERR_DIR_MUST_END_WITH_SLASH, path)); } + let _ = capacity; let resolved = self.resolve_path(path); - - let trimmed = resolved.trim_matches('/'); - let parts: Vec<&str> = if trimmed.is_empty() { - Vec::new() - } else { - trimmed.split('/').collect() - }; - let mut cur = PATH_SEP.to_string(); - for p in parts { - cur = format!("{}{}", join_path(&cur, p), DIR_INDEX_SUF); - if self.read_dir_index(&cur).is_empty() { - let (parent, name) = Self::parent_name(&cur); - self.add_child(&parent, &format!("{}{}", name, DIR_INDEX_SUF)); - } - } - // 叶目录预留容量：存 [0,cap,0]，首成员插入时 grow_cap(cap,1)=cap 物化 cap×M，不重分配。 - if capacity > 0 && cur != PATH_SEP { - let nodes = self.read_dir_index(&cur); - let old_m = self - .store - .get(&cur) - .map(|d| matrix_width(&decode_xvalue_head(&d).dims())) - .unwrap_or(0); - let cap = grow_cap(capacity as usize, nodes.len()); - let (d, b) = encode_index_grow(&nodes, cap, old_m); - self.store.set(&cur, &encode_head(KIND_INDEX, 0, &d, &b)); + let value = crate::headlenpow::encode(5, 0, 0, 0, "lib", &[], 0) + .ok_or_else(|| format!("Mkindex: invalid directory {resolved}"))?; + if self.store.get(&resolved).is_none() { + self.store.set(&resolved, &value); } Ok(()) } @@ -930,20 +663,31 @@ impl KVSpace for Backend { ERR_DIR_MUST_END_WITH_SLASH, path, ext_path )); } - if let Some(data) = self.store.get(ext_path) { - let head = decode_xvalue_head(&data); - if head.kind() == KIND_EXT_INDEX { - return Err(format!("{}: {}", ERR_EXT_CASCADE, ext_path)); - } + if !self.prefix_ext(ext_path).is_empty() { + return Err(format!("{}: {}", ERR_EXT_CASCADE, ext_path)); } let resolved = self.resolve_parent(path); - let (parent, name) = Self::parent_name(&resolved); - self.ensure_parent_dir(&parent); - - let v = new_ext_index(&[], ext_path); - self.store.set(&resolved, &v.encode()); - self.add_child(&parent, &format!("{}{}", name, DIR_INDEX_SUF)); + let source = self + .store + .get(ext_path) + .ok_or_else(|| format!("ExtIndex target missing: {ext_path}"))?; + let langtype = decode_xvalue_head(&source).langtype; + let pow = (5..=31) + .find(|&p| 18 + langtype.len() <= 1usize << p) + .ok_or_else(|| format!("ExtIndex type too long: {langtype}"))?; + let len = ext_path.len() as u64; + let value = crate::headlenpow::encode( + pow, + 3, + len, + len, + &langtype, + ext_path.as_bytes(), + ext_path.len(), + ) + .ok_or_else(|| format!("ExtIndex locator invalid: {ext_path}"))?; + self.store.set(&resolved, &value); Ok(()) } @@ -958,15 +702,13 @@ impl KVSpace for Backend { let head = decode_xvalue_head(&data); if head.is_ptr() { self.store.del(&[link_key]); - let (parent, name) = Self::parent_name(&resolved); - self.remove_child(&parent, &[name]); + self.set_metadata(link_key, false, 0)?; return Ok(()); } } self.store.del(&[&resolved]); - let (parent, name) = Self::parent_name(&resolved); - self.remove_child(&parent, &[name]); + self.set_metadata(&resolved, false, 0)?; Ok(()) } diff --git a/src/const.rs b/src/const.rs index 269a504..2abc294 100644 --- a/src/const.rs +++ b/src/const.rs @@ -4,10 +4,10 @@ // ── 路径结构 ────────────────────────────────────────────────────────────── pub const PATH_SEP: &str = "/"; // 路径分隔符 +pub const META_ROOT_NAME: &str = ".kvspace-meta"; pub const DIR_INDEX_SUF: &str = "/"; // 目录索引键后缀（尾斜杠 = 目录，必须以 / 开头的 key 保证不冲突） pub const RUNTIME_MEMBER_SEP: &str = "‥"; // 运行时保留字段前缀（U+2025）——‥ 的唯一定义处，List 时隐藏 -pub const INDEX_VALUE_SEP: &str = "\n"; // index XValueHead 中的路径分隔符 -pub const EXT_INDEX_HEAD: &str = "…"; // extindex XValueHead bytes 首元素前缀，如 …/lib/init/ +pub const EXT_INDEX_HEAD: &str = "…"; // Directory display marker. // ── 错误消息 ──────────────────────────────────────────────────────────── @@ -42,7 +42,3 @@ pub const KIND_FLOAT32: &str = "float32"; // → byte, 4B pub const KIND_INT64: &str = "int64"; // → byte, 8B pub const KIND_UINT64: &str = "uint64"; // → byte, 8B pub const KIND_FLOAT64: &str = "float64"; // → byte, 8B - -pub const KIND_MAP: &str = "stringkeymap"; -pub const KIND_INDEX: &str = "index"; -pub const KIND_EXT_INDEX: &str = "extindex"; // 扩展索引，写留在上层 diff --git a/src/ffi.rs b/src/ffi.rs index d00cb84..5c8b556 100644 --- a/src/ffi.rs +++ b/src/ffi.rs @@ -19,7 +19,7 @@ use std::time::Duration; use crate::conn::conn; use crate::kvspace::{KVPair, KVSpace}; -use crate::xvalue::{decode_xvalue, decode_xvalue_head, encode_head, encode_head_perm, new_ptr}; +use crate::xvalue::{decode_xvalue, decode_xvalue_head, encode_head, new_ptr}; use crate::xvalue_bool::new_bool; use crate::xvalue_byte::new_char_byte; use crate::xvalue_float::new_float64; @@ -39,6 +39,7 @@ pub struct Handle { read_bufs: Vec>, write_buf: Vec, pending_key: Option, + pending_meta: Option<(bool, u32)>, } impl Handle { @@ -46,13 +47,17 @@ impl Handle { /// 借用生命周期同该槽、跨指令内的写仍有效；由 VM 在指令边界显式调 kvspaceReadReset 回收。 fn flush(&mut self) -> Result<(), String> { if let Some(key) = self.pending_key.take() { + let meta = self.pending_meta.take(); let tlv = std::mem::take(&mut self.write_buf); let val = decode_xvalue(&tlv); self.kv.set(&[KVPair { - key, + key: key.clone(), val, raw: Some(tlv), }])?; + if let Some((ro, vid)) = meta { + self.kv.set_metadata(&key, ro, vid)?; + } } Ok(()) } @@ -75,54 +80,20 @@ unsafe fn kv_flush<'a>(h: *mut Handle) -> Result<&'a mut dyn KVSpace, String> { Ok(&mut *hd.kv) } -/// 由 (ref, storetype, ro, vid, langtype) + body_len 直接构造三正交轴 head 并预留 body_len 零字节。 -/// 与 kvspace-c kvspaceXvalueWriteHead 逐字节一致：ARRAYND 从 langtype 的 [dims] 落物理字段， -/// NONE/ATOM 无物理字段（index/extindex 不走本路径）。 fn build_tlv( r#ref: u8, storetype: u8, - ro: u8, - vid: u32, langtype: &str, body_len: usize, -) -> Vec { - let dims = parse_langtype_dims(langtype); - let lt = langtype.as_bytes(); - let has_dims = crate::xvalue::store_has_dims(storetype); - let phys = if has_dims { 1 + 4 * dims.len() } else { 0 }; - let headlen = crate::xvalue::HEAD_PREFIX + phys + lt.len(); - let mut v = vec![0u8; headlen + body_len]; - v[0..2].copy_from_slice(&(headlen as u16).to_le_bytes()); - v[2] = r#ref; - v[3] = storetype; - v[4] = ro; - v[5..9].copy_from_slice(&vid.to_le_bytes()); - v[9..13].copy_from_slice(&(body_len as u32).to_le_bytes()); - let mut o = crate::xvalue::HEAD_PREFIX; - if has_dims { - v[o] = dims.len() as u8; - o += 1; - for d in &dims { - v[o..o + 4].copy_from_slice(&(*d as u32).to_le_bytes()); - o += 4; - } - } - v[o..o + lt.len()].copy_from_slice(lt); - v -} - -/// 解析 langtype 前导 [dims]（仅 ARRAYND 携带；无则空）。 -fn parse_langtype_dims(lt: &str) -> Vec { - if lt.starts_with('[') { - if let Some(end) = lt.find(']') { - return lt[1..end] - .split(',') - .filter(|s| !s.is_empty()) - .map(|s| s.parse().unwrap_or(0)) - .collect(); - } - } - Vec::new() + body_cap: usize, +) -> Option> { + let flags = match (r#ref, storetype) { + (0, 0..=2) => storetype, + (1, 1) => 5, + (2, 3) => 3, + _ => return None, + }; + crate::headlenpow::reserve(flags, langtype, body_len, body_cap) } // ── 内部助手 ───────────────────────────────────────────────────────── @@ -196,7 +167,7 @@ fn result_to_code(r: Result<(), String>, err: *mut c_char, err_cap: u32) -> c_in pub struct kvspaceHead_t { pub headlen: u16, // head 总字节数 pub r#ref: u8, // 存储位置：0=inline 1=ptr 2=@ext - pub storetype: u8, // 物理布局：NONE/ATOM/ARRAYND/index/extindex + pub storetype: u8, // Wire storage class. pub ro: u8, // 1=只读，0=可写 pub vid: u32, // vthread id pub body_len: i32, // body 字节数 @@ -205,6 +176,7 @@ pub struct kvspaceHead_t { pub langtype: [u8; 256], // 完整 kindexpr 串（含 [dims]、无前缀），NUL 终止 pub langtype_len: i32, // langtype 字节数（不含 NUL） pub body_offset: i32, // body 在 data 内的起始偏移（= head_len） + pub body_cap: u64, } fn fill_head(head: &crate::xvalue::XValueHead, out: *mut kvspaceHead_t) { @@ -229,6 +201,7 @@ fn fill_head(head: &crate::xvalue::XValueHead, out: *mut kvspaceHead_t) { o.vid = head.vid; o.body_len = head.body_len; o.body_offset = head.head_len(); + o.body_cap = head.body_cap; } } @@ -243,6 +216,7 @@ pub extern "C" fn kvspaceConnect(dsn: *const c_char) -> *mut Handle { read_bufs: Vec::new(), write_buf: Vec::new(), pending_key: None, + pending_meta: None, })), Err(_) => std::ptr::null_mut(), } @@ -306,6 +280,53 @@ pub extern "C" fn kvspaceGet( } } +#[no_mangle] +pub extern "C" fn kvspaceSetValue( + h: *mut Handle, + key: *const c_char, + value: *const u8, + value_len: u32, + ro: u8, + vid: u32, + err: *mut c_char, + err_cap: u32, +) -> c_int { + let Some(hd) = (unsafe { h.as_mut() }) else { + return 1; + }; + if key.is_null() || value.is_null() || value_len == 0 { + write_err(err, err_cap, "invalid SetValue input"); + return 1; + } + if let Err(e) = hd.flush() { + write_err(err, err_cap, &e); + return 1; + } + let key = unsafe { cstr(key) }.to_string(); + let raw = unsafe { std::slice::from_raw_parts(value, value_len as usize) }; + if crate::headlenpow::decode(raw).map(|head| head.total) != Some(raw.len()) { + write_err(err, err_cap, "invalid XValue"); + return 1; + } + let raw = raw.to_vec(); + let result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { + let val = decode_xvalue(&raw); + hd.kv.set(&[KVPair { + key: key.clone(), + val, + raw: Some(raw), + }])?; + hd.kv.set_metadata(&key, ro != 0, vid) + })); + match result { + Ok(result) => result_to_code(result, err, err_cap), + Err(_) => { + write_err(err, err_cap, "SetValue backend panic"); + 1 + } + } +} + /// 借指令边界回收读借用池：VM 每条指令执行完调用一次。durable 惰性写不再清池， /// 全部读借用在此统一失效（shm 常驻映射侧为 no-op）。 #[no_mangle] @@ -406,22 +427,28 @@ pub extern "C" fn kvspaceGetHead( return 1; } let key = unsafe { cstr(key) }.to_string(); - let prefix = match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| { - hd.kv.get_part(&key, 0, 512) - })) { + let value = match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| hd.kv.get_raw(&key))) + { Ok(p) => p, Err(_) => return 1, }; - if prefix.is_empty() { + if value.is_empty() { return 1; } - // head 只需前缀即可解（body 不在手上，不能走整值校验）；head 长于已读前缀说明前缀太小， - // 报错而非给出空 head——绝不静默返回一个"看起来合法"的默认值。 - let head = crate::xvalue::decode_xvalue_head_prefix(&prefix); + let head = crate::xvalue::decode_xvalue_head(&value); if head.headlen == 0 { return 1; } + let (ro, vid) = + match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| hd.kv.get_metadata(&key))) { + Ok(Ok(meta)) => meta, + _ => return 1, + }; fill_head(&head, out); + unsafe { + (*out).ro = ro as u8; + (*out).vid = vid; + } 0 } @@ -453,12 +480,25 @@ pub extern "C" fn kvspaceWriteInPlace( } let head = decode_xvalue_head(&existing); let head_len = head.head_len() as usize; - if head.body_len as u32 != body_len || existing.len() != head_len + body_len as usize { + if head.headlen == 0 + || head.body_len < 0 + || head.body_len as u32 != body_len + || head.body_cap > usize::MAX as u64 + || head_len.checked_add(head.body_cap as usize) != Some(existing.len()) + { write_err(err, err_cap, "kvspace: write-in-place body_len mismatch"); return 1; } + let meta = match hd.kv.get_metadata(&key) { + Ok(value) => value, + Err(e) => { + write_err(err, err_cap, &e); + return 1; + } + }; hd.write_buf = existing; hd.pending_key = Some(key); + hd.pending_meta = Some(meta); unsafe { *body = hd.write_buf.as_mut_ptr().add(head_len) }; 0 } @@ -474,6 +514,7 @@ pub extern "C" fn kvspaceWriteNewPlace( vid: u32, langtype: *const c_char, body_len: u32, + body_cap: u64, body: *mut *mut u8, err: *mut c_char, err_cap: u32, @@ -488,10 +529,18 @@ pub extern "C" fn kvspaceWriteNewPlace( } let key = unsafe { cstr(key) }.to_string(); let lt = unsafe { cstr(langtype) }; - let tlv = build_tlv(r#ref, storetype, ro, vid, lt, body_len as usize); - let head_len = tlv.len() - body_len as usize; + let Ok(cap) = usize::try_from(body_cap) else { + write_err(err, err_cap, "capacity exceeds platform size"); + return 1; + }; + let Some(tlv) = build_tlv(r#ref, storetype, lt, body_len as usize, cap) else { + write_err(err, err_cap, "invalid XValue geometry"); + return 1; + }; + let head_len = tlv.len() - cap; hd.write_buf = tlv; hd.pending_key = Some(key); + hd.pending_meta = Some((ro != 0, vid)); unsafe { *body = hd.write_buf.as_mut_ptr().add(head_len) }; 0 } @@ -829,8 +878,11 @@ pub extern "C" fn kvspaceTlvEncodeMode( ) -> c_int { let raw = unsafe { std::slice::from_raw_parts(raw, raw_len as usize) }; let dims = ffi_dims(dims, ndim); + if ro != 0 || vid != 0 { + return 1; + } alloc( - encode_head_perm(unsafe { cstr(kind) }, r#ref, dims, raw, ro != 0, vid), + encode_head(unsafe { cstr(kind) }, r#ref, dims, raw), out, out_len, ) @@ -856,6 +908,9 @@ pub extern "C" fn kvspaceDecodeHead( return 1; } let head = decode_xvalue_head(unsafe { std::slice::from_raw_parts(data, data_len as usize) }); + if head.headlen == 0 { + return 1; + } fill_head(&head, out); 0 } diff --git a/src/fs/kvspace.rs b/src/fs/kvspace.rs index 104feab..238b23e 100644 --- a/src/fs/kvspace.rs +++ b/src/fs/kvspace.rs @@ -1,6 +1,6 @@ // fs/kvspace.rs — 结构感知的文件系统 KVSpace。 // 编码：kvspace 的 '·'（成员分隔）一律替换为 '·/'（父目录名带尾中点 + "/" 分隔成员），反向 '·/' → '·'。 -// "/" 与 "·" 的 index 都从 readdir 派生；ExtIndex 用目录内 __extindex__ 文件存 ext_target_path（第一行）。 +// Directory members come from physical keys. use std::fs; use std::os::unix::fs::FileExt; @@ -10,17 +10,13 @@ use std::time::Duration; use crate::coord::cmp_coord; use crate::kvspace::{KVPair, KVSpace}; use crate::kvspace_common::{ - join_path, sep_path, split_index, strip_dir_suf, validate_ptr, watch_value, + is_descendant, is_frame_operand_key, join_path, sep_path, split_index, strip_dir_suf, + validate_ptr, watch_value, }; use crate::r#const::*; use crate::xvalue::*; -use crate::xvalue_index::{new_ext_index, new_index}; - -const EXTINDEX_MARKER: &str = "__extindex__"; const SELF_MARKER: &str = "__self__"; -const ORDER_MARKER: &str = "__order__"; -/// stringkeymap 成员目录标记，内容为 dims（逗号分隔）。readdir 派生不出 kind 与 dims，故显式落盘。 -const MAP_MARKER: &str = "__map__"; +const DIR_MARKER: &str = "__dir__"; pub struct FsKVSpace { root: PathBuf, @@ -58,32 +54,75 @@ impl FsKVSpace { } prev = c; } - self.root.join(rel.trim_start_matches('/')) + let path = rel + .trim_start_matches('/') + .split('/') + .map(|part| { + if part == SELF_MARKER || part == DIR_MARKER || part.starts_with('~') { + let mut escaped = String::from("~"); + for byte in part.bytes() { + escaped.push_str(&format!("{byte:02x}")); + } + escaped + } else { + part.to_string() + } + }) + .collect::>() + .join("/"); + self.root.join(path) + } + + fn unescape_name(name: &str) -> String { + let Some(hex) = name.strip_prefix('~') else { + return name.to_string(); + }; + if hex.len() % 2 != 0 { + return name.to_string(); + } + let bytes = (0..hex.len()) + .step_by(2) + .map(|i| u8::from_str_radix(&hex[i..i + 2], 16)) + .collect::, _>>(); + bytes + .ok() + .and_then(|v| String::from_utf8(v).ok()) + .unwrap_or_else(|| name.to_string()) } fn is_dir_key(key: &str) -> bool { key.ends_with(DIR_INDEX_SUF) || key.ends_with(OBJ_SEP) } + fn leaf_marker(key: &str) -> &'static str { + if key != PATH_SEP && key.ends_with(DIR_INDEX_SUF) { + DIR_MARKER + } else { + SELF_MARKER + } + } + fn read_leaf(&self, key: &str) -> Option> { if key.contains("//") { return None; } let p = self.fs_path(key); - // 目录带值：值落在目录内的保留文件 __self__，readdir 派生的 index 之外。 if p.is_dir() { - fs::read(p.join(SELF_MARKER)).ok() + fs::read(p.join(Self::leaf_marker(key))).ok() } else { fs::read(p).ok() } } - /// 叶值实际所在文件（目录带值时为 __self__），供定位读写 pread/pwrite。存在才返回。 fn leaf_file(&self, key: &str) -> Option { if key.contains("//") { return None; } let p = self.fs_path(key); - let f = if p.is_dir() { p.join(SELF_MARKER) } else { p }; + let f = if p.is_dir() { + p.join(Self::leaf_marker(key)) + } else { + p + }; if f.is_file() { Some(f) } else { @@ -92,12 +131,19 @@ impl FsKVSpace { } /// 解析逻辑 key 到叶文件（先 meta，再 ext 扩展存储）。 fn physical_file(&mut self, key: &str) -> Option { + if crate::metadata::is_reserved(key) { + return None; + } let (mut p, l) = sep_path(key); if p != PATH_SEP { p.push_str(DIR_INDEX_SUF); } let p = self.resolve_path(&p); - if let Some(f) = self.leaf_file(&join_path(&p, &l)) { + let full = join_path(&p, &l); + if crate::metadata::is_reserved(&full) { + return None; + } + if let Some(f) = self.leaf_file(&full) { return Some(f); } let ext_t = self.prefix_ext(&p); @@ -110,9 +156,12 @@ impl FsKVSpace { } fn write_leaf(&self, key: &str, val: &[u8]) { + if key != PATH_SEP && key.ends_with(DIR_INDEX_SUF) { + self.ensure_dir(key); + } let p = self.fs_path(key); if p.is_dir() { - fs::write(p.join(SELF_MARKER), val) + fs::write(p.join(Self::leaf_marker(key)), val) .unwrap_or_else(|e| panic!("kvspace-fs: set {}: {}", key, e)); } else { if let Some(parent) = p.parent() { @@ -125,12 +174,92 @@ impl FsKVSpace { fn remove_leaf(&self, key: &str) { let p = self.fs_path(key); if p.is_dir() { - let _ = fs::remove_file(p.join(SELF_MARKER)); + let _ = fs::remove_file(p.join(Self::leaf_marker(key))); } else { let _ = fs::remove_file(p); } } + fn sync_metadata(&mut self, key: &str, raw: &[u8]) -> Result<(), String> { + let head = decode_xvalue_head(raw); + if !raw.is_empty() && head.headlen == 0 { + return Err(format!("invalid XValue at {key}")); + } + self.set_metadata(key, head.ro, head.vid) + } + + fn metadata_at(&self, key: &str) -> Result<(bool, u32), String> { + let meta = + crate::metadata::key_for(key).ok_or_else(|| format!("reserved metadata key: {key}"))?; + match self.read_leaf(&meta) { + None => Ok((false, 0)), + Some(data) => { + crate::metadata::decode(&data).ok_or_else(|| format!("invalid metadata at {key}")) + } + } + } + + fn metadata_entries(&self) -> Vec<(String, PathBuf)> { + let mut entries = Vec::new(); + if let Ok(dir) = fs::read_dir(self.fs_path("/.kvspace-meta/")) { + for item in dir.flatten() { + let name = item.file_name().to_string_lossy().into_owned(); + if let Some(key) = crate::metadata::original_key(&name) { + entries.push((key, item.path())); + } + } + } + entries + } + + fn in_tree(key: &str, base: &str) -> bool { + base == PATH_SEP + || key == base + || key + .strip_prefix(base) + .is_some_and(|rest| rest.starts_with(PATH_SEP) || rest.starts_with(OBJ_SEP)) + } + + fn remove_metadata_tree(&self, base: &str) -> Result<(), String> { + for (key, path) in self.metadata_entries() { + if Self::in_tree(&key, base) { + fs::remove_file(path).map_err(|e| format!("remove metadata for {key}: {e}"))?; + } + } + Ok(()) + } + + fn copy_metadata_tree(&self, src: &str, dst: &str, one_level: bool) -> Result<(), String> { + for (key, path) in self.metadata_entries() { + if !Self::in_tree(&key, src) { + continue; + } + let suffix = &key[src.len()..]; + if one_level && !suffix.is_empty() { + let Some(member) = suffix.strip_prefix(OBJ_SEP) else { + continue; + }; + if member.contains(OBJ_SEP) || member.contains(PATH_SEP) { + continue; + } + } + let target = if src == PATH_SEP { + format!( + "{}/{}", + dst.trim_end_matches('/'), + key.trim_start_matches('/') + ) + } else { + format!("{dst}{suffix}") + }; + let meta = crate::metadata::key_for(&target) + .ok_or_else(|| format!("reserved metadata key: {target}"))?; + let data = fs::read(path).map_err(|e| format!("copy metadata for {key}: {e}"))?; + self.write_leaf(&meta, &data); + } + Ok(()) + } + fn parent_name(path: &str) -> (String, String) { let mut path = path.to_string(); if Self::is_dir_key(&path) && path != PATH_SEP { @@ -191,7 +320,11 @@ impl FsKVSpace { let resolved = self.resolve_path(&parent); let mut result = join_path(&resolved, &last); if dir_suf { - result.push_str(DIR_INDEX_SUF); + result.push_str(if path.ends_with(OBJ_SEP) { + OBJ_SEP + } else { + DIR_INDEX_SUF + }); } result } @@ -226,16 +359,10 @@ impl FsKVSpace { fn prefix_ext(&self, prefix: &str) -> String { if let Some(data) = self.read_leaf(prefix) { let head = decode_xvalue_head(&data); - if head.kind() == KIND_EXT_INDEX { - let body = head.body(&data); - return crate::xvalue_index::decode_ext_index(body, &head.dims()).ext_path; + if head.r#ref == crate::xvalue::REF_EXT { + return String::from_utf8_lossy(head.body(&data)).into_owned(); } } - // 结构感知：ext 也可能存在 marker 文件里 - let marker = self.fs_path(prefix).join(EXTINDEX_MARKER); - if let Ok(b) = fs::read(&marker) { - return String::from_utf8_lossy(&b).into_owned(); - } String::new() } @@ -246,66 +373,32 @@ impl FsKVSpace { return Vec::new(); } let p = self.fs_path(dir_key); - // 目录名（末段）是否以 '·' 结尾：成员前缀目录（"math·"）内的成员不再扁平化。 - let name = dir_key - .trim_end_matches('/') - .rsplit('/') - .next() - .unwrap_or(""); - let is_member_dir = name.ends_with(OBJ_SEP); let mut children = Vec::new(); if let Ok(entries) = fs::read_dir(&p) { for e in entries.flatten() { let fname = e.file_name().to_string_lossy().into_owned(); - if fname == EXTINDEX_MARKER - || fname == SELF_MARKER - || fname == ORDER_MARKER - || fname == MAP_MARKER + if fname == SELF_MARKER + || fname == DIR_MARKER + || (dir_key == PATH_SEP && fname == META_ROOT_NAME) { continue; } - if fname.ends_with(OBJ_SEP) { - if is_member_dir { - // 成员目录内：obj 是完整 key，发射为 "init." - children.push(fname); - } else { - // 普通目录内：成员前缀目录（"string·"）→ **裸成员名** "string"。 - // 对齐 redis 后端（存储索引把成员前缀节点登记为去 · 的裸名）： - // 扁平展开会让父目录列出全部后代（如 "string·upper.src"）， - // 递归遍历/重建（layout dump、walk_lib）就再也对不上父子层级。 - children.push(fname.trim_end_matches(OBJ_SEP).to_string()); - } + let name = Self::unescape_name(&fname); + if name.ends_with(OBJ_SEP) { + children.push(name.trim_end_matches(OBJ_SEP).to_string()); } else if e.path().is_dir() { - children.push(format!("{}/", fname)); - // 目录带值：额外发射无尾斜杠的叶名（对应 __self__） - if e.path().join(SELF_MARKER).is_file() { - children.push(fname.clone()); - } + children.push(format!("{}/", name)); } else { - children.push(fname); + children.push(name); } } } // memindex 统一按 cmp_coord 规范排序（坐标 row-major 数值序、字符串键字典序），三后端一致。 children.sort_by(|a, b| cmp_coord(a, b)); + children.dedup(); children } - /// 目录 key 的 XValue：统一为 index（memindex）；ExtIndex 有 marker 时为 extindex。 - /// 不存在的目录返回 None（对齐 redis 后端：目录 key 不存在 → None）。 - fn dir_value(&self, dir_key: &str) -> XValue { - if dir_key.contains("//") || !self.fs_path(dir_key).is_dir() { - return XValue::None; - } - let children = self.dir_children(dir_key); - let marker = self.fs_path(dir_key).join(EXTINDEX_MARKER); - if let Ok(b) = fs::read(&marker) { - let ext_path = String::from_utf8_lossy(&b).into_owned(); - return new_ext_index(&children, &ext_path); - } - new_index(&children) - } - /// 递归复制文件/目录（extindex/self/order/map marker 作普通文件一并复制）。 fn copy_recursive(src: &Path, dst: &Path) -> std::io::Result<()> { if src.is_dir() { @@ -325,6 +418,18 @@ impl FsKVSpace { } impl KVSpace for FsKVSpace { + fn set_metadata(&mut self, key: &str, ro: bool, vid: u32) -> Result<(), String> { + let meta = + crate::metadata::key_for(key).ok_or_else(|| format!("reserved metadata key: {key}"))?; + if ro || vid != 0 { + let value = crate::metadata::encode(ro, vid) + .ok_or_else(|| format!("cannot encode metadata at {key}"))?; + self.write_leaf(&meta, &value); + } else { + self.remove_leaf(&meta); + } + Ok(()) + } fn get(&mut self, prefix: &str, keys: &[String], resolve: bool) -> Vec { if prefix != PATH_SEP && !Self::is_dir_key(prefix) { panic!("{}: {}", ERR_DIR_MUST_END_WITH_SLASH, prefix); @@ -339,17 +444,18 @@ impl KVSpace for FsKVSpace { keys.iter() .map(|k| { let full = join_path(&prefix, k); + if crate::metadata::is_reserved(&full) { + return XValue::None; + } if Self::is_dir_key(&full) { - return self.dir_value(&full); + return self + .read_leaf(&full) + .map(|raw| decode_xvalue(&raw)) + .unwrap_or(XValue::None); } if let Some(data) = self.read_leaf(&full) { return decode_xvalue(&data); } - // dict 形式回落：读 seen 回落 seen. - let dict_key = format!("{}{}", full, OBJ_SEP); - if self.fs_path(&dict_key).is_dir() { - return self.dir_value(&dict_key); - } if !ext_t.is_empty() { let target = join_path(&ext_t, k); if let Some(data) = self.read_leaf(&target) { @@ -362,12 +468,13 @@ impl KVSpace for FsKVSpace { } fn get_raw(&mut self, key: &str) -> Vec { - // 目录 key 无落盘索引（FS 自身即索引）：按需派生 index XValue 再返回， - // 对齐 redis 后端「目录 key get 返回容器 XValue」（issue #250）。 + if crate::metadata::is_reserved(key) { + return Vec::new(); + } if Self::is_dir_key(key) { let resolved = self.resolve_path(key); - if Self::is_dir_key(&resolved) && self.fs_path(&resolved).is_dir() { - return self.dir_value(&resolved).encode(); + if Self::is_dir_key(&resolved) { + return self.read_leaf(&resolved).unwrap_or_default(); } } let (mut p, l) = sep_path(key); @@ -376,6 +483,9 @@ impl KVSpace for FsKVSpace { } let p = self.resolve_path(&p); let full = join_path(&p, &l); + if crate::metadata::is_reserved(&full) { + return Vec::new(); + } if let Some(data) = self.read_leaf(&full) { return data; } @@ -389,13 +499,18 @@ impl KVSpace for FsKVSpace { Vec::new() } + fn get_metadata(&mut self, key: &str) -> Result<(bool, u32), String> { + let resolved = self.resolve_path(key); + self.metadata_at(&resolved) + } + fn get_part(&mut self, key: &str, off: u32, len: u32) -> Vec { - // 目录 key：索引是派生物（FS 自身即索引），先在内存里拼出 index XValue 再切片， - // 否则 head 读前缀会拿到空（issue #250：fs 目录 key head → nil）。 + if crate::metadata::is_reserved(key) { + return Vec::new(); + } if Self::is_dir_key(key) { let resolved = self.resolve_path(key); - if Self::is_dir_key(&resolved) && self.fs_path(&resolved).is_dir() { - let v = self.dir_value(&resolved).encode(); + if let Some(v) = self.read_leaf(&resolved) { let s = (off as usize).min(v.len()); let e = (s + len as usize).min(v.len()); return v[s..e].to_vec(); @@ -418,6 +533,9 @@ impl KVSpace for FsKVSpace { } fn set_part(&mut self, key: &str, off: u32, buf: &[u8]) -> Result<(), String> { + if crate::metadata::is_reserved(key) { + return Err(format!("reserved metadata key: {key}")); + } let pf = self .physical_file(key) .ok_or_else(|| format!("set_part: missing key {}", key))?; @@ -431,87 +549,56 @@ impl KVSpace for FsKVSpace { } fn set(&mut self, pairs: &[KVPair]) -> Result<(), String> { - for p in pairs { - // 只解析父路径，整键不穿透：写 Ptr 变量/帧槽时写的是指针本体（槽本身）， - // 解引用由 runtime 显式 `*` 掌控（对齐 backend.rs / kvspace-c）。 - let resolved = self.resolve_parent(&p.key); - if resolved.contains("//") { - return Err(format!("Set: double-slash in key {:?}", resolved)); - } - match &p.val { - XValue::Index(_) | XValue::ExtIndex(_) => { - if !Self::is_dir_key(&resolved) { - return Err(format!( - "Set: directory-kind value at non-directory key {:?}", - resolved - )); - } - } - _ => {} + for pair in pairs { + if crate::metadata::is_reserved(&pair.key) { + return Err(format!("reserved metadata key: {}", pair.key)); } - if let XValue::Ptr(ptr) = &p.val { - validate_ptr(self, &ptr.target, &ptr.target_kindexpr)?; - } - - // 目录 index 值：结构派生，无需存；ExtIndex 写 marker。 - if let XValue::ExtIndex(e) = &p.val { - self.ensure_dir(&resolved); - let marker = self.fs_path(&resolved).join(EXTINDEX_MARKER); - fs::write(&marker, e.ext_path.as_bytes()) - .map_err(|e| format!("kvspace-fs: extindex {}: {}", resolved, e))?; - continue; + let key = self.resolve_parent(&pair.key); + if key.contains("//") || crate::metadata::is_reserved(&key) { + return Err(format!("invalid key: {key}")); } - if let XValue::Map(_) = &p.val { - let base = if resolved == PATH_SEP { - resolved.clone() - } else { - strip_dir_suf(&resolved).to_string() - }; - let (parent, _name) = Self::parent_name(&base); - self.ensure_dir(&parent); // 父可能是同名叶文件（如 /lib/input def rwir）→ 提升为目录 - self.write_leaf(&base, &p.raw.clone().unwrap_or_else(|| p.val.encode())); - self.ensure_dir(&format!("{}{}", base, OBJ_SEP)); - continue; + let raw = pair.raw.clone().unwrap_or_else(|| pair.val.encode()); + let head = crate::headlenpow::decode(&raw) + .ok_or_else(|| format!("invalid XValue at {key}"))?; + if head.total != raw.len() { + return Err(format!("invalid XValue at {key}")); } - if let XValue::Index(_) = &p.val { - self.ensure_dir(&resolved); - continue; + if let XValue::Ptr(ptr) = decode_xvalue(&raw) { + validate_ptr(self, &ptr.target, &ptr.target_kindexpr)?; } - - // 叶值：确保父是目录（父可能是同名叶文件，如 /lib/println），写文件。 - let (parent, name, _) = split_index(&resolved); - // 成员写（父为尾 `·` 的成员目录）：memhead 必须已存在，容器值不在即拒绝—— - // 与 backend.rs / runtime 的 kvlangBuiltinCheckMemhead 同一规则，绝不兜底自动建。 - // `/lib` 下的 `·` 是**包·函数命名分隔符**（`/lib/pkg·func`），不是 memindex 标记， - // 与 runtime 的 kvlangBuiltinCheckMemhead 同款豁免——不豁免会把 `/lib/json·to` - // 当成「在容器 /lib/json 上写成员 to」，把扩展算子的注册整个拒掉。 - if parent.ends_with(OBJ_SEP) && !strip_dir_suf(&parent).starts_with("/lib") { + let (parent, _, _) = split_index(&key); + if parent.ends_with(OBJ_SEP) { let base = strip_dir_suf(&parent); - if !self.fs_path(base).exists() && !Self::is_dir_key(base) { + if !base.starts_with("/lib") && self.read_leaf(base).is_none() { return Err(format!( "{}: memhead {} does not exist — declare the container first", ERR_MEMHEAD_MISSING, base )); } } - self.ensure_dir(&parent); - // extindex 写保护：只读扩展层上的同名节点禁止写入（对齐 backend.rs）。 - let marker = self.fs_path(&parent).join(EXTINDEX_MARKER); - if let Ok(b) = fs::read(&marker) { - let ext_t = String::from_utf8_lossy(&b).into_owned(); - if !self.dir_children(&parent).iter().any(|n| n == &name) - && self.dir_children(&ext_t).iter().any(|n| n == &name) + let ext_target = self.prefix_ext(&parent); + if !ext_target.is_empty() && self.read_leaf(&key).is_none() { + let suffix = key.strip_prefix(&parent).unwrap_or(""); + if self.read_leaf(&format!("{ext_target}{suffix}")).is_some() + && !(head.flags & 4 != 0 && is_frame_operand_key(&key)) { - return Err(format!("{}: {}", ERR_EXT_WRITE, resolved)); + return Err(format!("{}: {}", ERR_EXT_WRITE, key)); } } - let bytes = p.raw.clone().unwrap_or_else(|| p.val.encode()); - self.write_leaf(&resolved, &bytes); + self.ensure_dir(&parent); + if Self::is_dir_key(&key) { + self.ensure_dir(&key); + } + self.sync_metadata(&key, &raw)?; + self.write_leaf(&key, &raw); } Ok(()) } fn list(&mut self, prefix: &str, expand_ext: bool, resolve: bool) -> Vec { + if crate::metadata::is_reserved(prefix) { + return Vec::new(); + } if prefix != PATH_SEP && !Self::is_dir_key(prefix) { panic!("{}: {}", ERR_DIR_MUST_END_WITH_SLASH, prefix); } @@ -523,6 +610,9 @@ impl KVSpace for FsKVSpace { if !Self::is_dir_key(&resolved) { return Vec::new(); } + if crate::metadata::is_reserved(&resolved) { + return Vec::new(); + } let mut members = self.dir_children(&resolved); if expand_ext { @@ -540,18 +630,27 @@ impl KVSpace for FsKVSpace { fn del(&mut self, keys: &[String]) -> Result<(), String> { for key in keys { + if crate::metadata::is_reserved(key) { + return Err(format!("reserved metadata key: {key}")); + } let resolved = self.resolve_parent(key); - if Self::is_dir_key(&resolved) { - let _ = fs::remove_dir_all(self.fs_path(&resolved)); - } else { - self.remove_leaf(&resolved); + if crate::metadata::is_reserved(&resolved) { + return Err(format!("reserved metadata key: {resolved}")); } + self.remove_leaf(&resolved); + self.set_metadata(&resolved, false, 0)?; } Ok(()) } fn del_tree(&mut self, prefix: &str) -> Result<(), String> { + if crate::metadata::is_reserved(prefix) { + return Err(format!("reserved metadata key: {prefix}")); + } let resolved = self.resolve_path(prefix); + if crate::metadata::is_reserved(&resolved) { + return Err(format!("reserved metadata key: {resolved}")); + } // 若 prefix 本身是链接（叶 Ptr），只删链接。 let link_key = if Self::is_dir_key(&resolved) && resolved != PATH_SEP { strip_dir_suf(&resolved) @@ -563,26 +662,50 @@ impl KVSpace for FsKVSpace { return self.del(&[resolved]); } } - let _ = fs::remove_dir_all(self.fs_path(&resolved)); + let base = if Self::is_dir_key(&resolved) && resolved != PATH_SEP { + strip_dir_suf(&resolved) + } else { + &resolved + }; + for key in [base.to_string(), format!("{base}{OBJ_SEP}")] { + let node = self.node_path(&key); + if node.is_dir() { + fs::remove_dir_all(&node).map_err(|e| format!("DelTree {key}: {e}"))?; + } else if node.is_file() { + fs::remove_file(&node).map_err(|e| format!("DelTree {key}: {e}"))?; + } + } + self.remove_metadata_tree(base)?; Ok(()) } fn cp(&mut self, src: &str, dst: &str) -> Result<(), String> { + if crate::metadata::is_reserved(src) || crate::metadata::is_reserved(dst) { + return Err("reserved metadata key".into()); + } let raw = self.get_raw(src); if raw.is_empty() { return Err(format!("Cp: source not found: {}", src)); } + let (ro, vid) = self.metadata_at(src)?; let v = decode_xvalue(&raw); self.set(&[KVPair { key: dst.to_string(), val: v, raw: Some(raw), - }]) + }])?; + self.set_metadata(dst, ro, vid) } fn cp_tree(&mut self, src: &str, dst: &str) -> Result<(), String> { + if crate::metadata::is_reserved(src) || crate::metadata::is_reserved(dst) { + return Err("reserved metadata key".into()); + } let src_res = self.resolve_path(src); let dst_res = self.resolve_path(dst); + if crate::metadata::is_reserved(&src_res) || crate::metadata::is_reserved(&dst_res) { + return Err("reserved metadata key".into()); + } let de_suffix = |k: &str| { if Self::is_dir_key(k) && k != PATH_SEP { strip_dir_suf(k).to_string() @@ -595,6 +718,9 @@ impl KVSpace for FsKVSpace { if src_base == dst_base { return Ok(()); } + if is_descendant(&src_base, &dst_base) { + return Err("CpTree: destination is inside source".into()); + } // 一个节点跨两条 fs 实体：base（值/层级子树）与兄弟成员目录 base·。 let sb = self.node_path(&src_base); let smem = self.node_path(&format!("{}{}", src_base, OBJ_SEP)); @@ -611,6 +737,7 @@ impl KVSpace for FsKVSpace { Self::copy_recursive(&smem, &dmem) .map_err(|e| format!("CpTree {}→{}: {}", src, dst, e))?; } + self.copy_metadata_tree(&src_base, &dst_base, false)?; // 确保 dst 父目录存在（成员名单结构派生，无需登记）。 let (parent, _name) = Self::parent_name(&dst_base); self.ensure_dir(&parent); @@ -618,8 +745,14 @@ impl KVSpace for FsKVSpace { } fn cp_list(&mut self, src: &str, dst: &str) -> Result<(), String> { + if crate::metadata::is_reserved(src) || crate::metadata::is_reserved(dst) { + return Err("reserved metadata key".into()); + } let src_res = self.resolve_path(src); let dst_res = self.resolve_path(dst); + if crate::metadata::is_reserved(&src_res) || crate::metadata::is_reserved(&dst_res) { + return Err("reserved metadata key".into()); + } let de_suffix = |k: &str| { if Self::is_dir_key(k) && k != PATH_SEP { strip_dir_suf(k).to_string() @@ -651,13 +784,18 @@ impl KVSpace for FsKVSpace { let dp = dmem.join(e.file_name()); if sp.is_file() { let _ = fs::copy(&sp, &dp); - } else if let Ok(c) = fs::read(sp.join(SELF_MARKER)) { - let _ = fs::create_dir_all(&dp); - let _ = fs::write(dp.join(SELF_MARKER), c); + } else if sp.is_dir() { + for marker in [SELF_MARKER, DIR_MARKER] { + if let Ok(c) = fs::read(sp.join(marker)) { + let _ = fs::create_dir_all(&dp); + let _ = fs::write(dp.join(marker), c); + } + } } } } } + self.copy_metadata_tree(&src_base, &dst_base, true)?; let (parent, _name) = Self::parent_name(&dst_base); self.ensure_dir(&parent); Ok(()) @@ -673,7 +811,12 @@ impl KVSpace for FsKVSpace { return Err(format!("{}: Mkindex {}", ERR_DIR_MUST_END_WITH_SLASH, path)); } let resolved = self.resolve_path(path); - let _ = fs::create_dir_all(self.fs_path(&resolved)); + self.ensure_dir(&resolved); + if self.read_leaf(&resolved).is_none() { + let value = crate::headlenpow::encode(5, 0, 0, 0, "lib", &[], 0) + .ok_or_else(|| format!("Mkindex: invalid directory {resolved}"))?; + self.write_leaf(&resolved, &value); + } Ok(()) } @@ -685,22 +828,36 @@ impl KVSpace for FsKVSpace { )); } let resolved = self.resolve_parent(path); - // 级联检查：ext_path 本身是 extindex → 不容许（对齐 backend.rs） - if self.fs_path(ext_path).join(EXTINDEX_MARKER).is_file() { + if !self.prefix_ext(ext_path).is_empty() { return Err(format!("{}: {}", ERR_EXT_CASCADE, ext_path)); } - let dir = self.fs_path(&resolved); - let _ = fs::create_dir_all(&dir); - let marker = dir.join(EXTINDEX_MARKER); - fs::write(&marker, ext_path.as_bytes()) - .map_err(|e| format!("kvspace-fs: extindex {}: {}", resolved, e))?; + let source = self + .read_leaf(ext_path) + .ok_or_else(|| format!("ExtIndex target missing: {ext_path}"))?; + let langtype = decode_xvalue_head(&source).langtype; + let pow = (5..=31) + .find(|&p| 18 + langtype.len() <= 1usize << p) + .ok_or_else(|| format!("ExtIndex type too long: {langtype}"))?; + let len = ext_path.len() as u64; + let value = crate::headlenpow::encode( + pow, + 3, + len, + len, + &langtype, + ext_path.as_bytes(), + ext_path.len(), + ) + .ok_or_else(|| format!("ExtIndex locator invalid: {ext_path}"))?; + self.ensure_dir(&resolved); + self.write_leaf(&resolved, &value); Ok(()) } fn del_ext_index(&mut self, path: &str) -> Result<(), String> { let resolved = self.resolve_parent(path); - let marker = self.fs_path(&resolved).join(EXTINDEX_MARKER); - let _ = fs::remove_file(marker); + self.remove_leaf(&resolved); + self.set_metadata(&resolved, false, 0)?; Ok(()) } diff --git a/src/headlenpow.rs b/src/headlenpow.rs new file mode 100644 index 0000000..f2f975f --- /dev/null +++ b/src/headlenpow.rs @@ -0,0 +1,355 @@ +pub const PREFIX: usize = 18; + +#[derive(Debug, PartialEq)] +pub struct Head<'a> { + pub pow: u8, + pub flags: u8, + pub a: u64, + pub b: u64, + pub langtype: &'a str, + pub body: &'a [u8], + pub content_len: usize, + pub total: usize, +} + +fn scalar_width(s: &str) -> Option { + match s { + "bool" | "int8" | "uint8" | "float8/e4m3" | "float8/e5m2" => Some(1), + "int16" | "uint16" | "float16" | "bfloat16" => Some(2), + "int32" | "uint32" | "float32" => Some(4), + "int64" | "uint64" | "float64" => Some(8), + _ => None, + } +} + +fn short_width(s: &str) -> Option { + scalar_width(s).or_else(|| match s { + "" | "def struct" | "lib" | "rwfunc" => Some(0), + "def rwir" => Some(5), + "time" | "duration" => Some(8), + _ if s.starts_with('/') || s.contains('·') => Some(0), + _ => None, + }) +} + +fn min_pow(n: usize) -> Option { + (5..=31).find(|&p| (n as u64 + PREFIX as u64) <= (1u64 << p)) +} + +fn decimal(s: &str) -> Option { + if s.is_empty() || (s.len() > 1 && s.starts_with('0')) || !s.bytes().all(|b| b.is_ascii_digit()) + { + return None; + } + s.parse().ok() +} + +fn slack_count(s: &str, body: &[u8]) -> Option { + match s { + "char/utf8" => Some(std::str::from_utf8(body).ok()?.chars().count() as u64), + "byte" => Some(body.len() as u64), + "char/ascii" if body.is_ascii() => Some(body.len() as u64), + "char/utf32" if body.len() % 4 == 0 => { + for chunk in body.chunks_exact(4) { + let cp = u32::from_le_bytes(chunk.try_into().ok()?); + char::from_u32(cp)?; + } + Some((body.len() / 4) as u64) + } + _ => None, + } +} + +fn valid_slack_type(langtype: &str, body: &[u8]) -> bool { + let Some(end) = langtype.find(']') else { + return false; + }; + if !langtype.starts_with('[') { + return false; + } + let Some(expected) = decimal(&langtype[1..end]) else { + return false; + }; + slack_count(&langtype[end + 1..], body) == Some(expected) +} + +fn tensor_width(langtype: &str) -> Option<(u64, u64)> { + let end = langtype.find(']')?; + if !langtype.starts_with('[') { + return None; + } + let dims = &langtype[1..end]; + if dims.is_empty() { + return None; + } + let mut numel = 1u64; + for dim in dims.split(',') { + numel = numel.checked_mul(decimal(dim)?)?; + } + let width = scalar_width(&langtype[end + 1..])? as u64; + Some((numel, width)) +} + +pub fn decode(data: &[u8]) -> Option> { + let pow = *data.first()?; + if !(5..=31).contains(&pow) { + return None; + } + let headlen = 1usize.checked_shl(pow as u32)?; + if data.len() < headlen { + return None; + } + let flags = data[1]; + if flags & !7 != 0 { + return None; + } + let a = u64::from_le_bytes(data[2..10].try_into().ok()?); + let b = u64::from_le_bytes(data[10..18].try_into().ok()?); + let type_region = &data[PREFIX..headlen]; + let type_len = type_region + .iter() + .position(|&v| v == 0) + .unwrap_or(type_region.len()); + let langtype = std::str::from_utf8(&type_region[..type_len]).ok()?; + let (content, cap) = match flags { + 0 => { + if a != 0 || b != 0 || min_pow(type_len)? != pow { + return None; + } + let width = short_width(langtype)? as u64; + (width, width) + } + 1 => { + if a > b { + return None; + } + let code = langtype == "rwir" || langtype == "def langtype" || langtype == "rwfunc"; + if pow != if code { 5 } else { 6 } { + return None; + } + if langtype == "rwir" && a < 5 { + return None; + } + if langtype == "rwfunc" && a < 5 { + return None; + } + (a, b) + } + 2 => { + if pow != 7 { + return None; + } + let (numel, width) = tensor_width(langtype)?; + if a != numel || b != width { + return None; + } + let size = a.checked_mul(b)?; + (size, size) + } + 3 | 5 => { + if langtype.is_empty() || a == 0 || a > b || min_pow(type_len)? != pow { + return None; + } + (a, b) + } + _ => return None, + }; + let content_len = usize::try_from(content).ok()?; + let cap = usize::try_from(cap).ok()?; + let total = headlen.checked_add(cap)?; + if total > data.len() { + return None; + } + let body = &data[headlen..total]; + if flags == 0 && langtype == "bool" && body[0] > 1 { + return None; + } + if flags == 1 + && langtype != "rwir" + && langtype != "def langtype" + && langtype != "rwfunc" + && !valid_slack_type(langtype, &body[..content_len]) + { + return None; + } + if (flags == 3 || flags == 5) + && (body[..content_len].contains(&0) || std::str::from_utf8(&body[..content_len]).is_err()) + { + return None; + } + Some(Head { + pow, + flags, + a, + b, + langtype, + body, + content_len, + total, + }) +} + +pub fn encode( + pow: u8, + flags: u8, + a: u64, + b: u64, + langtype: &str, + body: &[u8], + cap: usize, +) -> Option> { + if langtype.contains('\0') || body.len() > cap || !(5..=31).contains(&pow) { + return None; + } + let headlen = 1usize.checked_shl(pow as u32)?; + if langtype.len() > headlen.checked_sub(PREFIX)? { + return None; + } + let total = headlen.checked_add(cap)?; + let mut data = Vec::new(); + data.try_reserve_exact(total).ok()?; + data.resize(total, 0); + data[0] = pow; + data[1] = flags; + data[2..10].copy_from_slice(&a.to_le_bytes()); + data[10..18].copy_from_slice(&b.to_le_bytes()); + data[PREFIX..PREFIX + langtype.len()].copy_from_slice(langtype.as_bytes()); + data[headlen..headlen + body.len()].copy_from_slice(body); + if decode(&data)?.content_len != body.len() { + return None; + } + Some(data) +} + +pub fn reserve(flags: u8, langtype: &str, content: usize, cap: usize) -> Option> { + if langtype.contains('\0') || content > cap { + return None; + } + let (pow, a, b) = match flags { + 0 if short_width(langtype)? == content && cap == content => { + (min_pow(langtype.len())?, 0, 0) + } + 1 => { + let code = matches!(langtype, "rwir" | "def langtype" | "rwfunc"); + if code { + if (langtype == "rwir" && content < 5) || (langtype == "rwfunc" && content < 5) { + return None; + } + } else { + let end = langtype.find(']')?; + if !langtype.starts_with('[') + || decimal(&langtype[1..end]).is_none() + || !matches!( + &langtype[end + 1..], + "char/utf8" | "byte" | "char/utf32" | "char/ascii" + ) + { + return None; + } + } + (if code { 5 } else { 6 }, content as u64, cap as u64) + } + 2 => { + let (numel, width) = tensor_width(langtype)?; + if numel.checked_mul(width)? != content as u64 || cap != content { + return None; + } + (7, numel, width) + } + 3 | 5 if !langtype.is_empty() && content > 0 => { + (min_pow(langtype.len())?, content as u64, cap as u64) + } + _ => return None, + }; + let headlen = 1usize.checked_shl(pow as u32)?; + let total = headlen.checked_add(cap)?; + let mut data = Vec::new(); + data.try_reserve_exact(total).ok()?; + data.resize(total, 0); + data[0] = pow; + data[1] = flags; + data[2..10].copy_from_slice(&a.to_le_bytes()); + data[10..18].copy_from_slice(&b.to_le_bytes()); + data[PREFIX..PREFIX + langtype.len()].copy_from_slice(langtype.as_bytes()); + Some(data) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn fixtures() { + let scalar = encode(5, 0, 0, 0, "int64", &42i64.to_le_bytes(), 8).unwrap(); + assert_eq!(scalar.len(), 40); + assert_eq!(&scalar[..2], &[5, 0]); + assert_eq!(&scalar[18..23], b"int64"); + assert_eq!(decode(&scalar).unwrap().content_len, 8); + + let text = "你😀".as_bytes(); + let utf8 = encode(6, 1, text.len() as u64, 16, "[2]char/utf8", text, 16).unwrap(); + assert_eq!(utf8.len(), 80); + assert_eq!(decode(&utf8).unwrap().content_len, text.len()); + + let map_type = "[int64]·[]char/utf32"; + let map = encode(6, 0, 0, 0, map_type, &[], 0).unwrap(); + assert_eq!(decode(&map).unwrap().langtype, map_type); + + let def = encode(5, 0, 0, 0, "def rwir", &[0; 5], 5).unwrap(); + assert_eq!(decode(&def).unwrap().content_len, 5); + + let time = encode(5, 0, 0, 0, "time", &[1; 8], 8).unwrap(); + assert_eq!(decode(&time).unwrap().content_len, 8); + + let ptr = encode(5, 5, 5, 8, "int64", b"/varx", 8).unwrap(); + assert_eq!(decode(&ptr).unwrap().body.len(), 8); + + let ext = encode(5, 3, 5, 8, "int64", b"s3://", 8).unwrap(); + assert_eq!(decode(&ext).unwrap().content_len, 5); + + let rwir = encode(5, 1, 8, 8, "rwir", &[0; 8], 8).unwrap(); + assert_eq!(decode(&rwir).unwrap().content_len, 8); + + let def_type = encode(5, 1, 3, 3, "def langtype", b"int", 3).unwrap(); + assert_eq!(decode(&def_type).unwrap().content_len, 3); + + let func = encode(5, 0, 0, 0, "rwfunc", &[], 0).unwrap(); + let anchor = encode(5, 1, 5, 8, "rwfunc", &[0; 5], 8).unwrap(); + assert_eq!(decode(&func).unwrap().content_len, 0); + assert_eq!(decode(&anchor).unwrap().content_len, 5); + let call = encode(5, 1, 6, 6, "rwfunc", &[0, 0, 0, 0, 0, b'f'], 6).unwrap(); + assert_eq!(decode(&call).unwrap().body, &[0, 0, 0, 0, 0, b'f']); + + let tensor = encode(7, 2, 6, 4, "[2,3]float32", &[0; 24], 24).unwrap(); + assert_eq!(decode(&tensor).unwrap().total, 152); + } + + #[test] + fn malformed() { + assert!(encode(5, 0, 0, 0, "int64", &[0; 7], 7).is_none()); + assert!(encode(5, 0, 0, 0, "bool", &[2], 1).is_none()); + assert!(encode(6, 1, 2, 2, "[2]char/utf8", b"\xc0\xaf", 2).is_none()); + assert!(encode(5, 5, 5, 8, "int64", b"/a\0bc", 8).is_none()); + assert!(encode(5, 3, 5, 8, "int64", b"s3\0//", 8).is_none()); + assert!(encode(5, 3, 0, 8, "int64", b"", 8).is_none()); + assert!(encode(5, 3, 1, 8, "int64", b"\xff", 8).is_none()); + assert!(encode(5, 1, 4, 8, "rwir", &[0; 4], 8).is_none()); + assert!(encode(5, 1, 4, 8, "rwfunc", &[0; 4], 8).is_none()); + assert!(encode(6, 1, 1, 1, "[1]char/ascii", &[0x80], 1).is_none()); + assert!(encode(6, 1, 4, 4, "[1]char/utf32", &0xd800u32.to_le_bytes(), 4).is_none()); + assert!(encode(6, 1, 4, 4, "[1]char/utf32", &0x110000u32.to_le_bytes(), 4).is_none()); + let mut ptr = encode(5, 5, 5, 8, "int64", b"/varx", 8).unwrap(); + ptr[1] |= 0x80; + assert!(decode(&ptr).is_none()); + } + + #[test] + fn reserved_body() { + let mut v = reserve(1, "[2]char/utf8", 3, 9).unwrap(); + v[64..67].copy_from_slice("éa".as_bytes()); + let h = decode(&v).unwrap(); + assert_eq!((h.a, h.b, h.total), (3, 9, 73)); + assert!(reserve(2, "[2,3]float32", 23, 23).is_none()); + assert!(reserve(1, "rwfunc", 4, 5).is_none()); + } +} diff --git a/src/kvspace.rs b/src/kvspace.rs index e57d503..ed50cab 100644 --- a/src/kvspace.rs +++ b/src/kvspace.rs @@ -42,6 +42,8 @@ pub trait KVSpace { fn get(&mut self, prefix: &str, keys: &[String], resolve: bool) -> Vec; /// 单点读原始字节（不 decode/re-encode，保 head 权限位 ro/vid）。无值返回空。 fn get_raw(&mut self, key: &str) -> Vec; + fn get_metadata(&mut self, key: &str) -> Result<(bool, u32), String>; + fn set_metadata(&mut self, key: &str, ro: bool, vid: u32) -> Result<(), String>; /// 单点写：Set 写完整 XValue，并维护目录索引；只解析父路径 link，整键不穿透。 fn set(&mut self, pairs: &[KVPair]) -> Result<(), String>; diff --git a/src/kvspace_common.rs b/src/kvspace_common.rs index 02c3229..b89a046 100644 --- a/src/kvspace_common.rs +++ b/src/kvspace_common.rs @@ -2,10 +2,9 @@ use std::time::Duration; -use crate::kvspace::{KVPair, KVSpace}; +use crate::kvspace::KVSpace; use crate::r#const::*; use crate::xvalue::{body_bytes, is_none, plain, XValue}; -use crate::xvalue_index::new_index; /// JoinPath 拼接父子路径。 pub fn join_path(parent: &str, child: &str) -> String { @@ -29,6 +28,28 @@ pub fn strip_dir_suf(path: &str) -> &str { } } +pub fn is_descendant(base: &str, path: &str) -> bool { + if base == PATH_SEP { + return path != PATH_SEP && path.starts_with(PATH_SEP); + } + path.strip_prefix(base) + .is_some_and(|rest| rest.starts_with(PATH_SEP) || rest.starts_with(OBJ_SEP)) +} + +pub fn is_frame_operand_key(key: &str) -> bool { + if !key.starts_with("/vthread/") { + return false; + } + let slot = key.rsplit('/').next().unwrap_or(""); + let Some(inner) = slot.strip_prefix('[').and_then(|s| s.strip_suffix(']')) else { + return false; + }; + let Some((row, col)) = inner.split_once(',') else { + return false; + }; + row.parse::().is_ok_and(|n| n > 0) && col.parse::().is_ok_and(|n| n != 0) +} + /// SepPath 拆分路径为 (prefix, last)。 pub fn sep_path(path: &str) -> (String, String) { if path == PATH_SEP { @@ -73,42 +94,6 @@ pub fn split_index(path: &str) -> (String, String, SepKind) { (parent, last.to_string(), SepKind::SepDir) } -/// MkIndexRecursive 递归创建目录，已存在的目录跳过。 -/// 用 split_index（而非 sep_path）切父目录，使 `·` 成员目录（如 /lib/math·sum/）按 -/// 其真实父 memindex `/lib/math·` 判定存在性，避免把已有成员目录误判为缺失而重建覆盖。 -pub fn mk_index_recursive(kv: &mut dyn KVSpace, path: &str) { - if !path.ends_with(DIR_INDEX_SUF) { - panic!("MkIndex: path must end with {}", DIR_INDEX_SUF); - } - let mut i = 1; - while i < path.len() { - match path[i..].find('/') { - None => break, - Some(j) => { - i += j + 1; - let dir = &path[..i]; - let (p, n, _) = split_index(&dir[..dir.len() - 1]); - if !dir_exists(kv, &p, &n) { - let _ = kv.set(&[KVPair { - key: dir.to_string(), - val: new_index(&[]), - raw: None, - }]); - } - } - } - } -} - -pub fn dir_exists(kv: &mut dyn KVSpace, parent_dir: &str, name: &str) -> bool { - for m in kv.list(parent_dir, false, true) { - if m == name || m == format!("{}{}", name, DIR_INDEX_SUF) { - return true; - } - } - false -} - /// langtype 归一：**只**剥「数组形状前缀」（`[5]`、`[]`、`[?]`——方括号内是维度数字/空）。 /// `[int64]·int64`、`[float64,float64]·int32` 里的方括号是**元组/标量键类型**，不是形状 /// （见 [[map容器]]：`[float64,float64]` ≠ `[2]float64`），一律原样保留——剥掉会让 map 的 @@ -209,38 +194,21 @@ pub fn equal_xvalue(a: &XValue, b: &XValue) -> bool { body_bytes(a) == body_bytes(b) } -// ── 展示（对齐 FprintList / FprintTree / GetAt / ReadPrefixExt / StripExtChildren） ── +// Display helpers. pub fn get_at(kv: &mut dyn KVSpace, dir: &str, name: &str) -> XValue { kv.get(dir, &[name.to_string()], true).remove(0) } pub fn read_prefix_ext(kv: &mut dyn KVSpace, prefix: &str) -> String { - if let XValue::ExtIndex(e) = get_one(kv, prefix) { - return e.ext_path; + if let XValue::Ext(e) = get_one(kv, prefix) { + return e.locator; } String::new() } -pub fn strip_ext_children( - kv: &mut dyn KVSpace, - prefix: &str, - children: Vec, -) -> Vec { - let ext_target = read_prefix_ext(kv, prefix); - if ext_target.is_empty() { - return children; - } - let ext_children = kv.list(&ext_target, false, true); - let n = children.len().saturating_sub(ext_children.len()); - children[..n].to_vec() -} - pub fn fprint_list(kv: &mut dyn KVSpace, prefix: &str, show_ext: bool, show_kind: bool) { - let mut children = kv.list(prefix, true, true); - if !show_ext { - children = strip_ext_children(kv, prefix, children); - } + let children = kv.list(prefix, show_ext, true); for c in children { let mut v = get_at(kv, prefix, &c); let child_dir = format!("{}{}", join_path(prefix, &c), DIR_INDEX_SUF); @@ -279,10 +247,7 @@ pub fn fprint_tree( show_ext: bool, show_kind: bool, ) { - let mut children = kv.list(prefix, true, true); - if !show_ext { - children = strip_ext_children(kv, prefix, children); - } + let mut children = kv.list(prefix, show_ext, true); children.sort_by(|a, b| { let (a_, b_) = (a.trim_end_matches('/'), b.trim_end_matches('/')); if a_ == b_ { diff --git a/src/lib.rs b/src/lib.rs index cdb2354..3758598 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -7,17 +7,17 @@ pub mod xvalue; pub mod xvalue_bool; pub mod xvalue_byte; pub mod xvalue_float; -pub mod xvalue_index; pub mod xvalue_int; -pub mod xvalue_obj; pub mod xvalue_uint; pub mod backend; pub mod conn; pub mod ffi; pub mod fs; +pub mod headlenpow; pub mod kvspace; pub mod kvspace_common; +pub mod metadata; pub mod redis; pub mod s3; pub mod store; @@ -28,9 +28,7 @@ pub use xvalue::*; pub use xvalue_bool::*; pub use xvalue_byte::*; pub use xvalue_float::*; -pub use xvalue_index::*; pub use xvalue_int::*; -pub use xvalue_obj::*; pub use xvalue_uint::*; pub use backend::Backend; diff --git a/src/metadata.rs b/src/metadata.rs new file mode 100644 index 0000000..5568462 --- /dev/null +++ b/src/metadata.rs @@ -0,0 +1,77 @@ +use crate::headlenpow; +use crate::r#const::META_ROOT_NAME; + +pub fn key_for(key: &str) -> Option { + if !key.starts_with('/') || is_reserved(key) { + return None; + } + let mut out = format!("/{META_ROOT_NAME}/"); + const HEX: &[u8; 16] = b"0123456789abcdef"; + for b in key.bytes() { + out.push(HEX[(b >> 4) as usize] as char); + out.push(HEX[(b & 15) as usize] as char); + } + Some(out) +} + +pub fn is_reserved(key: &str) -> bool { + key == "/.kvspace-meta" || key.starts_with("/.kvspace-meta/") +} + +pub fn original_key(hex: &str) -> Option { + if hex.len() % 2 != 0 { + return None; + } + let mut bytes = Vec::with_capacity(hex.len() / 2); + for pair in hex.as_bytes().chunks_exact(2) { + let hi = (pair[0] as char).to_digit(16)?; + let lo = (pair[1] as char).to_digit(16)?; + bytes.push(((hi << 4) | lo) as u8); + } + let key = String::from_utf8(bytes).ok()?; + if key.starts_with('/') && !is_reserved(&key) { + Some(key) + } else { + None + } +} + +pub fn encode(ro: bool, vid: u32) -> Option> { + let mut body = [0u8; 5]; + body[0] = u8::from(ro); + body[1..].copy_from_slice(&vid.to_le_bytes()); + headlenpow::encode(6, 1, 5, 5, "[5]byte", &body, 5) +} + +pub fn decode(data: &[u8]) -> Option<(bool, u32)> { + let h = headlenpow::decode(data)?; + if h.total != data.len() + || h.flags != 1 + || h.a != 5 + || h.b != 5 + || h.langtype != "[5]byte" + || h.body[0] > 1 + { + return None; + } + Some(( + h.body[0] == 1, + u32::from_le_bytes(h.body[1..5].try_into().ok()?), + )) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn metadata_roundtrip() { + assert_eq!(key_for("/a").as_deref(), Some("/.kvspace-meta/2f61")); + assert_eq!(key_for("/.kvspace-meta/2f61"), None); + assert_eq!(original_key("2f61").as_deref(), Some("/a")); + let mut data = encode(true, 0x12345678).unwrap(); + assert_eq!(decode(&data), Some((true, 0x12345678))); + data[64] = 2; + assert_eq!(decode(&data), None); + } +} diff --git a/src/redis/store.rs b/src/redis/store.rs index f62b662..7dc1bf9 100644 --- a/src/redis/store.rs +++ b/src/redis/store.rs @@ -4,7 +4,7 @@ use std::cell::RefCell; use std::io::{BufRead, BufReader, Read, Write}; use std::net::TcpStream; -use crate::store::KVStore; +use crate::store::{is_subtree_key, KVStore}; pub struct RedisStore { stream: RefCell>, @@ -48,7 +48,11 @@ impl RedisStore { .write_all(&out) .unwrap_or_else(|e| panic!("kvspace-redis: write: {}", e)); } - self.read_resp() + let response = self.read_resp(); + if let Resp::Error(message) = &response { + panic!("kvspace-redis: {message}"); + } + response } fn read_line(&self) -> Vec { @@ -179,6 +183,40 @@ impl KVStore for RedisStore { let _ = self.cmd(&args); } + fn scan_keys(&self, prefix: &str) -> Vec { + let mut keys = Vec::new(); + let mut cursor: i64 = 0; + loop { + let cur = cursor.to_string(); + match self.cmd(&[b"SCAN", cur.as_bytes(), b"COUNT", b"1000"]) { + Resp::Array(parts) if parts.len() == 2 => { + cursor = match &parts[0] { + Resp::Bulk(Some(bytes)) => { + String::from_utf8_lossy(bytes).parse().unwrap_or(0) + } + _ => 0, + }; + if let Resp::Array(items) = &parts[1] { + for item in items { + if let Resp::Bulk(Some(bytes)) = item { + if let Ok(key) = String::from_utf8(bytes.clone()) { + if is_subtree_key(prefix, &key) { + keys.push(key); + } + } + } + } + } + if cursor == 0 { + break; + } + } + _ => break, + } + } + keys + } + fn flush(&self) { let _ = self.cmd(&[b"FLUSHDB"]); } diff --git a/src/s3/store.rs b/src/s3/store.rs index 8f86626..dcab989 100644 --- a/src/s3/store.rs +++ b/src/s3/store.rs @@ -8,7 +8,7 @@ use std::time::Duration; use super::dsn::{self, S3Dsn}; use super::sigv4::{self, uri_encode}; -use crate::store::KVStore; +use crate::store::{is_subtree_key, KVStore}; /// 5xx 与网络错误重试几次。4xx 一律不重试 —— 签名错、桶不存在这类问题重试一万次也不会好。 const MAX_ATTEMPTS: u32 = 3; @@ -51,33 +51,33 @@ fn hex_of(body: &[u8]) -> String { fn xml_unescape(s: &str) -> String { let mut out = String::with_capacity(s.len()); - let b = s.as_bytes(); - let mut i = 0; - while i < b.len() { - if b[i] == b'&' { - if let Some(semi) = s[i..].find(';') { - let ent = &s[i + 1..i + semi]; - let rep = match ent { - "amp" => Some('&'), - "lt" => Some('<'), - "gt" => Some('>'), - "quot" => Some('"'), - "apos" => Some('\''), - _ => ent - .strip_prefix('#') - .and_then(|n| n.parse::().ok()) - .and_then(char::from_u32), - }; - if let Some(c) = rep { - out.push(c); - i += semi + 1; - continue; - } + let mut rest = s; + while let Some(amp) = rest.find('&') { + out.push_str(&rest[..amp]); + rest = &rest[amp..]; + if let Some(semi) = rest.find(';') { + let ent = &rest[1..semi]; + let decoded = match ent { + "amp" => Some('&'), + "lt" => Some('<'), + "gt" => Some('>'), + "quot" => Some('"'), + "apos" => Some('\''), + _ => ent + .strip_prefix('#') + .and_then(|n| n.parse::().ok()) + .and_then(char::from_u32), + }; + if let Some(ch) = decoded { + out.push(ch); + rest = &rest[semi + 1..]; + continue; } } - out.push(b[i] as char); - i += 1; + out.push('&'); + rest = &rest[1..]; } + out.push_str(rest); out } @@ -328,6 +328,13 @@ impl KVStore for S3Store { } } + fn scan_keys(&self, prefix: &str) -> Vec { + self.list_all(prefix) + .into_iter() + .filter(|key| is_subtree_key(prefix, key)) + .collect() + } + fn exists(&self, key: &str) -> bool { // HEAD 不传 body，比整读便宜得多。这是覆盖默认实现最值的一处。 match self.send_empty("HEAD", key, &[], None) { @@ -392,6 +399,7 @@ mod tests { fn xml_数字实体() { assert_eq!(xml_unescape("好"), "好"); assert_eq!(xml_unescape("&lt;"), "<"); // 只解一层 + assert_eq!(xml_unescape("/f·键‥0&x"), "/f·键‥0&x"); } #[test] diff --git a/src/store.rs b/src/store.rs index c160a5f..3d14242 100644 --- a/src/store.rs +++ b/src/store.rs @@ -1,11 +1,11 @@ -// store.rs — 后端存储原语。redis/fs 各自实现，generic backend 只依赖它。 -// -// 注意这里**没有** `scan_keys` 这类「前缀扫描」原语。子树枚举由 `Backend::collect_subtree` -// 沿目录索引递归完成 —— 后端只需要 get/set/del 这几个点操作。 -// 这样 S3 后端不必依赖 `ListObjects(Prefix=)`（那是裸字节前缀，与 KVSpace 要的词边界 -// 匹配不是一回事），也不必为「有没有前缀扫描能力」做适配。 +pub(crate) fn is_subtree_key(prefix: &str, key: &str) -> bool { + if prefix.is_empty() || prefix == "/" { + return key.starts_with('/'); + } + key.strip_prefix(prefix) + .is_some_and(|rest| rest.is_empty() || rest.starts_with('/') || rest.starts_with('·')) +} -/// 单 key 字节级存储原语（无目录索引、无 link 语义，纯 get/set/del/scan/flush）。 pub trait KVStore { /// 读 key 的原始字节；None = 不存在。 fn get(&self, key: &str) -> Option>; @@ -15,6 +15,7 @@ pub trait KVStore { } fn set(&self, key: &str, val: &[u8]); fn del(&self, keys: &[&str]); + fn scan_keys(&self, prefix: &str) -> Vec; /// key 是否存在。默认整读判定，后端可覆盖为 EXISTS。 fn exists(&self, key: &str) -> bool { self.get(key).is_some() @@ -41,3 +42,18 @@ pub trait KVStore { } fn flush(&self); } + +#[cfg(test)] +mod tests { + use super::is_subtree_key; + + #[test] + fn subtree_scan_respects_key_boundaries() { + assert!(is_subtree_key("/proto", "/proto")); + assert!(is_subtree_key("/proto", "/proto/member")); + assert!(is_subtree_key("/proto", "/proto·field")); + assert!(!is_subtree_key("/proto", "/protocol")); + assert!(is_subtree_key("/", "/proto/member")); + assert!(is_subtree_key("", "/proto/member")); + } +} diff --git a/src/xvalue.rs b/src/xvalue.rs index 37f686b..c3a0c0c 100644 --- a/src/xvalue.rs +++ b/src/xvalue.rs @@ -4,102 +4,14 @@ use crate::r#const::*; -// ── XValueHead（三正交轴 ref × storetype × langtype，对齐 kvspace/frontend.c 黄金基准）──────────── -// head = [headlen u16 LE][ref u8][storetype u8][ro u8][vid u32 LE][body_len u32 LE] -// [storetype 物理字段][langtype kindexpr 串（占至 headlen）] -// body = [body_len B raw] -// ref 0=inline（body=值本体）/1=ptr（body=目标 key）/2=@ext（body=扩展定位符） -// storetype 物理布局（codec 唯一分派）：NONE/ATOM/ARRAYND/index/extindex。 -// 物理字段：ARRAYND / index / extindex 为 ndim u8 + dims[ndim] u32 LE -// （index/extindex 的 dims=[len,cap,M]）；NONE / ATOM 无物理字段。 -// langtype 完整 kindexpr 串（含 [dims]、无前缀），恒为 head 最后一段（长度 = headlen − 当前偏移）。 - -pub const HEAD_PREFIX: usize = 13; // headlen(2)+ref(1)+storetype(1)+ro(1)+vid(4)+body_len(4) - pub const REF_INLINE: u8 = 0; pub const REF_PTR: u8 = 1; pub const REF_EXT: u8 = 2; -pub const STORETYPE_NONE: u8 = 0; -pub const STORETYPE_ATOM: u8 = 1; -pub const STORETYPE_ARRAYND: u8 = 2; -pub const STORETYPE_INDEX: u8 = 3; -pub const STORETYPE_EXTINDEX: u8 = 4; - -/// ARRAYND / index / extindex 携带 ndim+dims 物理字段。 -pub fn store_has_dims(st: u8) -> bool { - st == STORETYPE_ARRAYND || st == STORETYPE_INDEX || st == STORETYPE_EXTINDEX -} - -fn is_index_kind(kind: &str) -> bool { - kind == KIND_INDEX || kind == KIND_EXT_INDEX || kind == "rwfunc" || kind == "def rwir" -} - -/// 由 base 种类名（+ndim）推 storetype。 -fn storetype_of(kind: &str, ndim: i32) -> u8 { - if kind.is_empty() { - return STORETYPE_NONE; - } - if kind == KIND_EXT_INDEX { - return STORETYPE_EXTINDEX; - } - if is_index_kind(kind) || is_map_langtype(kind) { - return STORETYPE_INDEX; - } - if ndim > 0 { - return STORETYPE_ARRAYND; - } - STORETYPE_ATOM -} - -/// map langtype：`{memitemkeylangtype}·{memitemvaluelangtype}`。值容器的物理布局恒 index -/// （成员名索引落兄弟槽 `{key}·`，见 [[map容器]]），与标量/张量截然不同。 pub fn is_map_langtype(kind: &str) -> bool { kind.contains(OBJ_SEP) } -/// 指针 head 的 storetype = 目标语义 storetype（据目标完整 kindexpr 推；指针自身物理字段恒空）。 -fn storetype_from_kindexpr(kx: &str) -> u8 { - if kx.is_empty() { - return STORETYPE_NONE; - } - let (has_dims, base) = if kx.starts_with('[') { - match kx.find(']') { - Some(e) => (true, &kx[e + 1..]), - None => (false, kx), - } - } else { - (false, kx) - }; - if base == KIND_EXT_INDEX { - return STORETYPE_EXTINDEX; - } - if is_index_kind(base) || base.starts_with('/') || base.contains(OBJ_SEP) { - return STORETYPE_INDEX; - } - if has_dims { - return STORETYPE_ARRAYND; - } - STORETYPE_ATOM -} - -/// langtype 串（ARRAYND 含 [dims]，其余为裸种类名/路径）。 -fn build_langtype(kind: &str, storetype: u8, dims: &[i32]) -> String { - let mut s = String::new(); - if storetype == STORETYPE_ARRAYND && !dims.is_empty() { - s.push('['); - for (i, d) in dims.iter().enumerate() { - if i > 0 { - s.push(','); - } - s.push_str(&d.to_string()); - } - s.push(']'); - } - s.push_str(kind); - s -} - /// langtype 解析 → (dims, kind)：`[dims]` 段表形状，其余为基 kind。 /// /// **map langtype 无形状段**：`{memitemkeylangtype}·{memitemvaluelangtype}`（见 [[map容器]]）里 @@ -138,13 +50,14 @@ pub(crate) fn parse_kindexpr(s: &str) -> (Vec, String) { #[derive(Default, Clone, Debug, PartialEq)] pub struct XValueHead { pub headlen: u16, - pub r#ref: u8, // 存储位置：REF_* - pub storetype: u8, // 物理布局：STORETYPE_* - pub langtype: String, // 完整 kindexpr（含 [dims]、无前缀） - pub phys_dims: Vec, // 物理字段 dims：ARRAYND=形状、index/extindex=[len,cap,M] + pub r#ref: u8, + pub storetype: u8, // Storage class. + pub langtype: String, + pub phys_dims: Vec, pub ro: bool, pub vid: u32, pub body_len: i32, + pub body_cap: u64, } impl XValueHead { @@ -193,14 +106,17 @@ impl XValueHead { target: String::from_utf8_lossy(body).into_owned(), }); } + if self.r#ref == REF_EXT { + return XValue::Ext(ExtHandle { + langtype: self.langtype.clone(), + locator: String::from_utf8_lossy(body).into_owned(), + }); + } let kind = self.kind(); let dims = self.dims(); - // 值容器：langtype 即完整 map langtype（`{keylt}·{valt}lt`），storetype=index，主槽 body 空。 + // Map members live under the physical member prefix. if is_map_langtype(&kind) { - return XValue::Map(MapValue { - langtype: kind, - dims, - }); + return XValue::Map(MapValue { langtype: kind }); } match kind.as_str() { KIND_BOOL => XValue::Bool(crate::xvalue_bool::decode_bool(body, &dims)), @@ -219,12 +135,6 @@ impl XValueHead { XValue::CharAscii(crate::xvalue_byte::decode_char_ascii(body, &dims)) } KIND_CHAR => XValue::Char32(crate::xvalue_byte::decode_char32(body, &dims)), - KIND_MAP => XValue::Map(MapValue { - langtype: kind.clone(), - dims: dims.clone(), - }), - KIND_INDEX => XValue::Index(crate::xvalue_index::decode_index(body, &dims)), - KIND_EXT_INDEX => XValue::ExtIndex(crate::xvalue_index::decode_ext_index(body, &dims)), _ => XValue::Opaque(Opaque { kind: kind.clone(), body: body.to_vec(), @@ -272,10 +182,9 @@ pub enum XValue { CharByte(Arr), // char/utf8，1B×N CharAscii(Arr), // char/ascii，1B×N Char32(Arr), // char/utf32，码点，4B×N - Map(MapValue), // stringkeymap 值容器：langtype 是完整 map langtype，成员在 memindex（p·） - Index(Vec), // index - ExtIndex(ExtIndex), // extindex - Opaque(Opaque), // 未知 kind（如 kvlang 的 rwir/rwfunc/scope），原样存取 + Map(MapValue), + Ext(ExtHandle), + Opaque(Opaque), } impl XValue { @@ -298,8 +207,7 @@ impl XValue { XValue::CharAscii(_) => KIND_CHAR_ASCII, XValue::Char32(_) => KIND_CHAR, XValue::Map(m) => m.langtype.as_str(), - XValue::Index(_) => KIND_INDEX, - XValue::ExtIndex(_) => KIND_EXT_INDEX, + XValue::Ext(e) => e.langtype.as_str(), XValue::Opaque(o) => o.kind.as_str(), } } @@ -326,16 +234,14 @@ impl XValue { XValue::CharByte(d) => d.data.len() as i32, XValue::CharAscii(d) => d.data.len() as i32, XValue::Char32(d) => d.data.len() as i32, - XValue::Map(m) => m.dims.iter().product(), - XValue::Index(_) => 1, - XValue::ExtIndex(_) => 1, + XValue::Map(_) | XValue::Ext(_) => 1, XValue::Opaque(o) => o.array_len, } } pub fn encode(&self) -> Vec { match self { - XValue::None => Vec::new(), + XValue::None => encode_head("None", 0, &[], &[]), XValue::Ptr(p) => tlv_encode_ptr(&p.target_kindexpr, p.target.as_bytes()), XValue::Bool(d) => crate::xvalue_bool::encode_bool(&d.data, &d.dims), XValue::Int8(d) => crate::xvalue_int::encode_int8(&d.data, &d.dims), @@ -351,15 +257,8 @@ impl XValue { XValue::CharByte(d) => crate::xvalue_byte::encode_char_byte(&d.data, &d.dims), XValue::CharAscii(d) => crate::xvalue_byte::encode_char_ascii(&d.data, &d.dims), XValue::Char32(d) => crate::xvalue_byte::encode_char32(&d.data, &d.dims), - XValue::Map(m) => encode_head(&m.langtype, 0, &m.dims, &[]), - XValue::Index(d) => { - let (dims, body) = crate::xvalue_index::encode_index(d); - encode_head(KIND_INDEX, 0, &dims, &body) - } - XValue::ExtIndex(e) => { - let (dims, body) = crate::xvalue_index::encode_ext_index(&e.ext_path, &e.childs); - encode_head(KIND_EXT_INDEX, 0, &dims, &body) - } + XValue::Map(m) => encode_head(&m.langtype, 0, &[], &[]), + XValue::Ext(e) => encode_head(&e.langtype, 2, &[], e.locator.as_bytes()), XValue::Opaque(o) => tlv_encode(&o.kind, &o.body, o.array_len), } } @@ -386,16 +285,8 @@ impl XValue { .iter() .map(|&c| char::from_u32(c).unwrap_or('\u{FFFD}')) .collect(), - XValue::Map(m) => format!( - "map[{}]", - m.dims - .iter() - .map(|d| d.to_string()) - .collect::>() - .join(",") - ), - XValue::Index(d) => index_value_string(d), - XValue::ExtIndex(e) => e.value_string(), + XValue::Map(_) => "map".to_string(), + XValue::Ext(e) => e.locator.clone(), XValue::Opaque(o) => String::from_utf8_lossy(&o.body).into_owned(), } } @@ -416,12 +307,22 @@ impl std::fmt::Display for XValue { } // ── Map 值容器 ───────────────────────────────────────────────────────────── -/// stringkeymap 值容器：`langtype` = 完整 map langtype（`{memitemkeylangtype}·{memitemvaluelangtype}`， -/// 见 [[map容器]]），恒非空；`dims` = 逻辑形状（无形状的空容器为 [0]），成员名索引落兄弟槽 `{key}·`。 +/// A map value stores only its full key/value langtype. #[derive(Clone, Debug, PartialEq)] pub struct MapValue { pub langtype: String, - pub dims: Vec, +} + +#[derive(Clone, Debug, PartialEq)] +pub struct ExtHandle { + pub langtype: String, + pub locator: String, +} + +pub fn new_map_langtype(langtype: &str) -> XValue { + XValue::Map(MapValue { + langtype: langtype.to_string(), + }) } // ── Ptr ──────────────────────────────────────────────────────────────────── @@ -447,26 +348,6 @@ pub fn ptr_target(v: &XValue) -> String { } } -// ── Dict / Index / ExtIndex 结构 ────────────────────────────────────────── - -#[derive(Clone, Debug, PartialEq)] -pub struct ExtIndex { - pub childs: Vec, - pub ext_path: String, -} - -impl ExtIndex { - pub fn value_string(&self) -> String { - if !self.ext_path.is_empty() { - format!("({}) …{}", self.childs.len(), self.ext_path) - } else if self.childs.is_empty() { - "(empty ext)".to_string() - } else { - format!("({})", self.childs.len()) - } - } -} - /// 未知 kind（非标准 XValue）的原样字节，供上层自定义 kind（如 kvlang 的 rwir/rwfunc）存取值。 #[derive(Clone, Debug, PartialEq)] pub struct Opaque { @@ -475,14 +356,6 @@ pub struct Opaque { pub array_len: i32, } -fn index_value_string(childs: &[String]) -> String { - if childs.len() == 1 && childs[0].is_empty() { - "(empty)".to_string() - } else { - format!("({})", childs.len()) - } -} - // ── 工具函数 ────────────────────────────────────────────────────────────── pub fn is_none(v: &XValue) -> bool { @@ -511,10 +384,7 @@ fn bool_string(b: bool) -> String { } } -// ── 三正交轴编解码（byte-identical 于 kvspace/frontend.c）─────────────────────────────── -// head = [headlen u16 LE][ref u8][storetype u8][ro u8][vid u32 LE][body_len u32 LE] -// [storetype 物理字段（ARRAYND/index/extindex 为 ndim u8 + dims[ndim] u32 LE）][langtype] -// body = [body_len B raw]，offset = headlen。None 编码为 nil。 +// Headlenpow XValue encoding. pub fn tlv_encode(kind: &str, raw: &[u8], array_len: i32) -> Vec { encode_head(kind, 0, &array_to_header(kind, array_len), raw) @@ -538,148 +408,118 @@ fn array_to_header(kind: &str, array_len: i32) -> Vec { } pub fn encode_head(kind: &str, r#ref: i32, dims: &[i32], raw: &[u8]) -> Vec { - encode_head_perm(kind, r#ref, dims, raw, false, 0) -} - -pub fn encode_head_perm( - kind: &str, - r#ref: i32, - dims: &[i32], - raw: &[u8], - ro: bool, - vid: u32, -) -> Vec { - // ptr：kind 参数即目标完整 kindexpr，storetype 从其推、物理字段恒空、langtype 原样。 - let (storetype, langtype, phys): (u8, String, Vec) = if r#ref == 1 { - (storetype_from_kindexpr(kind), kind.to_string(), Vec::new()) - } else { - let st = storetype_of(kind, dims.len() as i32); - let lt = build_langtype(kind, st, dims); - let pd = if store_has_dims(st) { - dims.to_vec() - } else { - Vec::new() + let (pow, flags, a, b, langtype) = if r#ref == 1 || r#ref == 2 { + let pow = (5..=31).find(|&p| kind.len() + crate::headlenpow::PREFIX <= 1usize << p); + let Some(pow) = pow else { return Vec::new() }; + ( + pow, + if r#ref == 1 { 5 } else { 3 }, + raw.len() as u64, + raw.len() as u64, + kind.to_string(), + ) + } else if r#ref != 0 { + return Vec::new(); + } else if kind.is_empty() || kind == "None" { + (5, 0, 0, 0, String::new()) + } else if matches!(kind, "char/utf8" | "char/ascii" | "char/utf32" | "byte") { + let count = match kind { + "char/utf8" => std::str::from_utf8(raw).ok().map(|s| s.chars().count()), + "char/ascii" if raw.is_ascii() => Some(raw.len()), + "char/utf32" + if raw.len() % 4 == 0 + && raw.chunks_exact(4).all(|c| { + char::from_u32(u32::from_le_bytes(c.try_into().unwrap())).is_some() + }) => + { + Some(raw.len() / 4) + } + "byte" => Some(raw.len()), + _ => None, }; - (st, lt, pd) - }; - let lt_bytes = langtype.as_bytes(); - let phys_bytes = if store_has_dims(storetype) { - 1 + 4 * phys.len() + let Some(count) = count else { + return Vec::new(); + }; + ( + 6, + 1, + raw.len() as u64, + raw.len() as u64, + format!("[{count}]{kind}"), + ) + } else if matches!(kind, "rwir" | "rwfunc") && raw.len() >= 5 || kind == "def langtype" { + (5, 1, raw.len() as u64, raw.len() as u64, kind.to_string()) + } else if !dims.is_empty() && !kind.contains('·') && !kind.starts_with('/') { + let Some(numel) = dims.iter().try_fold(1u64, |n, &d| { + u64::try_from(d).ok().and_then(|d| n.checked_mul(d)) + }) else { + return Vec::new(); + }; + let width = elem_size(kind); + if width <= 0 { + return Vec::new(); + } + ( + 7, + 2, + numel, + width as u64, + format!( + "[{}]{kind}", + dims.iter() + .map(i32::to_string) + .collect::>() + .join(",") + ), + ) } else { - 0 + let pow = (5..=31).find(|&p| kind.len() + crate::headlenpow::PREFIX <= 1usize << p); + let Some(pow) = pow else { return Vec::new() }; + (pow, 0, 0, 0, kind.to_string()) }; - let headlen = HEAD_PREFIX + phys_bytes + lt_bytes.len(); - let mut buf = vec![0u8; headlen + raw.len()]; - buf[0..2].copy_from_slice(&(headlen as u16).to_le_bytes()); - buf[2] = r#ref as u8; - buf[3] = storetype; - buf[4] = ro as u8; - buf[5..9].copy_from_slice(&vid.to_le_bytes()); - buf[9..13].copy_from_slice(&(raw.len() as u32).to_le_bytes()); - let mut o = HEAD_PREFIX; - if store_has_dims(storetype) { - buf[o] = phys.len() as u8; - o += 1; - for d in &phys { - buf[o..o + 4].copy_from_slice(&(*d as u32).to_le_bytes()); - o += 4; - } - } - buf[o..o + lt_bytes.len()].copy_from_slice(lt_bytes); - buf[headlen..].copy_from_slice(raw); - buf -} - -/// 只解 head、**不要求 body 到齐**：供 `kvspaceGetHead` 这类只读值前缀的调用点用 -/// （它按约定只取前若干字节，body 本就不在手上）。整值调用点仍走 [`decode_xvalue_head`]， -/// 那里的「headlen + body_len ≤ data.len()」是防截断校验。 -pub fn decode_xvalue_head_prefix(data: &[u8]) -> XValueHead { - if data.len() < HEAD_PREFIX { - return XValueHead::default(); - } - let headlen = u16::from_le_bytes(data[0..2].try_into().unwrap()) as usize; - let r#ref = data[2]; - let storetype = data[3]; - let ro = data[4] != 0; - let vid = u32::from_le_bytes(data[5..9].try_into().unwrap()); - let body_len = u32::from_le_bytes(data[9..13].try_into().unwrap()) as i32; - if headlen < HEAD_PREFIX || data.len() < headlen { - return XValueHead::default(); - } - let mut o = HEAD_PREFIX; - let mut phys_dims = Vec::new(); - if store_has_dims(storetype) { - let ndim = data[o] as usize; - o += 1; - for _ in 0..ndim { - if o + 4 > headlen { - break; - } - phys_dims.push(i32::from_le_bytes(data[o..o + 4].try_into().unwrap())); - o += 4; - } - } - let langtype = String::from_utf8_lossy(&data[o..headlen]).into_owned(); - XValueHead { - headlen: headlen as u16, - r#ref, - storetype, - langtype, - phys_dims, - ro, - vid, - body_len, - } + crate::headlenpow::encode(pow, flags, a, b, &langtype, raw, raw.len()).unwrap_or_default() } pub fn decode_xvalue_head(data: &[u8]) -> XValueHead { - if data.len() < HEAD_PREFIX { + let Some(h) = crate::headlenpow::decode(data) else { return XValueHead::default(); - } - let headlen = u16::from_le_bytes(data[0..2].try_into().unwrap()) as usize; - let r#ref = data[2]; - let storetype = data[3]; - let ro = data[4] != 0; - let vid = u32::from_le_bytes(data[5..9].try_into().unwrap()); - let body_len = u32::from_le_bytes(data[9..13].try_into().unwrap()) as i32; - if headlen < HEAD_PREFIX || data.len() < headlen { + }; + if h.total != data.len() { return XValueHead::default(); } - let mut o = HEAD_PREFIX; - let mut phys_dims = Vec::new(); - if store_has_dims(storetype) { - let ndim = data[o] as usize; - o += 1; - for _ in 0..ndim { - if o + 4 > headlen { - break; - } - phys_dims.push(i32::from_le_bytes(data[o..o + 4].try_into().unwrap())); - o += 4; - } - } - let langtype = String::from_utf8_lossy(&data[o..headlen]).into_owned(); - if data.len() < headlen + body_len as usize { + let (Ok(headlen), Ok(body_len)) = + (u16::try_from(1usize << h.pow), i32::try_from(h.content_len)) + else { return XValueHead::default(); - } + }; + let dims = if h.flags == 2 || (h.flags == 1 && h.langtype.starts_with('[')) { + parse_kindexpr(h.langtype).0 + } else { + Vec::new() + }; XValueHead { - headlen: headlen as u16, - r#ref, - storetype, - langtype, - phys_dims, - ro, - vid, + headlen, + r#ref: if h.flags == 5 { + REF_PTR + } else if h.flags == 3 { + REF_EXT + } else { + REF_INLINE + }, + storetype: h.flags & 3, + langtype: h.langtype.to_string(), + phys_dims: dims, + ro: false, + vid: 0, body_len, + body_cap: (h.total - headlen as usize) as u64, } } /// 解析完整 XValue（head + body）为 XValue。 pub fn decode_xvalue(data: &[u8]) -> XValue { let h = decode_xvalue_head(data); - // 空 TLV / 空 langtype 的**非指针**值 = None（写 None 落 1 字节空 kind TLV）。 - // 指针（ref=1）例外：它的 langtype 可能为空（如实参为无值容器时 runtime 推不出类型）， - // 但空 langtype 的 Ptr 仍是 Ptr——引用一个键，不是"无值"。否则指针会被读成 None。 + // An empty inline langtype denotes None. if h.langtype.is_empty() && !h.is_ptr() { return XValue::None; } @@ -746,8 +586,17 @@ mod tests { ("float64", vec![2, 3]), ("char/utf32", vec![0]), ] { - let st = storetype_of(kind, dims.len() as i32); - let s = build_langtype(kind, st, &dims); + let s = if dims.is_empty() { + kind.to_string() + } else { + format!( + "[{}]{kind}", + dims.iter() + .map(i32::to_string) + .collect::>() + .join(",") + ) + }; let (d2, k2) = parse_kindexpr(&s); assert_eq!((dims, kind.to_string()), (d2, k2), "langtype {}", s); } @@ -768,51 +617,25 @@ mod tests { } #[test] - fn index_matrix_roundtrip() { - use crate::xvalue_index::{matrix_at, matrix_count}; - // 乱序输入 → encode 规范排序（坐标数值序，非字节序：[2] 在 [10] 前）。 - let v = XValue::Index(vec!["[10]".into(), "[2]".into(), "[1]".into()]); - let bytes = v.encode(); - let h = decode_xvalue_head(&bytes); - assert_eq!(h.kind(), KIND_INDEX); - assert_eq!(h.dims(), vec![3, 3, 8]); // len=3, cap=3, M=align8(len("[10]")=4)=8 - let decoded = decode_xvalue(&bytes); + fn directory_and_extension_wire() { + let dir = encode_head("lib", 0, &[], &[]); + let head = crate::headlenpow::decode(&dir).unwrap(); + assert_eq!((head.flags, head.langtype, head.content_len), (0, "lib", 0)); + let ext = XValue::Ext(ExtHandle { + langtype: "rwfunc".into(), + locator: "/lib/f/".into(), + }) + .encode(); + let head = crate::headlenpow::decode(&ext).unwrap(); assert_eq!( - decoded, - XValue::Index(vec!["[1]".into(), "[2]".into(), "[10]".into()]) + (head.flags, head.langtype, head.body), + (3, "rwfunc", b"/lib/f/".as_slice()) ); - // O(1) 原语走 head+body。 - let body = h.body(&bytes); - assert_eq!(matrix_count(&h.dims()), 3); - assert_eq!(matrix_at(body, 8, 0).as_deref(), Some("[1]")); - assert_eq!(matrix_at(body, 8, 2).as_deref(), Some("[10]")); - assert_eq!(matrix_at(body, 8, 3), None); - } - - #[test] - fn index_empty() { - let bytes = XValue::Index(vec![]).encode(); - let h = decode_xvalue_head(&bytes); - assert_eq!(h.dims(), vec![0, 0, 0]); - assert_eq!(decode_xvalue(&bytes), XValue::Index(vec![])); - } - - #[test] - fn ext_index_roundtrip() { - // ext_path 置 body 头部、childs 尾部矩阵（规范排序）。 - let v = XValue::ExtIndex(ExtIndex { - childs: vec!["b".into(), "a".into()], - ext_path: "/lib/main·add/".into(), - }); - let bytes = v.encode(); - let h = decode_xvalue_head(&bytes); - assert_eq!(h.kind(), KIND_EXT_INDEX); - assert_eq!(h.dims(), vec![2, 2, 8]); // len=2, cap=2, M=align8(1)=8 assert_eq!( - decode_xvalue(&bytes), - XValue::ExtIndex(ExtIndex { - childs: vec!["a".into(), "b".into()], - ext_path: "/lib/main·add/".into(), + decode_xvalue(&ext), + XValue::Ext(ExtHandle { + langtype: "rwfunc".into(), + locator: "/lib/f/".into() }) ); } diff --git a/src/xvalue_index.rs b/src/xvalue_index.rs deleted file mode 100644 index 6c06346..0000000 --- a/src/xvalue_index.rs +++ /dev/null @@ -1,192 +0,0 @@ -// xvalue_index.rs — memindex 定宽排序矩阵（index / object / stringkeymap / extindex 共用） - -use crate::coord::cmp_coord; -use crate::r#const::KIND_MAP; -use crate::xvalue::{ExtIndex, MapValue, XValue}; - -pub fn new_index(children: &[String]) -> XValue { - XValue::Index(children.to_vec()) -} -/// stringkeymap 值容器：langtype 为完整 map langtype（逐字，见 [[map容器]]）。 -pub fn new_map_langtype(langtype: &str, dims: &[i32]) -> XValue { - XValue::Map(MapValue { - langtype: langtype.to_string(), - dims: dims.to_vec(), - }) -} -/// 无类型标注的散 key 字面量（`{v0,v1,…}`）：容器值 langtype 退化为种类名 `stringkeymap`。 -pub fn new_map_index(dims: &[i32]) -> XValue { - new_map_langtype(KIND_MAP, dims) -} -pub fn new_ext_index(children: &[String], ext_path: &str) -> XValue { - XValue::ExtIndex(ExtIndex { - childs: children.to_vec(), - ext_path: ext_path.to_string(), - }) -} - -// 成员名单编码为定宽排序矩阵（Go-slice cap/len 语义），几何 [len,cap,M] 落 XValueHead.dims： -// len = 有效成员数（listlen，O(1)）；cap ≥ len = 预留行数；M = 成员 UTF-8 字节最大长向上 8 对齐（行宽）。 -// body = cap×M：前 len 行 = 成员名 UTF-8 + NUL 补齐到 M（cmp_coord 有序），后 cap−len 行全 NUL。 -// 容量内增删 body 长度恒 cap×M → 就地覆写不重分配（减少扩容）；满则 cap 翻倍。 -// 全表规范排序 → 三后端 blob 逐字节一致，listat/listlen O(1)、成员二分 O(log len)。 - -/// 行宽向上 8 字节对齐。 -pub fn align8(n: usize) -> usize { - (n + 7) & !7 -} - -/// cap 增长：容量足够则不变；空则取 need；否则从旧容量翻倍直到覆盖 need（默认 2×）。 -pub fn grow_cap(old_cap: usize, need: usize) -> usize { - if need <= old_cap { - old_cap - } else if old_cap == 0 { - need - } else { - let mut c = old_cap; - while c < need { - c *= 2; - } - c - } -} - -/// 成员名单 → (dims=[len,cap,M], body=cap×M)。cap_hint/m_hint 为下限（只增不减，用于保留预留容量 -/// 与既有行宽）；encode 侧规范排序，调用方无需预排。 -pub fn encode_index_grow( - children: &[String], - cap_hint: usize, - m_hint: usize, -) -> (Vec, Vec) { - let mut c: Vec<&str> = children.iter().map(|s| s.as_str()).collect(); - c.sort_by(|a, b| cmp_coord(a, b)); - let len = c.len(); - let m = m_hint.max(align8(c.iter().map(|s| s.len()).max().unwrap_or(0))); - let cap = cap_hint.max(len); - let mut body = vec![0u8; cap * m]; - for (i, s) in c.iter().enumerate() { - body[i * m..i * m + s.len()].copy_from_slice(s.as_bytes()); - } - (vec![len as i32, cap as i32, m as i32], body) -} - -/// 成员名单 → 紧凑编码（cap=len）。用于直接 set/新建。 -pub fn encode_index(children: &[String]) -> (Vec, Vec) { - encode_index_grow(children, 0, 0) -} - -/// 定宽矩阵解码：len=dims[0]、M=dims[2]，读前 len 行去尾 NUL。 -pub fn decode_index(body: &[u8], dims: &[i32]) -> Vec { - let m = matrix_width(dims); - (0..matrix_count(dims)) - .filter_map(|i| matrix_at(body, m, i)) - .collect() -} - -pub fn decode_ext_index(body: &[u8], dims: &[i32]) -> ExtIndex { - let cap = matrix_cap(dims); - let m = matrix_width(dims); - let off = body.len().saturating_sub(cap * m); - let ext_path = String::from_utf8_lossy(&body[..off]).into_owned(); - let mat = &body[off..]; - let childs = (0..matrix_count(dims)) - .filter_map(|i| matrix_at(mat, m, i)) - .collect(); - ExtIndex { childs, ext_path } -} - -/// extindex → (dims=[len,cap,M], body)。body = 头部变长 ext_path + 尾部 cap×M 矩阵（childs，规范排序）。 -/// ext_path 置头部：帧生命周期内一次写定、childs 才 churn，矩阵起点 off=body_len−cap*M 稳定。 -pub fn encode_ext_index(ext_path: &str, children: &[String]) -> (Vec, Vec) { - encode_ext_index_grow(ext_path, children, 0, 0) -} - -/// extindex 带容量下限（cap/m 只增不减，对齐 encode_index_grow）。 -pub fn encode_ext_index_grow( - ext_path: &str, - children: &[String], - cap_hint: usize, - m_hint: usize, -) -> (Vec, Vec) { - let (dims, mat) = encode_index_grow(children, cap_hint, m_hint); - let mut body = ext_path.as_bytes().to_vec(); - body.extend(mat); - (dims, body) -} - -/// 有效成员数 len = head dims[0]（O(1)，不碰 body）。 -pub fn matrix_count(dims: &[i32]) -> usize { - dims.first().map(|&n| n.max(0) as usize).unwrap_or(0) -} - -/// 预留容量 cap = head dims[1]。 -pub fn matrix_cap(dims: &[i32]) -> usize { - dims.get(1).map(|&n| n.max(0) as usize).unwrap_or(0) -} - -/// 行宽 M = head dims[2]。 -pub fn matrix_width(dims: &[i32]) -> usize { - dims.get(2).map(|&m| m.max(0) as usize).unwrap_or(0) -} - -/// 第 idx 行（O(1)），去尾 NUL。越界返回 None。 -pub fn matrix_at(body: &[u8], m: usize, idx: usize) -> Option { - if m == 0 { - return Some(String::new()); - } - let s = idx * m; - let e = s + m; - if e > body.len() { - return None; - } - let row = &body[s..e]; - let end = row.iter().position(|&b| b == 0).unwrap_or(m); - Some(String::from_utf8_lossy(&row[..end]).into_owned()) -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn index_roundtrip_align8() { - let (dims, body) = encode_index(&["yy".into(), "x".into(), "zzz".into()]); - assert_eq!(dims, vec![3, 3, 8]); // len=3, cap=3, M=align8(3)=8 - assert_eq!(body.len(), 24); - assert_eq!(decode_index(&body, &dims), vec!["x", "yy", "zzz"]); - } - - #[test] - fn index_empty() { - let (dims, body) = encode_index(&[]); - assert_eq!(dims, vec![0, 0, 0]); - assert!(body.is_empty()); - } - - #[test] - fn grow_reserves_cap() { - // 预留 cap=4：body 恒 4×8=32，前 1 行有效。 - let (dims, body) = encode_index_grow(&["x".into()], 4, 0); - assert_eq!(dims, vec![1, 4, 8]); - assert_eq!(body.len(), 32); - assert_eq!(decode_index(&body, &dims), vec!["x"]); - } - - #[test] - fn grow_cap_doubling() { - assert_eq!(grow_cap(0, 1), 1); - assert_eq!(grow_cap(4, 4), 4); - assert_eq!(grow_cap(4, 5), 8); - assert_eq!(grow_cap(4, 20), 32); - } - - #[test] - fn ext_index_roundtrip() { - let (dims, body) = encode_ext_index("/p", &["a".into()]); - assert_eq!(dims, vec![1, 1, 8]); // len=1, cap=1, M=8 - assert_eq!(body.len(), 2 + 8); // "/p" + 1×8 矩阵 - let e = decode_ext_index(&body, &dims); - assert_eq!(e.ext_path, "/p"); - assert_eq!(e.childs, vec!["a"]); - } -} diff --git a/src/xvalue_obj.rs b/src/xvalue_obj.rs deleted file mode 100644 index 9d90f19..0000000 --- a/src/xvalue_obj.rs +++ /dev/null @@ -1,8 +0,0 @@ -// xvalue_obj.rs — 对齐 xvalue_obj.go（Obj{}，空 obj）+ map 构造 - -use crate::xvalue::XValue; - -/// 空的散 key ndarray：给定 dims（恒 ndim≥1），无成员。 -pub fn new_map(dims: &[i32]) -> XValue { - crate::xvalue_index::new_map_index(dims) -} diff --git a/tests/cp.rs b/tests/cp.rs index 4b6590a..824b8cb 100644 --- a/tests/cp.rs +++ b/tests/cp.rs @@ -28,16 +28,29 @@ fn run(dsn: &str) { kv.clear().unwrap(); // 原型 stringkeymap 容器：两个叶成员 + 一个嵌套容器成员。 - set(kv, "/proto", &new_map_index(&[0])); + set(kv, "/proto", &new_map_langtype("[]char/utf32·int64")); set(kv, "/proto·x", &new_int64(&[10])); set(kv, "/proto·y", &new_int64(&[20])); - set(kv, "/proto·sub", &new_map_index(&[0])); + set(kv, "/proto·sub", &new_map_langtype("[]char/utf32·int64")); set(kv, "/proto·sub·z", &new_int64(&[30])); + kv.cp_tree("/proto", "/proto").unwrap(); + assert!(kv.cp_tree("/proto", "/proto/child").is_err()); + assert_eq!(get_one(kv, "/proto·x"), new_int64(&[10])); + // 只读扩展源 + 原型上的 extindex 成员 ov 覆盖 /ext·（extindex 节点落 `/` 目录键 /proto·ov/）。 - set(kv, "/ext·", &new_map_index(&[0])); + set(kv, "/ext", &new_map_langtype("[]char/utf32·int64")); + set( + kv, + "/ext·", + &XValue::Opaque(Opaque { + kind: "lib".into(), + body: vec![], + array_len: 1, + }), + ); set(kv, "/ext·a", &new_int64(&[99])); - kv.ext_index("/proto·ov·", "/ext·").unwrap(); + kv.ext_index("/proto·ov/", "/ext·").unwrap(); // ── cp：单 key 拷贝，不带成员 ── kv.cp("/proto·x", "/px").unwrap(); @@ -81,10 +94,16 @@ fn run(dsn: &str) { "cp_tree 的 extindex 成员 overlay 读通" ); match kv.get("/inst·", &["ov/".into()], true).remove(0) { - XValue::ExtIndex(e) => assert_eq!(e.ext_path, "/ext·", "extindex 指向同一只读扩展"), + XValue::Ext(e) => assert_eq!((e.langtype.as_str(), e.locator.as_str()), ("lib", "/ext·")), other => panic!("target 侧应为 extindex: {:?}", other), } + kv.cp_list("/proto", "/shallow").unwrap(); + assert_eq!( + kv.get("/shallow·ov/", &["a".into()], true).remove(0), + new_int64(&[99]) + ); + // 改动源 extindex 目标，target 因共享同一只读扩展而同步可见（非深拷贝底层数据）。 set(kv, "/ext·a", &new_int64(&[123])); assert_eq!( @@ -96,7 +115,9 @@ fn run(dsn: &str) { #[test] fn cp_redis() { - run("redis://127.0.0.1:6379"); + let dsn = std::env::var("KVSPACE_TEST_REDIS_DSN") + .unwrap_or_else(|_| "redis://127.0.0.1:6379".to_string()); + run(&dsn); } #[test] diff --git a/tests/map_coord.rs b/tests/map_coord.rs index 1e05ee7..a0c4f82 100644 --- a/tests/map_coord.rs +++ b/tests/map_coord.rs @@ -27,17 +27,16 @@ fn run(dsn: &str) { let kv: &mut dyn KVSpace = kv.as_mut(); kv.clear().unwrap(); - // 显式创建 [2,3]stringkeymap 容器：值在 /m，memindex 在 /m·。 - set(kv, "/m·", &new_map_index(&[2, 3])); + set(kv, "/m", &new_map_langtype("[int64,int64]·float32")); // 乱序写坐标成员。 set(kv, "/m·[1,2]", &new_float32(&[6.28])); set(kv, "/m·[0,1]", &new_float32(&[3.14])); set(kv, "/m·[0,0]", &new_float32(&[1.0])); - // 容器值 round-trip：kind + dims 保留（值在 /m，成员在 memindex /m·）。 + // The map type is stored at /m; members are physical /m· keys. match get_one(kv, "/m") { - XValue::Map(m) => assert_eq!(m.dims, vec![2, 3], "dims round-trip"), + XValue::Map(m) => assert_eq!(m.langtype, "[int64,int64]·float32"), other => panic!("容器值 kind 丢失: {:?}", other), } @@ -61,7 +60,7 @@ fn run(dsn: &str) { assert!(is_none(&get_one(kv, "/n")), "被拒后不应留下容器值"); // 命名成员仍为裸名，与坐标段字面可分。 - set(kv, "/h/", &new_map_index(&[0])); + set(kv, "/h", &new_map_langtype("[]char/utf32·int64")); set(kv, "/h·x", &new_int64(&[1])); set(kv, "/h·[0]", &new_int64(&[2])); let names = kv.list("/h·", false, true); @@ -70,7 +69,9 @@ fn run(dsn: &str) { #[test] fn map_coord_redis() { - run("redis://127.0.0.1:6379"); + let dsn = std::env::var("KVSPACE_TEST_REDIS_DSN") + .unwrap_or_else(|_| "redis://127.0.0.1:6379".to_string()); + run(&dsn); } #[test] diff --git a/tests/metadata_fs.rs b/tests/metadata_fs.rs new file mode 100644 index 0000000..409be84 --- /dev/null +++ b/tests/metadata_fs.rs @@ -0,0 +1,166 @@ +use kvspace_durable::{conn, metadata, new_map_langtype, KVPair, KVSpace, XValue}; +use std::fs; + +fn set(kv: &mut dyn KVSpace, key: &str, ro: bool, vid: u32) { + let raw = kvspace_durable::xvalue::encode_head("int32", 0, &[], &7i32.to_le_bytes()); + kv.set(&[KVPair { + key: key.into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + kv.set_metadata(key, ro, vid).unwrap(); +} + +#[test] +fn metadata_tracks_fs_subtrees() { + let root = std::env::temp_dir().join(format!("kvspace-meta-fs-{}", std::process::id())); + let _ = fs::remove_dir_all(&root); + let mut kv = conn(&format!("fs://{}", root.display())); + let kv = kv.as_mut(); + set(kv, "/src/a", true, 7); + set(kv, "/src/sub/b", true, 8); + let meta = |key: &str| root.join(metadata::key_for(key).unwrap().trim_start_matches('/')); + assert_eq!( + metadata::decode(&fs::read(meta("/src/a")).unwrap()), + Some((true, 7)) + ); + assert_eq!(kv.get_metadata("/src/a").unwrap(), (true, 7)); + assert!(kv + .list("/", false, false) + .iter() + .all(|name| !name.contains(".kvspace-meta"))); + + kv.cp_tree("/src", "/dst").unwrap(); + assert_eq!( + metadata::decode(&fs::read(meta("/dst/a")).unwrap()), + Some((true, 7)) + ); + assert_eq!( + metadata::decode(&fs::read(meta("/dst/sub/b")).unwrap()), + Some((true, 8)) + ); + kv.del_tree("/src").unwrap(); + assert!(!meta("/src/a").exists()); + assert!(!meta("/src/sub/b").exists()); + assert!(meta("/dst/a").exists()); + kv.del_tree("/dst").unwrap(); + assert!(!meta("/dst/a").exists()); + assert!(!meta("/dst/sub/b").exists()); + + kv.set(&[KVPair { + key: "/m".into(), + val: new_map_langtype("[int64]·[]char/utf32"), + raw: None, + }]) + .unwrap(); + set(kv, "/m·x", true, 9); + kv.cp_list("/m", "/n").unwrap(); + assert_eq!( + metadata::decode(&fs::read(meta("/n·x")).unwrap()), + Some((true, 9)) + ); + kv.del_tree("/m").unwrap(); + assert!(!meta("/m·x").exists()); + assert!(meta("/n·x").exists()); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn reads_new_wire_directory_value() { + let root = std::env::temp_dir().join(format!("kvspace-new-dir-fs-{}", std::process::id())); + let _ = fs::remove_dir_all(&root); + let mut kv = conn(&format!("fs://{}", root.display())); + let raw = kvspace_durable::headlenpow::encode(5, 0, 0, 0, "rwfunc", &[], 0).unwrap(); + kv.set(&[KVPair { + key: "/lib/f/".into(), + val: kvspace_durable::xvalue::decode_xvalue(&raw), + raw: Some(raw.clone()), + }]) + .unwrap(); + assert_eq!(kv.get_raw("/lib/f/"), raw); + assert_eq!(kv.get_part("/lib/f/", 0, 32), raw); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn copies_pointer_slot_metadata() { + let root = std::env::temp_dir().join(format!("kvspace-pointer-meta-fs-{}", std::process::id())); + let _ = fs::remove_dir_all(&root); + let mut kv = conn(&format!("fs://{}", root.display())); + let kv = kv.as_mut(); + let target = + kvspace_durable::headlenpow::encode(5, 0, 0, 0, "int32", &7i32.to_le_bytes(), 4).unwrap(); + let pointer = kvspace_durable::headlenpow::encode(5, 5, 7, 7, "int32", b"/target", 7).unwrap(); + for (key, raw) in [("/target", target), ("/link", pointer)] { + kv.set(&[KVPair { + key: key.into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + } + kv.set_metadata("/target", true, 42).unwrap(); + kv.set_metadata("/link", true, 17).unwrap(); + kv.cp("/link", "/copy").unwrap(); + assert_eq!(kv.get_metadata("/copy").unwrap(), (true, 42)); + let key = metadata::key_for("/copy").unwrap(); + assert_eq!( + metadata::decode(&fs::read(root.join(key.trim_start_matches('/'))).unwrap()), + Some((true, 17)) + ); + let _ = fs::remove_dir_all(&root); +} + +#[test] +fn value_and_directory_keys_coexist() { + let root = std::env::temp_dir().join(format!("kvspace-dir-value-fs-{}", std::process::id())); + let _ = fs::remove_dir_all(&root); + let mut kv = conn(&format!("fs://{}", root.display())); + let value = kvspace_durable::xvalue::encode_head("int64", 0, &[], &7i64.to_le_bytes()); + kv.set(&[KVPair { + key: "/node".into(), + val: XValue::None, + raw: Some(value.clone()), + }]) + .unwrap(); + kv.mkindex("/node/", 0).unwrap(); + let directory = kv.get_raw("/node/"); + assert_eq!( + kvspace_durable::headlenpow::decode(&directory) + .unwrap() + .langtype, + "lib" + ); + assert_eq!(kv.get_raw("/node"), value); + for name in ["__dir__", "__self__", "~abc"] { + kv.set(&[KVPair { + key: format!("/node/{name}"), + val: XValue::None, + raw: Some(value.clone()), + }]) + .unwrap(); + assert_eq!(kv.get_raw(&format!("/node/{name}")), value); + kv.set(&[KVPair { + key: format!("/node·{name}"), + val: XValue::None, + raw: Some(value.clone()), + }]) + .unwrap(); + assert_eq!(kv.get_raw(&format!("/node·{name}")), value); + } + let names = kv.list("/node/", false, false); + for name in ["__dir__", "__self__", "~abc"] { + assert!(names.contains(&name.to_string())); + assert!(kv.list("/node·", false, false).contains(&name.to_string())); + } + assert_eq!(kv.get_raw("/node/"), directory); + kv.cp_tree("/node", "/copy").unwrap(); + assert_eq!(kv.get_raw("/copy"), value); + assert_eq!(kv.get_raw("/copy/"), directory); + for name in ["__dir__", "__self__", "~abc"] { + assert_eq!(kv.get_raw(&format!("/copy/{name}")), value); + assert_eq!(kv.get_raw(&format!("/copy·{name}")), value); + } + let _ = fs::remove_dir_all(&root); +} diff --git a/tests/prefix_list.rs b/tests/prefix_list.rs new file mode 100644 index 0000000..597a74c --- /dev/null +++ b/tests/prefix_list.rs @@ -0,0 +1,253 @@ +use kvspace_durable::{metadata, Backend, KVPair, KVSpace, KVStore, XValue}; +use std::cell::RefCell; +use std::collections::HashMap; +use std::rc::Rc; + +#[derive(Default, Clone)] +struct MemoryStore(Rc>>>); + +impl KVStore for MemoryStore { + fn get(&self, key: &str) -> Option> { + self.0.borrow().get(key).cloned() + } + + fn set(&self, key: &str, value: &[u8]) { + self.0.borrow_mut().insert(key.to_string(), value.to_vec()); + } + + fn del(&self, keys: &[&str]) { + let mut values = self.0.borrow_mut(); + for key in keys { + values.remove(*key); + } + } + + fn scan_keys(&self, prefix: &str) -> Vec { + self.0 + .borrow() + .keys() + .filter(|key| { + (key.len() > prefix.len() + && key.starts_with(prefix) + && (key[prefix.len()..].starts_with('/') + || key[prefix.len()..].starts_with('·'))) + || *key == prefix + }) + .cloned() + .collect() + } + + fn flush(&self) { + self.0.borrow_mut().clear(); + } +} + +#[test] +fn frame_pointer_shadows_code_operand() { + let store = MemoryStore::default(); + let mut kv = Backend::new(store); + let dir = kvspace_durable::headlenpow::encode(5, 0, 0, 0, "lib", &[], 0).unwrap(); + let code = + kvspace_durable::headlenpow::encode(5, 1, 6, 6, "rwir", &[0, 0, 0, 0, 0, b'x'], 6).unwrap(); + for (key, raw) in [ + ("/lib/f/", dir), + ("/lib/f/[1,-1]", code.clone()), + ("/lib/f/[1,0]", code), + ] { + kv.set(&[KVPair { + key: key.into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + } + kv.ext_index("/vthread/1/[1]/", "/lib/f/").unwrap(); + let ptr = kvspace_durable::headlenpow::encode(5, 5, 2, 2, "int64", b"/x", 2).unwrap(); + assert!(kv + .set(&[KVPair { + key: "/vthread/1/[1]/[1,0]".into(), + val: XValue::None, + raw: Some(ptr.clone()), + }]) + .is_err()); + kv.set(&[KVPair { + key: "/vthread/1/[1]/[1,-1]".into(), + val: XValue::None, + raw: Some(ptr.clone()), + }]) + .unwrap(); + assert_eq!(kv.get_raw("/vthread/1/[1]/[1,-1]"), ptr); +} + +#[test] +fn stores_and_removes_metadata_sidecars() { + let store = MemoryStore::default(); + let inspect = store.clone(); + let mut backend = Backend::new(store); + let key = "/private"; + let meta = metadata::key_for(key).unwrap(); + let raw = kvspace_durable::xvalue::encode_head("int32", 0, &[], &7i32.to_le_bytes()); + backend + .set(&[KVPair { + key: key.into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + backend.set_metadata(key, true, 42).unwrap(); + assert_eq!( + metadata::decode(&inspect.get(&meta).unwrap()), + Some((true, 42)) + ); + assert_eq!(backend.get_metadata(key).unwrap(), (true, 42)); + backend.cp(key, "/copy").unwrap(); + let copy_meta = metadata::key_for("/copy").unwrap(); + assert_eq!( + metadata::decode(&inspect.get(©_meta).unwrap()), + Some((true, 42)) + ); + backend.del(&["/copy".into()]).unwrap(); + assert!(inspect.get(©_meta).is_none()); + let raw = kvspace_durable::xvalue::encode_head("int32", 0, &[], &8i32.to_le_bytes()); + backend + .set(&[KVPair { + key: key.into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + assert!(inspect.get(&meta).is_none()); + assert_eq!(backend.get_metadata(key).unwrap(), (false, 0)); + + let raw = kvspace_durable::xvalue::encode_head("int32", 0, &[], &9i32.to_le_bytes()); + backend + .set(&[KVPair { + key: "/tree/a".into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + backend.set_metadata("/tree/a", true, 19).unwrap(); + backend.cp_tree("/tree", "/tree_copy").unwrap(); + let copied = metadata::key_for("/tree_copy/a").unwrap(); + assert_eq!( + metadata::decode(&inspect.get(&copied).unwrap()), + Some((true, 19)) + ); + backend.del_tree("/tree_copy").unwrap(); + assert!(inspect.get(&copied).is_none()); +} + +#[test] +fn lists_physical_children() { + let store = MemoryStore::default(); + store.set( + "/m", + &kvspace_durable::xvalue::encode_head("[int64]·int64", 0, &[], &[]), + ); + store.set("/m·[10]", b"value"); + store.set("/m·[2]", b"value"); + store.set("/m·[2]·nested", b"value"); + store.set("/d/x", b"value"); + store.set("/d/sub/", b"directory"); + store.set("/d/sub/y", b"value"); + store.set("/.kvspace-meta/2f", b"value"); + let mut backend = Backend::new(store); + assert_eq!(backend.list("/m·", false, false), vec!["[2]", "[10]"]); + assert_eq!(backend.list_len("/m·", false, false), 2); + assert_eq!(backend.list_at("/m·", 1, false, false), Some("[10]".into())); + assert_eq!(backend.list("/d/", false, false), vec!["sub/", "x"]); + assert_eq!(backend.list("/", false, false), vec!["d", "m"]); +} + +#[test] +fn copies_new_wire_metadata() { + let store = MemoryStore::default(); + let inspect = store.clone(); + let mut backend = Backend::new(store); + let raw = + kvspace_durable::headlenpow::encode(5, 0, 0, 0, "int32", &7i32.to_le_bytes(), 4).unwrap(); + backend + .set(&[KVPair { + key: "/src/a".into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + backend.set_metadata("/src/a", true, 29).unwrap(); + backend.cp("/src/a", "/copy").unwrap(); + backend.cp_tree("/src", "/tree").unwrap(); + for key in ["/copy", "/tree/a"] { + let meta = metadata::key_for(key).unwrap(); + assert_eq!( + metadata::decode(&inspect.get(&meta).unwrap()), + Some((true, 29)) + ); + } +} + +#[test] +fn copies_pointer_slot_metadata() { + let store = MemoryStore::default(); + let inspect = store.clone(); + let mut backend = Backend::new(store); + let target = + kvspace_durable::headlenpow::encode(5, 0, 0, 0, "int32", &7i32.to_le_bytes(), 4).unwrap(); + let pointer = kvspace_durable::headlenpow::encode(5, 5, 7, 7, "int32", b"/target", 7).unwrap(); + for (key, raw) in [("/target", target), ("/link", pointer)] { + backend + .set(&[KVPair { + key: key.into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + } + backend.set_metadata("/target", true, 42).unwrap(); + backend.set_metadata("/link", true, 17).unwrap(); + backend.cp("/link", "/copy").unwrap(); + assert_eq!(backend.get_metadata("/copy").unwrap(), (true, 42)); + let key = metadata::key_for("/copy").unwrap(); + assert_eq!( + metadata::decode(&inspect.get(&key).unwrap()), + Some((true, 17)) + ); +} + +#[test] +fn new_wire_directories_and_extensions_use_physical_keys() { + let store = MemoryStore::default(); + let inspect = store.clone(); + let mut backend = Backend::new(store); + backend.mkindex("/lib/", 32).unwrap(); + let dir = kvspace_durable::headlenpow::encode(5, 0, 0, 0, "rwfunc", &[], 0).unwrap(); + let slot = + kvspace_durable::headlenpow::encode(5, 1, 5, 5, "rwir", &[0, 0, 0, 0, 0], 5).unwrap(); + let map = kvspace_durable::headlenpow::encode(5, 0, 0, 0, "[int64]·int64", &[], 0).unwrap(); + let scalar = + kvspace_durable::headlenpow::encode(5, 0, 0, 0, "int64", &7i64.to_le_bytes(), 8).unwrap(); + for (key, raw) in [ + ("/lib/f/", dir), + ("/lib/f/[1,0]", slot.clone()), + ("/m", map), + ("/m·1", scalar), + ] { + backend + .set(&[KVPair { + key: key.into(), + val: XValue::None, + raw: Some(raw), + }]) + .unwrap(); + } + backend.ext_index("/stack/", "/lib/f/").unwrap(); + assert_eq!(backend.get_raw("/stack/[1,0]"), slot); + assert_eq!(backend.list("/m·", false, false), vec!["1"]); + assert_eq!(backend.list("/stack/", true, false), vec!["[1,0]"]); + assert!(inspect + .0 + .borrow() + .values() + .all(|raw| kvspace_durable::headlenpow::decode(raw).is_some())); + assert!(inspect.get("/m·").is_none()); +}