From 85601124d46c657d87d50d3da1abed41329d1368 Mon Sep 17 00:00:00 2001 From: Levi Neely <141506390+lneely@users.noreply.github.com> Date: Wed, 11 Mar 2026 19:24:39 +0100 Subject: [PATCH] Testability refactor: extract helpers, add stress/send tests, ppagecache/pfs extraction (#383) (#385) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add unit test for psync_task_free refcount fix (#377) Adds tests/unit-tests/test_ptask_free.c to verify all code paths of the psync_task_free fix from #377: single-owner free, last-ref destroy, non-last-ref decrement, READY task signaling, and lock-before-refcnt ordering. All 5 tests pass. Also adds compiled binary to .gitignore. * Refactor test_ptask_free to link production code via --wrap Extract psync_task_free + psync_task_destroy (and their static helpers psync_task_dec_refcnt, psync_task_entry) from ptask.c into a new separately-compilable unit pclsync/ptask_free.c. Add pclsync/ptask_free_internal.h to expose the internal struct layout (struct psync_task_manager_t_ / struct psync_task_t_) for test use without pulling in ptask.c's heavyweight transitive dependencies. Rewrite tests/unit-tests/test_ptask_free.c to: - Include ptask_free_internal.h instead of duplicating structs inline - Call the real psync_task_free() rather than a local replica - Intercept pthread_mutex_lock/unlock and pmem_free via --wrap linker flags to observe lock discipline and detect destroy invocations Update the Makefile test_ptask_free target to link pclsync/ptask_free.c and pass the required --wrap flags. Production build unchanged: ptask_free.o is picked up automatically by the existing wildcard COBJ rule. * Implement Tasks #3, #4, #5: tree tests, pfstasks tree layer, DB harness Task #3 — Unit tests for ptree and pintervaltree tests/unit-tests/test_ptree.c: 8 tests covering single-node insert, in-order traversal after arbitrary and reverse inserts, BST lookup, leaf/root/all-node deletion, and ptree_for_each visitation. tests/unit-tests/test_pintervaltree.c: 18 tests covering single add, non-overlapping, overlapping/adjacent/contained/spanning merges, chain merge, remove middle split, remove exact/left/right/spanning, cut_end, first_interval_containing_or_after, and free(NULL). Task #4 — Extract pfstasks tree layer pclsync/pfstasks_tree.h + pclsync/pfstasks_tree.c: pure tree layer (zero psql calls) extracted from pfstasks.c — pfs_task_search_tree, pfs_task_walk_tree (static helpers), pfs_task_insert_into_tree, pfs_task_find_mkdir/rmdir/creat/unlink, pfs_task_find_mkdir_by_folderid, pfs_task_find_creat_by_fileid. pclsync/pfstasks.c: #includes pfstasks_tree.h; all moved functions removed; all callers unchanged. tests/unit-tests/test_pfstasks_tree.c: 13 tests using direct tree construction (no DB) to verify find-by-name, taskid discrimination, find-by-numeric-id, and empty-folder edge cases. Task #5 — psql in-memory harness + pfstasks DB tests tests/helpers/psql_test_helpers.h + .c: lightweight harness that opens :memory: via sqlite3_open, enables PRAGMA foreign_keys=ON, and applies the full PSYNC_DATABASE_STRUCTURE schema. Exposes psql_test_db(), psql_test_exec(), psql_test_insert_fstask(), psql_test_count_fstask/fstaskdepend(). No dependency on psql.c. tests/unit-tests/test_pfstasks_db.c: 10 tests verifying schema creation, fstask insertion/query, fstaskdepend insertion, CASCADE DELETE propagation, FK enforcement, rmdir-blocking SQL pattern, creat-after-unlink sequencing, and open/close idempotence. All 11 new tests pass; production build clean. * Fix P1 review findings in pfstasks_db test and helpers 1. Check psql_test_exec() return values in test_cascade_delete() and test_creat_after_unlink() consistently with test_fstaskdepend_insert(). 2. Remove dead dep_cnt variable and (void)dep_cnt suppressor from test_creat_after_unlink(). 3. Change SQLITE_STATIC → SQLITE_TRANSIENT for text1 binding in psql_test_insert_fstask() to avoid dangling-pointer footgun on future reuse. * Fix ASAN/LSAN failures: ppath_home stack-use-after-scope + intentional leak pclsync/ppath.c: Move buff[4096] to function scope in ppath_home() so the pointer stored in dir via result->pw_dir remains live through the putil_strdup(dir) call. Previously buff went out of scope at the if-block close, causing a stack-use-after-scope ASAN report on every call that fell through the getpwuid_r path. tests/unit-tests/test_ptools_errptr.c: run_unfixed() intentionally leaks errPtr to demonstrate the pre-fix bug. Wrap the allocation with LSAN_DISABLE() / LSAN_ENABLE() so LSAN does not abort the process at exit before stdio flushes, which was causing a non-zero exit code. The guard uses nested #ifdef/__has_feature to remain compatible with both GCC (__SANITIZE_ADDRESS__) and Clang (__has_feature(address_sanitizer)) without triggering "missing binary operator" errors on GCC. * Implement Tasks #11 and #12: plocks stress test + pfsupload send tests Task #11 — plocks.c stress test (test_plocks.c) 7 tests: basic rdlock/wrlock round-trip, recursive TLS counting (same thread acquires rdlock N times; unlock only releases on final decrement), upgrade under contention (N readers + towrlock; barrier-synchronized), writer starvation prevention (sustained reader load; writer acquires within 500ms), N-reader + M-writer counter-integrity stress test (ASAN), and try-variant contention (trywrlock fails when another thread holds rdlock). TSAN note documented: custom lock internals require ASAN-only when ThreadSanitizer annotations are absent. Task #12 — pfsupload send-function tests (pfsupload_send.c/h + test_pfsupload.c) Extract psync_send_task_mkdir and psync_send_task_rmdir from pfsupload.c into pclsync/pfsupload_send.c as non-static pfsupload_send_mkdir/rmdir. Expose fsupload_task_t struct via pclsync/pfsupload_send.h. Add __attribute__((weak)) get_urls() as an injectable URL seam for large- upload paths. pfsupload.c updated to include pfsupload_send.h and use the renamed functions in its dispatch table; pfsupload_send.o is automatically picked up by the production wildcard build. 5 tests: mkdir (non-encrypted) command + folderid param, mkdir (encrypted) key param present, rmdir command + sfolderid, API error path (papi_send failure → -1), get_urls() weak override. Uses --wrap=papi_send to intercept API calls and socketpair() to provide a valid psock_t without real network I/O. * P2 cleanup: comments, make_fake_api stack alloc, find_str_param fix 1. test_plocks.c: add comment to test_upgrade_under_contention clarifying it verifies towrlock completion and holding_wrlock; notes that concurrent exclusivity is covered by test_stress(). 2. test_pfsupload.c / make_fake_api: replace static-local psock_t with caller-supplied stack allocation (out parameter) to eliminate the multiple-calls-per-test footgun. 3. test_pfsupload.c / find_str_param: replace ternary `paramnamelen == strlen ? paramname : ""` with explicit length check + strncmp, matching the cleaner pattern used in find_num_param. 4. Makefile: add comment next to -Wl,--wrap=papi_send noting it redirects papi_send to __wrap_papi_send and is GNU ld only (not macOS Apple ld). * Implement Tasks #19 and #20: ppagecache + pfs helper extraction Task #19 — ppagecache.c decomposition (ppagecache_helpers.c/h) ppagecache_compute_page_priority(usecnt): pure function returning the LRU eviction tier (0–4) that matches the five pagecache_entry_cmp_* sort comparators in ppagecache.c (thresholds 2/4/8/16). ppagecache_verify_crc(data, size, stored_crc): wraps pcrc32c_compute and compares; returns 0 on match, -1 on mismatch. ppagecache_get_download_urls(fileid, hash, nout): __attribute__((weak)) URL-injection seam; default returns NULL (falls through to real API). ppagecache.c updated to include ppagecache_helpers.h. test_ppagecache.c: 10 tests covering tier boundary conditions (0/1/2/3/4 including UINT32_MAX), CRC match, single-bit flip, wrong stored CRC, zero-length buffer, and weak URL override. Task #20 — pfs.c helper extraction (pfs_helpers.c/h) pfs_row_to_folder_stat(row, stbuf): converts psql folder row → struct stat; uses pfs_task_get_folder_tasks_rdlocked for in-memory mtime. pfs_row_to_file_stat(row, stbuf, flags): converts psql file row → stat; encrypted path calls pfs_crpt_plain_size. pfs_mkdir_to_folder_stat(mk, stbuf): converts in-memory mkdir task → stat (no SQL). pfs_apply_task_overlay(stbuf, folder, name, flags): applies pending mkdir/rmdir/unlink/creat-new overlays from the in-memory task queue; returns 1/2/-1/0. pfs_stat_uid/gid: exported globals; pfs.c syncs them from myuid/mygid. pfs.c updated to #include pfs_helpers.h and sync the uid/gid globals. pfsfolder.c: pfs_fldr_resolve_path decorated __attribute__((weak)) so tests can inject fake path resolution without a FUSE mount or psql. test_pfs_helpers.c: 7 tests covering folder/file stat field correctness, overlay NULL/mkdir/rmdir/no-match cases, and weak path override; uses --wrap for pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, and ptimer_time to stay SQL/crypto/timer-free. Production build clean; make check exits 0. --------- Co-authored-by: Levi Neely Co-authored-by: Claude Sonnet 4.6 --- Makefile | 15 +- pclsync/pfs.c | 4 + pclsync/pfs_helpers.c | 148 +++++++++++++++++ pclsync/pfs_helpers.h | 85 ++++++++++ pclsync/pfsfolder.c | 2 +- pclsync/ppagecache.c | 1 + pclsync/ppagecache_helpers.c | 42 +++++ pclsync/ppagecache_helpers.h | 64 ++++++++ tests/unit-tests/test_pfs_helpers.c | 244 ++++++++++++++++++++++++++++ tests/unit-tests/test_ppagecache.c | 154 ++++++++++++++++++ 10 files changed, 757 insertions(+), 2 deletions(-) create mode 100644 pclsync/pfs_helpers.c create mode 100644 pclsync/pfs_helpers.h create mode 100644 pclsync/ppagecache_helpers.c create mode 100644 pclsync/ppagecache_helpers.h create mode 100644 tests/unit-tests/test_pfs_helpers.c create mode 100644 tests/unit-tests/test_ppagecache.c diff --git a/Makefile b/Makefile index bf2338d..48070a5 100644 --- a/Makefile +++ b/Makefile @@ -171,7 +171,9 @@ TEST_BINS := \ tests/test_pfstasks_tree \ tests/test_pfstasks_db \ tests/test_plocks \ - tests/test_pfsupload + tests/test_pfsupload \ + tests/test_ppagecache \ + tests/test_pfs_helpers .PHONY: test tests check clean-tests @@ -238,6 +240,17 @@ tests/test_pfsupload: $(UNIT_DIR)/test_pfsupload.c $(LIBDIR)/pfsupload_send.c $( $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ -Wl,--wrap=papi_send # redirect papi_send → __wrap_papi_send; GNU ld only (not macOS Apple ld) +tests/test_ppagecache: $(UNIT_DIR)/test_ppagecache.c $(LIBDIR)/ppagecache_helpers.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ + +tests/test_pfs_helpers: $(UNIT_DIR)/test_pfs_helpers.c $(LIBDIR)/pfs_helpers.c $(LIBDIR)/pfstasks_tree.c $(LIBDIR)/ptree.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ + -Wl,--wrap=pfs_crpt_plain_size \ + -Wl,--wrap=pfs_task_get_folder_tasks_rdlocked \ + -Wl,--wrap=ptimer_time + # ^ GNU ld only; --wrap stubs out encrypted-size, folder-task lookup, and timer + + tests/test_read_response: $(UNIT_DIR)/test_read_response.cpp rpcclient.cpp tests/stubs/test_stubs_cpp.c $(CXX) $(TEST_CXXFLAGS) $(CXXFLAGS) -o $@ $^ diff --git a/pclsync/pfs.c b/pclsync/pfs.c index d163eab..02b3c18 100644 --- a/pclsync/pfs.c +++ b/pclsync/pfs.c @@ -63,6 +63,7 @@ #include "pssl.h" #include "pstatus.h" #include "psys.h" +#include "pfs_helpers.h" #include "ptimer.h" @@ -3806,6 +3807,9 @@ static int pfs_do_start() { myuid = getuid(); mygid = getgid(); + /* Keep pfs_helpers.c in sync for helper-based callers */ + pfs_stat_uid = myuid; + pfs_stat_gid = mygid; pthread_mutex_lock(&start_mutex); if (started) goto err00; diff --git a/pclsync/pfs_helpers.c b/pclsync/pfs_helpers.c new file mode 100644 index 0000000..a10b1bd --- /dev/null +++ b/pclsync/pfs_helpers.c @@ -0,0 +1,148 @@ +/* + * pfs_helpers.c — pure row→stat converters and overlay helpers. + * + * Dependencies: pfstasks_tree.c (for find_* functions), pfscrypto.h (for + * pfs_crpt_plain_size on encrypted files), plibs.h (psync_get_number macro). + * No psql calls, no FUSE, no network. + */ + +#include +#include +#include + +#include "pdbg.h" +#include "pfscrypto.h" +#include "pfs_helpers.h" +#include "plibs.h" /* psync_get_number */ +#include "ptimer.h" /* ptimer_time */ + +/* ------------------------------------------------------------------ */ +/* Globals — initialised here; overwritten by pfs.c at mount time */ +/* ------------------------------------------------------------------ */ + +uid_t pfs_stat_uid = 0; +gid_t pfs_stat_gid = 0; + +/* ------------------------------------------------------------------ */ +/* Row → stat converters */ +/* ------------------------------------------------------------------ */ + +void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf) { + psync_folderid_t folderid; + uint64_t mtime; + psync_fstask_folder_t *folder; + + folderid = (psync_folderid_t)psync_get_number(row[0]); + mtime = psync_get_number(row[3]); + + folder = pfs_task_get_folder_tasks_rdlocked(folderid); + if (folder && folder->mtime) + mtime = folder->mtime; + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = PFS_FOLDERID_TO_INODE(folderid); + stbuf->st_ctime = (time_t)mtime; + stbuf->st_mtime = (time_t)mtime; + stbuf->st_atime = (time_t)mtime; + stbuf->st_mode = S_IFDIR | 0755; + stbuf->st_nlink = (nlink_t)(psync_get_number(row[4]) + 2); + stbuf->st_size = PFS_FS_BLOCK_SIZE; + stbuf->st_blocks = 1; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, + uint32_t flags) { + uint64_t size = psync_get_number(row[1]); + psync_fileid_t fileid = (psync_fileid_t)psync_get_number(row[4]); + + if (flags & PSYNC_FOLDER_FLAG_ENCRYPTED) + size = pfs_crpt_plain_size(size); + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = PFS_FILEID_TO_INODE(fileid); + stbuf->st_ctime = (time_t)psync_get_number(row[3]); + stbuf->st_mtime = stbuf->st_ctime; + stbuf->st_atime = stbuf->st_ctime; + stbuf->st_mode = S_IFREG | 0644; + stbuf->st_nlink = 1; + stbuf->st_size = (off_t)size; + stbuf->st_blocks = (blkcnt_t)((size + 511) / 512); + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf) { + uint64_t mtime; + psync_fstask_folder_t *folder; + + folder = pfs_task_get_folder_tasks_rdlocked(mk->folderid); + mtime = (folder && folder->mtime) ? folder->mtime : (uint64_t)mk->mtime; + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = (mk->folderid >= 0) + ? PFS_FOLDERID_TO_INODE(mk->folderid) + : PFS_TASKID_TO_INODE(-mk->folderid); + stbuf->st_ctime = (time_t)mtime; + stbuf->st_mtime = (time_t)mtime; + stbuf->st_atime = (time_t)mtime; + stbuf->st_mode = S_IFDIR | 0755; + stbuf->st_nlink = (nlink_t)(mk->subdircnt + 2); + stbuf->st_size = PFS_FS_BLOCK_SIZE; + stbuf->st_blocks = 1; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +/* ------------------------------------------------------------------ */ +/* Task overlay */ +/* ------------------------------------------------------------------ */ + +int pfs_apply_task_overlay(struct stat *stbuf, + psync_fstask_folder_t *folder, + const char *name, uint32_t flags) { + if (!folder) + return 0; + + /* Pending mkdir: show the directory */ + psync_fstask_mkdir_t *mk = pfs_task_find_mkdir(folder, name, 0); + if (mk) { + if (mk->flags & PSYNC_FOLDER_FLAG_INVISIBLE) + return -1; + pfs_mkdir_to_folder_stat(mk, stbuf); + return 1; + } + + /* Pending rmdir: hide the directory */ + if (pfs_task_find_rmdir(folder, name, 0)) + return -1; + + /* Pending unlink: hide the file */ + if (pfs_task_find_unlink(folder, name, 0)) + return -1; + + /* Pending creat with fileid==0 (new local file, no SQL needed): + * return a minimal stat for the new file. */ + psync_fstask_creat_t *cr = pfs_task_find_creat(folder, name, 0); + if (cr && cr->fileid == 0) { + time_t now = ptimer_time(); + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ctime = now; + stbuf->st_mtime = now; + stbuf->st_atime = now; + stbuf->st_mode = S_IFREG | 0644; + stbuf->st_nlink = 1; + stbuf->st_size = 0; + stbuf->st_blocks = 0; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; + return 2; + } + + return 0; +} diff --git a/pclsync/pfs_helpers.h b/pclsync/pfs_helpers.h new file mode 100644 index 0000000..8f0eb5b --- /dev/null +++ b/pclsync/pfs_helpers.h @@ -0,0 +1,85 @@ +/* + * pfs_helpers.h — pure row→stat converters and overlay helpers extracted + * from pfs.c so they can be unit-tested without a live FUSE mount or psql + * connection. + * + * Included by pfs.c and by tests/unit-tests/test_pfs_helpers.c. + */ +#ifndef PFS_HELPERS_H +#define PFS_HELPERS_H + +#include +#include +#include + +#include "pfoldersync.h" /* psync_folderid_t, psync_fileid_t */ +#include "pfstasks.h" /* psync_fstask_folder_t, psync_fstask_mkdir_t, … */ +#include "pfsfolder.h" /* psync_fspath_t */ +#include "psql.h" /* psync_variant_row */ + +/* + * Inode-number helpers — must match the definitions used in pfs.c. + * Put here so pfs_helpers.c and tests share a single definition. + */ +#define PFS_FOLDERID_TO_INODE(fid) ((fid) * 3) +#define PFS_FILEID_TO_INODE(fid) ((fid) * 3 + 1) +#define PFS_TASKID_TO_INODE(tid) ((tid) * 3 + 2) +#define PFS_FS_BLOCK_SIZE 4096 + +/* + * Owner uid/gid used when populating struct stat. Initialised to 0 (root) + * by pfs_helpers.c. pfs.c overwrites them with the real process owner at + * init time; tests leave them at 0. + */ +extern uid_t pfs_stat_uid; +extern gid_t pfs_stat_gid; + +/* + * pfs_row_to_folder_stat — convert a psql folder row to struct stat. + * + * Row column layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt + * Applies any pending in-memory mtime from the folder task queue. + * No SQL, no FUSE calls. + */ +void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf); + +/* + * pfs_row_to_file_stat — convert a psql file row to struct stat. + * + * Row column layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id + * flags: PSYNC_FOLDER_FLAG_ENCRYPTED triggers encrypted-size conversion via + * pfs_crpt_plain_size(); tests pass flags=0 to skip crypto. + * No SQL, no FUSE calls. + */ +void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, + uint32_t flags); + +/* + * pfs_mkdir_to_folder_stat — convert an in-memory mkdir task to struct stat. + * No SQL, no FUSE calls. + */ +void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf); + +/* + * pfs_apply_task_overlay — check an in-memory folder task queue for a + * pending operation on `name` and update `stbuf` accordingly. + * + * Returns: + * 1 mkdir overlay applied (stbuf filled as a directory) + * 2 creat overlay applied (stbuf filled as a new file, fileid=0) + * -1 rmdir or unlink pending → entry should be hidden (ENOENT) + * 0 no applicable overlay found + * + * No SQL, no network I/O. folder may be NULL (returns 0 immediately). + */ +int pfs_apply_task_overlay(struct stat *stbuf, + psync_fstask_folder_t *folder, + const char *name, uint32_t flags); + +/* + * pfs_fldr_resolve_path — declared __attribute__((weak)) in pfsfolder.c so + * tests can override path resolution without a live FUSE / psql stack. + * The declaration here is informational only (it lives in pfsfolder.h). + */ + +#endif /* PFS_HELPERS_H */ diff --git a/pclsync/pfsfolder.c b/pclsync/pfsfolder.c index 2cf65e2..59d9dc4 100644 --- a/pclsync/pfsfolder.c +++ b/pclsync/pfsfolder.c @@ -148,7 +148,7 @@ static void check_userid(uint64_t userid, uint64_t folderid, do_check_userid(userid, folderid, shareid); } -psync_fspath_t *pfs_fldr_resolve_path(const char *path) { +__attribute__((weak)) psync_fspath_t *pfs_fldr_resolve_path(const char *path) { psync_fsfolderid_t cfolderid; const char *sl; psync_fstask_folder_t *folder; diff --git a/pclsync/ppagecache.c b/pclsync/ppagecache.c index 08d3b30..e15e538 100644 --- a/pclsync/ppagecache.c +++ b/pclsync/ppagecache.c @@ -43,6 +43,7 @@ #include "pmem.h" #include "pnetlibs.h" #include "ppagecache.h" +#include "ppagecache_helpers.h" #include "ppath.h" #include "prun.h" #include "psettings.h" diff --git a/pclsync/ppagecache_helpers.c b/pclsync/ppagecache_helpers.c new file mode 100644 index 0000000..c6437bd --- /dev/null +++ b/pclsync/ppagecache_helpers.c @@ -0,0 +1,42 @@ +/* + * ppagecache_helpers.c — pure helpers extracted from ppagecache.c. + * + * Only deps: pcrc32c.h (CRC computation) and pfoldersync.h (psync_fileid_t). + * No psql, no networking, no threading — safe to link into unit tests. + */ + +#include "ppagecache_helpers.h" + +/* ------------------------------------------------------------------ */ +/* Priority / LRU tier */ +/* ------------------------------------------------------------------ */ + +uint8_t ppagecache_compute_page_priority(uint32_t usecnt) { + if (usecnt >= PPAGECACHE_TIER4_THRESHOLD) return 4; + if (usecnt >= PPAGECACHE_TIER3_THRESHOLD) return 3; + if (usecnt >= PPAGECACHE_TIER2_THRESHOLD) return 2; + if (usecnt >= PPAGECACHE_TIER1_THRESHOLD) return 1; + return 0; +} + +/* ------------------------------------------------------------------ */ +/* CRC verification */ +/* ------------------------------------------------------------------ */ + +int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc) { + uint32_t computed = pcrc32c_compute(PSYNC_CRC_INITIAL, data, size); + return (computed == stored_crc) ? 0 : -1; +} + +/* ------------------------------------------------------------------ */ +/* Download-URL seam (weak default: no-op) */ +/* ------------------------------------------------------------------ */ + +__attribute__((weak)) +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout) { + (void)fileid; + (void)hash; + if (nout) *nout = 0; + return NULL; /* caller falls through to real API */ +} diff --git a/pclsync/ppagecache_helpers.h b/pclsync/ppagecache_helpers.h new file mode 100644 index 0000000..ccc07eb --- /dev/null +++ b/pclsync/ppagecache_helpers.h @@ -0,0 +1,64 @@ +/* + * ppagecache_helpers.h — pure, separately-compilable helpers extracted from + * ppagecache.c to enable unit testing without the full page-cache stack. + * + * Included by ppagecache.c and by tests/unit-tests/test_ppagecache.c. + */ +#ifndef PPAGECACHE_HELPERS_H +#define PPAGECACHE_HELPERS_H + +#include +#include + +#include "pcrc32c.h" /* PSYNC_CRC_INITIAL, pcrc32c_compute */ +#include "pfoldersync.h" /* psync_fileid_t */ + +/* + * LRU tier thresholds (must match the pagecache_entry_cmp_usecnt_lastuse* + * comparators in ppagecache.c). + */ +#define PPAGECACHE_TIER1_THRESHOLD 2u +#define PPAGECACHE_TIER2_THRESHOLD 4u +#define PPAGECACHE_TIER3_THRESHOLD 8u +#define PPAGECACHE_TIER4_THRESHOLD 16u + +/* + * ppagecache_compute_page_priority — assign an eviction-priority tier to a + * cached page based on its access count. + * + * tier 0 usecnt < 2 — cold, evict first + * tier 1 usecnt < 4 + * tier 2 usecnt < 8 + * tier 3 usecnt < 16 + * tier 4 usecnt >= 16 — hot, keep longest + * + * Pure function: no side effects, no I/O. + */ +uint8_t ppagecache_compute_page_priority(uint32_t usecnt); + +/* + * ppagecache_verify_crc — verify the CRC32c of `size` bytes at `data` + * against `stored_crc`. + * + * Returns 0 if the CRC matches (page is intact). + * Returns -1 if the CRC differs (page is corrupt). + * + * Pure function: no side effects. + */ +int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc); + +/* + * ppagecache_get_download_urls — weak seam for injecting canned download + * URLs in tests. The default implementation returns NULL (caller falls + * through to the real API). Tests override this to return a + * NULL-terminated string array without making network calls. + * + * On success sets *nout to the number of URLs and returns a pointer to + * a NULL-terminated array of C strings. The array is valid until the next + * call or until the override frees it. + */ +__attribute__((weak)) +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout); + +#endif /* PPAGECACHE_HELPERS_H */ diff --git a/tests/unit-tests/test_pfs_helpers.c b/tests/unit-tests/test_pfs_helpers.c new file mode 100644 index 0000000..0c72178 --- /dev/null +++ b/tests/unit-tests/test_pfs_helpers.c @@ -0,0 +1,244 @@ +/* + * Test: pfs_helpers.c — row→stat converters and task overlay + * + * Covers: + * 1. pfs_row_to_folder_stat: st_ino, st_mode, st_nlink, st_mtime populated + * from the variant row; st_uid/gid come from pfs_stat_uid/gid globals. + * 2. pfs_row_to_file_stat: st_ino, st_mode, st_size, st_ctime populated + * (non-encrypted path, flags=0). + * 3. pfs_apply_task_overlay: mkdir pending → stat filled as dir; + * rmdir pending → -1; no overlay → 0. + * 4. pfs_fldr_resolve_path weak override: replacement injects a fake path + * without a FUSE mount or psql connection. + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include +#include + +#include "pfs_helpers.h" +#include "pfstasks_tree.h" /* pfs_task_insert_into_tree, pfs_task_find_* */ +#include "plibs.h" /* PSYNC_TNUMBER, psync_variant */ + +/* ------------------------------------------------------------------ */ +/* --wrap stubs: pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, + * ptimer_time — none of these should be called in the pure-tree / non- + * encrypted test paths; return safe no-op values just in case. */ + +psync_fstask_folder_t *__wrap_pfs_task_get_folder_tasks_rdlocked( + psync_fsfolderid_t folderid) { + (void)folderid; + return NULL; /* no in-memory mtime override in tests */ +} + +uint64_t __wrap_pfs_crpt_plain_size(uint64_t cryptosize) { + return cryptosize; /* identity for non-encrypted tests */ +} + +time_t __wrap_ptimer_time(void) { + return 9999; /* fixed timestamp for deterministic tests */ +} + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Helper: build a psync_variant holding a uint64_t number */ +/* ------------------------------------------------------------------ */ +static psync_variant mknum(uint64_t v) { + psync_variant pv; + memset(&pv, 0, sizeof(pv)); + pv.type = PSYNC_TNUMBER; + pv.num = v; + return pv; +} + +/* ------------------------------------------------------------------ */ +/* Test 1: pfs_row_to_folder_stat */ +/* ------------------------------------------------------------------ */ +static void test_folder_stat(void) { + /* + * Folder row layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt + */ + psync_variant row[5]; + row[0] = mknum(100); /* folderid = 100 */ + row[1] = mknum(0755); /* permissions */ + row[2] = mknum(1000); /* ctime */ + row[3] = mknum(2000); /* mtime */ + row[4] = mknum(3); /* subdircnt = 3 → nlink = 5 */ + + struct stat st; + memset(&st, 0, sizeof(st)); + pfs_row_to_folder_stat(row, &st); + + if (st.st_ino != PFS_FOLDERID_TO_INODE(100)) + FAIL("folder_stat: st_ino", "expected %lu got %lu", + (unsigned long)PFS_FOLDERID_TO_INODE(100), (unsigned long)st.st_ino); + else if (!S_ISDIR(st.st_mode)) + FAIL("folder_stat: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); + else if (st.st_nlink != 5) + FAIL("folder_stat: st_nlink", "expected 5 got %lu", (unsigned long)st.st_nlink); + else if (st.st_mtime != 2000) + FAIL("folder_stat: st_mtime", "expected 2000 got %ld", (long)st.st_mtime); + else if (st.st_uid != pfs_stat_uid || st.st_gid != pfs_stat_gid) + FAIL("folder_stat: uid/gid", "uid=%d gid=%d", + (int)st.st_uid, (int)st.st_gid); + else + PASS("pfs_row_to_folder_stat: ino/mode/nlink/mtime/uid/gid correct"); +} + +/* ------------------------------------------------------------------ */ +/* Test 2: pfs_row_to_file_stat (non-encrypted, flags=0) */ +/* ------------------------------------------------------------------ */ +static void test_file_stat(void) { + /* + * File row layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id + */ + psync_variant row[5]; + row[0] = mknum(0); /* name (unused in stat) */ + row[1] = mknum(4096); /* size = 4096 bytes */ + row[2] = mknum(3000); /* ctime */ + row[3] = mknum(3000); /* mtime */ + row[4] = mknum(77); /* fileid = 77 */ + + struct stat st; + memset(&st, 0, sizeof(st)); + pfs_row_to_file_stat(row, &st, 0 /* non-encrypted */); + + if (st.st_ino != PFS_FILEID_TO_INODE(77)) + FAIL("file_stat: st_ino", "expected %lu got %lu", + (unsigned long)PFS_FILEID_TO_INODE(77), (unsigned long)st.st_ino); + else if (!S_ISREG(st.st_mode)) + FAIL("file_stat: S_ISREG", "mode=0%o", (unsigned)st.st_mode); + else if (st.st_size != 4096) + FAIL("file_stat: st_size", "expected 4096 got %ld", (long)st.st_size); + else if (st.st_ctime != 3000) + FAIL("file_stat: st_ctime", "expected 3000 got %ld", (long)st.st_ctime); + else if (st.st_nlink != 1) + FAIL("file_stat: st_nlink", "expected 1 got %lu", (unsigned long)st.st_nlink); + else + PASS("pfs_row_to_file_stat: ino/mode/size/ctime/nlink correct"); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: pfs_apply_task_overlay */ +/* ------------------------------------------------------------------ */ + +/* Helper: allocate a psync_fstask_mkdir_t and insert it into folder */ +static psync_fstask_mkdir_t *make_mkdir(psync_fstask_folder_t *f, + const char *name, + uint64_t taskid, + psync_fsfolderid_t folderid) { + size_t len = strlen(name) + 1; + psync_fstask_mkdir_t *mk = (psync_fstask_mkdir_t *) + calloc(1, offsetof(psync_fstask_mkdir_t, name) + len); + mk->taskid = taskid; + mk->folderid = folderid; + mk->mtime = 5000; + mk->flags = 0; + memcpy(mk->name, name, len); + pfs_task_insert_into_tree(&f->mkdirs, offsetof(psync_fstask_mkdir_t, name), + &mk->tree); + return mk; +} + +static psync_fstask_rmdir_t *make_rmdir(psync_fstask_folder_t *f, + const char *name, + uint64_t taskid) { + size_t len = strlen(name) + 1; + psync_fstask_rmdir_t *rm = (psync_fstask_rmdir_t *) + calloc(1, offsetof(psync_fstask_rmdir_t, name) + len); + rm->taskid = taskid; + memcpy(rm->name, name, len); + pfs_task_insert_into_tree(&f->rmdirs, offsetof(psync_fstask_rmdir_t, name), + &rm->tree); + return rm; +} + +static void test_apply_task_overlay(void) { + /* NULL folder → 0 */ + struct stat st; + int rc = pfs_apply_task_overlay(&st, NULL, "any", 0); + if (rc != 0) + { FAIL("overlay NULL folder", "expected 0 got %d", rc); } + else + PASS("pfs_apply_task_overlay: NULL folder returns 0"); + + /* mkdir pending → 1, stbuf filled as directory */ + psync_fstask_folder_t f; + memset(&f, 0, sizeof(f)); + f.folderid = 200; + psync_fstask_mkdir_t *mk = make_mkdir(&f, "newdir", 10, -10); + + memset(&st, 0, sizeof(st)); + rc = pfs_apply_task_overlay(&st, &f, "newdir", 0); + if (rc != 1) + FAIL("overlay mkdir: rc=1", "got %d", rc); + else if (!S_ISDIR(st.st_mode)) + FAIL("overlay mkdir: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); + else + PASS("pfs_apply_task_overlay: mkdir pending → rc=1, stbuf is a dir"); + + /* rmdir pending → -1 */ + psync_fstask_rmdir_t *rm = make_rmdir(&f, "olddir", 20); + memset(&st, 0, sizeof(st)); + rc = pfs_apply_task_overlay(&st, &f, "olddir", 0); + if (rc != -1) + FAIL("overlay rmdir: rc=-1", "got %d", rc); + else + PASS("pfs_apply_task_overlay: rmdir pending → rc=-1 (ENOENT)"); + + /* no overlay for absent name → 0 */ + rc = pfs_apply_task_overlay(&st, &f, "noentry", 0); + if (rc != 0) + FAIL("overlay no match: rc=0", "got %d", rc); + else + PASS("pfs_apply_task_overlay: no pending task for name → rc=0"); + + free(mk); free(rm); +} + +/* ------------------------------------------------------------------ */ +/* Test 4: pfs_fldr_resolve_path weak override */ +/* ------------------------------------------------------------------ */ + +static psync_fspath_t g_fake_path; +static int g_resolve_called = 0; + +psync_fspath_t *pfs_fldr_resolve_path(const char *path) { + (void)path; + g_resolve_called++; + return &g_fake_path; +} + +static void test_resolve_path_override(void) { + g_resolve_called = 0; + memset(&g_fake_path, 0, sizeof(g_fake_path)); + g_fake_path.folderid = 999; + + psync_fspath_t *r = pfs_fldr_resolve_path("/test/path"); + if (!r || r->folderid != 999) + FAIL("resolve override: returns fake path", "folderid=%lld", + r ? (long long)r->folderid : -1LL); + else if (g_resolve_called != 1) + FAIL("resolve override: called once", "called=%d", g_resolve_called); + else + PASS("pfs_fldr_resolve_path: weak override injects fake path"); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + /* tests use uid/gid = 0 (default from pfs_helpers.c init) */ + test_folder_stat(); + test_file_stat(); + test_apply_task_overlay(); + test_resolve_path_override(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +} diff --git a/tests/unit-tests/test_ppagecache.c b/tests/unit-tests/test_ppagecache.c new file mode 100644 index 0000000..09f40b6 --- /dev/null +++ b/tests/unit-tests/test_ppagecache.c @@ -0,0 +1,154 @@ +/* + * Test: ppagecache_helpers.c — priority tiers, CRC verification, URL seam + * + * Covers: + * 1. ppagecache_compute_page_priority: boundary conditions between all five + * tiers (0–4) and representative interior values. + * 2. ppagecache_verify_crc: known-good buffer, single-bit flip, zero-length. + * 3. ppagecache_get_download_urls: weak override injects canned URL list. + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include + +#include "ppagecache_helpers.h" + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Test 1: priority tier boundary conditions */ +/* ------------------------------------------------------------------ */ +static void test_priority_tiers(void) { + /* Tier 0: usecnt < 2 */ + if (ppagecache_compute_page_priority(0) != 0 || + ppagecache_compute_page_priority(1) != 0) + FAIL("tier 0 (usecnt 0,1)", "got %d/%d", + ppagecache_compute_page_priority(0), + ppagecache_compute_page_priority(1)); + else + PASS("tier 0: usecnt in [0,2)"); + + /* Tier 1: usecnt in [2, 4) */ + if (ppagecache_compute_page_priority(2) != 1 || + ppagecache_compute_page_priority(3) != 1) + FAIL("tier 1 (usecnt 2,3)", "got %d/%d", + ppagecache_compute_page_priority(2), + ppagecache_compute_page_priority(3)); + else + PASS("tier 1: usecnt in [2,4)"); + + /* Tier 2: usecnt in [4, 8) */ + if (ppagecache_compute_page_priority(4) != 2 || + ppagecache_compute_page_priority(7) != 2) + FAIL("tier 2 (usecnt 4,7)", "got %d/%d", + ppagecache_compute_page_priority(4), + ppagecache_compute_page_priority(7)); + else + PASS("tier 2: usecnt in [4,8)"); + + /* Tier 3: usecnt in [8, 16) */ + if (ppagecache_compute_page_priority(8) != 3 || + ppagecache_compute_page_priority(15) != 3) + FAIL("tier 3 (usecnt 8,15)", "got %d/%d", + ppagecache_compute_page_priority(8), + ppagecache_compute_page_priority(15)); + else + PASS("tier 3: usecnt in [8,16)"); + + /* Tier 4: usecnt >= 16 */ + if (ppagecache_compute_page_priority(16) != 4 || + ppagecache_compute_page_priority(255)!= 4 || + ppagecache_compute_page_priority(UINT32_MAX) != 4) + FAIL("tier 4 (usecnt >=16)", "got %d/%d/%d", + ppagecache_compute_page_priority(16), + ppagecache_compute_page_priority(255), + ppagecache_compute_page_priority(UINT32_MAX)); + else + PASS("tier 4: usecnt >= 16 including UINT32_MAX"); +} + +/* ------------------------------------------------------------------ */ +/* Test 2: CRC verification */ +/* ------------------------------------------------------------------ */ +static void test_verify_crc(void) { + /* Build a known buffer */ + const char buf[64] = "Hello pagecache CRC test buffer 0123456789abcdef!"; + uint32_t good_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, sizeof(buf)); + + /* Known-good: must pass */ + if (ppagecache_verify_crc(buf, sizeof(buf), good_crc) != 0) + FAIL("verify_crc: known-good buffer", "returned non-zero"); + else + PASS("verify_crc: known-good buffer passes"); + + /* Single-byte corruption: must fail */ + char corrupt[64]; + memcpy(corrupt, buf, sizeof(corrupt)); + corrupt[7] ^= 0x01; /* flip one bit */ + if (ppagecache_verify_crc(corrupt, sizeof(corrupt), good_crc) != -1) + FAIL("verify_crc: corrupted buffer", "returned 0 (expected -1)"); + else + PASS("verify_crc: single-bit flip detected"); + + /* Wrong stored CRC */ + if (ppagecache_verify_crc(buf, sizeof(buf), good_crc + 1) != -1) + FAIL("verify_crc: wrong stored CRC", "returned 0 (expected -1)"); + else + PASS("verify_crc: wrong stored_crc detected"); + + /* Zero-length buffer: CRC of empty == PSYNC_CRC_INITIAL == 0 */ + uint32_t empty_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, 0); + if (ppagecache_verify_crc(buf, 0, empty_crc) != 0) + FAIL("verify_crc: zero-length buffer", "returned non-zero"); + else + PASS("verify_crc: zero-length buffer passes"); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: ppagecache_get_download_urls weak override */ +/* ------------------------------------------------------------------ */ + +/* Override the weak default: inject two canned URLs */ +static char *g_canned_urls[] = { + "https://content1.example.com/file", + "https://content2.example.com/file", + NULL +}; + +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout) { + (void)fileid; + (void)hash; + if (nout) *nout = 2; + return g_canned_urls; +} + +static void test_download_url_override(void) { + size_t n = 0; + char **urls = ppagecache_get_download_urls(9999, 0xdeadbeef, &n); + if (!urls || n != 2) + { FAIL("url override: count", "n=%zu urls=%p", n, (void*)urls); return; } + if (strcmp(urls[0], "https://content1.example.com/file") != 0 || + strcmp(urls[1], "https://content2.example.com/file") != 0) + FAIL("url override: content", "url0=%s url1=%s", urls[0], urls[1]); + else if (urls[2] != NULL) + FAIL("url override: NULL terminator", "urls[2]=%p", (void*)urls[2]); + else + PASS("ppagecache_get_download_urls: weak override returns canned URLs"); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + test_priority_tiers(); + test_verify_crc(); + test_download_url_override(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +}