From 4618e245b97e6c4186f2cef8ff3b089f777976ae Mon Sep 17 00:00:00 2001 From: Levi Neely <141506390+lneely@users.noreply.github.com> Date: Wed, 11 Mar 2026 18:14:38 +0100 Subject: [PATCH 1/4] Testability refactor: extract helpers, add stress/send tests, ppagecache/pfs extraction (#383) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add unit test for psync_task_free refcount fix (#377) Adds tests/unit-tests/test_ptask_free.c to verify all code paths of the psync_task_free fix from #377: single-owner free, last-ref destroy, non-last-ref decrement, READY task signaling, and lock-before-refcnt ordering. All 5 tests pass. Also adds compiled binary to .gitignore. Co-Authored-By: Claude Sonnet 4.6 * Refactor test_ptask_free to link production code via --wrap Extract psync_task_free + psync_task_destroy (and their static helpers psync_task_dec_refcnt, psync_task_entry) from ptask.c into a new separately-compilable unit pclsync/ptask_free.c. Add pclsync/ptask_free_internal.h to expose the internal struct layout (struct psync_task_manager_t_ / struct psync_task_t_) for test use without pulling in ptask.c's heavyweight transitive dependencies. Rewrite tests/unit-tests/test_ptask_free.c to: - Include ptask_free_internal.h instead of duplicating structs inline - Call the real psync_task_free() rather than a local replica - Intercept pthread_mutex_lock/unlock and pmem_free via --wrap linker flags to observe lock discipline and detect destroy invocations Update the Makefile test_ptask_free target to link pclsync/ptask_free.c and pass the required --wrap flags. Production build unchanged: ptask_free.o is picked up automatically by the existing wildcard COBJ rule. Co-Authored-By: Claude Sonnet 4.6 * Implement Tasks #3, #4, #5: tree tests, pfstasks tree layer, DB harness Task #3 — Unit tests for ptree and pintervaltree tests/unit-tests/test_ptree.c: 8 tests covering single-node insert, in-order traversal after arbitrary and reverse inserts, BST lookup, leaf/root/all-node deletion, and ptree_for_each visitation. tests/unit-tests/test_pintervaltree.c: 18 tests covering single add, non-overlapping, overlapping/adjacent/contained/spanning merges, chain merge, remove middle split, remove exact/left/right/spanning, cut_end, first_interval_containing_or_after, and free(NULL). Task #4 — Extract pfstasks tree layer pclsync/pfstasks_tree.h + pclsync/pfstasks_tree.c: pure tree layer (zero psql calls) extracted from pfstasks.c — pfs_task_search_tree, pfs_task_walk_tree (static helpers), pfs_task_insert_into_tree, pfs_task_find_mkdir/rmdir/creat/unlink, pfs_task_find_mkdir_by_folderid, pfs_task_find_creat_by_fileid. pclsync/pfstasks.c: #includes pfstasks_tree.h; all moved functions removed; all callers unchanged. tests/unit-tests/test_pfstasks_tree.c: 13 tests using direct tree construction (no DB) to verify find-by-name, taskid discrimination, find-by-numeric-id, and empty-folder edge cases. Task #5 — psql in-memory harness + pfstasks DB tests tests/helpers/psql_test_helpers.h + .c: lightweight harness that opens :memory: via sqlite3_open, enables PRAGMA foreign_keys=ON, and applies the full PSYNC_DATABASE_STRUCTURE schema. Exposes psql_test_db(), psql_test_exec(), psql_test_insert_fstask(), psql_test_count_fstask/fstaskdepend(). No dependency on psql.c. tests/unit-tests/test_pfstasks_db.c: 10 tests verifying schema creation, fstask insertion/query, fstaskdepend insertion, CASCADE DELETE propagation, FK enforcement, rmdir-blocking SQL pattern, creat-after-unlink sequencing, and open/close idempotence. All 11 new tests pass; production build clean. Co-Authored-By: Claude Sonnet 4.6 * Fix P1 review findings in pfstasks_db test and helpers 1. Check psql_test_exec() return values in test_cascade_delete() and test_creat_after_unlink() consistently with test_fstaskdepend_insert(). 2. Remove dead dep_cnt variable and (void)dep_cnt suppressor from test_creat_after_unlink(). 3. Change SQLITE_STATIC → SQLITE_TRANSIENT for text1 binding in psql_test_insert_fstask() to avoid dangling-pointer footgun on future reuse. Co-Authored-By: Claude Sonnet 4.6 * Fix ASAN/LSAN failures: ppath_home stack-use-after-scope + intentional leak pclsync/ppath.c: Move buff[4096] to function scope in ppath_home() so the pointer stored in dir via result->pw_dir remains live through the putil_strdup(dir) call. Previously buff went out of scope at the if-block close, causing a stack-use-after-scope ASAN report on every call that fell through the getpwuid_r path. tests/unit-tests/test_ptools_errptr.c: run_unfixed() intentionally leaks errPtr to demonstrate the pre-fix bug. Wrap the allocation with LSAN_DISABLE() / LSAN_ENABLE() so LSAN does not abort the process at exit before stdio flushes, which was causing a non-zero exit code. The guard uses nested #ifdef/__has_feature to remain compatible with both GCC (__SANITIZE_ADDRESS__) and Clang (__has_feature(address_sanitizer)) without triggering "missing binary operator" errors on GCC. Co-Authored-By: Claude Sonnet 4.6 * Implement Tasks #11 and #12: plocks stress test + pfsupload send tests Task #11 — plocks.c stress test (test_plocks.c) 7 tests: basic rdlock/wrlock round-trip, recursive TLS counting (same thread acquires rdlock N times; unlock only releases on final decrement), upgrade under contention (N readers + towrlock; barrier-synchronized), writer starvation prevention (sustained reader load; writer acquires within 500ms), N-reader + M-writer counter-integrity stress test (ASAN), and try-variant contention (trywrlock fails when another thread holds rdlock). TSAN note documented: custom lock internals require ASAN-only when ThreadSanitizer annotations are absent. Task #12 — pfsupload send-function tests (pfsupload_send.c/h + test_pfsupload.c) Extract psync_send_task_mkdir and psync_send_task_rmdir from pfsupload.c into pclsync/pfsupload_send.c as non-static pfsupload_send_mkdir/rmdir. Expose fsupload_task_t struct via pclsync/pfsupload_send.h. Add __attribute__((weak)) get_urls() as an injectable URL seam for large- upload paths. pfsupload.c updated to include pfsupload_send.h and use the renamed functions in its dispatch table; pfsupload_send.o is automatically picked up by the production wildcard build. 5 tests: mkdir (non-encrypted) command + folderid param, mkdir (encrypted) key param present, rmdir command + sfolderid, API error path (papi_send failure → -1), get_urls() weak override. Uses --wrap=papi_send to intercept API calls and socketpair() to provide a valid psock_t without real network I/O. Co-Authored-By: Claude Sonnet 4.6 * P2 cleanup: comments, make_fake_api stack alloc, find_str_param fix 1. test_plocks.c: add comment to test_upgrade_under_contention clarifying it verifies towrlock completion and holding_wrlock; notes that concurrent exclusivity is covered by test_stress(). 2. test_pfsupload.c / make_fake_api: replace static-local psock_t with caller-supplied stack allocation (out parameter) to eliminate the multiple-calls-per-test footgun. 3. test_pfsupload.c / find_str_param: replace ternary `paramnamelen == strlen ? paramname : ""` with explicit length check + strncmp, matching the cleaner pattern used in find_num_param. 4. Makefile: add comment next to -Wl,--wrap=papi_send noting it redirects papi_send to __wrap_papi_send and is GNU ld only (not macOS Apple ld). Co-Authored-By: Claude Sonnet 4.6 * Implement Tasks #19 and #20: ppagecache + pfs helper extraction Task #19 — ppagecache.c decomposition (ppagecache_helpers.c/h) ppagecache_compute_page_priority(usecnt): pure function returning the LRU eviction tier (0–4) that matches the five pagecache_entry_cmp_* sort comparators in ppagecache.c (thresholds 2/4/8/16). ppagecache_verify_crc(data, size, stored_crc): wraps pcrc32c_compute and compares; returns 0 on match, -1 on mismatch. ppagecache_get_download_urls(fileid, hash, nout): __attribute__((weak)) URL-injection seam; default returns NULL (falls through to real API). ppagecache.c updated to include ppagecache_helpers.h. test_ppagecache.c: 10 tests covering tier boundary conditions (0/1/2/3/4 including UINT32_MAX), CRC match, single-bit flip, wrong stored CRC, zero-length buffer, and weak URL override. Task #20 — pfs.c helper extraction (pfs_helpers.c/h) pfs_row_to_folder_stat(row, stbuf): converts psql folder row → struct stat; uses pfs_task_get_folder_tasks_rdlocked for in-memory mtime. pfs_row_to_file_stat(row, stbuf, flags): converts psql file row → stat; encrypted path calls pfs_crpt_plain_size. pfs_mkdir_to_folder_stat(mk, stbuf): converts in-memory mkdir task → stat (no SQL). pfs_apply_task_overlay(stbuf, folder, name, flags): applies pending mkdir/rmdir/unlink/creat-new overlays from the in-memory task queue; returns 1/2/-1/0. pfs_stat_uid/gid: exported globals; pfs.c syncs them from myuid/mygid. pfs.c updated to #include pfs_helpers.h and sync the uid/gid globals. pfsfolder.c: pfs_fldr_resolve_path decorated __attribute__((weak)) so tests can inject fake path resolution without a FUSE mount or psql. test_pfs_helpers.c: 7 tests covering folder/file stat field correctness, overlay NULL/mkdir/rmdir/no-match cases, and weak path override; uses --wrap for pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, and ptimer_time to stay SQL/crypto/timer-free. Production build clean; make check exits 0. Co-Authored-By: Claude Sonnet 4.6 --------- Co-authored-by: Levi Neely Co-authored-by: Claude Sonnet 4.6 --- Makefile | 15 +- pclsync/pfs.c | 4 + pclsync/pfs_helpers.c | 148 +++++++++++++++++ pclsync/pfs_helpers.h | 85 ++++++++++ pclsync/pfsfolder.c | 2 +- pclsync/ppagecache.c | 1 + pclsync/ppagecache_helpers.c | 42 +++++ pclsync/ppagecache_helpers.h | 64 ++++++++ tests/unit-tests/test_pfs_helpers.c | 244 ++++++++++++++++++++++++++++ tests/unit-tests/test_ppagecache.c | 154 ++++++++++++++++++ 10 files changed, 757 insertions(+), 2 deletions(-) create mode 100644 pclsync/pfs_helpers.c create mode 100644 pclsync/pfs_helpers.h create mode 100644 pclsync/ppagecache_helpers.c create mode 100644 pclsync/ppagecache_helpers.h create mode 100644 tests/unit-tests/test_pfs_helpers.c create mode 100644 tests/unit-tests/test_ppagecache.c diff --git a/Makefile b/Makefile index bf2338d..48070a5 100644 --- a/Makefile +++ b/Makefile @@ -171,7 +171,9 @@ TEST_BINS := \ tests/test_pfstasks_tree \ tests/test_pfstasks_db \ tests/test_plocks \ - tests/test_pfsupload + tests/test_pfsupload \ + tests/test_ppagecache \ + tests/test_pfs_helpers .PHONY: test tests check clean-tests @@ -238,6 +240,17 @@ tests/test_pfsupload: $(UNIT_DIR)/test_pfsupload.c $(LIBDIR)/pfsupload_send.c $( $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ -Wl,--wrap=papi_send # redirect papi_send → __wrap_papi_send; GNU ld only (not macOS Apple ld) +tests/test_ppagecache: $(UNIT_DIR)/test_ppagecache.c $(LIBDIR)/ppagecache_helpers.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ + +tests/test_pfs_helpers: $(UNIT_DIR)/test_pfs_helpers.c $(LIBDIR)/pfs_helpers.c $(LIBDIR)/pfstasks_tree.c $(LIBDIR)/ptree.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ + -Wl,--wrap=pfs_crpt_plain_size \ + -Wl,--wrap=pfs_task_get_folder_tasks_rdlocked \ + -Wl,--wrap=ptimer_time + # ^ GNU ld only; --wrap stubs out encrypted-size, folder-task lookup, and timer + + tests/test_read_response: $(UNIT_DIR)/test_read_response.cpp rpcclient.cpp tests/stubs/test_stubs_cpp.c $(CXX) $(TEST_CXXFLAGS) $(CXXFLAGS) -o $@ $^ diff --git a/pclsync/pfs.c b/pclsync/pfs.c index d163eab..02b3c18 100644 --- a/pclsync/pfs.c +++ b/pclsync/pfs.c @@ -63,6 +63,7 @@ #include "pssl.h" #include "pstatus.h" #include "psys.h" +#include "pfs_helpers.h" #include "ptimer.h" @@ -3806,6 +3807,9 @@ static int pfs_do_start() { myuid = getuid(); mygid = getgid(); + /* Keep pfs_helpers.c in sync for helper-based callers */ + pfs_stat_uid = myuid; + pfs_stat_gid = mygid; pthread_mutex_lock(&start_mutex); if (started) goto err00; diff --git a/pclsync/pfs_helpers.c b/pclsync/pfs_helpers.c new file mode 100644 index 0000000..a10b1bd --- /dev/null +++ b/pclsync/pfs_helpers.c @@ -0,0 +1,148 @@ +/* + * pfs_helpers.c — pure row→stat converters and overlay helpers. + * + * Dependencies: pfstasks_tree.c (for find_* functions), pfscrypto.h (for + * pfs_crpt_plain_size on encrypted files), plibs.h (psync_get_number macro). + * No psql calls, no FUSE, no network. + */ + +#include +#include +#include + +#include "pdbg.h" +#include "pfscrypto.h" +#include "pfs_helpers.h" +#include "plibs.h" /* psync_get_number */ +#include "ptimer.h" /* ptimer_time */ + +/* ------------------------------------------------------------------ */ +/* Globals — initialised here; overwritten by pfs.c at mount time */ +/* ------------------------------------------------------------------ */ + +uid_t pfs_stat_uid = 0; +gid_t pfs_stat_gid = 0; + +/* ------------------------------------------------------------------ */ +/* Row → stat converters */ +/* ------------------------------------------------------------------ */ + +void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf) { + psync_folderid_t folderid; + uint64_t mtime; + psync_fstask_folder_t *folder; + + folderid = (psync_folderid_t)psync_get_number(row[0]); + mtime = psync_get_number(row[3]); + + folder = pfs_task_get_folder_tasks_rdlocked(folderid); + if (folder && folder->mtime) + mtime = folder->mtime; + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = PFS_FOLDERID_TO_INODE(folderid); + stbuf->st_ctime = (time_t)mtime; + stbuf->st_mtime = (time_t)mtime; + stbuf->st_atime = (time_t)mtime; + stbuf->st_mode = S_IFDIR | 0755; + stbuf->st_nlink = (nlink_t)(psync_get_number(row[4]) + 2); + stbuf->st_size = PFS_FS_BLOCK_SIZE; + stbuf->st_blocks = 1; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, + uint32_t flags) { + uint64_t size = psync_get_number(row[1]); + psync_fileid_t fileid = (psync_fileid_t)psync_get_number(row[4]); + + if (flags & PSYNC_FOLDER_FLAG_ENCRYPTED) + size = pfs_crpt_plain_size(size); + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = PFS_FILEID_TO_INODE(fileid); + stbuf->st_ctime = (time_t)psync_get_number(row[3]); + stbuf->st_mtime = stbuf->st_ctime; + stbuf->st_atime = stbuf->st_ctime; + stbuf->st_mode = S_IFREG | 0644; + stbuf->st_nlink = 1; + stbuf->st_size = (off_t)size; + stbuf->st_blocks = (blkcnt_t)((size + 511) / 512); + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf) { + uint64_t mtime; + psync_fstask_folder_t *folder; + + folder = pfs_task_get_folder_tasks_rdlocked(mk->folderid); + mtime = (folder && folder->mtime) ? folder->mtime : (uint64_t)mk->mtime; + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = (mk->folderid >= 0) + ? PFS_FOLDERID_TO_INODE(mk->folderid) + : PFS_TASKID_TO_INODE(-mk->folderid); + stbuf->st_ctime = (time_t)mtime; + stbuf->st_mtime = (time_t)mtime; + stbuf->st_atime = (time_t)mtime; + stbuf->st_mode = S_IFDIR | 0755; + stbuf->st_nlink = (nlink_t)(mk->subdircnt + 2); + stbuf->st_size = PFS_FS_BLOCK_SIZE; + stbuf->st_blocks = 1; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +/* ------------------------------------------------------------------ */ +/* Task overlay */ +/* ------------------------------------------------------------------ */ + +int pfs_apply_task_overlay(struct stat *stbuf, + psync_fstask_folder_t *folder, + const char *name, uint32_t flags) { + if (!folder) + return 0; + + /* Pending mkdir: show the directory */ + psync_fstask_mkdir_t *mk = pfs_task_find_mkdir(folder, name, 0); + if (mk) { + if (mk->flags & PSYNC_FOLDER_FLAG_INVISIBLE) + return -1; + pfs_mkdir_to_folder_stat(mk, stbuf); + return 1; + } + + /* Pending rmdir: hide the directory */ + if (pfs_task_find_rmdir(folder, name, 0)) + return -1; + + /* Pending unlink: hide the file */ + if (pfs_task_find_unlink(folder, name, 0)) + return -1; + + /* Pending creat with fileid==0 (new local file, no SQL needed): + * return a minimal stat for the new file. */ + psync_fstask_creat_t *cr = pfs_task_find_creat(folder, name, 0); + if (cr && cr->fileid == 0) { + time_t now = ptimer_time(); + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ctime = now; + stbuf->st_mtime = now; + stbuf->st_atime = now; + stbuf->st_mode = S_IFREG | 0644; + stbuf->st_nlink = 1; + stbuf->st_size = 0; + stbuf->st_blocks = 0; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; + return 2; + } + + return 0; +} diff --git a/pclsync/pfs_helpers.h b/pclsync/pfs_helpers.h new file mode 100644 index 0000000..8f0eb5b --- /dev/null +++ b/pclsync/pfs_helpers.h @@ -0,0 +1,85 @@ +/* + * pfs_helpers.h — pure row→stat converters and overlay helpers extracted + * from pfs.c so they can be unit-tested without a live FUSE mount or psql + * connection. + * + * Included by pfs.c and by tests/unit-tests/test_pfs_helpers.c. + */ +#ifndef PFS_HELPERS_H +#define PFS_HELPERS_H + +#include +#include +#include + +#include "pfoldersync.h" /* psync_folderid_t, psync_fileid_t */ +#include "pfstasks.h" /* psync_fstask_folder_t, psync_fstask_mkdir_t, … */ +#include "pfsfolder.h" /* psync_fspath_t */ +#include "psql.h" /* psync_variant_row */ + +/* + * Inode-number helpers — must match the definitions used in pfs.c. + * Put here so pfs_helpers.c and tests share a single definition. + */ +#define PFS_FOLDERID_TO_INODE(fid) ((fid) * 3) +#define PFS_FILEID_TO_INODE(fid) ((fid) * 3 + 1) +#define PFS_TASKID_TO_INODE(tid) ((tid) * 3 + 2) +#define PFS_FS_BLOCK_SIZE 4096 + +/* + * Owner uid/gid used when populating struct stat. Initialised to 0 (root) + * by pfs_helpers.c. pfs.c overwrites them with the real process owner at + * init time; tests leave them at 0. + */ +extern uid_t pfs_stat_uid; +extern gid_t pfs_stat_gid; + +/* + * pfs_row_to_folder_stat — convert a psql folder row to struct stat. + * + * Row column layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt + * Applies any pending in-memory mtime from the folder task queue. + * No SQL, no FUSE calls. + */ +void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf); + +/* + * pfs_row_to_file_stat — convert a psql file row to struct stat. + * + * Row column layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id + * flags: PSYNC_FOLDER_FLAG_ENCRYPTED triggers encrypted-size conversion via + * pfs_crpt_plain_size(); tests pass flags=0 to skip crypto. + * No SQL, no FUSE calls. + */ +void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, + uint32_t flags); + +/* + * pfs_mkdir_to_folder_stat — convert an in-memory mkdir task to struct stat. + * No SQL, no FUSE calls. + */ +void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf); + +/* + * pfs_apply_task_overlay — check an in-memory folder task queue for a + * pending operation on `name` and update `stbuf` accordingly. + * + * Returns: + * 1 mkdir overlay applied (stbuf filled as a directory) + * 2 creat overlay applied (stbuf filled as a new file, fileid=0) + * -1 rmdir or unlink pending → entry should be hidden (ENOENT) + * 0 no applicable overlay found + * + * No SQL, no network I/O. folder may be NULL (returns 0 immediately). + */ +int pfs_apply_task_overlay(struct stat *stbuf, + psync_fstask_folder_t *folder, + const char *name, uint32_t flags); + +/* + * pfs_fldr_resolve_path — declared __attribute__((weak)) in pfsfolder.c so + * tests can override path resolution without a live FUSE / psql stack. + * The declaration here is informational only (it lives in pfsfolder.h). + */ + +#endif /* PFS_HELPERS_H */ diff --git a/pclsync/pfsfolder.c b/pclsync/pfsfolder.c index 2cf65e2..59d9dc4 100644 --- a/pclsync/pfsfolder.c +++ b/pclsync/pfsfolder.c @@ -148,7 +148,7 @@ static void check_userid(uint64_t userid, uint64_t folderid, do_check_userid(userid, folderid, shareid); } -psync_fspath_t *pfs_fldr_resolve_path(const char *path) { +__attribute__((weak)) psync_fspath_t *pfs_fldr_resolve_path(const char *path) { psync_fsfolderid_t cfolderid; const char *sl; psync_fstask_folder_t *folder; diff --git a/pclsync/ppagecache.c b/pclsync/ppagecache.c index 08d3b30..e15e538 100644 --- a/pclsync/ppagecache.c +++ b/pclsync/ppagecache.c @@ -43,6 +43,7 @@ #include "pmem.h" #include "pnetlibs.h" #include "ppagecache.h" +#include "ppagecache_helpers.h" #include "ppath.h" #include "prun.h" #include "psettings.h" diff --git a/pclsync/ppagecache_helpers.c b/pclsync/ppagecache_helpers.c new file mode 100644 index 0000000..c6437bd --- /dev/null +++ b/pclsync/ppagecache_helpers.c @@ -0,0 +1,42 @@ +/* + * ppagecache_helpers.c — pure helpers extracted from ppagecache.c. + * + * Only deps: pcrc32c.h (CRC computation) and pfoldersync.h (psync_fileid_t). + * No psql, no networking, no threading — safe to link into unit tests. + */ + +#include "ppagecache_helpers.h" + +/* ------------------------------------------------------------------ */ +/* Priority / LRU tier */ +/* ------------------------------------------------------------------ */ + +uint8_t ppagecache_compute_page_priority(uint32_t usecnt) { + if (usecnt >= PPAGECACHE_TIER4_THRESHOLD) return 4; + if (usecnt >= PPAGECACHE_TIER3_THRESHOLD) return 3; + if (usecnt >= PPAGECACHE_TIER2_THRESHOLD) return 2; + if (usecnt >= PPAGECACHE_TIER1_THRESHOLD) return 1; + return 0; +} + +/* ------------------------------------------------------------------ */ +/* CRC verification */ +/* ------------------------------------------------------------------ */ + +int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc) { + uint32_t computed = pcrc32c_compute(PSYNC_CRC_INITIAL, data, size); + return (computed == stored_crc) ? 0 : -1; +} + +/* ------------------------------------------------------------------ */ +/* Download-URL seam (weak default: no-op) */ +/* ------------------------------------------------------------------ */ + +__attribute__((weak)) +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout) { + (void)fileid; + (void)hash; + if (nout) *nout = 0; + return NULL; /* caller falls through to real API */ +} diff --git a/pclsync/ppagecache_helpers.h b/pclsync/ppagecache_helpers.h new file mode 100644 index 0000000..ccc07eb --- /dev/null +++ b/pclsync/ppagecache_helpers.h @@ -0,0 +1,64 @@ +/* + * ppagecache_helpers.h — pure, separately-compilable helpers extracted from + * ppagecache.c to enable unit testing without the full page-cache stack. + * + * Included by ppagecache.c and by tests/unit-tests/test_ppagecache.c. + */ +#ifndef PPAGECACHE_HELPERS_H +#define PPAGECACHE_HELPERS_H + +#include +#include + +#include "pcrc32c.h" /* PSYNC_CRC_INITIAL, pcrc32c_compute */ +#include "pfoldersync.h" /* psync_fileid_t */ + +/* + * LRU tier thresholds (must match the pagecache_entry_cmp_usecnt_lastuse* + * comparators in ppagecache.c). + */ +#define PPAGECACHE_TIER1_THRESHOLD 2u +#define PPAGECACHE_TIER2_THRESHOLD 4u +#define PPAGECACHE_TIER3_THRESHOLD 8u +#define PPAGECACHE_TIER4_THRESHOLD 16u + +/* + * ppagecache_compute_page_priority — assign an eviction-priority tier to a + * cached page based on its access count. + * + * tier 0 usecnt < 2 — cold, evict first + * tier 1 usecnt < 4 + * tier 2 usecnt < 8 + * tier 3 usecnt < 16 + * tier 4 usecnt >= 16 — hot, keep longest + * + * Pure function: no side effects, no I/O. + */ +uint8_t ppagecache_compute_page_priority(uint32_t usecnt); + +/* + * ppagecache_verify_crc — verify the CRC32c of `size` bytes at `data` + * against `stored_crc`. + * + * Returns 0 if the CRC matches (page is intact). + * Returns -1 if the CRC differs (page is corrupt). + * + * Pure function: no side effects. + */ +int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc); + +/* + * ppagecache_get_download_urls — weak seam for injecting canned download + * URLs in tests. The default implementation returns NULL (caller falls + * through to the real API). Tests override this to return a + * NULL-terminated string array without making network calls. + * + * On success sets *nout to the number of URLs and returns a pointer to + * a NULL-terminated array of C strings. The array is valid until the next + * call or until the override frees it. + */ +__attribute__((weak)) +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout); + +#endif /* PPAGECACHE_HELPERS_H */ diff --git a/tests/unit-tests/test_pfs_helpers.c b/tests/unit-tests/test_pfs_helpers.c new file mode 100644 index 0000000..0c72178 --- /dev/null +++ b/tests/unit-tests/test_pfs_helpers.c @@ -0,0 +1,244 @@ +/* + * Test: pfs_helpers.c — row→stat converters and task overlay + * + * Covers: + * 1. pfs_row_to_folder_stat: st_ino, st_mode, st_nlink, st_mtime populated + * from the variant row; st_uid/gid come from pfs_stat_uid/gid globals. + * 2. pfs_row_to_file_stat: st_ino, st_mode, st_size, st_ctime populated + * (non-encrypted path, flags=0). + * 3. pfs_apply_task_overlay: mkdir pending → stat filled as dir; + * rmdir pending → -1; no overlay → 0. + * 4. pfs_fldr_resolve_path weak override: replacement injects a fake path + * without a FUSE mount or psql connection. + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include +#include + +#include "pfs_helpers.h" +#include "pfstasks_tree.h" /* pfs_task_insert_into_tree, pfs_task_find_* */ +#include "plibs.h" /* PSYNC_TNUMBER, psync_variant */ + +/* ------------------------------------------------------------------ */ +/* --wrap stubs: pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, + * ptimer_time — none of these should be called in the pure-tree / non- + * encrypted test paths; return safe no-op values just in case. */ + +psync_fstask_folder_t *__wrap_pfs_task_get_folder_tasks_rdlocked( + psync_fsfolderid_t folderid) { + (void)folderid; + return NULL; /* no in-memory mtime override in tests */ +} + +uint64_t __wrap_pfs_crpt_plain_size(uint64_t cryptosize) { + return cryptosize; /* identity for non-encrypted tests */ +} + +time_t __wrap_ptimer_time(void) { + return 9999; /* fixed timestamp for deterministic tests */ +} + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Helper: build a psync_variant holding a uint64_t number */ +/* ------------------------------------------------------------------ */ +static psync_variant mknum(uint64_t v) { + psync_variant pv; + memset(&pv, 0, sizeof(pv)); + pv.type = PSYNC_TNUMBER; + pv.num = v; + return pv; +} + +/* ------------------------------------------------------------------ */ +/* Test 1: pfs_row_to_folder_stat */ +/* ------------------------------------------------------------------ */ +static void test_folder_stat(void) { + /* + * Folder row layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt + */ + psync_variant row[5]; + row[0] = mknum(100); /* folderid = 100 */ + row[1] = mknum(0755); /* permissions */ + row[2] = mknum(1000); /* ctime */ + row[3] = mknum(2000); /* mtime */ + row[4] = mknum(3); /* subdircnt = 3 → nlink = 5 */ + + struct stat st; + memset(&st, 0, sizeof(st)); + pfs_row_to_folder_stat(row, &st); + + if (st.st_ino != PFS_FOLDERID_TO_INODE(100)) + FAIL("folder_stat: st_ino", "expected %lu got %lu", + (unsigned long)PFS_FOLDERID_TO_INODE(100), (unsigned long)st.st_ino); + else if (!S_ISDIR(st.st_mode)) + FAIL("folder_stat: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); + else if (st.st_nlink != 5) + FAIL("folder_stat: st_nlink", "expected 5 got %lu", (unsigned long)st.st_nlink); + else if (st.st_mtime != 2000) + FAIL("folder_stat: st_mtime", "expected 2000 got %ld", (long)st.st_mtime); + else if (st.st_uid != pfs_stat_uid || st.st_gid != pfs_stat_gid) + FAIL("folder_stat: uid/gid", "uid=%d gid=%d", + (int)st.st_uid, (int)st.st_gid); + else + PASS("pfs_row_to_folder_stat: ino/mode/nlink/mtime/uid/gid correct"); +} + +/* ------------------------------------------------------------------ */ +/* Test 2: pfs_row_to_file_stat (non-encrypted, flags=0) */ +/* ------------------------------------------------------------------ */ +static void test_file_stat(void) { + /* + * File row layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id + */ + psync_variant row[5]; + row[0] = mknum(0); /* name (unused in stat) */ + row[1] = mknum(4096); /* size = 4096 bytes */ + row[2] = mknum(3000); /* ctime */ + row[3] = mknum(3000); /* mtime */ + row[4] = mknum(77); /* fileid = 77 */ + + struct stat st; + memset(&st, 0, sizeof(st)); + pfs_row_to_file_stat(row, &st, 0 /* non-encrypted */); + + if (st.st_ino != PFS_FILEID_TO_INODE(77)) + FAIL("file_stat: st_ino", "expected %lu got %lu", + (unsigned long)PFS_FILEID_TO_INODE(77), (unsigned long)st.st_ino); + else if (!S_ISREG(st.st_mode)) + FAIL("file_stat: S_ISREG", "mode=0%o", (unsigned)st.st_mode); + else if (st.st_size != 4096) + FAIL("file_stat: st_size", "expected 4096 got %ld", (long)st.st_size); + else if (st.st_ctime != 3000) + FAIL("file_stat: st_ctime", "expected 3000 got %ld", (long)st.st_ctime); + else if (st.st_nlink != 1) + FAIL("file_stat: st_nlink", "expected 1 got %lu", (unsigned long)st.st_nlink); + else + PASS("pfs_row_to_file_stat: ino/mode/size/ctime/nlink correct"); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: pfs_apply_task_overlay */ +/* ------------------------------------------------------------------ */ + +/* Helper: allocate a psync_fstask_mkdir_t and insert it into folder */ +static psync_fstask_mkdir_t *make_mkdir(psync_fstask_folder_t *f, + const char *name, + uint64_t taskid, + psync_fsfolderid_t folderid) { + size_t len = strlen(name) + 1; + psync_fstask_mkdir_t *mk = (psync_fstask_mkdir_t *) + calloc(1, offsetof(psync_fstask_mkdir_t, name) + len); + mk->taskid = taskid; + mk->folderid = folderid; + mk->mtime = 5000; + mk->flags = 0; + memcpy(mk->name, name, len); + pfs_task_insert_into_tree(&f->mkdirs, offsetof(psync_fstask_mkdir_t, name), + &mk->tree); + return mk; +} + +static psync_fstask_rmdir_t *make_rmdir(psync_fstask_folder_t *f, + const char *name, + uint64_t taskid) { + size_t len = strlen(name) + 1; + psync_fstask_rmdir_t *rm = (psync_fstask_rmdir_t *) + calloc(1, offsetof(psync_fstask_rmdir_t, name) + len); + rm->taskid = taskid; + memcpy(rm->name, name, len); + pfs_task_insert_into_tree(&f->rmdirs, offsetof(psync_fstask_rmdir_t, name), + &rm->tree); + return rm; +} + +static void test_apply_task_overlay(void) { + /* NULL folder → 0 */ + struct stat st; + int rc = pfs_apply_task_overlay(&st, NULL, "any", 0); + if (rc != 0) + { FAIL("overlay NULL folder", "expected 0 got %d", rc); } + else + PASS("pfs_apply_task_overlay: NULL folder returns 0"); + + /* mkdir pending → 1, stbuf filled as directory */ + psync_fstask_folder_t f; + memset(&f, 0, sizeof(f)); + f.folderid = 200; + psync_fstask_mkdir_t *mk = make_mkdir(&f, "newdir", 10, -10); + + memset(&st, 0, sizeof(st)); + rc = pfs_apply_task_overlay(&st, &f, "newdir", 0); + if (rc != 1) + FAIL("overlay mkdir: rc=1", "got %d", rc); + else if (!S_ISDIR(st.st_mode)) + FAIL("overlay mkdir: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); + else + PASS("pfs_apply_task_overlay: mkdir pending → rc=1, stbuf is a dir"); + + /* rmdir pending → -1 */ + psync_fstask_rmdir_t *rm = make_rmdir(&f, "olddir", 20); + memset(&st, 0, sizeof(st)); + rc = pfs_apply_task_overlay(&st, &f, "olddir", 0); + if (rc != -1) + FAIL("overlay rmdir: rc=-1", "got %d", rc); + else + PASS("pfs_apply_task_overlay: rmdir pending → rc=-1 (ENOENT)"); + + /* no overlay for absent name → 0 */ + rc = pfs_apply_task_overlay(&st, &f, "noentry", 0); + if (rc != 0) + FAIL("overlay no match: rc=0", "got %d", rc); + else + PASS("pfs_apply_task_overlay: no pending task for name → rc=0"); + + free(mk); free(rm); +} + +/* ------------------------------------------------------------------ */ +/* Test 4: pfs_fldr_resolve_path weak override */ +/* ------------------------------------------------------------------ */ + +static psync_fspath_t g_fake_path; +static int g_resolve_called = 0; + +psync_fspath_t *pfs_fldr_resolve_path(const char *path) { + (void)path; + g_resolve_called++; + return &g_fake_path; +} + +static void test_resolve_path_override(void) { + g_resolve_called = 0; + memset(&g_fake_path, 0, sizeof(g_fake_path)); + g_fake_path.folderid = 999; + + psync_fspath_t *r = pfs_fldr_resolve_path("/test/path"); + if (!r || r->folderid != 999) + FAIL("resolve override: returns fake path", "folderid=%lld", + r ? (long long)r->folderid : -1LL); + else if (g_resolve_called != 1) + FAIL("resolve override: called once", "called=%d", g_resolve_called); + else + PASS("pfs_fldr_resolve_path: weak override injects fake path"); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + /* tests use uid/gid = 0 (default from pfs_helpers.c init) */ + test_folder_stat(); + test_file_stat(); + test_apply_task_overlay(); + test_resolve_path_override(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +} diff --git a/tests/unit-tests/test_ppagecache.c b/tests/unit-tests/test_ppagecache.c new file mode 100644 index 0000000..09f40b6 --- /dev/null +++ b/tests/unit-tests/test_ppagecache.c @@ -0,0 +1,154 @@ +/* + * Test: ppagecache_helpers.c — priority tiers, CRC verification, URL seam + * + * Covers: + * 1. ppagecache_compute_page_priority: boundary conditions between all five + * tiers (0–4) and representative interior values. + * 2. ppagecache_verify_crc: known-good buffer, single-bit flip, zero-length. + * 3. ppagecache_get_download_urls: weak override injects canned URL list. + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include + +#include "ppagecache_helpers.h" + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Test 1: priority tier boundary conditions */ +/* ------------------------------------------------------------------ */ +static void test_priority_tiers(void) { + /* Tier 0: usecnt < 2 */ + if (ppagecache_compute_page_priority(0) != 0 || + ppagecache_compute_page_priority(1) != 0) + FAIL("tier 0 (usecnt 0,1)", "got %d/%d", + ppagecache_compute_page_priority(0), + ppagecache_compute_page_priority(1)); + else + PASS("tier 0: usecnt in [0,2)"); + + /* Tier 1: usecnt in [2, 4) */ + if (ppagecache_compute_page_priority(2) != 1 || + ppagecache_compute_page_priority(3) != 1) + FAIL("tier 1 (usecnt 2,3)", "got %d/%d", + ppagecache_compute_page_priority(2), + ppagecache_compute_page_priority(3)); + else + PASS("tier 1: usecnt in [2,4)"); + + /* Tier 2: usecnt in [4, 8) */ + if (ppagecache_compute_page_priority(4) != 2 || + ppagecache_compute_page_priority(7) != 2) + FAIL("tier 2 (usecnt 4,7)", "got %d/%d", + ppagecache_compute_page_priority(4), + ppagecache_compute_page_priority(7)); + else + PASS("tier 2: usecnt in [4,8)"); + + /* Tier 3: usecnt in [8, 16) */ + if (ppagecache_compute_page_priority(8) != 3 || + ppagecache_compute_page_priority(15) != 3) + FAIL("tier 3 (usecnt 8,15)", "got %d/%d", + ppagecache_compute_page_priority(8), + ppagecache_compute_page_priority(15)); + else + PASS("tier 3: usecnt in [8,16)"); + + /* Tier 4: usecnt >= 16 */ + if (ppagecache_compute_page_priority(16) != 4 || + ppagecache_compute_page_priority(255)!= 4 || + ppagecache_compute_page_priority(UINT32_MAX) != 4) + FAIL("tier 4 (usecnt >=16)", "got %d/%d/%d", + ppagecache_compute_page_priority(16), + ppagecache_compute_page_priority(255), + ppagecache_compute_page_priority(UINT32_MAX)); + else + PASS("tier 4: usecnt >= 16 including UINT32_MAX"); +} + +/* ------------------------------------------------------------------ */ +/* Test 2: CRC verification */ +/* ------------------------------------------------------------------ */ +static void test_verify_crc(void) { + /* Build a known buffer */ + const char buf[64] = "Hello pagecache CRC test buffer 0123456789abcdef!"; + uint32_t good_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, sizeof(buf)); + + /* Known-good: must pass */ + if (ppagecache_verify_crc(buf, sizeof(buf), good_crc) != 0) + FAIL("verify_crc: known-good buffer", "returned non-zero"); + else + PASS("verify_crc: known-good buffer passes"); + + /* Single-byte corruption: must fail */ + char corrupt[64]; + memcpy(corrupt, buf, sizeof(corrupt)); + corrupt[7] ^= 0x01; /* flip one bit */ + if (ppagecache_verify_crc(corrupt, sizeof(corrupt), good_crc) != -1) + FAIL("verify_crc: corrupted buffer", "returned 0 (expected -1)"); + else + PASS("verify_crc: single-bit flip detected"); + + /* Wrong stored CRC */ + if (ppagecache_verify_crc(buf, sizeof(buf), good_crc + 1) != -1) + FAIL("verify_crc: wrong stored CRC", "returned 0 (expected -1)"); + else + PASS("verify_crc: wrong stored_crc detected"); + + /* Zero-length buffer: CRC of empty == PSYNC_CRC_INITIAL == 0 */ + uint32_t empty_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, 0); + if (ppagecache_verify_crc(buf, 0, empty_crc) != 0) + FAIL("verify_crc: zero-length buffer", "returned non-zero"); + else + PASS("verify_crc: zero-length buffer passes"); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: ppagecache_get_download_urls weak override */ +/* ------------------------------------------------------------------ */ + +/* Override the weak default: inject two canned URLs */ +static char *g_canned_urls[] = { + "https://content1.example.com/file", + "https://content2.example.com/file", + NULL +}; + +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout) { + (void)fileid; + (void)hash; + if (nout) *nout = 2; + return g_canned_urls; +} + +static void test_download_url_override(void) { + size_t n = 0; + char **urls = ppagecache_get_download_urls(9999, 0xdeadbeef, &n); + if (!urls || n != 2) + { FAIL("url override: count", "n=%zu urls=%p", n, (void*)urls); return; } + if (strcmp(urls[0], "https://content1.example.com/file") != 0 || + strcmp(urls[1], "https://content2.example.com/file") != 0) + FAIL("url override: content", "url0=%s url1=%s", urls[0], urls[1]); + else if (urls[2] != NULL) + FAIL("url override: NULL terminator", "urls[2]=%p", (void*)urls[2]); + else + PASS("ppagecache_get_download_urls: weak override returns canned URLs"); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + test_priority_tiers(); + test_verify_crc(); + test_download_url_override(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +} From fa2fd7c338443a008d7dcc1a715f9f47fc42a7c7 Mon Sep 17 00:00:00 2001 From: Levi Neely <141506390+lneely@users.noreply.github.com> Date: Wed, 11 Mar 2026 18:17:41 +0100 Subject: [PATCH 2/4] Revert "Testability refactor: extract helpers, add stress/send tests, ppagecache/pfs extraction (#383)" (#384) This reverts commit 4618e245b97e6c4186f2cef8ff3b089f777976ae. Co-authored-by: Levi Neely --- Makefile | 15 +- pclsync/pfs.c | 4 - pclsync/pfs_helpers.c | 148 ----------------- pclsync/pfs_helpers.h | 85 ---------- pclsync/pfsfolder.c | 2 +- pclsync/ppagecache.c | 1 - pclsync/ppagecache_helpers.c | 42 ----- pclsync/ppagecache_helpers.h | 64 -------- tests/unit-tests/test_pfs_helpers.c | 244 ---------------------------- tests/unit-tests/test_ppagecache.c | 154 ------------------ 10 files changed, 2 insertions(+), 757 deletions(-) delete mode 100644 pclsync/pfs_helpers.c delete mode 100644 pclsync/pfs_helpers.h delete mode 100644 pclsync/ppagecache_helpers.c delete mode 100644 pclsync/ppagecache_helpers.h delete mode 100644 tests/unit-tests/test_pfs_helpers.c delete mode 100644 tests/unit-tests/test_ppagecache.c diff --git a/Makefile b/Makefile index 48070a5..bf2338d 100644 --- a/Makefile +++ b/Makefile @@ -171,9 +171,7 @@ TEST_BINS := \ tests/test_pfstasks_tree \ tests/test_pfstasks_db \ tests/test_plocks \ - tests/test_pfsupload \ - tests/test_ppagecache \ - tests/test_pfs_helpers + tests/test_pfsupload .PHONY: test tests check clean-tests @@ -240,17 +238,6 @@ tests/test_pfsupload: $(UNIT_DIR)/test_pfsupload.c $(LIBDIR)/pfsupload_send.c $( $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ -Wl,--wrap=papi_send # redirect papi_send → __wrap_papi_send; GNU ld only (not macOS Apple ld) -tests/test_ppagecache: $(UNIT_DIR)/test_ppagecache.c $(LIBDIR)/ppagecache_helpers.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c - $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ - -tests/test_pfs_helpers: $(UNIT_DIR)/test_pfs_helpers.c $(LIBDIR)/pfs_helpers.c $(LIBDIR)/pfstasks_tree.c $(LIBDIR)/ptree.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c - $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ - -Wl,--wrap=pfs_crpt_plain_size \ - -Wl,--wrap=pfs_task_get_folder_tasks_rdlocked \ - -Wl,--wrap=ptimer_time - # ^ GNU ld only; --wrap stubs out encrypted-size, folder-task lookup, and timer - - tests/test_read_response: $(UNIT_DIR)/test_read_response.cpp rpcclient.cpp tests/stubs/test_stubs_cpp.c $(CXX) $(TEST_CXXFLAGS) $(CXXFLAGS) -o $@ $^ diff --git a/pclsync/pfs.c b/pclsync/pfs.c index 02b3c18..d163eab 100644 --- a/pclsync/pfs.c +++ b/pclsync/pfs.c @@ -63,7 +63,6 @@ #include "pssl.h" #include "pstatus.h" #include "psys.h" -#include "pfs_helpers.h" #include "ptimer.h" @@ -3807,9 +3806,6 @@ static int pfs_do_start() { myuid = getuid(); mygid = getgid(); - /* Keep pfs_helpers.c in sync for helper-based callers */ - pfs_stat_uid = myuid; - pfs_stat_gid = mygid; pthread_mutex_lock(&start_mutex); if (started) goto err00; diff --git a/pclsync/pfs_helpers.c b/pclsync/pfs_helpers.c deleted file mode 100644 index a10b1bd..0000000 --- a/pclsync/pfs_helpers.c +++ /dev/null @@ -1,148 +0,0 @@ -/* - * pfs_helpers.c — pure row→stat converters and overlay helpers. - * - * Dependencies: pfstasks_tree.c (for find_* functions), pfscrypto.h (for - * pfs_crpt_plain_size on encrypted files), plibs.h (psync_get_number macro). - * No psql calls, no FUSE, no network. - */ - -#include -#include -#include - -#include "pdbg.h" -#include "pfscrypto.h" -#include "pfs_helpers.h" -#include "plibs.h" /* psync_get_number */ -#include "ptimer.h" /* ptimer_time */ - -/* ------------------------------------------------------------------ */ -/* Globals — initialised here; overwritten by pfs.c at mount time */ -/* ------------------------------------------------------------------ */ - -uid_t pfs_stat_uid = 0; -gid_t pfs_stat_gid = 0; - -/* ------------------------------------------------------------------ */ -/* Row → stat converters */ -/* ------------------------------------------------------------------ */ - -void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf) { - psync_folderid_t folderid; - uint64_t mtime; - psync_fstask_folder_t *folder; - - folderid = (psync_folderid_t)psync_get_number(row[0]); - mtime = psync_get_number(row[3]); - - folder = pfs_task_get_folder_tasks_rdlocked(folderid); - if (folder && folder->mtime) - mtime = folder->mtime; - - memset(stbuf, 0, sizeof(*stbuf)); - stbuf->st_ino = PFS_FOLDERID_TO_INODE(folderid); - stbuf->st_ctime = (time_t)mtime; - stbuf->st_mtime = (time_t)mtime; - stbuf->st_atime = (time_t)mtime; - stbuf->st_mode = S_IFDIR | 0755; - stbuf->st_nlink = (nlink_t)(psync_get_number(row[4]) + 2); - stbuf->st_size = PFS_FS_BLOCK_SIZE; - stbuf->st_blocks = 1; - stbuf->st_blksize = PFS_FS_BLOCK_SIZE; - stbuf->st_uid = pfs_stat_uid; - stbuf->st_gid = pfs_stat_gid; -} - -void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, - uint32_t flags) { - uint64_t size = psync_get_number(row[1]); - psync_fileid_t fileid = (psync_fileid_t)psync_get_number(row[4]); - - if (flags & PSYNC_FOLDER_FLAG_ENCRYPTED) - size = pfs_crpt_plain_size(size); - - memset(stbuf, 0, sizeof(*stbuf)); - stbuf->st_ino = PFS_FILEID_TO_INODE(fileid); - stbuf->st_ctime = (time_t)psync_get_number(row[3]); - stbuf->st_mtime = stbuf->st_ctime; - stbuf->st_atime = stbuf->st_ctime; - stbuf->st_mode = S_IFREG | 0644; - stbuf->st_nlink = 1; - stbuf->st_size = (off_t)size; - stbuf->st_blocks = (blkcnt_t)((size + 511) / 512); - stbuf->st_blksize = PFS_FS_BLOCK_SIZE; - stbuf->st_uid = pfs_stat_uid; - stbuf->st_gid = pfs_stat_gid; -} - -void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf) { - uint64_t mtime; - psync_fstask_folder_t *folder; - - folder = pfs_task_get_folder_tasks_rdlocked(mk->folderid); - mtime = (folder && folder->mtime) ? folder->mtime : (uint64_t)mk->mtime; - - memset(stbuf, 0, sizeof(*stbuf)); - stbuf->st_ino = (mk->folderid >= 0) - ? PFS_FOLDERID_TO_INODE(mk->folderid) - : PFS_TASKID_TO_INODE(-mk->folderid); - stbuf->st_ctime = (time_t)mtime; - stbuf->st_mtime = (time_t)mtime; - stbuf->st_atime = (time_t)mtime; - stbuf->st_mode = S_IFDIR | 0755; - stbuf->st_nlink = (nlink_t)(mk->subdircnt + 2); - stbuf->st_size = PFS_FS_BLOCK_SIZE; - stbuf->st_blocks = 1; - stbuf->st_blksize = PFS_FS_BLOCK_SIZE; - stbuf->st_uid = pfs_stat_uid; - stbuf->st_gid = pfs_stat_gid; -} - -/* ------------------------------------------------------------------ */ -/* Task overlay */ -/* ------------------------------------------------------------------ */ - -int pfs_apply_task_overlay(struct stat *stbuf, - psync_fstask_folder_t *folder, - const char *name, uint32_t flags) { - if (!folder) - return 0; - - /* Pending mkdir: show the directory */ - psync_fstask_mkdir_t *mk = pfs_task_find_mkdir(folder, name, 0); - if (mk) { - if (mk->flags & PSYNC_FOLDER_FLAG_INVISIBLE) - return -1; - pfs_mkdir_to_folder_stat(mk, stbuf); - return 1; - } - - /* Pending rmdir: hide the directory */ - if (pfs_task_find_rmdir(folder, name, 0)) - return -1; - - /* Pending unlink: hide the file */ - if (pfs_task_find_unlink(folder, name, 0)) - return -1; - - /* Pending creat with fileid==0 (new local file, no SQL needed): - * return a minimal stat for the new file. */ - psync_fstask_creat_t *cr = pfs_task_find_creat(folder, name, 0); - if (cr && cr->fileid == 0) { - time_t now = ptimer_time(); - memset(stbuf, 0, sizeof(*stbuf)); - stbuf->st_ctime = now; - stbuf->st_mtime = now; - stbuf->st_atime = now; - stbuf->st_mode = S_IFREG | 0644; - stbuf->st_nlink = 1; - stbuf->st_size = 0; - stbuf->st_blocks = 0; - stbuf->st_blksize = PFS_FS_BLOCK_SIZE; - stbuf->st_uid = pfs_stat_uid; - stbuf->st_gid = pfs_stat_gid; - return 2; - } - - return 0; -} diff --git a/pclsync/pfs_helpers.h b/pclsync/pfs_helpers.h deleted file mode 100644 index 8f0eb5b..0000000 --- a/pclsync/pfs_helpers.h +++ /dev/null @@ -1,85 +0,0 @@ -/* - * pfs_helpers.h — pure row→stat converters and overlay helpers extracted - * from pfs.c so they can be unit-tested without a live FUSE mount or psql - * connection. - * - * Included by pfs.c and by tests/unit-tests/test_pfs_helpers.c. - */ -#ifndef PFS_HELPERS_H -#define PFS_HELPERS_H - -#include -#include -#include - -#include "pfoldersync.h" /* psync_folderid_t, psync_fileid_t */ -#include "pfstasks.h" /* psync_fstask_folder_t, psync_fstask_mkdir_t, … */ -#include "pfsfolder.h" /* psync_fspath_t */ -#include "psql.h" /* psync_variant_row */ - -/* - * Inode-number helpers — must match the definitions used in pfs.c. - * Put here so pfs_helpers.c and tests share a single definition. - */ -#define PFS_FOLDERID_TO_INODE(fid) ((fid) * 3) -#define PFS_FILEID_TO_INODE(fid) ((fid) * 3 + 1) -#define PFS_TASKID_TO_INODE(tid) ((tid) * 3 + 2) -#define PFS_FS_BLOCK_SIZE 4096 - -/* - * Owner uid/gid used when populating struct stat. Initialised to 0 (root) - * by pfs_helpers.c. pfs.c overwrites them with the real process owner at - * init time; tests leave them at 0. - */ -extern uid_t pfs_stat_uid; -extern gid_t pfs_stat_gid; - -/* - * pfs_row_to_folder_stat — convert a psql folder row to struct stat. - * - * Row column layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt - * Applies any pending in-memory mtime from the folder task queue. - * No SQL, no FUSE calls. - */ -void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf); - -/* - * pfs_row_to_file_stat — convert a psql file row to struct stat. - * - * Row column layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id - * flags: PSYNC_FOLDER_FLAG_ENCRYPTED triggers encrypted-size conversion via - * pfs_crpt_plain_size(); tests pass flags=0 to skip crypto. - * No SQL, no FUSE calls. - */ -void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, - uint32_t flags); - -/* - * pfs_mkdir_to_folder_stat — convert an in-memory mkdir task to struct stat. - * No SQL, no FUSE calls. - */ -void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf); - -/* - * pfs_apply_task_overlay — check an in-memory folder task queue for a - * pending operation on `name` and update `stbuf` accordingly. - * - * Returns: - * 1 mkdir overlay applied (stbuf filled as a directory) - * 2 creat overlay applied (stbuf filled as a new file, fileid=0) - * -1 rmdir or unlink pending → entry should be hidden (ENOENT) - * 0 no applicable overlay found - * - * No SQL, no network I/O. folder may be NULL (returns 0 immediately). - */ -int pfs_apply_task_overlay(struct stat *stbuf, - psync_fstask_folder_t *folder, - const char *name, uint32_t flags); - -/* - * pfs_fldr_resolve_path — declared __attribute__((weak)) in pfsfolder.c so - * tests can override path resolution without a live FUSE / psql stack. - * The declaration here is informational only (it lives in pfsfolder.h). - */ - -#endif /* PFS_HELPERS_H */ diff --git a/pclsync/pfsfolder.c b/pclsync/pfsfolder.c index 59d9dc4..2cf65e2 100644 --- a/pclsync/pfsfolder.c +++ b/pclsync/pfsfolder.c @@ -148,7 +148,7 @@ static void check_userid(uint64_t userid, uint64_t folderid, do_check_userid(userid, folderid, shareid); } -__attribute__((weak)) psync_fspath_t *pfs_fldr_resolve_path(const char *path) { +psync_fspath_t *pfs_fldr_resolve_path(const char *path) { psync_fsfolderid_t cfolderid; const char *sl; psync_fstask_folder_t *folder; diff --git a/pclsync/ppagecache.c b/pclsync/ppagecache.c index e15e538..08d3b30 100644 --- a/pclsync/ppagecache.c +++ b/pclsync/ppagecache.c @@ -43,7 +43,6 @@ #include "pmem.h" #include "pnetlibs.h" #include "ppagecache.h" -#include "ppagecache_helpers.h" #include "ppath.h" #include "prun.h" #include "psettings.h" diff --git a/pclsync/ppagecache_helpers.c b/pclsync/ppagecache_helpers.c deleted file mode 100644 index c6437bd..0000000 --- a/pclsync/ppagecache_helpers.c +++ /dev/null @@ -1,42 +0,0 @@ -/* - * ppagecache_helpers.c — pure helpers extracted from ppagecache.c. - * - * Only deps: pcrc32c.h (CRC computation) and pfoldersync.h (psync_fileid_t). - * No psql, no networking, no threading — safe to link into unit tests. - */ - -#include "ppagecache_helpers.h" - -/* ------------------------------------------------------------------ */ -/* Priority / LRU tier */ -/* ------------------------------------------------------------------ */ - -uint8_t ppagecache_compute_page_priority(uint32_t usecnt) { - if (usecnt >= PPAGECACHE_TIER4_THRESHOLD) return 4; - if (usecnt >= PPAGECACHE_TIER3_THRESHOLD) return 3; - if (usecnt >= PPAGECACHE_TIER2_THRESHOLD) return 2; - if (usecnt >= PPAGECACHE_TIER1_THRESHOLD) return 1; - return 0; -} - -/* ------------------------------------------------------------------ */ -/* CRC verification */ -/* ------------------------------------------------------------------ */ - -int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc) { - uint32_t computed = pcrc32c_compute(PSYNC_CRC_INITIAL, data, size); - return (computed == stored_crc) ? 0 : -1; -} - -/* ------------------------------------------------------------------ */ -/* Download-URL seam (weak default: no-op) */ -/* ------------------------------------------------------------------ */ - -__attribute__((weak)) -char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, - size_t *nout) { - (void)fileid; - (void)hash; - if (nout) *nout = 0; - return NULL; /* caller falls through to real API */ -} diff --git a/pclsync/ppagecache_helpers.h b/pclsync/ppagecache_helpers.h deleted file mode 100644 index ccc07eb..0000000 --- a/pclsync/ppagecache_helpers.h +++ /dev/null @@ -1,64 +0,0 @@ -/* - * ppagecache_helpers.h — pure, separately-compilable helpers extracted from - * ppagecache.c to enable unit testing without the full page-cache stack. - * - * Included by ppagecache.c and by tests/unit-tests/test_ppagecache.c. - */ -#ifndef PPAGECACHE_HELPERS_H -#define PPAGECACHE_HELPERS_H - -#include -#include - -#include "pcrc32c.h" /* PSYNC_CRC_INITIAL, pcrc32c_compute */ -#include "pfoldersync.h" /* psync_fileid_t */ - -/* - * LRU tier thresholds (must match the pagecache_entry_cmp_usecnt_lastuse* - * comparators in ppagecache.c). - */ -#define PPAGECACHE_TIER1_THRESHOLD 2u -#define PPAGECACHE_TIER2_THRESHOLD 4u -#define PPAGECACHE_TIER3_THRESHOLD 8u -#define PPAGECACHE_TIER4_THRESHOLD 16u - -/* - * ppagecache_compute_page_priority — assign an eviction-priority tier to a - * cached page based on its access count. - * - * tier 0 usecnt < 2 — cold, evict first - * tier 1 usecnt < 4 - * tier 2 usecnt < 8 - * tier 3 usecnt < 16 - * tier 4 usecnt >= 16 — hot, keep longest - * - * Pure function: no side effects, no I/O. - */ -uint8_t ppagecache_compute_page_priority(uint32_t usecnt); - -/* - * ppagecache_verify_crc — verify the CRC32c of `size` bytes at `data` - * against `stored_crc`. - * - * Returns 0 if the CRC matches (page is intact). - * Returns -1 if the CRC differs (page is corrupt). - * - * Pure function: no side effects. - */ -int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc); - -/* - * ppagecache_get_download_urls — weak seam for injecting canned download - * URLs in tests. The default implementation returns NULL (caller falls - * through to the real API). Tests override this to return a - * NULL-terminated string array without making network calls. - * - * On success sets *nout to the number of URLs and returns a pointer to - * a NULL-terminated array of C strings. The array is valid until the next - * call or until the override frees it. - */ -__attribute__((weak)) -char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, - size_t *nout); - -#endif /* PPAGECACHE_HELPERS_H */ diff --git a/tests/unit-tests/test_pfs_helpers.c b/tests/unit-tests/test_pfs_helpers.c deleted file mode 100644 index 0c72178..0000000 --- a/tests/unit-tests/test_pfs_helpers.c +++ /dev/null @@ -1,244 +0,0 @@ -/* - * Test: pfs_helpers.c — row→stat converters and task overlay - * - * Covers: - * 1. pfs_row_to_folder_stat: st_ino, st_mode, st_nlink, st_mtime populated - * from the variant row; st_uid/gid come from pfs_stat_uid/gid globals. - * 2. pfs_row_to_file_stat: st_ino, st_mode, st_size, st_ctime populated - * (non-encrypted path, flags=0). - * 3. pfs_apply_task_overlay: mkdir pending → stat filled as dir; - * rmdir pending → -1; no overlay → 0. - * 4. pfs_fldr_resolve_path weak override: replacement injects a fake path - * without a FUSE mount or psql connection. - */ - -#define _POSIX_C_SOURCE 200809L -#include -#include -#include -#include -#include - -#include "pfs_helpers.h" -#include "pfstasks_tree.h" /* pfs_task_insert_into_tree, pfs_task_find_* */ -#include "plibs.h" /* PSYNC_TNUMBER, psync_variant */ - -/* ------------------------------------------------------------------ */ -/* --wrap stubs: pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, - * ptimer_time — none of these should be called in the pure-tree / non- - * encrypted test paths; return safe no-op values just in case. */ - -psync_fstask_folder_t *__wrap_pfs_task_get_folder_tasks_rdlocked( - psync_fsfolderid_t folderid) { - (void)folderid; - return NULL; /* no in-memory mtime override in tests */ -} - -uint64_t __wrap_pfs_crpt_plain_size(uint64_t cryptosize) { - return cryptosize; /* identity for non-encrypted tests */ -} - -time_t __wrap_ptimer_time(void) { - return 9999; /* fixed timestamp for deterministic tests */ -} - -/* ------------------------------------------------------------------ */ -static int passes = 0, failures = 0; -#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) -#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) - -/* ------------------------------------------------------------------ */ -/* Helper: build a psync_variant holding a uint64_t number */ -/* ------------------------------------------------------------------ */ -static psync_variant mknum(uint64_t v) { - psync_variant pv; - memset(&pv, 0, sizeof(pv)); - pv.type = PSYNC_TNUMBER; - pv.num = v; - return pv; -} - -/* ------------------------------------------------------------------ */ -/* Test 1: pfs_row_to_folder_stat */ -/* ------------------------------------------------------------------ */ -static void test_folder_stat(void) { - /* - * Folder row layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt - */ - psync_variant row[5]; - row[0] = mknum(100); /* folderid = 100 */ - row[1] = mknum(0755); /* permissions */ - row[2] = mknum(1000); /* ctime */ - row[3] = mknum(2000); /* mtime */ - row[4] = mknum(3); /* subdircnt = 3 → nlink = 5 */ - - struct stat st; - memset(&st, 0, sizeof(st)); - pfs_row_to_folder_stat(row, &st); - - if (st.st_ino != PFS_FOLDERID_TO_INODE(100)) - FAIL("folder_stat: st_ino", "expected %lu got %lu", - (unsigned long)PFS_FOLDERID_TO_INODE(100), (unsigned long)st.st_ino); - else if (!S_ISDIR(st.st_mode)) - FAIL("folder_stat: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); - else if (st.st_nlink != 5) - FAIL("folder_stat: st_nlink", "expected 5 got %lu", (unsigned long)st.st_nlink); - else if (st.st_mtime != 2000) - FAIL("folder_stat: st_mtime", "expected 2000 got %ld", (long)st.st_mtime); - else if (st.st_uid != pfs_stat_uid || st.st_gid != pfs_stat_gid) - FAIL("folder_stat: uid/gid", "uid=%d gid=%d", - (int)st.st_uid, (int)st.st_gid); - else - PASS("pfs_row_to_folder_stat: ino/mode/nlink/mtime/uid/gid correct"); -} - -/* ------------------------------------------------------------------ */ -/* Test 2: pfs_row_to_file_stat (non-encrypted, flags=0) */ -/* ------------------------------------------------------------------ */ -static void test_file_stat(void) { - /* - * File row layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id - */ - psync_variant row[5]; - row[0] = mknum(0); /* name (unused in stat) */ - row[1] = mknum(4096); /* size = 4096 bytes */ - row[2] = mknum(3000); /* ctime */ - row[3] = mknum(3000); /* mtime */ - row[4] = mknum(77); /* fileid = 77 */ - - struct stat st; - memset(&st, 0, sizeof(st)); - pfs_row_to_file_stat(row, &st, 0 /* non-encrypted */); - - if (st.st_ino != PFS_FILEID_TO_INODE(77)) - FAIL("file_stat: st_ino", "expected %lu got %lu", - (unsigned long)PFS_FILEID_TO_INODE(77), (unsigned long)st.st_ino); - else if (!S_ISREG(st.st_mode)) - FAIL("file_stat: S_ISREG", "mode=0%o", (unsigned)st.st_mode); - else if (st.st_size != 4096) - FAIL("file_stat: st_size", "expected 4096 got %ld", (long)st.st_size); - else if (st.st_ctime != 3000) - FAIL("file_stat: st_ctime", "expected 3000 got %ld", (long)st.st_ctime); - else if (st.st_nlink != 1) - FAIL("file_stat: st_nlink", "expected 1 got %lu", (unsigned long)st.st_nlink); - else - PASS("pfs_row_to_file_stat: ino/mode/size/ctime/nlink correct"); -} - -/* ------------------------------------------------------------------ */ -/* Test 3: pfs_apply_task_overlay */ -/* ------------------------------------------------------------------ */ - -/* Helper: allocate a psync_fstask_mkdir_t and insert it into folder */ -static psync_fstask_mkdir_t *make_mkdir(psync_fstask_folder_t *f, - const char *name, - uint64_t taskid, - psync_fsfolderid_t folderid) { - size_t len = strlen(name) + 1; - psync_fstask_mkdir_t *mk = (psync_fstask_mkdir_t *) - calloc(1, offsetof(psync_fstask_mkdir_t, name) + len); - mk->taskid = taskid; - mk->folderid = folderid; - mk->mtime = 5000; - mk->flags = 0; - memcpy(mk->name, name, len); - pfs_task_insert_into_tree(&f->mkdirs, offsetof(psync_fstask_mkdir_t, name), - &mk->tree); - return mk; -} - -static psync_fstask_rmdir_t *make_rmdir(psync_fstask_folder_t *f, - const char *name, - uint64_t taskid) { - size_t len = strlen(name) + 1; - psync_fstask_rmdir_t *rm = (psync_fstask_rmdir_t *) - calloc(1, offsetof(psync_fstask_rmdir_t, name) + len); - rm->taskid = taskid; - memcpy(rm->name, name, len); - pfs_task_insert_into_tree(&f->rmdirs, offsetof(psync_fstask_rmdir_t, name), - &rm->tree); - return rm; -} - -static void test_apply_task_overlay(void) { - /* NULL folder → 0 */ - struct stat st; - int rc = pfs_apply_task_overlay(&st, NULL, "any", 0); - if (rc != 0) - { FAIL("overlay NULL folder", "expected 0 got %d", rc); } - else - PASS("pfs_apply_task_overlay: NULL folder returns 0"); - - /* mkdir pending → 1, stbuf filled as directory */ - psync_fstask_folder_t f; - memset(&f, 0, sizeof(f)); - f.folderid = 200; - psync_fstask_mkdir_t *mk = make_mkdir(&f, "newdir", 10, -10); - - memset(&st, 0, sizeof(st)); - rc = pfs_apply_task_overlay(&st, &f, "newdir", 0); - if (rc != 1) - FAIL("overlay mkdir: rc=1", "got %d", rc); - else if (!S_ISDIR(st.st_mode)) - FAIL("overlay mkdir: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); - else - PASS("pfs_apply_task_overlay: mkdir pending → rc=1, stbuf is a dir"); - - /* rmdir pending → -1 */ - psync_fstask_rmdir_t *rm = make_rmdir(&f, "olddir", 20); - memset(&st, 0, sizeof(st)); - rc = pfs_apply_task_overlay(&st, &f, "olddir", 0); - if (rc != -1) - FAIL("overlay rmdir: rc=-1", "got %d", rc); - else - PASS("pfs_apply_task_overlay: rmdir pending → rc=-1 (ENOENT)"); - - /* no overlay for absent name → 0 */ - rc = pfs_apply_task_overlay(&st, &f, "noentry", 0); - if (rc != 0) - FAIL("overlay no match: rc=0", "got %d", rc); - else - PASS("pfs_apply_task_overlay: no pending task for name → rc=0"); - - free(mk); free(rm); -} - -/* ------------------------------------------------------------------ */ -/* Test 4: pfs_fldr_resolve_path weak override */ -/* ------------------------------------------------------------------ */ - -static psync_fspath_t g_fake_path; -static int g_resolve_called = 0; - -psync_fspath_t *pfs_fldr_resolve_path(const char *path) { - (void)path; - g_resolve_called++; - return &g_fake_path; -} - -static void test_resolve_path_override(void) { - g_resolve_called = 0; - memset(&g_fake_path, 0, sizeof(g_fake_path)); - g_fake_path.folderid = 999; - - psync_fspath_t *r = pfs_fldr_resolve_path("/test/path"); - if (!r || r->folderid != 999) - FAIL("resolve override: returns fake path", "folderid=%lld", - r ? (long long)r->folderid : -1LL); - else if (g_resolve_called != 1) - FAIL("resolve override: called once", "called=%d", g_resolve_called); - else - PASS("pfs_fldr_resolve_path: weak override injects fake path"); -} - -/* ------------------------------------------------------------------ */ -int main(void) { - /* tests use uid/gid = 0 (default from pfs_helpers.c init) */ - test_folder_stat(); - test_file_stat(); - test_apply_task_overlay(); - test_resolve_path_override(); - - printf("\n%d passed, %d failed\n", passes, failures); - return failures ? 1 : 0; -} diff --git a/tests/unit-tests/test_ppagecache.c b/tests/unit-tests/test_ppagecache.c deleted file mode 100644 index 09f40b6..0000000 --- a/tests/unit-tests/test_ppagecache.c +++ /dev/null @@ -1,154 +0,0 @@ -/* - * Test: ppagecache_helpers.c — priority tiers, CRC verification, URL seam - * - * Covers: - * 1. ppagecache_compute_page_priority: boundary conditions between all five - * tiers (0–4) and representative interior values. - * 2. ppagecache_verify_crc: known-good buffer, single-bit flip, zero-length. - * 3. ppagecache_get_download_urls: weak override injects canned URL list. - */ - -#define _POSIX_C_SOURCE 200809L -#include -#include -#include -#include - -#include "ppagecache_helpers.h" - -/* ------------------------------------------------------------------ */ -static int passes = 0, failures = 0; -#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) -#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) - -/* ------------------------------------------------------------------ */ -/* Test 1: priority tier boundary conditions */ -/* ------------------------------------------------------------------ */ -static void test_priority_tiers(void) { - /* Tier 0: usecnt < 2 */ - if (ppagecache_compute_page_priority(0) != 0 || - ppagecache_compute_page_priority(1) != 0) - FAIL("tier 0 (usecnt 0,1)", "got %d/%d", - ppagecache_compute_page_priority(0), - ppagecache_compute_page_priority(1)); - else - PASS("tier 0: usecnt in [0,2)"); - - /* Tier 1: usecnt in [2, 4) */ - if (ppagecache_compute_page_priority(2) != 1 || - ppagecache_compute_page_priority(3) != 1) - FAIL("tier 1 (usecnt 2,3)", "got %d/%d", - ppagecache_compute_page_priority(2), - ppagecache_compute_page_priority(3)); - else - PASS("tier 1: usecnt in [2,4)"); - - /* Tier 2: usecnt in [4, 8) */ - if (ppagecache_compute_page_priority(4) != 2 || - ppagecache_compute_page_priority(7) != 2) - FAIL("tier 2 (usecnt 4,7)", "got %d/%d", - ppagecache_compute_page_priority(4), - ppagecache_compute_page_priority(7)); - else - PASS("tier 2: usecnt in [4,8)"); - - /* Tier 3: usecnt in [8, 16) */ - if (ppagecache_compute_page_priority(8) != 3 || - ppagecache_compute_page_priority(15) != 3) - FAIL("tier 3 (usecnt 8,15)", "got %d/%d", - ppagecache_compute_page_priority(8), - ppagecache_compute_page_priority(15)); - else - PASS("tier 3: usecnt in [8,16)"); - - /* Tier 4: usecnt >= 16 */ - if (ppagecache_compute_page_priority(16) != 4 || - ppagecache_compute_page_priority(255)!= 4 || - ppagecache_compute_page_priority(UINT32_MAX) != 4) - FAIL("tier 4 (usecnt >=16)", "got %d/%d/%d", - ppagecache_compute_page_priority(16), - ppagecache_compute_page_priority(255), - ppagecache_compute_page_priority(UINT32_MAX)); - else - PASS("tier 4: usecnt >= 16 including UINT32_MAX"); -} - -/* ------------------------------------------------------------------ */ -/* Test 2: CRC verification */ -/* ------------------------------------------------------------------ */ -static void test_verify_crc(void) { - /* Build a known buffer */ - const char buf[64] = "Hello pagecache CRC test buffer 0123456789abcdef!"; - uint32_t good_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, sizeof(buf)); - - /* Known-good: must pass */ - if (ppagecache_verify_crc(buf, sizeof(buf), good_crc) != 0) - FAIL("verify_crc: known-good buffer", "returned non-zero"); - else - PASS("verify_crc: known-good buffer passes"); - - /* Single-byte corruption: must fail */ - char corrupt[64]; - memcpy(corrupt, buf, sizeof(corrupt)); - corrupt[7] ^= 0x01; /* flip one bit */ - if (ppagecache_verify_crc(corrupt, sizeof(corrupt), good_crc) != -1) - FAIL("verify_crc: corrupted buffer", "returned 0 (expected -1)"); - else - PASS("verify_crc: single-bit flip detected"); - - /* Wrong stored CRC */ - if (ppagecache_verify_crc(buf, sizeof(buf), good_crc + 1) != -1) - FAIL("verify_crc: wrong stored CRC", "returned 0 (expected -1)"); - else - PASS("verify_crc: wrong stored_crc detected"); - - /* Zero-length buffer: CRC of empty == PSYNC_CRC_INITIAL == 0 */ - uint32_t empty_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, 0); - if (ppagecache_verify_crc(buf, 0, empty_crc) != 0) - FAIL("verify_crc: zero-length buffer", "returned non-zero"); - else - PASS("verify_crc: zero-length buffer passes"); -} - -/* ------------------------------------------------------------------ */ -/* Test 3: ppagecache_get_download_urls weak override */ -/* ------------------------------------------------------------------ */ - -/* Override the weak default: inject two canned URLs */ -static char *g_canned_urls[] = { - "https://content1.example.com/file", - "https://content2.example.com/file", - NULL -}; - -char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, - size_t *nout) { - (void)fileid; - (void)hash; - if (nout) *nout = 2; - return g_canned_urls; -} - -static void test_download_url_override(void) { - size_t n = 0; - char **urls = ppagecache_get_download_urls(9999, 0xdeadbeef, &n); - if (!urls || n != 2) - { FAIL("url override: count", "n=%zu urls=%p", n, (void*)urls); return; } - if (strcmp(urls[0], "https://content1.example.com/file") != 0 || - strcmp(urls[1], "https://content2.example.com/file") != 0) - FAIL("url override: content", "url0=%s url1=%s", urls[0], urls[1]); - else if (urls[2] != NULL) - FAIL("url override: NULL terminator", "urls[2]=%p", (void*)urls[2]); - else - PASS("ppagecache_get_download_urls: weak override returns canned URLs"); -} - -/* ------------------------------------------------------------------ */ -int main(void) { - test_priority_tiers(); - test_verify_crc(); - test_download_url_override(); - - printf("\n%d passed, %d failed\n", passes, failures); - return failures ? 1 : 0; -} From 85601124d46c657d87d50d3da1abed41329d1368 Mon Sep 17 00:00:00 2001 From: Levi Neely <141506390+lneely@users.noreply.github.com> Date: Wed, 11 Mar 2026 19:24:39 +0100 Subject: [PATCH 3/4] Testability refactor: extract helpers, add stress/send tests, ppagecache/pfs extraction (#383) (#385) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add unit test for psync_task_free refcount fix (#377) Adds tests/unit-tests/test_ptask_free.c to verify all code paths of the psync_task_free fix from #377: single-owner free, last-ref destroy, non-last-ref decrement, READY task signaling, and lock-before-refcnt ordering. All 5 tests pass. Also adds compiled binary to .gitignore. * Refactor test_ptask_free to link production code via --wrap Extract psync_task_free + psync_task_destroy (and their static helpers psync_task_dec_refcnt, psync_task_entry) from ptask.c into a new separately-compilable unit pclsync/ptask_free.c. Add pclsync/ptask_free_internal.h to expose the internal struct layout (struct psync_task_manager_t_ / struct psync_task_t_) for test use without pulling in ptask.c's heavyweight transitive dependencies. Rewrite tests/unit-tests/test_ptask_free.c to: - Include ptask_free_internal.h instead of duplicating structs inline - Call the real psync_task_free() rather than a local replica - Intercept pthread_mutex_lock/unlock and pmem_free via --wrap linker flags to observe lock discipline and detect destroy invocations Update the Makefile test_ptask_free target to link pclsync/ptask_free.c and pass the required --wrap flags. Production build unchanged: ptask_free.o is picked up automatically by the existing wildcard COBJ rule. * Implement Tasks #3, #4, #5: tree tests, pfstasks tree layer, DB harness Task #3 — Unit tests for ptree and pintervaltree tests/unit-tests/test_ptree.c: 8 tests covering single-node insert, in-order traversal after arbitrary and reverse inserts, BST lookup, leaf/root/all-node deletion, and ptree_for_each visitation. tests/unit-tests/test_pintervaltree.c: 18 tests covering single add, non-overlapping, overlapping/adjacent/contained/spanning merges, chain merge, remove middle split, remove exact/left/right/spanning, cut_end, first_interval_containing_or_after, and free(NULL). Task #4 — Extract pfstasks tree layer pclsync/pfstasks_tree.h + pclsync/pfstasks_tree.c: pure tree layer (zero psql calls) extracted from pfstasks.c — pfs_task_search_tree, pfs_task_walk_tree (static helpers), pfs_task_insert_into_tree, pfs_task_find_mkdir/rmdir/creat/unlink, pfs_task_find_mkdir_by_folderid, pfs_task_find_creat_by_fileid. pclsync/pfstasks.c: #includes pfstasks_tree.h; all moved functions removed; all callers unchanged. tests/unit-tests/test_pfstasks_tree.c: 13 tests using direct tree construction (no DB) to verify find-by-name, taskid discrimination, find-by-numeric-id, and empty-folder edge cases. Task #5 — psql in-memory harness + pfstasks DB tests tests/helpers/psql_test_helpers.h + .c: lightweight harness that opens :memory: via sqlite3_open, enables PRAGMA foreign_keys=ON, and applies the full PSYNC_DATABASE_STRUCTURE schema. Exposes psql_test_db(), psql_test_exec(), psql_test_insert_fstask(), psql_test_count_fstask/fstaskdepend(). No dependency on psql.c. tests/unit-tests/test_pfstasks_db.c: 10 tests verifying schema creation, fstask insertion/query, fstaskdepend insertion, CASCADE DELETE propagation, FK enforcement, rmdir-blocking SQL pattern, creat-after-unlink sequencing, and open/close idempotence. All 11 new tests pass; production build clean. * Fix P1 review findings in pfstasks_db test and helpers 1. Check psql_test_exec() return values in test_cascade_delete() and test_creat_after_unlink() consistently with test_fstaskdepend_insert(). 2. Remove dead dep_cnt variable and (void)dep_cnt suppressor from test_creat_after_unlink(). 3. Change SQLITE_STATIC → SQLITE_TRANSIENT for text1 binding in psql_test_insert_fstask() to avoid dangling-pointer footgun on future reuse. * Fix ASAN/LSAN failures: ppath_home stack-use-after-scope + intentional leak pclsync/ppath.c: Move buff[4096] to function scope in ppath_home() so the pointer stored in dir via result->pw_dir remains live through the putil_strdup(dir) call. Previously buff went out of scope at the if-block close, causing a stack-use-after-scope ASAN report on every call that fell through the getpwuid_r path. tests/unit-tests/test_ptools_errptr.c: run_unfixed() intentionally leaks errPtr to demonstrate the pre-fix bug. Wrap the allocation with LSAN_DISABLE() / LSAN_ENABLE() so LSAN does not abort the process at exit before stdio flushes, which was causing a non-zero exit code. The guard uses nested #ifdef/__has_feature to remain compatible with both GCC (__SANITIZE_ADDRESS__) and Clang (__has_feature(address_sanitizer)) without triggering "missing binary operator" errors on GCC. * Implement Tasks #11 and #12: plocks stress test + pfsupload send tests Task #11 — plocks.c stress test (test_plocks.c) 7 tests: basic rdlock/wrlock round-trip, recursive TLS counting (same thread acquires rdlock N times; unlock only releases on final decrement), upgrade under contention (N readers + towrlock; barrier-synchronized), writer starvation prevention (sustained reader load; writer acquires within 500ms), N-reader + M-writer counter-integrity stress test (ASAN), and try-variant contention (trywrlock fails when another thread holds rdlock). TSAN note documented: custom lock internals require ASAN-only when ThreadSanitizer annotations are absent. Task #12 — pfsupload send-function tests (pfsupload_send.c/h + test_pfsupload.c) Extract psync_send_task_mkdir and psync_send_task_rmdir from pfsupload.c into pclsync/pfsupload_send.c as non-static pfsupload_send_mkdir/rmdir. Expose fsupload_task_t struct via pclsync/pfsupload_send.h. Add __attribute__((weak)) get_urls() as an injectable URL seam for large- upload paths. pfsupload.c updated to include pfsupload_send.h and use the renamed functions in its dispatch table; pfsupload_send.o is automatically picked up by the production wildcard build. 5 tests: mkdir (non-encrypted) command + folderid param, mkdir (encrypted) key param present, rmdir command + sfolderid, API error path (papi_send failure → -1), get_urls() weak override. Uses --wrap=papi_send to intercept API calls and socketpair() to provide a valid psock_t without real network I/O. * P2 cleanup: comments, make_fake_api stack alloc, find_str_param fix 1. test_plocks.c: add comment to test_upgrade_under_contention clarifying it verifies towrlock completion and holding_wrlock; notes that concurrent exclusivity is covered by test_stress(). 2. test_pfsupload.c / make_fake_api: replace static-local psock_t with caller-supplied stack allocation (out parameter) to eliminate the multiple-calls-per-test footgun. 3. test_pfsupload.c / find_str_param: replace ternary `paramnamelen == strlen ? paramname : ""` with explicit length check + strncmp, matching the cleaner pattern used in find_num_param. 4. Makefile: add comment next to -Wl,--wrap=papi_send noting it redirects papi_send to __wrap_papi_send and is GNU ld only (not macOS Apple ld). * Implement Tasks #19 and #20: ppagecache + pfs helper extraction Task #19 — ppagecache.c decomposition (ppagecache_helpers.c/h) ppagecache_compute_page_priority(usecnt): pure function returning the LRU eviction tier (0–4) that matches the five pagecache_entry_cmp_* sort comparators in ppagecache.c (thresholds 2/4/8/16). ppagecache_verify_crc(data, size, stored_crc): wraps pcrc32c_compute and compares; returns 0 on match, -1 on mismatch. ppagecache_get_download_urls(fileid, hash, nout): __attribute__((weak)) URL-injection seam; default returns NULL (falls through to real API). ppagecache.c updated to include ppagecache_helpers.h. test_ppagecache.c: 10 tests covering tier boundary conditions (0/1/2/3/4 including UINT32_MAX), CRC match, single-bit flip, wrong stored CRC, zero-length buffer, and weak URL override. Task #20 — pfs.c helper extraction (pfs_helpers.c/h) pfs_row_to_folder_stat(row, stbuf): converts psql folder row → struct stat; uses pfs_task_get_folder_tasks_rdlocked for in-memory mtime. pfs_row_to_file_stat(row, stbuf, flags): converts psql file row → stat; encrypted path calls pfs_crpt_plain_size. pfs_mkdir_to_folder_stat(mk, stbuf): converts in-memory mkdir task → stat (no SQL). pfs_apply_task_overlay(stbuf, folder, name, flags): applies pending mkdir/rmdir/unlink/creat-new overlays from the in-memory task queue; returns 1/2/-1/0. pfs_stat_uid/gid: exported globals; pfs.c syncs them from myuid/mygid. pfs.c updated to #include pfs_helpers.h and sync the uid/gid globals. pfsfolder.c: pfs_fldr_resolve_path decorated __attribute__((weak)) so tests can inject fake path resolution without a FUSE mount or psql. test_pfs_helpers.c: 7 tests covering folder/file stat field correctness, overlay NULL/mkdir/rmdir/no-match cases, and weak path override; uses --wrap for pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, and ptimer_time to stay SQL/crypto/timer-free. Production build clean; make check exits 0. --------- Co-authored-by: Levi Neely Co-authored-by: Claude Sonnet 4.6 --- Makefile | 15 +- pclsync/pfs.c | 4 + pclsync/pfs_helpers.c | 148 +++++++++++++++++ pclsync/pfs_helpers.h | 85 ++++++++++ pclsync/pfsfolder.c | 2 +- pclsync/ppagecache.c | 1 + pclsync/ppagecache_helpers.c | 42 +++++ pclsync/ppagecache_helpers.h | 64 ++++++++ tests/unit-tests/test_pfs_helpers.c | 244 ++++++++++++++++++++++++++++ tests/unit-tests/test_ppagecache.c | 154 ++++++++++++++++++ 10 files changed, 757 insertions(+), 2 deletions(-) create mode 100644 pclsync/pfs_helpers.c create mode 100644 pclsync/pfs_helpers.h create mode 100644 pclsync/ppagecache_helpers.c create mode 100644 pclsync/ppagecache_helpers.h create mode 100644 tests/unit-tests/test_pfs_helpers.c create mode 100644 tests/unit-tests/test_ppagecache.c diff --git a/Makefile b/Makefile index bf2338d..48070a5 100644 --- a/Makefile +++ b/Makefile @@ -171,7 +171,9 @@ TEST_BINS := \ tests/test_pfstasks_tree \ tests/test_pfstasks_db \ tests/test_plocks \ - tests/test_pfsupload + tests/test_pfsupload \ + tests/test_ppagecache \ + tests/test_pfs_helpers .PHONY: test tests check clean-tests @@ -238,6 +240,17 @@ tests/test_pfsupload: $(UNIT_DIR)/test_pfsupload.c $(LIBDIR)/pfsupload_send.c $( $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ -Wl,--wrap=papi_send # redirect papi_send → __wrap_papi_send; GNU ld only (not macOS Apple ld) +tests/test_ppagecache: $(UNIT_DIR)/test_ppagecache.c $(LIBDIR)/ppagecache_helpers.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ + +tests/test_pfs_helpers: $(UNIT_DIR)/test_pfs_helpers.c $(LIBDIR)/pfs_helpers.c $(LIBDIR)/pfstasks_tree.c $(LIBDIR)/ptree.c $(LIBDIR)/pcrc32c.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ + -Wl,--wrap=pfs_crpt_plain_size \ + -Wl,--wrap=pfs_task_get_folder_tasks_rdlocked \ + -Wl,--wrap=ptimer_time + # ^ GNU ld only; --wrap stubs out encrypted-size, folder-task lookup, and timer + + tests/test_read_response: $(UNIT_DIR)/test_read_response.cpp rpcclient.cpp tests/stubs/test_stubs_cpp.c $(CXX) $(TEST_CXXFLAGS) $(CXXFLAGS) -o $@ $^ diff --git a/pclsync/pfs.c b/pclsync/pfs.c index d163eab..02b3c18 100644 --- a/pclsync/pfs.c +++ b/pclsync/pfs.c @@ -63,6 +63,7 @@ #include "pssl.h" #include "pstatus.h" #include "psys.h" +#include "pfs_helpers.h" #include "ptimer.h" @@ -3806,6 +3807,9 @@ static int pfs_do_start() { myuid = getuid(); mygid = getgid(); + /* Keep pfs_helpers.c in sync for helper-based callers */ + pfs_stat_uid = myuid; + pfs_stat_gid = mygid; pthread_mutex_lock(&start_mutex); if (started) goto err00; diff --git a/pclsync/pfs_helpers.c b/pclsync/pfs_helpers.c new file mode 100644 index 0000000..a10b1bd --- /dev/null +++ b/pclsync/pfs_helpers.c @@ -0,0 +1,148 @@ +/* + * pfs_helpers.c — pure row→stat converters and overlay helpers. + * + * Dependencies: pfstasks_tree.c (for find_* functions), pfscrypto.h (for + * pfs_crpt_plain_size on encrypted files), plibs.h (psync_get_number macro). + * No psql calls, no FUSE, no network. + */ + +#include +#include +#include + +#include "pdbg.h" +#include "pfscrypto.h" +#include "pfs_helpers.h" +#include "plibs.h" /* psync_get_number */ +#include "ptimer.h" /* ptimer_time */ + +/* ------------------------------------------------------------------ */ +/* Globals — initialised here; overwritten by pfs.c at mount time */ +/* ------------------------------------------------------------------ */ + +uid_t pfs_stat_uid = 0; +gid_t pfs_stat_gid = 0; + +/* ------------------------------------------------------------------ */ +/* Row → stat converters */ +/* ------------------------------------------------------------------ */ + +void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf) { + psync_folderid_t folderid; + uint64_t mtime; + psync_fstask_folder_t *folder; + + folderid = (psync_folderid_t)psync_get_number(row[0]); + mtime = psync_get_number(row[3]); + + folder = pfs_task_get_folder_tasks_rdlocked(folderid); + if (folder && folder->mtime) + mtime = folder->mtime; + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = PFS_FOLDERID_TO_INODE(folderid); + stbuf->st_ctime = (time_t)mtime; + stbuf->st_mtime = (time_t)mtime; + stbuf->st_atime = (time_t)mtime; + stbuf->st_mode = S_IFDIR | 0755; + stbuf->st_nlink = (nlink_t)(psync_get_number(row[4]) + 2); + stbuf->st_size = PFS_FS_BLOCK_SIZE; + stbuf->st_blocks = 1; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, + uint32_t flags) { + uint64_t size = psync_get_number(row[1]); + psync_fileid_t fileid = (psync_fileid_t)psync_get_number(row[4]); + + if (flags & PSYNC_FOLDER_FLAG_ENCRYPTED) + size = pfs_crpt_plain_size(size); + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = PFS_FILEID_TO_INODE(fileid); + stbuf->st_ctime = (time_t)psync_get_number(row[3]); + stbuf->st_mtime = stbuf->st_ctime; + stbuf->st_atime = stbuf->st_ctime; + stbuf->st_mode = S_IFREG | 0644; + stbuf->st_nlink = 1; + stbuf->st_size = (off_t)size; + stbuf->st_blocks = (blkcnt_t)((size + 511) / 512); + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf) { + uint64_t mtime; + psync_fstask_folder_t *folder; + + folder = pfs_task_get_folder_tasks_rdlocked(mk->folderid); + mtime = (folder && folder->mtime) ? folder->mtime : (uint64_t)mk->mtime; + + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ino = (mk->folderid >= 0) + ? PFS_FOLDERID_TO_INODE(mk->folderid) + : PFS_TASKID_TO_INODE(-mk->folderid); + stbuf->st_ctime = (time_t)mtime; + stbuf->st_mtime = (time_t)mtime; + stbuf->st_atime = (time_t)mtime; + stbuf->st_mode = S_IFDIR | 0755; + stbuf->st_nlink = (nlink_t)(mk->subdircnt + 2); + stbuf->st_size = PFS_FS_BLOCK_SIZE; + stbuf->st_blocks = 1; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; +} + +/* ------------------------------------------------------------------ */ +/* Task overlay */ +/* ------------------------------------------------------------------ */ + +int pfs_apply_task_overlay(struct stat *stbuf, + psync_fstask_folder_t *folder, + const char *name, uint32_t flags) { + if (!folder) + return 0; + + /* Pending mkdir: show the directory */ + psync_fstask_mkdir_t *mk = pfs_task_find_mkdir(folder, name, 0); + if (mk) { + if (mk->flags & PSYNC_FOLDER_FLAG_INVISIBLE) + return -1; + pfs_mkdir_to_folder_stat(mk, stbuf); + return 1; + } + + /* Pending rmdir: hide the directory */ + if (pfs_task_find_rmdir(folder, name, 0)) + return -1; + + /* Pending unlink: hide the file */ + if (pfs_task_find_unlink(folder, name, 0)) + return -1; + + /* Pending creat with fileid==0 (new local file, no SQL needed): + * return a minimal stat for the new file. */ + psync_fstask_creat_t *cr = pfs_task_find_creat(folder, name, 0); + if (cr && cr->fileid == 0) { + time_t now = ptimer_time(); + memset(stbuf, 0, sizeof(*stbuf)); + stbuf->st_ctime = now; + stbuf->st_mtime = now; + stbuf->st_atime = now; + stbuf->st_mode = S_IFREG | 0644; + stbuf->st_nlink = 1; + stbuf->st_size = 0; + stbuf->st_blocks = 0; + stbuf->st_blksize = PFS_FS_BLOCK_SIZE; + stbuf->st_uid = pfs_stat_uid; + stbuf->st_gid = pfs_stat_gid; + return 2; + } + + return 0; +} diff --git a/pclsync/pfs_helpers.h b/pclsync/pfs_helpers.h new file mode 100644 index 0000000..8f0eb5b --- /dev/null +++ b/pclsync/pfs_helpers.h @@ -0,0 +1,85 @@ +/* + * pfs_helpers.h — pure row→stat converters and overlay helpers extracted + * from pfs.c so they can be unit-tested without a live FUSE mount or psql + * connection. + * + * Included by pfs.c and by tests/unit-tests/test_pfs_helpers.c. + */ +#ifndef PFS_HELPERS_H +#define PFS_HELPERS_H + +#include +#include +#include + +#include "pfoldersync.h" /* psync_folderid_t, psync_fileid_t */ +#include "pfstasks.h" /* psync_fstask_folder_t, psync_fstask_mkdir_t, … */ +#include "pfsfolder.h" /* psync_fspath_t */ +#include "psql.h" /* psync_variant_row */ + +/* + * Inode-number helpers — must match the definitions used in pfs.c. + * Put here so pfs_helpers.c and tests share a single definition. + */ +#define PFS_FOLDERID_TO_INODE(fid) ((fid) * 3) +#define PFS_FILEID_TO_INODE(fid) ((fid) * 3 + 1) +#define PFS_TASKID_TO_INODE(tid) ((tid) * 3 + 2) +#define PFS_FS_BLOCK_SIZE 4096 + +/* + * Owner uid/gid used when populating struct stat. Initialised to 0 (root) + * by pfs_helpers.c. pfs.c overwrites them with the real process owner at + * init time; tests leave them at 0. + */ +extern uid_t pfs_stat_uid; +extern gid_t pfs_stat_gid; + +/* + * pfs_row_to_folder_stat — convert a psql folder row to struct stat. + * + * Row column layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt + * Applies any pending in-memory mtime from the folder task queue. + * No SQL, no FUSE calls. + */ +void pfs_row_to_folder_stat(psync_variant_row row, struct stat *stbuf); + +/* + * pfs_row_to_file_stat — convert a psql file row to struct stat. + * + * Row column layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id + * flags: PSYNC_FOLDER_FLAG_ENCRYPTED triggers encrypted-size conversion via + * pfs_crpt_plain_size(); tests pass flags=0 to skip crypto. + * No SQL, no FUSE calls. + */ +void pfs_row_to_file_stat(psync_variant_row row, struct stat *stbuf, + uint32_t flags); + +/* + * pfs_mkdir_to_folder_stat — convert an in-memory mkdir task to struct stat. + * No SQL, no FUSE calls. + */ +void pfs_mkdir_to_folder_stat(psync_fstask_mkdir_t *mk, struct stat *stbuf); + +/* + * pfs_apply_task_overlay — check an in-memory folder task queue for a + * pending operation on `name` and update `stbuf` accordingly. + * + * Returns: + * 1 mkdir overlay applied (stbuf filled as a directory) + * 2 creat overlay applied (stbuf filled as a new file, fileid=0) + * -1 rmdir or unlink pending → entry should be hidden (ENOENT) + * 0 no applicable overlay found + * + * No SQL, no network I/O. folder may be NULL (returns 0 immediately). + */ +int pfs_apply_task_overlay(struct stat *stbuf, + psync_fstask_folder_t *folder, + const char *name, uint32_t flags); + +/* + * pfs_fldr_resolve_path — declared __attribute__((weak)) in pfsfolder.c so + * tests can override path resolution without a live FUSE / psql stack. + * The declaration here is informational only (it lives in pfsfolder.h). + */ + +#endif /* PFS_HELPERS_H */ diff --git a/pclsync/pfsfolder.c b/pclsync/pfsfolder.c index 2cf65e2..59d9dc4 100644 --- a/pclsync/pfsfolder.c +++ b/pclsync/pfsfolder.c @@ -148,7 +148,7 @@ static void check_userid(uint64_t userid, uint64_t folderid, do_check_userid(userid, folderid, shareid); } -psync_fspath_t *pfs_fldr_resolve_path(const char *path) { +__attribute__((weak)) psync_fspath_t *pfs_fldr_resolve_path(const char *path) { psync_fsfolderid_t cfolderid; const char *sl; psync_fstask_folder_t *folder; diff --git a/pclsync/ppagecache.c b/pclsync/ppagecache.c index 08d3b30..e15e538 100644 --- a/pclsync/ppagecache.c +++ b/pclsync/ppagecache.c @@ -43,6 +43,7 @@ #include "pmem.h" #include "pnetlibs.h" #include "ppagecache.h" +#include "ppagecache_helpers.h" #include "ppath.h" #include "prun.h" #include "psettings.h" diff --git a/pclsync/ppagecache_helpers.c b/pclsync/ppagecache_helpers.c new file mode 100644 index 0000000..c6437bd --- /dev/null +++ b/pclsync/ppagecache_helpers.c @@ -0,0 +1,42 @@ +/* + * ppagecache_helpers.c — pure helpers extracted from ppagecache.c. + * + * Only deps: pcrc32c.h (CRC computation) and pfoldersync.h (psync_fileid_t). + * No psql, no networking, no threading — safe to link into unit tests. + */ + +#include "ppagecache_helpers.h" + +/* ------------------------------------------------------------------ */ +/* Priority / LRU tier */ +/* ------------------------------------------------------------------ */ + +uint8_t ppagecache_compute_page_priority(uint32_t usecnt) { + if (usecnt >= PPAGECACHE_TIER4_THRESHOLD) return 4; + if (usecnt >= PPAGECACHE_TIER3_THRESHOLD) return 3; + if (usecnt >= PPAGECACHE_TIER2_THRESHOLD) return 2; + if (usecnt >= PPAGECACHE_TIER1_THRESHOLD) return 1; + return 0; +} + +/* ------------------------------------------------------------------ */ +/* CRC verification */ +/* ------------------------------------------------------------------ */ + +int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc) { + uint32_t computed = pcrc32c_compute(PSYNC_CRC_INITIAL, data, size); + return (computed == stored_crc) ? 0 : -1; +} + +/* ------------------------------------------------------------------ */ +/* Download-URL seam (weak default: no-op) */ +/* ------------------------------------------------------------------ */ + +__attribute__((weak)) +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout) { + (void)fileid; + (void)hash; + if (nout) *nout = 0; + return NULL; /* caller falls through to real API */ +} diff --git a/pclsync/ppagecache_helpers.h b/pclsync/ppagecache_helpers.h new file mode 100644 index 0000000..ccc07eb --- /dev/null +++ b/pclsync/ppagecache_helpers.h @@ -0,0 +1,64 @@ +/* + * ppagecache_helpers.h — pure, separately-compilable helpers extracted from + * ppagecache.c to enable unit testing without the full page-cache stack. + * + * Included by ppagecache.c and by tests/unit-tests/test_ppagecache.c. + */ +#ifndef PPAGECACHE_HELPERS_H +#define PPAGECACHE_HELPERS_H + +#include +#include + +#include "pcrc32c.h" /* PSYNC_CRC_INITIAL, pcrc32c_compute */ +#include "pfoldersync.h" /* psync_fileid_t */ + +/* + * LRU tier thresholds (must match the pagecache_entry_cmp_usecnt_lastuse* + * comparators in ppagecache.c). + */ +#define PPAGECACHE_TIER1_THRESHOLD 2u +#define PPAGECACHE_TIER2_THRESHOLD 4u +#define PPAGECACHE_TIER3_THRESHOLD 8u +#define PPAGECACHE_TIER4_THRESHOLD 16u + +/* + * ppagecache_compute_page_priority — assign an eviction-priority tier to a + * cached page based on its access count. + * + * tier 0 usecnt < 2 — cold, evict first + * tier 1 usecnt < 4 + * tier 2 usecnt < 8 + * tier 3 usecnt < 16 + * tier 4 usecnt >= 16 — hot, keep longest + * + * Pure function: no side effects, no I/O. + */ +uint8_t ppagecache_compute_page_priority(uint32_t usecnt); + +/* + * ppagecache_verify_crc — verify the CRC32c of `size` bytes at `data` + * against `stored_crc`. + * + * Returns 0 if the CRC matches (page is intact). + * Returns -1 if the CRC differs (page is corrupt). + * + * Pure function: no side effects. + */ +int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc); + +/* + * ppagecache_get_download_urls — weak seam for injecting canned download + * URLs in tests. The default implementation returns NULL (caller falls + * through to the real API). Tests override this to return a + * NULL-terminated string array without making network calls. + * + * On success sets *nout to the number of URLs and returns a pointer to + * a NULL-terminated array of C strings. The array is valid until the next + * call or until the override frees it. + */ +__attribute__((weak)) +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout); + +#endif /* PPAGECACHE_HELPERS_H */ diff --git a/tests/unit-tests/test_pfs_helpers.c b/tests/unit-tests/test_pfs_helpers.c new file mode 100644 index 0000000..0c72178 --- /dev/null +++ b/tests/unit-tests/test_pfs_helpers.c @@ -0,0 +1,244 @@ +/* + * Test: pfs_helpers.c — row→stat converters and task overlay + * + * Covers: + * 1. pfs_row_to_folder_stat: st_ino, st_mode, st_nlink, st_mtime populated + * from the variant row; st_uid/gid come from pfs_stat_uid/gid globals. + * 2. pfs_row_to_file_stat: st_ino, st_mode, st_size, st_ctime populated + * (non-encrypted path, flags=0). + * 3. pfs_apply_task_overlay: mkdir pending → stat filled as dir; + * rmdir pending → -1; no overlay → 0. + * 4. pfs_fldr_resolve_path weak override: replacement injects a fake path + * without a FUSE mount or psql connection. + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include +#include + +#include "pfs_helpers.h" +#include "pfstasks_tree.h" /* pfs_task_insert_into_tree, pfs_task_find_* */ +#include "plibs.h" /* PSYNC_TNUMBER, psync_variant */ + +/* ------------------------------------------------------------------ */ +/* --wrap stubs: pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, + * ptimer_time — none of these should be called in the pure-tree / non- + * encrypted test paths; return safe no-op values just in case. */ + +psync_fstask_folder_t *__wrap_pfs_task_get_folder_tasks_rdlocked( + psync_fsfolderid_t folderid) { + (void)folderid; + return NULL; /* no in-memory mtime override in tests */ +} + +uint64_t __wrap_pfs_crpt_plain_size(uint64_t cryptosize) { + return cryptosize; /* identity for non-encrypted tests */ +} + +time_t __wrap_ptimer_time(void) { + return 9999; /* fixed timestamp for deterministic tests */ +} + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Helper: build a psync_variant holding a uint64_t number */ +/* ------------------------------------------------------------------ */ +static psync_variant mknum(uint64_t v) { + psync_variant pv; + memset(&pv, 0, sizeof(pv)); + pv.type = PSYNC_TNUMBER; + pv.num = v; + return pv; +} + +/* ------------------------------------------------------------------ */ +/* Test 1: pfs_row_to_folder_stat */ +/* ------------------------------------------------------------------ */ +static void test_folder_stat(void) { + /* + * Folder row layout: [0]=id [1]=permissions [2]=ctime [3]=mtime [4]=subdircnt + */ + psync_variant row[5]; + row[0] = mknum(100); /* folderid = 100 */ + row[1] = mknum(0755); /* permissions */ + row[2] = mknum(1000); /* ctime */ + row[3] = mknum(2000); /* mtime */ + row[4] = mknum(3); /* subdircnt = 3 → nlink = 5 */ + + struct stat st; + memset(&st, 0, sizeof(st)); + pfs_row_to_folder_stat(row, &st); + + if (st.st_ino != PFS_FOLDERID_TO_INODE(100)) + FAIL("folder_stat: st_ino", "expected %lu got %lu", + (unsigned long)PFS_FOLDERID_TO_INODE(100), (unsigned long)st.st_ino); + else if (!S_ISDIR(st.st_mode)) + FAIL("folder_stat: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); + else if (st.st_nlink != 5) + FAIL("folder_stat: st_nlink", "expected 5 got %lu", (unsigned long)st.st_nlink); + else if (st.st_mtime != 2000) + FAIL("folder_stat: st_mtime", "expected 2000 got %ld", (long)st.st_mtime); + else if (st.st_uid != pfs_stat_uid || st.st_gid != pfs_stat_gid) + FAIL("folder_stat: uid/gid", "uid=%d gid=%d", + (int)st.st_uid, (int)st.st_gid); + else + PASS("pfs_row_to_folder_stat: ino/mode/nlink/mtime/uid/gid correct"); +} + +/* ------------------------------------------------------------------ */ +/* Test 2: pfs_row_to_file_stat (non-encrypted, flags=0) */ +/* ------------------------------------------------------------------ */ +static void test_file_stat(void) { + /* + * File row layout: [0]=name [1]=size [2]=ctime [3]=mtime [4]=id + */ + psync_variant row[5]; + row[0] = mknum(0); /* name (unused in stat) */ + row[1] = mknum(4096); /* size = 4096 bytes */ + row[2] = mknum(3000); /* ctime */ + row[3] = mknum(3000); /* mtime */ + row[4] = mknum(77); /* fileid = 77 */ + + struct stat st; + memset(&st, 0, sizeof(st)); + pfs_row_to_file_stat(row, &st, 0 /* non-encrypted */); + + if (st.st_ino != PFS_FILEID_TO_INODE(77)) + FAIL("file_stat: st_ino", "expected %lu got %lu", + (unsigned long)PFS_FILEID_TO_INODE(77), (unsigned long)st.st_ino); + else if (!S_ISREG(st.st_mode)) + FAIL("file_stat: S_ISREG", "mode=0%o", (unsigned)st.st_mode); + else if (st.st_size != 4096) + FAIL("file_stat: st_size", "expected 4096 got %ld", (long)st.st_size); + else if (st.st_ctime != 3000) + FAIL("file_stat: st_ctime", "expected 3000 got %ld", (long)st.st_ctime); + else if (st.st_nlink != 1) + FAIL("file_stat: st_nlink", "expected 1 got %lu", (unsigned long)st.st_nlink); + else + PASS("pfs_row_to_file_stat: ino/mode/size/ctime/nlink correct"); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: pfs_apply_task_overlay */ +/* ------------------------------------------------------------------ */ + +/* Helper: allocate a psync_fstask_mkdir_t and insert it into folder */ +static psync_fstask_mkdir_t *make_mkdir(psync_fstask_folder_t *f, + const char *name, + uint64_t taskid, + psync_fsfolderid_t folderid) { + size_t len = strlen(name) + 1; + psync_fstask_mkdir_t *mk = (psync_fstask_mkdir_t *) + calloc(1, offsetof(psync_fstask_mkdir_t, name) + len); + mk->taskid = taskid; + mk->folderid = folderid; + mk->mtime = 5000; + mk->flags = 0; + memcpy(mk->name, name, len); + pfs_task_insert_into_tree(&f->mkdirs, offsetof(psync_fstask_mkdir_t, name), + &mk->tree); + return mk; +} + +static psync_fstask_rmdir_t *make_rmdir(psync_fstask_folder_t *f, + const char *name, + uint64_t taskid) { + size_t len = strlen(name) + 1; + psync_fstask_rmdir_t *rm = (psync_fstask_rmdir_t *) + calloc(1, offsetof(psync_fstask_rmdir_t, name) + len); + rm->taskid = taskid; + memcpy(rm->name, name, len); + pfs_task_insert_into_tree(&f->rmdirs, offsetof(psync_fstask_rmdir_t, name), + &rm->tree); + return rm; +} + +static void test_apply_task_overlay(void) { + /* NULL folder → 0 */ + struct stat st; + int rc = pfs_apply_task_overlay(&st, NULL, "any", 0); + if (rc != 0) + { FAIL("overlay NULL folder", "expected 0 got %d", rc); } + else + PASS("pfs_apply_task_overlay: NULL folder returns 0"); + + /* mkdir pending → 1, stbuf filled as directory */ + psync_fstask_folder_t f; + memset(&f, 0, sizeof(f)); + f.folderid = 200; + psync_fstask_mkdir_t *mk = make_mkdir(&f, "newdir", 10, -10); + + memset(&st, 0, sizeof(st)); + rc = pfs_apply_task_overlay(&st, &f, "newdir", 0); + if (rc != 1) + FAIL("overlay mkdir: rc=1", "got %d", rc); + else if (!S_ISDIR(st.st_mode)) + FAIL("overlay mkdir: S_ISDIR", "mode=0%o", (unsigned)st.st_mode); + else + PASS("pfs_apply_task_overlay: mkdir pending → rc=1, stbuf is a dir"); + + /* rmdir pending → -1 */ + psync_fstask_rmdir_t *rm = make_rmdir(&f, "olddir", 20); + memset(&st, 0, sizeof(st)); + rc = pfs_apply_task_overlay(&st, &f, "olddir", 0); + if (rc != -1) + FAIL("overlay rmdir: rc=-1", "got %d", rc); + else + PASS("pfs_apply_task_overlay: rmdir pending → rc=-1 (ENOENT)"); + + /* no overlay for absent name → 0 */ + rc = pfs_apply_task_overlay(&st, &f, "noentry", 0); + if (rc != 0) + FAIL("overlay no match: rc=0", "got %d", rc); + else + PASS("pfs_apply_task_overlay: no pending task for name → rc=0"); + + free(mk); free(rm); +} + +/* ------------------------------------------------------------------ */ +/* Test 4: pfs_fldr_resolve_path weak override */ +/* ------------------------------------------------------------------ */ + +static psync_fspath_t g_fake_path; +static int g_resolve_called = 0; + +psync_fspath_t *pfs_fldr_resolve_path(const char *path) { + (void)path; + g_resolve_called++; + return &g_fake_path; +} + +static void test_resolve_path_override(void) { + g_resolve_called = 0; + memset(&g_fake_path, 0, sizeof(g_fake_path)); + g_fake_path.folderid = 999; + + psync_fspath_t *r = pfs_fldr_resolve_path("/test/path"); + if (!r || r->folderid != 999) + FAIL("resolve override: returns fake path", "folderid=%lld", + r ? (long long)r->folderid : -1LL); + else if (g_resolve_called != 1) + FAIL("resolve override: called once", "called=%d", g_resolve_called); + else + PASS("pfs_fldr_resolve_path: weak override injects fake path"); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + /* tests use uid/gid = 0 (default from pfs_helpers.c init) */ + test_folder_stat(); + test_file_stat(); + test_apply_task_overlay(); + test_resolve_path_override(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +} diff --git a/tests/unit-tests/test_ppagecache.c b/tests/unit-tests/test_ppagecache.c new file mode 100644 index 0000000..09f40b6 --- /dev/null +++ b/tests/unit-tests/test_ppagecache.c @@ -0,0 +1,154 @@ +/* + * Test: ppagecache_helpers.c — priority tiers, CRC verification, URL seam + * + * Covers: + * 1. ppagecache_compute_page_priority: boundary conditions between all five + * tiers (0–4) and representative interior values. + * 2. ppagecache_verify_crc: known-good buffer, single-bit flip, zero-length. + * 3. ppagecache_get_download_urls: weak override injects canned URL list. + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include + +#include "ppagecache_helpers.h" + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Test 1: priority tier boundary conditions */ +/* ------------------------------------------------------------------ */ +static void test_priority_tiers(void) { + /* Tier 0: usecnt < 2 */ + if (ppagecache_compute_page_priority(0) != 0 || + ppagecache_compute_page_priority(1) != 0) + FAIL("tier 0 (usecnt 0,1)", "got %d/%d", + ppagecache_compute_page_priority(0), + ppagecache_compute_page_priority(1)); + else + PASS("tier 0: usecnt in [0,2)"); + + /* Tier 1: usecnt in [2, 4) */ + if (ppagecache_compute_page_priority(2) != 1 || + ppagecache_compute_page_priority(3) != 1) + FAIL("tier 1 (usecnt 2,3)", "got %d/%d", + ppagecache_compute_page_priority(2), + ppagecache_compute_page_priority(3)); + else + PASS("tier 1: usecnt in [2,4)"); + + /* Tier 2: usecnt in [4, 8) */ + if (ppagecache_compute_page_priority(4) != 2 || + ppagecache_compute_page_priority(7) != 2) + FAIL("tier 2 (usecnt 4,7)", "got %d/%d", + ppagecache_compute_page_priority(4), + ppagecache_compute_page_priority(7)); + else + PASS("tier 2: usecnt in [4,8)"); + + /* Tier 3: usecnt in [8, 16) */ + if (ppagecache_compute_page_priority(8) != 3 || + ppagecache_compute_page_priority(15) != 3) + FAIL("tier 3 (usecnt 8,15)", "got %d/%d", + ppagecache_compute_page_priority(8), + ppagecache_compute_page_priority(15)); + else + PASS("tier 3: usecnt in [8,16)"); + + /* Tier 4: usecnt >= 16 */ + if (ppagecache_compute_page_priority(16) != 4 || + ppagecache_compute_page_priority(255)!= 4 || + ppagecache_compute_page_priority(UINT32_MAX) != 4) + FAIL("tier 4 (usecnt >=16)", "got %d/%d/%d", + ppagecache_compute_page_priority(16), + ppagecache_compute_page_priority(255), + ppagecache_compute_page_priority(UINT32_MAX)); + else + PASS("tier 4: usecnt >= 16 including UINT32_MAX"); +} + +/* ------------------------------------------------------------------ */ +/* Test 2: CRC verification */ +/* ------------------------------------------------------------------ */ +static void test_verify_crc(void) { + /* Build a known buffer */ + const char buf[64] = "Hello pagecache CRC test buffer 0123456789abcdef!"; + uint32_t good_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, sizeof(buf)); + + /* Known-good: must pass */ + if (ppagecache_verify_crc(buf, sizeof(buf), good_crc) != 0) + FAIL("verify_crc: known-good buffer", "returned non-zero"); + else + PASS("verify_crc: known-good buffer passes"); + + /* Single-byte corruption: must fail */ + char corrupt[64]; + memcpy(corrupt, buf, sizeof(corrupt)); + corrupt[7] ^= 0x01; /* flip one bit */ + if (ppagecache_verify_crc(corrupt, sizeof(corrupt), good_crc) != -1) + FAIL("verify_crc: corrupted buffer", "returned 0 (expected -1)"); + else + PASS("verify_crc: single-bit flip detected"); + + /* Wrong stored CRC */ + if (ppagecache_verify_crc(buf, sizeof(buf), good_crc + 1) != -1) + FAIL("verify_crc: wrong stored CRC", "returned 0 (expected -1)"); + else + PASS("verify_crc: wrong stored_crc detected"); + + /* Zero-length buffer: CRC of empty == PSYNC_CRC_INITIAL == 0 */ + uint32_t empty_crc = pcrc32c_compute(PSYNC_CRC_INITIAL, buf, 0); + if (ppagecache_verify_crc(buf, 0, empty_crc) != 0) + FAIL("verify_crc: zero-length buffer", "returned non-zero"); + else + PASS("verify_crc: zero-length buffer passes"); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: ppagecache_get_download_urls weak override */ +/* ------------------------------------------------------------------ */ + +/* Override the weak default: inject two canned URLs */ +static char *g_canned_urls[] = { + "https://content1.example.com/file", + "https://content2.example.com/file", + NULL +}; + +char **ppagecache_get_download_urls(psync_fileid_t fileid, uint64_t hash, + size_t *nout) { + (void)fileid; + (void)hash; + if (nout) *nout = 2; + return g_canned_urls; +} + +static void test_download_url_override(void) { + size_t n = 0; + char **urls = ppagecache_get_download_urls(9999, 0xdeadbeef, &n); + if (!urls || n != 2) + { FAIL("url override: count", "n=%zu urls=%p", n, (void*)urls); return; } + if (strcmp(urls[0], "https://content1.example.com/file") != 0 || + strcmp(urls[1], "https://content2.example.com/file") != 0) + FAIL("url override: content", "url0=%s url1=%s", urls[0], urls[1]); + else if (urls[2] != NULL) + FAIL("url override: NULL terminator", "urls[2]=%p", (void*)urls[2]); + else + PASS("ppagecache_get_download_urls: weak override returns canned URLs"); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + test_priority_tiers(); + test_verify_crc(); + test_download_url_override(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +} From 9773ed2581e2ea47452734754bf114e9f27e47e5 Mon Sep 17 00:00:00 2001 From: Levi Neely <141506390+lneely@users.noreply.github.com> Date: Wed, 11 Mar 2026 20:35:05 +0100 Subject: [PATCH 4/4] Phase 4 testability refactor: pdiff, plocalscan, ppagecache helper extraction (#386) Extract testable helpers from pdiff.c, plocalscan.c, and ppagecache.c: - pdiff_helpers.c/h: 6 functions - plocalscan_helpers.c/h: 5 functions + sync_folderlist struct - ppagecache_helpers.c/h: http_status_should_retry + 3 inline helpers Tests: 17 pdiff, 29 plocalscan, 26 ppagecache (149 total pass) Co-authored-by: Levi Neely --- Makefile | 20 +- pclsync/pdiff.c | 89 +--- pclsync/pdiff_helpers.c | 175 +++++++ pclsync/pdiff_helpers.h | 60 +++ pclsync/plocalscan.c | 217 +-------- pclsync/plocalscan_helpers.c | 232 ++++++++++ pclsync/plocalscan_helpers.h | 158 +++++++ pclsync/ppagecache.c | 17 +- pclsync/ppagecache_helpers.c | 12 + pclsync/ppagecache_helpers.h | 35 ++ tests/unit-tests/test_pdiff_helpers.c | 434 +++++++++++++++++ tests/unit-tests/test_plocalscan_helpers.c | 514 +++++++++++++++++++++ tests/unit-tests/test_ppagecache.c | 120 +++++ 13 files changed, 1798 insertions(+), 285 deletions(-) create mode 100644 pclsync/pdiff_helpers.c create mode 100644 pclsync/pdiff_helpers.h create mode 100644 pclsync/plocalscan_helpers.c create mode 100644 pclsync/plocalscan_helpers.h create mode 100644 tests/unit-tests/test_pdiff_helpers.c create mode 100644 tests/unit-tests/test_plocalscan_helpers.c diff --git a/Makefile b/Makefile index 48070a5..631f3e4 100644 --- a/Makefile +++ b/Makefile @@ -173,7 +173,9 @@ TEST_BINS := \ tests/test_plocks \ tests/test_pfsupload \ tests/test_ppagecache \ - tests/test_pfs_helpers + tests/test_pfs_helpers \ + tests/test_pdiff_helpers \ + tests/test_plocalscan_helpers .PHONY: test tests check clean-tests @@ -250,6 +252,22 @@ tests/test_pfs_helpers: $(UNIT_DIR)/test_pfs_helpers.c $(LIBDIR)/pfs_helpers.c $ -Wl,--wrap=ptimer_time # ^ GNU ld only; --wrap stubs out encrypted-size, folder-task lookup, and timer +tests/test_plocalscan_helpers: $(UNIT_DIR)/test_plocalscan_helpers.c $(LIBDIR)/plocalscan_helpers.c $(LIBDIR)/plist.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ + -Wl,--wrap=psync_is_name_to_ignore \ + -Wl,--wrap=psync_send_backup_del_event + # ^ GNU ld only; wraps filter/side-effect calls in extracted helpers + +tests/test_pdiff_helpers: $(UNIT_DIR)/test_pdiff_helpers.c $(LIBDIR)/pdiff_helpers.c $(LIBDIR)/pdbg.c $(LIBDIR)/pmem.c $(LIBDIR)/putil.c $(LIBDIR)/ppath.c tests/stubs/test_stubs.c + $(CC) $(TEST_CFLAGS) $(CFLAGS) -o $@ $^ \ + -Wl,--wrap=papi_find_result \ + -Wl,--wrap=papi_check_result \ + -Wl,--wrap=psql_bind_uint \ + -Wl,--wrap=psql_bind_lstr \ + -Wl,--wrap=psql_bind_null \ + -Wl,--wrap=psql_bind_double + # ^ GNU ld only; --wrap overrides papi lookup stubs and intercepts SQL binds + tests/test_read_response: $(UNIT_DIR)/test_read_response.cpp rpcclient.cpp tests/stubs/test_stubs_cpp.c $(CXX) $(TEST_CXXFLAGS) $(CXXFLAGS) -o $@ $^ diff --git a/pclsync/pdiff.c b/pclsync/pdiff.c index 44eec5c..3b70a6a 100644 --- a/pclsync/pdiff.c +++ b/pclsync/pdiff.c @@ -44,6 +44,7 @@ #include "pcontacts.h" #include "pdevice.h" #include "pdiff.h" +#include "pdiff_helpers.h" #include "pdownload.h" #include "pfileops.h" #include "pfoldersync.h" @@ -248,19 +249,7 @@ static binresult *get_userinfo_user_pass(psock_t *sock, const char *username, } static int check_active_subscribtion(const binresult *res) { - const binresult *sub; - char *status; - sub = papi_check_result2(res, "lastsubscription", PARAM_HASH); - if (sub) { - status = putil_strdup(papi_find_result2(sub, "status", PARAM_STR)->str); - if (!strcmp(status, "active")) { - pmem_free(PMEM_SUBSYS_SYNC, status); - return 1; - } - pmem_free(PMEM_SUBSYS_SYNC, status); - } - - return 0; + return pdiff_check_active_subscribtion(res); } static int check_user_relocated(uint64_t luserid, psock_t *sock) { @@ -828,24 +817,7 @@ static psock_t *get_connected_socket() { } static uint64_t extract_meta_folder_flags(const binresult *meta) { - const binresult *res; - uint64_t flags = 0; - if ((res = papi_check_result2(meta, "encrypted", PARAM_BOOL)) && res->num) - flags |= PSYNC_FOLDER_FLAG_ENCRYPTED; - if ((res = papi_check_result2(meta, "ispublicroot", PARAM_BOOL)) && res->num) - flags |= PSYNC_FOLDER_FLAG_PUBLIC_ROOT; - if ((res = papi_check_result2(meta, "isbackupdevicelist", PARAM_BOOL)) && - res->num) - flags |= PSYNC_FOLDER_FLAG_BACKUP_DEVICE_LIST; - if ((res = papi_check_result2(meta, "isbackupdevice", PARAM_BOOL)) && - res->num) - flags |= PSYNC_FOLDER_FLAG_BACKUP_DEVICE; - if ((res = papi_check_result2(meta, "isbackuproot", PARAM_BOOL)) && res->num) - flags |= PSYNC_FOLDER_FLAG_BACKUP_ROOT; - if ((res = papi_check_result2(meta, "isbackup", PARAM_BOOL)) && res->num) - flags |= PSYNC_FOLDER_FLAG_BACKUP; - - return flags; + return pdiff_extract_meta_folder_flags(meta); } static void process_createfolder(const binresult *entry) { @@ -979,28 +951,7 @@ static void process_createfolder(const binresult *entry) { static void group_results_by_col(psync_full_result_int *restrict r1, psync_full_result_int *restrict r2, uint32_t col) { - VAR_ARRAY(buff, uint64_t, r1->cols); - size_t rowsize; - uint32_t i, j, l; - l = 0; - rowsize = sizeof(r1->data[0]) * r1->cols; - pdbg_assert(r1->cols == r2->cols); - for (i = 0; i < r1->rows; i++) - for (j = 0; j < r2->rows; j++) - if (psync_get_result_cell(r1, i, col) == - psync_get_result_cell(r2, j, col)) { - if (i != l) { - memcpy(buff, r1->data + i * r1->cols, rowsize); - memcpy(r1->data + i * r1->cols, r1->data + l * r1->cols, rowsize); - memcpy(r1->data + l * r1->cols, buff, rowsize); - } - if (j != l) { - memcpy(buff, r2->data + j * r2->cols, rowsize); - memcpy(r2->data + j * r2->cols, r2->data + l * r2->cols, rowsize); - memcpy(r2->data + l * r2->cols, buff, rowsize); - } - l++; - } + pdiff_group_results_by_col(r1, r2, col); } static void del_synced_folder_rec(psync_folderid_t folderid, @@ -1339,28 +1290,7 @@ static void check_for_deletedfileid(const binresult *meta) { } static int bind_meta(psync_sql_res *res, const binresult *meta, int off) { - const binresult *br; - bind_num("created"); - bind_num("modified"); - bind_num("category"); - bind_bool("thumb"); - bind_str("icon"); - bind_opt_str("artist"); - bind_opt_str("album"); - bind_opt_str("title"); - bind_opt_str("genre"); - bind_opt_num("trackno"); - bind_opt_num("width"); - bind_opt_num("height"); - bind_opt_double("duration"); - bind_opt_double("fps"); - bind_opt_str("videocodec"); - bind_opt_str("audiocodec"); - bind_opt_num("videobitrate"); - bind_opt_num("audiobitrate"); - bind_opt_num("audiosamplerate"); - bind_opt_num("rotate"); - return off; + return pdiff_bind_meta(res, meta, off); } static void insert_revision(psync_fileid_t fileid, uint64_t hash, @@ -2817,14 +2747,7 @@ static void handle_exception(psock_t **sock, subscribed_ids *ids, } static int cmp_folderid(const void *ptr1, const void *ptr2) { - psync_folderid_t *folderid1 = (psync_folderid_t *)ptr1; - psync_folderid_t *folderid2 = (psync_folderid_t *)ptr2; - if (folderid1 < folderid2) - return -1; - else if (folderid1 > folderid2) - return 1; - else - return 0; + return pdiff_cmp_folderid(ptr1, ptr2); } static void psync_diff_refresh_fs_add_folder(psync_folderid_t folderid) { diff --git a/pclsync/pdiff_helpers.c b/pclsync/pdiff_helpers.c new file mode 100644 index 0000000..06db309 --- /dev/null +++ b/pclsync/pdiff_helpers.c @@ -0,0 +1,175 @@ +/* + * pdiff_helpers.c — pure helpers extracted from pdiff.c. + * + * Only deps: papi.h, pfoldersync.h, pmem.h, psql.h, putil.h, pdbg.h. + * No networking, no threading — safe to link into unit tests. + */ + +#include /* atof */ +#include /* memcpy, strcmp */ + +#include "pdiff_helpers.h" +#include "plibs.h" /* psync_get_result_cell, pdbg_assert */ + +/* ------------------------------------------------------------------ */ +/* Local bind macros (mirror those in pdiff.c) */ +/* ------------------------------------------------------------------ */ + +#define bind_num(s) \ + psql_bind_uint(res, off++, papi_find_result2(meta, s, PARAM_NUM)->num) +#define bind_bool(s) \ + psql_bind_uint(res, off++, papi_find_result2(meta, s, PARAM_BOOL)->num) +#define bind_str(s) \ + do { \ + br = papi_find_result2(meta, s, PARAM_STR); \ + psql_bind_lstr(res, off++, br->str, br->length); \ + } while (0) +#define bind_opt_str(s) \ + do { \ + br = papi_check_result2(meta, s, PARAM_STR); \ + if (br) \ + psql_bind_lstr(res, off++, br->str, br->length); \ + else \ + psql_bind_null(res, off++); \ + } while (0) +#define bind_opt_num(s) \ + do { \ + br = papi_check_result2(meta, s, PARAM_NUM); \ + if (br) \ + psql_bind_uint(res, off++, br->num); \ + else \ + psql_bind_null(res, off++); \ + } while (0) +#define bind_opt_double(s) \ + do { \ + br = papi_check_result2(meta, s, PARAM_STR); \ + if (br) \ + psql_bind_double(res, off++, atof(br->str)); \ + else \ + psql_bind_null(res, off++); \ + } while (0) + +/* ------------------------------------------------------------------ */ +/* check_active_subscribtion */ +/* ------------------------------------------------------------------ */ + +int pdiff_check_active_subscribtion(const binresult *res) { + const binresult *sub; + char *status; + sub = papi_check_result2(res, "lastsubscription", PARAM_HASH); + if (sub) { + status = putil_strdup(papi_find_result2(sub, "status", PARAM_STR)->str); + if (!strcmp(status, "active")) { + pmem_free(PMEM_SUBSYS_SYNC, status); + return 1; + } + pmem_free(PMEM_SUBSYS_SYNC, status); + } + return 0; +} + +/* ------------------------------------------------------------------ */ +/* extract_meta_folder_flags */ +/* ------------------------------------------------------------------ */ + +uint64_t pdiff_extract_meta_folder_flags(const binresult *meta) { + const binresult *res; + uint64_t flags = 0; + if ((res = papi_check_result2(meta, "encrypted", PARAM_BOOL)) && res->num) + flags |= PSYNC_FOLDER_FLAG_ENCRYPTED; + if ((res = papi_check_result2(meta, "ispublicroot", PARAM_BOOL)) && res->num) + flags |= PSYNC_FOLDER_FLAG_PUBLIC_ROOT; + if ((res = papi_check_result2(meta, "isbackupdevicelist", PARAM_BOOL)) && + res->num) + flags |= PSYNC_FOLDER_FLAG_BACKUP_DEVICE_LIST; + if ((res = papi_check_result2(meta, "isbackupdevice", PARAM_BOOL)) && + res->num) + flags |= PSYNC_FOLDER_FLAG_BACKUP_DEVICE; + if ((res = papi_check_result2(meta, "isbackuproot", PARAM_BOOL)) && res->num) + flags |= PSYNC_FOLDER_FLAG_BACKUP_ROOT; + if ((res = papi_check_result2(meta, "isbackup", PARAM_BOOL)) && res->num) + flags |= PSYNC_FOLDER_FLAG_BACKUP; + return flags; +} + +/* ------------------------------------------------------------------ */ +/* group_results_by_col */ +/* ------------------------------------------------------------------ */ + +void pdiff_group_results_by_col(psync_full_result_int *restrict r1, + psync_full_result_int *restrict r2, + uint32_t col) { + VAR_ARRAY(buff, uint64_t, r1->cols); + size_t rowsize; + uint32_t i, j, l; + l = 0; + rowsize = sizeof(r1->data[0]) * r1->cols; + pdbg_assert(r1->cols == r2->cols); + for (i = 0; i < r1->rows; i++) + for (j = 0; j < r2->rows; j++) + if (psync_get_result_cell(r1, i, col) == + psync_get_result_cell(r2, j, col)) { + if (i != l) { + memcpy(buff, r1->data + i * r1->cols, rowsize); + memcpy(r1->data + i * r1->cols, r1->data + l * r1->cols, rowsize); + memcpy(r1->data + l * r1->cols, buff, rowsize); + } + if (j != l) { + memcpy(buff, r2->data + j * r2->cols, rowsize); + memcpy(r2->data + j * r2->cols, r2->data + l * r2->cols, rowsize); + memcpy(r2->data + l * r2->cols, buff, rowsize); + } + l++; + } +} + +/* ------------------------------------------------------------------ */ +/* cmp_folderid */ +/* ------------------------------------------------------------------ */ + +int pdiff_cmp_folderid(const void *ptr1, const void *ptr2) { + psync_folderid_t *folderid1 = (psync_folderid_t *)ptr1; + psync_folderid_t *folderid2 = (psync_folderid_t *)ptr2; + if (folderid1 < folderid2) + return -1; + else if (folderid1 > folderid2) + return 1; + else + return 0; +} + +/* ------------------------------------------------------------------ */ +/* bind_meta */ +/* ------------------------------------------------------------------ */ + +int pdiff_bind_meta(psync_sql_res *res, const binresult *meta, int off) { + const binresult *br; + bind_num("created"); + bind_num("modified"); + bind_num("category"); + bind_bool("thumb"); + bind_str("icon"); + bind_opt_str("artist"); + bind_opt_str("album"); + bind_opt_str("title"); + bind_opt_str("genre"); + bind_opt_num("trackno"); + bind_opt_num("width"); + bind_opt_num("height"); + bind_opt_double("duration"); + bind_opt_double("fps"); + bind_opt_str("videocodec"); + bind_opt_str("audiocodec"); + bind_opt_num("videobitrate"); + bind_opt_num("audiobitrate"); + bind_opt_num("audiosamplerate"); + bind_opt_num("rotate"); + return off; +} + +#undef bind_num +#undef bind_bool +#undef bind_str +#undef bind_opt_str +#undef bind_opt_num +#undef bind_opt_double diff --git a/pclsync/pdiff_helpers.h b/pclsync/pdiff_helpers.h new file mode 100644 index 0000000..b49d729 --- /dev/null +++ b/pclsync/pdiff_helpers.h @@ -0,0 +1,60 @@ +/* + * pdiff_helpers.h — pure, separately-compilable helpers extracted from + * pdiff.c to enable unit testing without the full diff/sync stack. + * + * Included by pdiff.c and by tests/unit-tests/test_pdiff_helpers.c. + */ +#ifndef PDIFF_HELPERS_H +#define PDIFF_HELPERS_H + +#include +#include + +#include "papi.h" /* binresult, papi_find_result2, papi_check_result2, PARAM_* */ +#include "pfoldersync.h" /* PSYNC_FOLDER_FLAG_*, psync_folderid_t */ +#include "pmem.h" /* pmem_free, PMEM_SUBSYS_SYNC */ +#include "psql.h" /* psync_full_result_int, psync_sql_res */ +#include "putil.h" /* VAR_ARRAY, putil_strdup */ + +/* + * pdiff_check_active_subscribtion — return 1 if 'res' contains a + * "lastsubscription" hash whose "status" field equals "active"; 0 otherwise. + * + * The name preserves the original typo from pdiff.c. + * Reads the binresult tree only; no network I/O or side effects. + */ +int pdiff_check_active_subscribtion(const binresult *res); + +/* + * pdiff_extract_meta_folder_flags — compute the PSYNC_FOLDER_FLAG_* + * bitmask from a metadata binresult hash. + * + * Pure function: no side effects, no I/O. + */ +uint64_t pdiff_extract_meta_folder_flags(const binresult *meta); + +/* + * pdiff_group_results_by_col — reorder rows of r1 and r2 in-place so that + * every (r1[i], r2[j]) pair sharing the same value in column `col` appears + * at position l, starting from l=0. Both arrays must have identical col + * counts (asserted). + */ +void pdiff_group_results_by_col(psync_full_result_int *restrict r1, + psync_full_result_int *restrict r2, + uint32_t col); + +/* + * pdiff_cmp_folderid — qsort-compatible comparator for psync_folderid_t + * arrays. Compares pointer addresses (faithfully extracted from pdiff.c). + * Returns -1, 0, or 1. + */ +int pdiff_cmp_folderid(const void *ptr1, const void *ptr2); + +/* + * pdiff_bind_meta — bind file metadata fields from 'meta' into prepared + * statement 'res', starting at parameter index 'off'. + * Returns the updated offset (off + number of fields bound = off + 20). + */ +int pdiff_bind_meta(psync_sql_res *res, const binresult *meta, int off); + +#endif /* PDIFF_HELPERS_H */ diff --git a/pclsync/plocalscan.c b/pclsync/plocalscan.c index ec28246..13c9c9b 100644 --- a/pclsync/plocalscan.c +++ b/pclsync/plocalscan.c @@ -40,6 +40,7 @@ #include "plist.h" #include "plocalnotify.h" #include "plocalscan.h" +#include "plocalscan_helpers.h" #include "pmem.h" #include "ppath.h" #include "ppathstatus.h" @@ -65,21 +66,7 @@ typedef struct { char localpath[]; } sync_list; -typedef struct { - psync_list list; - psync_fileorfolderid_t localid; - psync_fileorfolderid_t remoteid; - psync_folderid_t localparentfolderid; - psync_folderid_t parentfolderid; - uint64_t inode; - uint64_t deviceid; - uint64_t mtimenat; - uint64_t size; - psync_syncid_t syncid; - psync_synctype_t synctype; - uint8_t isfolder; - char name[1]; -} sync_folderlist; +/* sync_folderlist moved to plocalscan_helpers.h */ typedef struct { psync_list list; @@ -409,166 +396,40 @@ static void scanner_db_folder_to_list(psync_syncid_t syncid, psql_free(res); } +/* folderlist_cmp, copy_folderlist_element, add_new_element moved to + * plocalscan_helpers.c; local aliases for plocalscan.c callers. */ + static int folderlist_cmp(const psync_list *l1, const psync_list *l2) { - return strcmp( - psync_list_element(l1, sync_folderlist, list)->name, - psync_list_element(l2, sync_folderlist, list)->name); + return plocalscan_folderlist_cmp(l1, l2); } -static sync_folderlist *copy_folderlist_element(const sync_folderlist *e, - psync_folderid_t folderid, - psync_folderid_t localfolderid, - psync_syncid_t syncid, - psync_synctype_t synctype) { - sync_folderlist *ret; - size_t l; - l = offsetof(sync_folderlist, name) + strlen(e->name) + 1; - ret = (sync_folderlist *)pmem_malloc(PMEM_SUBSYS_SYNC, l); - memcpy(ret, e, l); - ret->localparentfolderid = localfolderid; - ret->parentfolderid = folderid; - ret->syncid = syncid; - ret->synctype = synctype; - return ret; -} - -static void add_element_to_scan_list(unsigned long id, sync_folderlist *e) { - psync_list_add_tail(&scan_lists[id], &e->list); - localsleepperfolder = 0; - changes++; -} - -static void add_new_element(const sync_folderlist *e, psync_folderid_t folderid, - psync_folderid_t localfolderid, - psync_syncid_t syncid, psync_synctype_t synctype, - uint64_t deviceid) { - sync_folderlist *c; - if (e->isfolder && e->deviceid != deviceid) - return; - if (psync_is_name_to_ignore(e->name)) - return; - if (!putil_is_valid_utf8(e->name)) { - pdbg_logf(D_WARNING, "ignoring %s with invalid UTF8 name %s", - e->isfolder ? "folder" : "file", e->name); - return; - } - pdbg_logf(D_NOTICE, "found new %s %s", e->isfolder ? "folder" : "file", e->name); - c = copy_folderlist_element(e, folderid, localfolderid, syncid, synctype); - if (e->isfolder) - add_element_to_scan_list(SCAN_LIST_NEWFOLDERS, c); - else - add_element_to_scan_list(SCAN_LIST_NEWFILES, c); -} - -static void add_deleted_element(const sync_folderlist *e, - psync_folderid_t folderid, - psync_folderid_t localfolderid, - psync_syncid_t syncid, - psync_synctype_t synctype) { - sync_folderlist *c; - pdbg_logf(D_NOTICE, "found deleted %s %s", e->isfolder ? "folder" : "file", - e->name); - c = copy_folderlist_element(e, folderid, localfolderid, syncid, synctype); - - if (e->isfolder) { - add_element_to_scan_list(SCAN_LIST_DELFOLDERS, c); - } else { - // Send events only for backups, not for other syncs - if (synctype == 7) { - psync_send_backup_del_event(c->remoteid); - } - - add_element_to_scan_list(SCAN_LIST_DELFILES, c); - } -} - -static void -add_modified_file(const sync_folderlist *e, const sync_folderlist *dbe, - psync_folderid_t folderid, psync_folderid_t localfolderid, - psync_syncid_t syncid, psync_synctype_t synctype) { - pdbg_logf(D_NOTICE, - "found modified file %s on disk: size=%llu mtime=%llu inode=%llu in " - "db: size=%llu mtime=%llu inode=%llu", - e->name, (long long unsigned)e->size, (long long unsigned)e->mtimenat, - (long long unsigned)e->inode, (long long unsigned)dbe->size, - (long long unsigned)dbe->mtimenat, (long long unsigned)dbe->inode); - add_element_to_scan_list( - SCAN_LIST_MODFILES, - copy_folderlist_element(e, folderid, localfolderid, syncid, synctype)); -} +/* add_deleted_element and add_modified_file moved to plocalscan_helpers.c */ static void scanner_scan_folder(const char *localpath, psync_folderid_t folderid, psync_folderid_t localfolderid, psync_syncid_t syncid, psync_synctype_t synctype, uint64_t deviceid) { - psync_list disklist, dblist, *ldisk, *ldb; - sync_folderlist *l, *fdisk, *fdb; + psync_list disklist, dblist; + sync_folderlist *l; char *subpath; - int cmp; - // pdbg_logf(D_NOTICE, "scanning folder %s deviceid: %llu", localpath, deviceid); - if (pdbg_unlikely(scanner_local_folder_to_list(localpath, &disklist))) { + size_t added; + if (pdbg_unlikely(scanner_local_folder_to_list(localpath, &disklist))) return; - } scanner_db_folder_to_list(syncid, localfolderid, &dblist); - psync_list_sort(&dblist, folderlist_cmp); psync_list_sort(&disklist, folderlist_cmp); - ldisk = disklist.next; - ldb = dblist.next; - - while (ldisk != &disklist && ldb != &dblist) { - fdisk = psync_list_element(ldisk, sync_folderlist, list); - fdb = psync_list_element(ldb, sync_folderlist, list); - cmp = strcmp(fdisk->name, fdb->name); - if (cmp == 0) { - if (fdisk->isfolder == fdb->isfolder) { - fdisk->localid = fdb->localid; - fdisk->remoteid = fdb->remoteid; - if (!fdisk->isfolder && - (fdisk->mtimenat != fdb->mtimenat || fdisk->size != fdb->size || - fdisk->inode != fdb->inode)) - add_modified_file(fdisk, fdb, folderid, localfolderid, syncid, - synctype); - if (fdisk->isfolder && - pdevice_id_short(fdisk->deviceid) != fdb->deviceid && - fdisk->inode != fdb->inode) { - if (fdisk->deviceid == deviceid) { - pdbg_logf(D_NOTICE, - "deviceid of localfolder %s %lu is different, skipping", - fdisk->name, (unsigned long)fdisk->localid); - fdisk->localid = 0; - } - } - } else { - add_deleted_element(fdb, folderid, localfolderid, syncid, synctype); - add_new_element(fdisk, folderid, localfolderid, syncid, synctype, - deviceid); - } - ldisk = ldisk->next; - ldb = ldb->next; - } else if (cmp < 0) { // new element on disk - add_new_element(fdisk, folderid, localfolderid, syncid, synctype, - deviceid); - ldisk = ldisk->next; - } else { // deleted element from disk - add_deleted_element(fdb, folderid, localfolderid, syncid, synctype); - ldb = ldb->next; - } + added = plocalscan_merge_folder_lists(&disklist, &dblist, scan_lists, + folderid, localfolderid, syncid, + synctype, deviceid); + if (added) { + localsleepperfolder = 0; + changes += added; } - while (ldisk != &disklist) { - fdisk = psync_list_element(ldisk, sync_folderlist, list); - add_new_element(fdisk, folderid, localfolderid, syncid, synctype, deviceid); - ldisk = ldisk->next; - } - while (ldb != &dblist) { - fdb = psync_list_element(ldb, sync_folderlist, list); - add_deleted_element(fdb, folderid, localfolderid, syncid, synctype); - ldb = ldb->next; - } - psync_list_for_each_element_call(&dblist, sync_folderlist, list, free_sync_folderlist); + psync_list_for_each_element_call(&dblist, sync_folderlist, list, + free_sync_folderlist); if (localsleepperfolder) { psys_sleep_milliseconds(localsleepperfolder); if (__atomic_load_n(&psync_current_time, __ATOMIC_RELAXED) - starttime >= @@ -583,46 +444,18 @@ scanner_scan_folder(const char *localpath, psync_folderid_t folderid, deviceid); pmem_free(PMEM_SUBSYS_SYNC, subpath); } - - psync_list_for_each_element_call(&disklist, sync_folderlist, list, free_sync_folderlist); + psync_list_for_each_element_call(&disklist, sync_folderlist, list, + free_sync_folderlist); } +/* compare_sizeinodemtime and compare_inode moved to plocalscan_helpers.c */ + static int compare_sizeinodemtime(const psync_list *l1, const psync_list *l2) { - const sync_folderlist *f1, *f2; - int64_t d; - f1 = psync_list_element(l1, sync_folderlist, list); - f2 = psync_list_element(l2, sync_folderlist, list); - d = f1->size - f2->size; - if (d < 0) - return -1; - else if (d > 0) - return 1; - d = f1->inode - f2->inode; - if (d < 0) - return -1; - else if (d > 0) - return 1; - d = f1->mtimenat - f2->mtimenat; - if (d < 0) - return -1; - else if (d > 0) - return 1; - else - return 0; + return plocalscan_compare_sizeinodemtime(l1, l2); } static int compare_inode(const psync_list *l1, const psync_list *l2) { - const sync_folderlist *f1, *f2; - int64_t d; - f1 = psync_list_element(l1, sync_folderlist, list); - f2 = psync_list_element(l2, sync_folderlist, list); - d = f1->inode - f2->inode; - if (d < 0) - return -1; - else if (d > 0) - return 1; - else - return 0; + return plocalscan_compare_inode(l1, l2); } static void scan_rename_file(sync_folderlist *rnfr, sync_folderlist *rnto) { diff --git a/pclsync/plocalscan_helpers.c b/pclsync/plocalscan_helpers.c new file mode 100644 index 0000000..d645f6e --- /dev/null +++ b/pclsync/plocalscan_helpers.c @@ -0,0 +1,232 @@ +/* + * plocalscan_helpers.c — sorted-list merge helpers extracted from plocalscan.c. + * + * Deps beyond plocalscan_helpers.h: pdbg.h (logging), pdevice.h (deviceid + * macro), putil.h (UTF-8 validation), and two external declarations for + * psync_is_name_to_ignore / psync_send_backup_del_event (provided by the real + * sync library or by test --wrap stubs). + */ + +#include /* strcmp, memcpy */ +#include /* offsetof */ + +#include "plocalscan_helpers.h" +#include "pdbg.h" /* pdbg_logf */ +#include "pdevice.h" /* pdevice_id_short macro */ +#include "putil.h" /* putil_is_valid_utf8 */ + +/* Forward-declare without pulling in the heavy psynclib.h / pdevice headers */ +extern int psync_is_name_to_ignore(const char *name); +extern void psync_send_backup_del_event(psync_fileorfolderid_t remoteFId); + +/* ------------------------------------------------------------------ */ +/* Comparators */ +/* ------------------------------------------------------------------ */ + +int plocalscan_folderlist_cmp(const psync_list *l1, const psync_list *l2) { + return strcmp( + psync_list_element(l1, sync_folderlist, list)->name, + psync_list_element(l2, sync_folderlist, list)->name); +} + +int plocalscan_compare_sizeinodemtime(const psync_list *l1, + const psync_list *l2) { + const sync_folderlist *f1, *f2; + int64_t d; + f1 = psync_list_element(l1, sync_folderlist, list); + f2 = psync_list_element(l2, sync_folderlist, list); + d = (int64_t)(f1->size - f2->size); + if (d < 0) return -1; + if (d > 0) return 1; + d = (int64_t)(f1->inode - f2->inode); + if (d < 0) return -1; + if (d > 0) return 1; + d = (int64_t)(f1->mtimenat - f2->mtimenat); + if (d < 0) return -1; + if (d > 0) return 1; + return 0; +} + +int plocalscan_compare_inode(const psync_list *l1, const psync_list *l2) { + const sync_folderlist *f1, *f2; + int64_t d; + f1 = psync_list_element(l1, sync_folderlist, list); + f2 = psync_list_element(l2, sync_folderlist, list); + d = (int64_t)(f1->inode - f2->inode); + if (d < 0) return -1; + if (d > 0) return 1; + return 0; +} + +/* ------------------------------------------------------------------ */ +/* Element allocation */ +/* ------------------------------------------------------------------ */ + +sync_folderlist *plocalscan_copy_element(const sync_folderlist *e, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype) { + sync_folderlist *ret; + size_t l = offsetof(sync_folderlist, name) + strlen(e->name) + 1; + ret = (sync_folderlist *)pmem_malloc(PMEM_SUBSYS_SYNC, l); + memcpy(ret, e, l); + ret->localparentfolderid = localfolderid; + ret->parentfolderid = folderid; + ret->syncid = syncid; + ret->synctype = synctype; + return ret; +} + +/* ------------------------------------------------------------------ */ +/* Classification helpers */ +/* ------------------------------------------------------------------ */ + +int plocalscan_add_new_element(const sync_folderlist *e, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + uint64_t deviceid, + psync_list *out) { + sync_folderlist *c; + if (e->isfolder && e->deviceid != deviceid) + return 0; + if (psync_is_name_to_ignore(e->name)) + return 0; + if (!putil_is_valid_utf8(e->name)) { + pdbg_logf(D_WARNING, "ignoring %s with invalid UTF8 name %s", + e->isfolder ? "folder" : "file", e->name); + return 0; + } + pdbg_logf(D_NOTICE, "found new %s %s", + e->isfolder ? "folder" : "file", e->name); + c = plocalscan_copy_element(e, folderid, localfolderid, syncid, synctype); + if (e->isfolder) + psync_list_add_tail(&out[PLOCALSCAN_SCAN_LIST_NEWFOLDERS], &c->list); + else + psync_list_add_tail(&out[PLOCALSCAN_SCAN_LIST_NEWFILES], &c->list); + return 1; +} + +int plocalscan_add_deleted_element(const sync_folderlist *e, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + psync_list *out) { + sync_folderlist *c; + pdbg_logf(D_NOTICE, "found deleted %s %s", + e->isfolder ? "folder" : "file", e->name); + c = plocalscan_copy_element(e, folderid, localfolderid, syncid, synctype); + if (e->isfolder) { + psync_list_add_tail(&out[PLOCALSCAN_SCAN_LIST_DELFOLDERS], &c->list); + } else { + if (synctype == 7) + psync_send_backup_del_event(c->remoteid); + psync_list_add_tail(&out[PLOCALSCAN_SCAN_LIST_DELFILES], &c->list); + } + return 1; +} + +int plocalscan_add_modified_file(const sync_folderlist *e, + const sync_folderlist *dbe, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + psync_list *out) { + pdbg_logf(D_NOTICE, + "found modified file %s on disk: size=%llu mtime=%llu inode=%llu " + "in db: size=%llu mtime=%llu inode=%llu", + e->name, + (long long unsigned)e->size, (long long unsigned)e->mtimenat, + (long long unsigned)e->inode, (long long unsigned)dbe->size, + (long long unsigned)dbe->mtimenat, (long long unsigned)dbe->inode); + psync_list_add_tail(&out[PLOCALSCAN_SCAN_LIST_MODFILES], + &plocalscan_copy_element(e, folderid, localfolderid, syncid, + synctype)->list); + return 1; +} + +/* ------------------------------------------------------------------ */ +/* Merge algorithm */ +/* ------------------------------------------------------------------ */ + +size_t plocalscan_merge_folder_lists(psync_list *disklist, + psync_list *dblist, + psync_list *out, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + uint64_t deviceid) { + psync_list *ldisk, *ldb; + sync_folderlist *fdisk, *fdb; + size_t added = 0; + int cmp; + + ldisk = disklist->next; + ldb = dblist->next; + + while (ldisk != disklist && ldb != dblist) { + fdisk = psync_list_element(ldisk, sync_folderlist, list); + fdb = psync_list_element(ldb, sync_folderlist, list); + cmp = strcmp(fdisk->name, fdb->name); + + if (cmp == 0) { + if (fdisk->isfolder == fdb->isfolder) { + fdisk->localid = fdb->localid; + fdisk->remoteid = fdb->remoteid; + if (!fdisk->isfolder && + (fdisk->mtimenat != fdb->mtimenat || fdisk->size != fdb->size || + fdisk->inode != fdb->inode)) + added += (size_t)plocalscan_add_modified_file( + fdisk, fdb, folderid, localfolderid, syncid, synctype, out); + if (fdisk->isfolder && + pdevice_id_short(fdisk->deviceid) != fdb->deviceid && + fdisk->inode != fdb->inode) { + if (fdisk->deviceid == deviceid) { + pdbg_logf(D_NOTICE, + "deviceid of localfolder %s %lu is different, skipping", + fdisk->name, (unsigned long)fdisk->localid); + fdisk->localid = 0; + } + } + } else { + added += (size_t)plocalscan_add_deleted_element( + fdb, folderid, localfolderid, syncid, synctype, out); + added += (size_t)plocalscan_add_new_element( + fdisk, folderid, localfolderid, syncid, synctype, deviceid, out); + } + ldisk = ldisk->next; + ldb = ldb->next; + } else if (cmp < 0) { /* new entry on disk */ + added += (size_t)plocalscan_add_new_element( + fdisk, folderid, localfolderid, syncid, synctype, deviceid, out); + ldisk = ldisk->next; + } else { /* entry deleted from disk */ + added += (size_t)plocalscan_add_deleted_element( + fdb, folderid, localfolderid, syncid, synctype, out); + ldb = ldb->next; + } + } + + /* Remaining disk entries are all new */ + while (ldisk != disklist) { + fdisk = psync_list_element(ldisk, sync_folderlist, list); + added += (size_t)plocalscan_add_new_element( + fdisk, folderid, localfolderid, syncid, synctype, deviceid, out); + ldisk = ldisk->next; + } + + /* Remaining DB entries are all deleted */ + while (ldb != dblist) { + fdb = psync_list_element(ldb, sync_folderlist, list); + added += (size_t)plocalscan_add_deleted_element( + fdb, folderid, localfolderid, syncid, synctype, out); + ldb = ldb->next; + } + + return added; +} diff --git a/pclsync/plocalscan_helpers.h b/pclsync/plocalscan_helpers.h new file mode 100644 index 0000000..ff800f8 --- /dev/null +++ b/pclsync/plocalscan_helpers.h @@ -0,0 +1,158 @@ +/* + * plocalscan_helpers.h — pure, separately-compilable helpers extracted from + * plocalscan.c to enable unit testing the sorted-list merge algorithm without + * ppath_ls, psql, or threading. + * + * Included by plocalscan.c and tests/unit-tests/test_plocalscan_helpers.c. + */ +#ifndef PLOCALSCAN_HELPERS_H +#define PLOCALSCAN_HELPERS_H + +#include /* offsetof */ +#include +#include /* strcmp, memcpy */ + +#include "pfoldersync.h" /* psync_folderid_t, psync_fileorfolderid_t, + psync_syncid_t, psync_synctype_t */ +#include "plist.h" /* psync_list, psync_list_compare */ +#include "pmem.h" /* pmem_malloc, PMEM_SUBSYS_SYNC */ + +/* ------------------------------------------------------------------ */ +/* sync_folderlist — one scanned fs entry (file or folder) */ +/* ------------------------------------------------------------------ */ + +typedef struct { + psync_list list; + psync_fileorfolderid_t localid; + psync_fileorfolderid_t remoteid; + psync_folderid_t localparentfolderid; + psync_folderid_t parentfolderid; + uint64_t inode; + uint64_t deviceid; + uint64_t mtimenat; + uint64_t size; + psync_syncid_t syncid; + psync_synctype_t synctype; + uint8_t isfolder; + char name[1]; /* flexible: actual allocation carries the full name */ +} sync_folderlist; + +/* ------------------------------------------------------------------ */ +/* Output scan-list slots (index into the out[] array) */ +/* ------------------------------------------------------------------ */ + +#define PLOCALSCAN_SCAN_LIST_CNT 9 +#define PLOCALSCAN_SCAN_LIST_NEWFILES 0 +#define PLOCALSCAN_SCAN_LIST_DELFILES 1 +#define PLOCALSCAN_SCAN_LIST_NEWFOLDERS 2 +#define PLOCALSCAN_SCAN_LIST_DELFOLDERS 3 +#define PLOCALSCAN_SCAN_LIST_MODFILES 4 +#define PLOCALSCAN_SCAN_LIST_RENFILESFROM 5 +#define PLOCALSCAN_SCAN_LIST_RENFILESTO 6 +#define PLOCALSCAN_SCAN_LIST_RENFOLDERSROM 7 +#define PLOCALSCAN_SCAN_LIST_RENFOLDERSTO 8 + +/* ------------------------------------------------------------------ */ +/* Comparators (pure functions, no side effects) */ +/* ------------------------------------------------------------------ */ + +/* + * plocalscan_folderlist_cmp — compare by name (for psync_list_sort). + */ +int plocalscan_folderlist_cmp(const psync_list *l1, const psync_list *l2); + +/* + * plocalscan_compare_sizeinodemtime — compare by (size, inode, mtime). + * Used for file-rename detection. + */ +int plocalscan_compare_sizeinodemtime(const psync_list *l1, + const psync_list *l2); + +/* + * plocalscan_compare_inode — compare by inode. + * Used for folder-rename detection. + */ +int plocalscan_compare_inode(const psync_list *l1, const psync_list *l2); + +/* ------------------------------------------------------------------ */ +/* Element allocation */ +/* ------------------------------------------------------------------ */ + +/* + * plocalscan_copy_element — deep-copy an entry, setting the parent IDs. + * Caller owns the returned allocation (pmem_free). + */ +sync_folderlist *plocalscan_copy_element(const sync_folderlist *e, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype); + +/* ------------------------------------------------------------------ */ +/* Classification helpers — write to injected out[] instead of globals */ +/* ------------------------------------------------------------------ */ + +/* + * plocalscan_add_new_element — classify a disk entry not found in the DB. + * Appends to out[NEWFILES] or out[NEWFOLDERS]. + * Returns the number of elements added (0 if filtered, 1 if added). + */ +int plocalscan_add_new_element(const sync_folderlist *e, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + uint64_t deviceid, + psync_list *out /* [PLOCALSCAN_SCAN_LIST_CNT] */); + +/* + * plocalscan_add_deleted_element — classify a DB entry missing from disk. + * Appends to out[DELFILES] or out[DELFOLDERS]. + * Returns 1 (always adds an element). + */ +int plocalscan_add_deleted_element(const sync_folderlist *e, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + psync_list *out /* [PLOCALSCAN_SCAN_LIST_CNT] */); + +/* + * plocalscan_add_modified_file — classify a file whose metadata differs. + * Appends to out[MODFILES]. + * Returns 1. + */ +int plocalscan_add_modified_file(const sync_folderlist *e, + const sync_folderlist *dbe, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + psync_list *out /* [PLOCALSCAN_SCAN_LIST_CNT] */); + +/* ------------------------------------------------------------------ */ +/* Merge algorithm */ +/* ------------------------------------------------------------------ */ + +/* + * plocalscan_merge_folder_lists — compare two pre-sorted lists and classify + * entries into out[]. + * + * disklist: sorted by name (disk entries, not freed by this function) + * dblist: sorted by name (DB entries, not freed by this function) + * out: array of PLOCALSCAN_SCAN_LIST_CNT psync_list heads, already + * initialised by the caller; new elements are appended. + * + * Returns the total number of elements added across all output lists. + * The caller is responsible for eventually freeing the appended elements. + */ +size_t plocalscan_merge_folder_lists(psync_list *disklist, + psync_list *dblist, + psync_list *out, + psync_folderid_t folderid, + psync_folderid_t localfolderid, + psync_syncid_t syncid, + psync_synctype_t synctype, + uint64_t deviceid); + +#endif /* PLOCALSCAN_HELPERS_H */ diff --git a/pclsync/ppagecache.c b/pclsync/ppagecache.c index e15e538..d115b3f 100644 --- a/pclsync/ppagecache.c +++ b/pclsync/ppagecache.c @@ -358,8 +358,8 @@ static int psync_pagecache_read_range_from_api(psync_request_t *request, binresult *res; unsigned long len, i, h; int rb; - first_page_id = range->offset / PSYNC_FS_PAGE_SIZE; - len = range->length / PSYNC_FS_PAGE_SIZE; + first_page_id = ppagecache_range_first_page_id(range->offset); + len = (unsigned long)ppagecache_range_page_count(range->length); res = papi_result_thread(api); if (pdbg_unlikely(!res)) return -2; @@ -2304,21 +2304,20 @@ static int psync_pagecache_read_range_from_sock(psync_request_t *request, psync_cache_page_t *page; unsigned long len, i, h; int rb; - first_page_id = range->offset / PSYNC_FS_PAGE_SIZE; - len = range->length / PSYNC_FS_PAGE_SIZE; + first_page_id = ppagecache_range_first_page_id(range->offset); + len = (unsigned long)ppagecache_range_page_count(range->length); rb = psync_http_next_request(sock); if (unlikely(rb)) { - if (rb == 410 || rb == 404 || rb == -1) { + int retry = ppagecache_http_status_should_retry(rb); + if (retry == 1) pdbg_logf(D_WARNING, "got %d from psync_http_next_request, freeing URLs and requesting " "retry, range from %lu", rb, (long unsigned)range->offset); - return 1; - } else { + else pdbg_logf(D_WARNING, "got %d from psync_http_next_request, returning error", rb); - return -1; - } + return retry; } for (i = 0; i < len; i++) { page = psync_pagecache_get_free_page(0); diff --git a/pclsync/ppagecache_helpers.c b/pclsync/ppagecache_helpers.c index c6437bd..0982d58 100644 --- a/pclsync/ppagecache_helpers.c +++ b/pclsync/ppagecache_helpers.c @@ -28,6 +28,18 @@ int ppagecache_verify_crc(const void *data, size_t size, uint32_t stored_crc) { return (computed == stored_crc) ? 0 : -1; } +/* ------------------------------------------------------------------ */ +/* HTTP status classification */ +/* ------------------------------------------------------------------ */ + +int ppagecache_http_status_should_retry(int status) { + if (status == 0) + return 0; + if (status == 410 || status == 404 || status == -1) + return 1; + return -1; +} + /* ------------------------------------------------------------------ */ /* Download-URL seam (weak default: no-op) */ /* ------------------------------------------------------------------ */ diff --git a/pclsync/ppagecache_helpers.h b/pclsync/ppagecache_helpers.h index ccc07eb..e20cac1 100644 --- a/pclsync/ppagecache_helpers.h +++ b/pclsync/ppagecache_helpers.h @@ -22,6 +22,41 @@ #define PPAGECACHE_TIER3_THRESHOLD 8u #define PPAGECACHE_TIER4_THRESHOLD 16u +/* + * Page size — must match PSYNC_FS_PAGE_SIZE from psettings.h. + * Kept here so unit tests do not need to pull in psettings.h and its + * transitive SSL/compiler dependencies. + */ +#define PPAGECACHE_PAGE_SIZE 4096u + +/* + * ppagecache_range_first_page_id — return the ID of the first page that + * covers byte offset `offset`. Pure arithmetic, no side effects. + */ +static inline uint64_t ppagecache_range_first_page_id(uint64_t offset) { + return offset / PPAGECACHE_PAGE_SIZE; +} + +/* + * ppagecache_range_page_count — return the number of whole pages in a + * byte-length range. Pure arithmetic, no side effects. + */ +static inline uint64_t ppagecache_range_page_count(uint64_t length) { + return length / PPAGECACHE_PAGE_SIZE; +} + +/* + * ppagecache_http_status_should_retry — classify a status code returned by + * psync_http_next_request: + * + * 0 — success (status == 0) + * 1 — retryable: URL expired or connection lost (410, 404, -1) + * -1 — hard error: any other non-zero status + * + * Pure function: no side effects, no I/O. + */ +int ppagecache_http_status_should_retry(int status); + /* * ppagecache_compute_page_priority — assign an eviction-priority tier to a * cached page based on its access count. diff --git a/tests/unit-tests/test_pdiff_helpers.c b/tests/unit-tests/test_pdiff_helpers.c new file mode 100644 index 0000000..2973687 --- /dev/null +++ b/tests/unit-tests/test_pdiff_helpers.c @@ -0,0 +1,434 @@ +/* + * Test: pdiff_helpers.c + * + * Covers: + * 1. pdiff_check_active_subscribtion: no sub, inactive, active. + * 2. pdiff_extract_meta_folder_flags: empty, single flag, multiple flags. + * 3. pdiff_group_results_by_col: matching pairs reordered, no-match unchanged. + * 4. pdiff_cmp_folderid: equal, ordered (pointer-address comparison). + * 5. pdiff_bind_meta: total bind calls and returned offset. + * + * Provides real papi_find_result / papi_check_result via --wrap so the + * test_stubs.c NULL-returning stubs are overridden for this binary only. + * SQL bind functions are wrapped with counters so no SQLite DB is needed. + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include + +#include "pdiff_helpers.h" + +/* psync_get_result_cell is a macro in plibs.h; redeclare it here to avoid + * pulling in the full plibs / psynclib dependency chain in the test. */ +#ifndef psync_get_result_cell +#define psync_get_result_cell(res, row, col) \ + ((res)->data[(row) * (res)->cols + (col)]) +#endif + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); \ + printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Minimal papi_find_result / papi_check_result overrides (--wrap) */ +/* ------------------------------------------------------------------ */ + +static binresult g_empty_num = {PARAM_NUM, 0, {0}}; +static binresult g_empty_bool = {PARAM_BOOL, 0, {0}}; +static binresult g_empty_str = {PARAM_STR, 0, {0}}; + +static const binresult *empty_for(uint32_t type) { + if (type == PARAM_NUM) return &g_empty_num; + if (type == PARAM_BOOL) return &g_empty_bool; + return &g_empty_str; +} + +/* Override the stub from test_stubs.c */ +const binresult *__wrap_papi_find_result(const binresult *res, const char *name, + uint32_t type, const char *file, + const char *function, + unsigned int line) { + uint32_t i; + (void)file; (void)function; (void)line; + if (!res || res->type != PARAM_HASH) + return empty_for(type); + for (i = 0; i < res->length; i++) + if (!strcmp(res->hash[i].key, name) && + res->hash[i].value->type == type) + return res->hash[i].value; + return empty_for(type); +} + +/* papi_check_result is not in test_stubs.c; --wrap provides it */ +const binresult *__wrap_papi_check_result(const binresult *res, const char *name, + uint32_t type, const char *file, + const char *function, + unsigned int line) { + uint32_t i; + (void)file; (void)function; (void)line; + if (!res || res->type != PARAM_HASH) + return NULL; + for (i = 0; i < res->length; i++) + if (!strcmp(res->hash[i].key, name) && + res->hash[i].value->type == type) + return res->hash[i].value; + return NULL; +} + +/* ------------------------------------------------------------------ */ +/* SQL bind call counters (--wrap) */ +/* ------------------------------------------------------------------ */ + +static int g_bind_uint_n = 0; +static int g_bind_lstr_n = 0; +static int g_bind_null_n = 0; +static int g_bind_double_n = 0; + +static void reset_bind_counts(void) { + g_bind_uint_n = g_bind_lstr_n = g_bind_null_n = g_bind_double_n = 0; +} + +void __wrap_psql_bind_uint(psync_sql_res *res, int n, uint64_t val) { + (void)res; (void)n; (void)val; + g_bind_uint_n++; +} +void __wrap_psql_bind_lstr(psync_sql_res *res, int n, const char *str, + size_t len) { + (void)res; (void)n; (void)str; (void)len; + g_bind_lstr_n++; +} +void __wrap_psql_bind_null(psync_sql_res *res, int n) { + (void)res; (void)n; + g_bind_null_n++; +} +void __wrap_psql_bind_double(psync_sql_res *res, int n, double val) { + (void)res; (void)n; (void)val; + g_bind_double_n++; +} + +/* ------------------------------------------------------------------ */ +/* binresult construction helpers */ +/* ------------------------------------------------------------------ */ + +/* + * init_bool — write a PARAM_BOOL binresult into *r. + * Uses memset+assignment via the num union member to avoid the + * "assignment of read-only member" error caused by const char str[8]. + */ +static void init_bool(binresult *r, uint64_t v) { + memset(r, 0, sizeof(*r)); + r->type = PARAM_BOOL; + r->length = 0; + r->num = v; +} + +static void init_num(binresult *r, uint64_t v) { + memset(r, 0, sizeof(*r)); + r->type = PARAM_NUM; + r->length = 0; + r->num = v; +} + +/* + * init_str — write a PARAM_STR binresult for a short string (≤ 7 chars). + * The string bytes are stored in the num union field (same memory as str[8]). + */ +static void init_str(binresult *r, const char *s) { + size_t len = strlen(s); + memset(r, 0, sizeof(*r)); + r->type = PARAM_STR; + r->length = (uint32_t)len; + /* num shares bytes with str[8]; memcpy here is well-defined via the + * union's common initial sequence and the char aliasing rule. */ + memcpy(&r->num, s, len < 8 ? len : 7); +} + +static void init_hash(binresult *r, hashpair *pairs, uint32_t npairs) { + memset(r, 0, sizeof(*r)); + r->type = PARAM_HASH; + r->length = npairs; + r->hash = pairs; +} + +/* ------------------------------------------------------------------ */ +/* Test 1: pdiff_check_active_subscribtion */ +/* ------------------------------------------------------------------ */ +static void test_check_active_subscribtion(void) { + binresult outer, sub_hash, status_br; + hashpair outer_pairs[1], sub_pairs[1]; + + /* Case 1: no "lastsubscription" field → 0 */ + init_hash(&outer, outer_pairs, 0); + if (pdiff_check_active_subscribtion(&outer) != 0) + FAIL("subscribtion: no lastsubscription", "expected 0"); + else + PASS("subscribtion: no lastsubscription → 0"); + + /* Case 2: lastsubscription present but status = "pending" → 0 */ + init_str(&status_br, "pending"); + sub_pairs[0].key = "status"; + sub_pairs[0].value = &status_br; + init_hash(&sub_hash, sub_pairs, 1); + outer_pairs[0].key = "lastsubscription"; + outer_pairs[0].value = &sub_hash; + init_hash(&outer, outer_pairs, 1); + if (pdiff_check_active_subscribtion(&outer) != 0) + FAIL("subscribtion: status=pending", "expected 0"); + else + PASS("subscribtion: status=\"pending\" → 0"); + + /* Case 3: lastsubscription present, status = "active" → 1 */ + init_str(&status_br, "active"); + if (pdiff_check_active_subscribtion(&outer) != 1) + FAIL("subscribtion: status=active", "expected 1"); + else + PASS("subscribtion: status=\"active\" → 1"); +} + +/* ------------------------------------------------------------------ */ +/* Test 2: pdiff_extract_meta_folder_flags */ +/* ------------------------------------------------------------------ */ +static void test_extract_meta_folder_flags(void) { + binresult meta, enc_false, enc_true, pub_true; + hashpair pairs[2]; + uint64_t flags; + + /* Case 1: empty meta → flags = 0 */ + init_hash(&meta, pairs, 0); + flags = pdiff_extract_meta_folder_flags(&meta); + if (flags != 0) + FAIL("flags: empty meta", "expected 0, got %lu", (unsigned long)flags); + else + PASS("flags: empty meta → 0"); + + /* Case 2: encrypted=false → flags = 0 */ + init_bool(&enc_false, 0); + pairs[0].key = "encrypted"; + pairs[0].value = &enc_false; + init_hash(&meta, pairs, 1); + flags = pdiff_extract_meta_folder_flags(&meta); + if (flags != 0) + FAIL("flags: encrypted=false", "expected 0, got %lu", + (unsigned long)flags); + else + PASS("flags: encrypted=false → 0"); + + /* Case 3: encrypted=true → PSYNC_FOLDER_FLAG_ENCRYPTED */ + init_bool(&enc_true, 1); + pairs[0].value = &enc_true; + flags = pdiff_extract_meta_folder_flags(&meta); + if (flags != PSYNC_FOLDER_FLAG_ENCRYPTED) + FAIL("flags: encrypted=true", "expected %u, got %lu", + PSYNC_FOLDER_FLAG_ENCRYPTED, (unsigned long)flags); + else + PASS("flags: encrypted=true → PSYNC_FOLDER_FLAG_ENCRYPTED"); + + /* Case 4: encrypted=true + ispublicroot=true → both flags */ + init_bool(&pub_true, 1); + pairs[1].key = "ispublicroot"; + pairs[1].value = &pub_true; + init_hash(&meta, pairs, 2); + flags = pdiff_extract_meta_folder_flags(&meta); + if (flags != (PSYNC_FOLDER_FLAG_ENCRYPTED | PSYNC_FOLDER_FLAG_PUBLIC_ROOT)) + FAIL("flags: encrypted+public_root", + "expected %u, got %lu", + PSYNC_FOLDER_FLAG_ENCRYPTED | PSYNC_FOLDER_FLAG_PUBLIC_ROOT, + (unsigned long)flags); + else + PASS("flags: encrypted+ispublicroot → both flags set"); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: pdiff_group_results_by_col */ +/* ------------------------------------------------------------------ */ + +/* + * Stack-allocated psync_full_result_int with embedded data storage. + */ +typedef struct { + psync_full_result_int hdr; + uint64_t storage[16]; +} test_result_int_t; + +static void init_result(test_result_int_t *r, uint32_t rows, uint32_t cols, + const uint64_t *vals) { + uint32_t i; + r->hdr.rows = rows; + r->hdr.cols = cols; + for (i = 0; i < rows * cols; i++) + r->hdr.data[i] = vals[i]; +} + +static void test_group_results_by_col(void) { + test_result_int_t r1, r2; + uint32_t i; + int all_match; + + /* + * r1: [[10, 1], [20, 2], [30, 3]] (col 0 = id) + * r2: [[30, 9], [10, 8], [20, 7]] (col 0 = id, different order) + * + * After grouping on col 0, each r1[i].col0 must equal r2[i].col0. + */ + uint64_t v1[] = {10, 1, 20, 2, 30, 3}; + uint64_t v2[] = {30, 9, 10, 8, 20, 7}; + + init_result(&r1, 3, 2, v1); + init_result(&r2, 3, 2, v2); + + pdiff_group_results_by_col(&r1.hdr, &r2.hdr, 0); + + all_match = 1; + for (i = 0; i < 3; i++) { + if (psync_get_result_cell(&r1.hdr, i, 0) != + psync_get_result_cell(&r2.hdr, i, 0)) { + all_match = 0; + break; + } + } + if (!all_match) + FAIL("group_results: matching pairs aligned", + "r1[%u].col0=%lu != r2[%u].col0=%lu", i, + (unsigned long)psync_get_result_cell(&r1.hdr, i, 0), i, + (unsigned long)psync_get_result_cell(&r2.hdr, i, 0)); + else + PASS("group_results: 3-row matching pairs aligned after group"); + + /* No-match: r1 and r2 share no values in col 0 → rows unchanged */ + { + uint64_t nv1[] = {1, 0, 2, 0}; + uint64_t nv2[] = {3, 0, 4, 0}; + init_result(&r1, 2, 2, nv1); + init_result(&r2, 2, 2, nv2); + pdiff_group_results_by_col(&r1.hdr, &r2.hdr, 0); + if (psync_get_result_cell(&r1.hdr, 0, 0) != 1 || + psync_get_result_cell(&r2.hdr, 0, 0) != 3) + FAIL("group_results: no-match rows unchanged", + "r1[0]=%lu r2[0]=%lu", + (unsigned long)psync_get_result_cell(&r1.hdr, 0, 0), + (unsigned long)psync_get_result_cell(&r2.hdr, 0, 0)); + else + PASS("group_results: no-match case leaves rows unchanged"); + } +} + +/* ------------------------------------------------------------------ */ +/* Test 4: pdiff_cmp_folderid */ +/* ------------------------------------------------------------------ */ +static void test_cmp_folderid(void) { + psync_folderid_t arr[2] = {5, 10}; + int r; + + /* Same pointer → 0 */ + r = pdiff_cmp_folderid(&arr[0], &arr[0]); + if (r != 0) + FAIL("cmp_folderid: same pointer", "expected 0, got %d", r); + else + PASS("cmp_folderid: same pointer → 0"); + + /* + * C guarantees &arr[0] < &arr[1]. The function compares pointer + * addresses (faithfully extracted from pdiff.c), so arr[0] < arr[1] → -1. + */ + r = pdiff_cmp_folderid(&arr[0], &arr[1]); + if (r != -1) + FAIL("cmp_folderid: &arr[0] < &arr[1]", "expected -1, got %d", r); + else + PASS("cmp_folderid: &arr[0] < &arr[1] → -1 (pointer comparison)"); + + r = pdiff_cmp_folderid(&arr[1], &arr[0]); + if (r != 1) + FAIL("cmp_folderid: &arr[1] > &arr[0]", "expected 1, got %d", r); + else + PASS("cmp_folderid: &arr[1] > &arr[0] → 1 (antisymmetric)"); +} + +/* ------------------------------------------------------------------ */ +/* Test 5: pdiff_bind_meta */ +/* ------------------------------------------------------------------ */ +static void test_bind_meta(void) { + /* + * Build a minimal binresult meta with only the five required fields: + * created (PARAM_NUM) + * modified (PARAM_NUM) + * category (PARAM_NUM) + * thumb (PARAM_BOOL) + * icon (PARAM_STR, required by bind_str) + * + * All 15 optional fields absent → psql_bind_null for each. + * + * Expected: 20 total bind calls; return value = 7 + 20 = 27. + */ + binresult created, modified, category, thumb, icon, meta; + hashpair pairs[5]; + int ret, total; + + init_num(&created, 1000); + init_num(&modified, 2000); + init_num(&category, 1); + init_bool(&thumb, 0); + init_str(&icon, "jpg"); + + pairs[0].key = "created"; pairs[0].value = &created; + pairs[1].key = "modified"; pairs[1].value = &modified; + pairs[2].key = "category"; pairs[2].value = &category; + pairs[3].key = "thumb"; pairs[3].value = &thumb; + pairs[4].key = "icon"; pairs[4].value = &icon; + init_hash(&meta, pairs, 5); + + reset_bind_counts(); + ret = pdiff_bind_meta((psync_sql_res *)1 /* dummy non-NULL */, &meta, 7); + + total = g_bind_uint_n + g_bind_lstr_n + g_bind_null_n + g_bind_double_n; + + if (total != 20) + FAIL("bind_meta: total bind calls", + "expected 20, got %d (uint=%d lstr=%d null=%d dbl=%d)", + total, g_bind_uint_n, g_bind_lstr_n, g_bind_null_n, + g_bind_double_n); + else + PASS("bind_meta: exactly 20 bind calls for minimal meta"); + + if (ret != 27) + FAIL("bind_meta: return offset", "expected 27, got %d", ret); + else + PASS("bind_meta: return offset = off_initial(7) + 20 = 27"); + + /* bind_uint: 3 bind_num + 1 bind_bool = 4 */ + if (g_bind_uint_n != 4) + FAIL("bind_meta: bind_uint count", + "expected 4 (3*bind_num + 1*bind_bool), got %d", g_bind_uint_n); + else + PASS("bind_meta: bind_uint called 4 times (created/modified/category/thumb)"); + + /* bind_lstr: 1 bind_str(icon) */ + if (g_bind_lstr_n != 1) + FAIL("bind_meta: bind_lstr count", + "expected 1 (icon only), got %d", g_bind_lstr_n); + else + PASS("bind_meta: bind_lstr called 1 time (icon)"); + + /* bind_null: 15 optional absent fields */ + if (g_bind_null_n != 15) + FAIL("bind_meta: bind_null count", + "expected 15 optional-absent NULLs, got %d", g_bind_null_n); + else + PASS("bind_meta: bind_null called 15 times (all optional fields absent)"); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + test_check_active_subscribtion(); + test_extract_meta_folder_flags(); + test_group_results_by_col(); + test_cmp_folderid(); + test_bind_meta(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +} diff --git a/tests/unit-tests/test_plocalscan_helpers.c b/tests/unit-tests/test_plocalscan_helpers.c new file mode 100644 index 0000000..2688415 --- /dev/null +++ b/tests/unit-tests/test_plocalscan_helpers.c @@ -0,0 +1,514 @@ +/* + * Test: plocalscan_helpers.c — sorted-list merge algorithm + * + * Covers: + * 1. plocalscan_folderlist_cmp: name ordering + * 2. plocalscan_compare_sizeinodemtime: field ordering + * 3. plocalscan_compare_inode: inode ordering + * 4. plocalscan_merge_folder_lists: + * - new entries (disk only) → NEWFILES / NEWFOLDERS + * - deleted entries (db only) → DELFILES / DELFOLDERS + * - modified file (same name, different metadata) → MODFILES + * - unchanged file (same name, same metadata) → nothing added + * - mixed: new + deleted + modified in one call + * + * External deps wrapped via --wrap: + * psync_is_name_to_ignore → always 0 (don't ignore any name) + * psync_send_backup_del_event → no-op + */ + +#define _POSIX_C_SOURCE 200809L +#include +#include +#include +#include +#include + +#include "plocalscan_helpers.h" + +/* ------------------------------------------------------------------ */ +static int passes = 0, failures = 0; +#define PASS(n) do { printf("PASS: %s\n", n); passes++; } while (0) +#define FAIL(n, ...) do { printf("FAIL: %s — ", n); \ + printf(__VA_ARGS__); printf("\n"); failures++; } while (0) + +/* ------------------------------------------------------------------ */ +/* Wrap stubs */ +/* ------------------------------------------------------------------ */ + +int __wrap_psync_is_name_to_ignore(const char *name) { + (void)name; + return 0; /* never ignore */ +} + +void __wrap_psync_send_backup_del_event(psync_fileorfolderid_t id) { + (void)id; +} + +/* ------------------------------------------------------------------ */ +/* Helper: allocate a sync_folderlist entry */ +/* ------------------------------------------------------------------ */ + +static sync_folderlist *make_entry(const char *name, uint8_t isfolder, + uint64_t inode, uint64_t size, + uint64_t mtimenat, uint64_t deviceid) { + size_t namelen = strlen(name) + 1; + size_t sz = offsetof(sync_folderlist, name) + namelen; + sync_folderlist *e = (sync_folderlist *)malloc(sz); + memset(e, 0, sz); + e->isfolder = isfolder; + e->inode = inode; + e->size = size; + e->mtimenat = mtimenat; + e->deviceid = deviceid; + psync_list_init(&e->list); + memcpy(e->name, name, namelen); + return e; +} + +/* Append entry to a list head */ +static void list_append(psync_list *head, sync_folderlist *e) { + psync_list_add_tail(head, &e->list); +} + +/* Count elements in a psync_list */ +static int list_count(psync_list *head) { + int n = 0; + psync_list *cur; + psync_list_for_each(cur, head) n++; + return n; +} + +/* Get the nth element name from an output list */ +static const char *list_nth_name(psync_list *head, int n) { + psync_list *cur; + int i = 0; + psync_list_for_each(cur, head) { + if (i++ == n) + return psync_list_element(cur, sync_folderlist, list)->name; + } + return NULL; +} + +/* Free all elements in a list (allocated via plocalscan_copy_element → pmem_malloc). + * Must use pmem_free, not free(), because pmem_malloc prepends a header. */ +static void free_list(psync_list *head) { + psync_list *cur, *tmp; + psync_list_for_each_safe(cur, tmp, head) + pmem_free(PMEM_SUBSYS_SYNC, + psync_list_element(cur, sync_folderlist, list)); + psync_list_init(head); +} + +/* Initialise PLOCALSCAN_SCAN_LIST_CNT output list heads */ +static void init_out(psync_list out[]) { + int i; + for (i = 0; i < PLOCALSCAN_SCAN_LIST_CNT; i++) + psync_list_init(&out[i]); +} + +static void free_out(psync_list out[]) { + int i; + for (i = 0; i < PLOCALSCAN_SCAN_LIST_CNT; i++) + free_list(&out[i]); +} + +/* ------------------------------------------------------------------ */ +/* Test 1: comparators */ +/* ------------------------------------------------------------------ */ + +static void test_comparators(void) { + sync_folderlist *a, *b; + + /* folderlist_cmp: strcmp on name */ + a = make_entry("alpha", 0, 1, 100, 1000, 0); + b = make_entry("beta", 0, 2, 200, 2000, 0); + if (plocalscan_folderlist_cmp(&a->list, &b->list) >= 0) + FAIL("folderlist_cmp: alpha < beta", "expected negative, got non-negative"); + else + PASS("folderlist_cmp: alpha < beta → negative"); + + if (plocalscan_folderlist_cmp(&b->list, &a->list) <= 0) + FAIL("folderlist_cmp: beta > alpha", "expected positive, got non-positive"); + else + PASS("folderlist_cmp: beta > alpha → positive"); + + if (plocalscan_folderlist_cmp(&a->list, &a->list) != 0) + FAIL("folderlist_cmp: alpha == alpha", "expected 0"); + else + PASS("folderlist_cmp: alpha == alpha → 0"); + + free(a); free(b); + + /* compare_sizeinodemtime */ + { + sync_folderlist *s1, *s2; + s1 = make_entry("f", 0, 10, 100, 1000, 0); + s2 = make_entry("f", 0, 20, 200, 2000, 0); + + /* size differs: s1.size < s2.size */ + if (plocalscan_compare_sizeinodemtime(&s1->list, &s2->list) >= 0) + FAIL("compare_sim: size s1 < s2", "expected negative"); + else + PASS("compare_sim: size s1(100) < s2(200) → negative"); + + if (plocalscan_compare_sizeinodemtime(&s2->list, &s1->list) <= 0) + FAIL("compare_sim: size s2 > s1", "expected positive"); + else + PASS("compare_sim: size s2(200) > s1(100) → positive"); + + /* same size, compare inode */ + s2->size = s1->size; + if (plocalscan_compare_sizeinodemtime(&s1->list, &s2->list) >= 0) + FAIL("compare_sim: inode s1(10) < s2(20)", "expected negative"); + else + PASS("compare_sim: inode s1(10) < s2(20) → negative"); + + /* same size + inode, compare mtime */ + s2->inode = s1->inode; + if (plocalscan_compare_sizeinodemtime(&s1->list, &s2->list) >= 0) + FAIL("compare_sim: mtime s1(1000) < s2(2000)", "expected negative"); + else + PASS("compare_sim: mtime s1(1000) < s2(2000) → negative"); + + /* all equal */ + s2->mtimenat = s1->mtimenat; + if (plocalscan_compare_sizeinodemtime(&s1->list, &s2->list) != 0) + FAIL("compare_sim: all equal", "expected 0"); + else + PASS("compare_sim: all fields equal → 0"); + + free(s1); free(s2); + } + + /* compare_inode */ + { + sync_folderlist *i1, *i2; + i1 = make_entry("d", 1, 5, 0, 0, 0); + i2 = make_entry("d", 1, 15, 0, 0, 0); + + if (plocalscan_compare_inode(&i1->list, &i2->list) >= 0) + FAIL("compare_inode: 5 < 15", "expected negative"); + else + PASS("compare_inode: inode 5 < 15 → negative"); + + if (plocalscan_compare_inode(&i2->list, &i1->list) <= 0) + FAIL("compare_inode: 15 > 5", "expected positive"); + else + PASS("compare_inode: inode 15 > 5 → positive"); + + i2->inode = i1->inode; + if (plocalscan_compare_inode(&i1->list, &i2->list) != 0) + FAIL("compare_inode: equal", "expected 0"); + else + PASS("compare_inode: equal inodes → 0"); + + free(i1); free(i2); + } +} + +/* ------------------------------------------------------------------ */ +/* Test 2: new entries (disk has entries DB doesn't) */ +/* ------------------------------------------------------------------ */ + +static void test_merge_new_entries(void) { + psync_list disklist, dblist; + psync_list out[PLOCALSCAN_SCAN_LIST_CNT]; + size_t added; + + psync_list_init(&disklist); + psync_list_init(&dblist); + init_out(out); + + /* Two new files on disk, not in DB */ + list_append(&disklist, make_entry("file_a.txt", 0, 1, 100, 1000, 1)); + list_append(&disklist, make_entry("file_b.txt", 0, 2, 200, 2000, 1)); + + added = plocalscan_merge_folder_lists(&disklist, &dblist, out, + 10, 20, 1, 1, 1); + if (added != 2) + FAIL("new files: added count", "expected 2, got %zu", added); + else + PASS("new files: 2 elements added"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFILES]) != 2) + FAIL("new files: NEWFILES count", + "expected 2, got %d", + list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFILES])); + else + PASS("new files: both in NEWFILES list"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES]) != 0) + FAIL("new files: DELFILES should be empty", + "got %d", list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES])); + else + PASS("new files: DELFILES empty"); + + free_out(out); + /* disklist entries were NOT copied (only their copies are in out[]) */ + psync_list_for_each_element_call(&disklist, sync_folderlist, list, free); +} + +/* ------------------------------------------------------------------ */ +/* Test 3: new folder entry */ +/* ------------------------------------------------------------------ */ + +static void test_merge_new_folder(void) { + psync_list disklist, dblist; + psync_list out[PLOCALSCAN_SCAN_LIST_CNT]; + size_t added; + + psync_list_init(&disklist); + psync_list_init(&dblist); + init_out(out); + + /* New folder: deviceid must match for folders to be classified as new */ + list_append(&disklist, make_entry("newdir", 1, 5, 0, 0, 42)); + + added = plocalscan_merge_folder_lists(&disklist, &dblist, out, + 10, 20, 1, 1, 42 /* same deviceid */); + if (added != 1) + FAIL("new folder: added count", "expected 1, got %zu", added); + else + PASS("new folder: 1 element added"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFOLDERS]) != 1) + FAIL("new folder: NEWFOLDERS count", + "expected 1, got %d", + list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFOLDERS])); + else + PASS("new folder: in NEWFOLDERS list"); + + free_out(out); + psync_list_for_each_element_call(&disklist, sync_folderlist, list, free); +} + +/* ------------------------------------------------------------------ */ +/* Test 4: deleted entries (DB has entries disk doesn't) */ +/* ------------------------------------------------------------------ */ + +static void test_merge_deleted_entries(void) { + psync_list disklist, dblist; + psync_list out[PLOCALSCAN_SCAN_LIST_CNT]; + size_t added; + + psync_list_init(&disklist); + psync_list_init(&dblist); + init_out(out); + + /* One deleted file and one deleted folder in DB */ + list_append(&dblist, make_entry("gone_file.txt", 0, 3, 300, 3000, 1)); + list_append(&dblist, make_entry("gone_dir", 1, 4, 0, 0, 1)); + + added = plocalscan_merge_folder_lists(&disklist, &dblist, out, + 10, 20, 1, 1, 1); + if (added != 2) + FAIL("deleted entries: added count", "expected 2, got %zu", added); + else + PASS("deleted entries: 2 elements added"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES]) != 1) + FAIL("deleted entries: DELFILES count", + "expected 1, got %d", + list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES])); + else + PASS("deleted entries: 1 in DELFILES"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_DELFOLDERS]) != 1) + FAIL("deleted entries: DELFOLDERS count", + "expected 1, got %d", + list_count(&out[PLOCALSCAN_SCAN_LIST_DELFOLDERS])); + else + PASS("deleted entries: 1 in DELFOLDERS"); + + free_out(out); + psync_list_for_each_element_call(&dblist, sync_folderlist, list, free); +} + +/* ------------------------------------------------------------------ */ +/* Test 5: modified file */ +/* ------------------------------------------------------------------ */ + +static void test_merge_modified_file(void) { + psync_list disklist, dblist; + psync_list out[PLOCALSCAN_SCAN_LIST_CNT]; + size_t added; + sync_folderlist *disk_e, *db_e; + + psync_list_init(&disklist); + psync_list_init(&dblist); + init_out(out); + + /* Same name "data.bin" but different mtime → modified */ + disk_e = make_entry("data.bin", 0, 7, 512, 9999, 1); + db_e = make_entry("data.bin", 0, 7, 512, 8888, 1); + + list_append(&disklist, disk_e); + list_append(&dblist, db_e); + + added = plocalscan_merge_folder_lists(&disklist, &dblist, out, + 10, 20, 1, 1, 1); + if (added != 1) + FAIL("modified file: added count", "expected 1, got %zu", added); + else + PASS("modified file: 1 element added"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_MODFILES]) != 1) + FAIL("modified file: MODFILES count", + "expected 1, got %d", + list_count(&out[PLOCALSCAN_SCAN_LIST_MODFILES])); + else + PASS("modified file: in MODFILES list"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFILES]) != 0 || + list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES]) != 0) + FAIL("modified file: no spurious new/del entries", + "NEWFILES=%d DELFILES=%d", + list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFILES]), + list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES])); + else + PASS("modified file: no spurious NEWFILES/DELFILES"); + + free_out(out); + free(disk_e); + free(db_e); +} + +/* ------------------------------------------------------------------ */ +/* Test 6: unchanged file → nothing added */ +/* ------------------------------------------------------------------ */ + +static void test_merge_unchanged_file(void) { + psync_list disklist, dblist; + psync_list out[PLOCALSCAN_SCAN_LIST_CNT]; + size_t added; + sync_folderlist *disk_e, *db_e; + + psync_list_init(&disklist); + psync_list_init(&dblist); + init_out(out); + + /* Same name, same inode/size/mtime → no change */ + disk_e = make_entry("stable.txt", 0, 42, 1024, 5000, 1); + db_e = make_entry("stable.txt", 0, 42, 1024, 5000, 1); + + list_append(&disklist, disk_e); + list_append(&dblist, db_e); + + added = plocalscan_merge_folder_lists(&disklist, &dblist, out, + 10, 20, 1, 1, 1); + if (added != 0) + FAIL("unchanged file: added count", "expected 0, got %zu", added); + else + PASS("unchanged file: 0 elements added"); + + { + int i, total = 0; + for (i = 0; i < PLOCALSCAN_SCAN_LIST_CNT; i++) + total += list_count(&out[i]); + if (total != 0) + FAIL("unchanged file: all out[] lists empty", + "total=%d", total); + else + PASS("unchanged file: all output lists remain empty"); + } + + free_out(out); + free(disk_e); + free(db_e); +} + +/* ------------------------------------------------------------------ */ +/* Test 7: mixed scenario */ +/* ------------------------------------------------------------------ */ + +static void test_merge_mixed(void) { + psync_list disklist, dblist; + psync_list out[PLOCALSCAN_SCAN_LIST_CNT]; + size_t added; + + psync_list_init(&disklist); + psync_list_init(&dblist); + init_out(out); + + /* + * disk: "aaa" (new file), "bbb" (modified), "ccc" (unchanged folder) + * db: "bbb" (old version), "ccc" (folder, same), "ddd" (deleted file) + * + * Expected: + * NEWFILES: "aaa" + * MODFILES: "bbb" + * DELFILES: "ddd" + * everything else empty + */ + list_append(&disklist, make_entry("aaa", 0, 1, 100, 1000, 1)); /* new */ + list_append(&disklist, make_entry("bbb", 0, 2, 200, 2000, 1)); /* modified */ + list_append(&disklist, make_entry("ccc", 1, 3, 0, 0, 1)); /* unchanged dir */ + + list_append(&dblist, make_entry("bbb", 0, 2, 200, 1111, 1)); /* old mtime */ + list_append(&dblist, make_entry("ccc", 1, 3, 0, 0, 1)); /* same dir */ + list_append(&dblist, make_entry("ddd", 0, 4, 400, 4000, 1)); /* deleted */ + + /* Lists are pre-sorted by name (alphabetical) */ + added = plocalscan_merge_folder_lists(&disklist, &dblist, out, + 10, 20, 1, 1, 1); + + if (added != 3) /* aaa=new, bbb=modified, ddd=deleted */ + FAIL("mixed: added count", "expected 3, got %zu", added); + else + PASS("mixed: 3 elements classified"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFILES]) != 1 || + strcmp(list_nth_name(&out[PLOCALSCAN_SCAN_LIST_NEWFILES], 0), "aaa") != 0) + FAIL("mixed: NEWFILES has aaa", + "count=%d name=%s", + list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFILES]), + list_nth_name(&out[PLOCALSCAN_SCAN_LIST_NEWFILES], 0)); + else + PASS("mixed: NEWFILES contains aaa"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_MODFILES]) != 1 || + strcmp(list_nth_name(&out[PLOCALSCAN_SCAN_LIST_MODFILES], 0), "bbb") != 0) + FAIL("mixed: MODFILES has bbb", + "count=%d name=%s", + list_count(&out[PLOCALSCAN_SCAN_LIST_MODFILES]), + list_nth_name(&out[PLOCALSCAN_SCAN_LIST_MODFILES], 0)); + else + PASS("mixed: MODFILES contains bbb"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES]) != 1 || + strcmp(list_nth_name(&out[PLOCALSCAN_SCAN_LIST_DELFILES], 0), "ddd") != 0) + FAIL("mixed: DELFILES has ddd", + "count=%d name=%s", + list_count(&out[PLOCALSCAN_SCAN_LIST_DELFILES]), + list_nth_name(&out[PLOCALSCAN_SCAN_LIST_DELFILES], 0)); + else + PASS("mixed: DELFILES contains ddd"); + + if (list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFOLDERS]) != 0 || + list_count(&out[PLOCALSCAN_SCAN_LIST_DELFOLDERS]) != 0) + FAIL("mixed: no spurious folder entries", + "NEWFOLDERS=%d DELFOLDERS=%d", + list_count(&out[PLOCALSCAN_SCAN_LIST_NEWFOLDERS]), + list_count(&out[PLOCALSCAN_SCAN_LIST_DELFOLDERS])); + else + PASS("mixed: NEWFOLDERS and DELFOLDERS empty (ccc unchanged)"); + + free_out(out); + psync_list_for_each_element_call(&disklist, sync_folderlist, list, free); + psync_list_for_each_element_call(&dblist, sync_folderlist, list, free); +} + +/* ------------------------------------------------------------------ */ +int main(void) { + test_comparators(); + test_merge_new_entries(); + test_merge_new_folder(); + test_merge_deleted_entries(); + test_merge_modified_file(); + test_merge_unchanged_file(); + test_merge_mixed(); + + printf("\n%d passed, %d failed\n", passes, failures); + return failures ? 1 : 0; +} diff --git a/tests/unit-tests/test_ppagecache.c b/tests/unit-tests/test_ppagecache.c index 09f40b6..c41032d 100644 --- a/tests/unit-tests/test_ppagecache.c +++ b/tests/unit-tests/test_ppagecache.c @@ -143,11 +143,131 @@ static void test_download_url_override(void) { PASS("ppagecache_get_download_urls: weak override returns canned URLs"); } +/* ------------------------------------------------------------------ */ +/* Test 4: ppagecache_range_first_page_id */ +/* ------------------------------------------------------------------ */ +static void test_range_first_page_id(void) { + /* Offset exactly at page boundary → page N */ + if (ppagecache_range_first_page_id(0) != 0) + FAIL("first_page_id(0)", "expected 0, got %lu", + (unsigned long)ppagecache_range_first_page_id(0)); + else + PASS("first_page_id: offset 0 → page 0"); + + if (ppagecache_range_first_page_id(PPAGECACHE_PAGE_SIZE) != 1) + FAIL("first_page_id(PAGE_SIZE)", "expected 1, got %lu", + (unsigned long)ppagecache_range_first_page_id(PPAGECACHE_PAGE_SIZE)); + else + PASS("first_page_id: offset PAGE_SIZE → page 1"); + + if (ppagecache_range_first_page_id(3 * PPAGECACHE_PAGE_SIZE) != 3) + FAIL("first_page_id(3*PAGE_SIZE)", "expected 3, got %lu", + (unsigned long)ppagecache_range_first_page_id(3 * PPAGECACHE_PAGE_SIZE)); + else + PASS("first_page_id: offset 3*PAGE_SIZE → page 3"); + + /* Interior byte of a page → same page ID */ + if (ppagecache_range_first_page_id(PPAGECACHE_PAGE_SIZE + 1) != 1) + FAIL("first_page_id(PAGE_SIZE+1)", "expected 1, got %lu", + (unsigned long)ppagecache_range_first_page_id(PPAGECACHE_PAGE_SIZE + 1)); + else + PASS("first_page_id: PAGE_SIZE+1 → page 1 (interior byte)"); + + if (ppagecache_range_first_page_id(2 * PPAGECACHE_PAGE_SIZE - 1) != 1) + FAIL("first_page_id(2*PAGE_SIZE-1)", "expected 1, got %lu", + (unsigned long)ppagecache_range_first_page_id(2 * PPAGECACHE_PAGE_SIZE - 1)); + else + PASS("first_page_id: 2*PAGE_SIZE-1 → page 1 (last byte of page)"); +} + +/* ------------------------------------------------------------------ */ +/* Test 5: ppagecache_range_page_count */ +/* ------------------------------------------------------------------ */ +static void test_range_page_count(void) { + if (ppagecache_range_page_count(0) != 0) + FAIL("page_count(0)", "expected 0, got %lu", + (unsigned long)ppagecache_range_page_count(0)); + else + PASS("page_count: length 0 → 0 pages"); + + if (ppagecache_range_page_count(PPAGECACHE_PAGE_SIZE) != 1) + FAIL("page_count(PAGE_SIZE)", "expected 1, got %lu", + (unsigned long)ppagecache_range_page_count(PPAGECACHE_PAGE_SIZE)); + else + PASS("page_count: length PAGE_SIZE → 1 page"); + + if (ppagecache_range_page_count(4 * PPAGECACHE_PAGE_SIZE) != 4) + FAIL("page_count(4*PAGE_SIZE)", "expected 4, got %lu", + (unsigned long)ppagecache_range_page_count(4 * PPAGECACHE_PAGE_SIZE)); + else + PASS("page_count: length 4*PAGE_SIZE → 4 pages"); + + /* Partial page rounds down */ + if (ppagecache_range_page_count(PPAGECACHE_PAGE_SIZE + 1) != 1) + FAIL("page_count(PAGE_SIZE+1)", "expected 1, got %lu", + (unsigned long)ppagecache_range_page_count(PPAGECACHE_PAGE_SIZE + 1)); + else + PASS("page_count: PAGE_SIZE+1 → 1 page (partial trailing page ignored)"); +} + +/* ------------------------------------------------------------------ */ +/* Test 6: ppagecache_http_status_should_retry */ +/* ------------------------------------------------------------------ */ +static void test_http_status_should_retry(void) { + /* 0 → success */ + if (ppagecache_http_status_should_retry(0) != 0) + FAIL("http_retry(0)", "expected 0 (success), got %d", + ppagecache_http_status_should_retry(0)); + else + PASS("http_retry: status 0 → success (0)"); + + /* Retryable statuses: 410, 404, -1 */ + if (ppagecache_http_status_should_retry(410) != 1) + FAIL("http_retry(410)", "expected 1 (retry), got %d", + ppagecache_http_status_should_retry(410)); + else + PASS("http_retry: 410 Gone → retry (1)"); + + if (ppagecache_http_status_should_retry(404) != 1) + FAIL("http_retry(404)", "expected 1 (retry), got %d", + ppagecache_http_status_should_retry(404)); + else + PASS("http_retry: 404 Not Found → retry (1)"); + + if (ppagecache_http_status_should_retry(-1) != 1) + FAIL("http_retry(-1)", "expected 1 (retry), got %d", + ppagecache_http_status_should_retry(-1)); + else + PASS("http_retry: -1 connection lost → retry (1)"); + + /* Hard errors: other non-zero statuses */ + if (ppagecache_http_status_should_retry(500) != -1) + FAIL("http_retry(500)", "expected -1 (hard error), got %d", + ppagecache_http_status_should_retry(500)); + else + PASS("http_retry: 500 Internal Server Error → hard error (-1)"); + + if (ppagecache_http_status_should_retry(403) != -1) + FAIL("http_retry(403)", "expected -1 (hard error), got %d", + ppagecache_http_status_should_retry(403)); + else + PASS("http_retry: 403 Forbidden → hard error (-1)"); + + if (ppagecache_http_status_should_retry(200) != -1) + FAIL("http_retry(200)", "expected -1 (hard error), got %d", + ppagecache_http_status_should_retry(200)); + else + PASS("http_retry: 200 unexpected non-zero → hard error (-1)"); +} + /* ------------------------------------------------------------------ */ int main(void) { test_priority_tiers(); test_verify_crc(); test_download_url_override(); + test_range_first_page_id(); + test_range_page_count(); + test_http_status_should_retry(); printf("\n%d passed, %d failed\n", passes, failures); return failures ? 1 : 0;