* Fix bad-free heap corruption in ppathstatus and psyncer (#391)
Both ppathstatus.c and psyncer.c used bare free() via
ptree_for_each_element_call_safe to bulk-free tree nodes that were
allocated with pmem_malloc. pmem_malloc prepends a pmem_header_t to
every allocation, so the returned pointer is an interior pointer to the
underlying glibc chunk. Passing it to free() makes glibc read a garbage
size field from the pmem header and abort with "free(): invalid size".
This is the same class of bug fixed earlier in pintervaltree.c and
pfscrypto.c. The crash manifests reliably with larger files because
more sync-queue and path-status churn occurs, increasing the likelihood
that one of these bulk-free paths is hit while a non-empty tree exists.
Fix: add free_folder_tasks_node() (ppathstatus.c) and
free_synced_down_folder() (psyncer.c) helpers that call pmem_free, and
use them as the ptree_for_each_element_call_safe callback in all three
affected call sites.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix bad-free in pfs.c and ppagecache.c (pcl-26j)
Two more call sites passed bare free() as a callback to tree/list
traversal macros, but the nodes were allocated via pmem_malloc which
prepends a pmem_header_t. Freeing the data pointer directly skips the
header and corrupts the heap.
- pfs.c: ptree_for_each_element_call_safe on sectorsinlog used free()
on psync_sector_inlog_t; replaced with free_sector_inlog_node() that
calls pmem_free(PMEM_SUBSYS_OTHER, e).
- ppagecache.c: psync_list_for_each_element_call on request->ranges
used free() on psync_request_range_t; replaced with
free_request_range() that calls pmem_free(PMEM_SUBSYS_CACHE, range).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add unit and smoke tests for pcl-26j bad-free (ppagecache, pfs, psyncer)
Unit test (test_pcl26j_free): exercises all four fixed call sites —
psync_request_range_t, synced_down_folder, folder_tasks_t, and
psync_sector_inlog_t — using --wrap=malloc/free to verify that
pmem_free() passes the header pointer to free(), not the data pointer.
Includes a harness self-check that confirms bare free(data_ptr) is
detected. Would have failed against pre-fix code.
Smoke test (smoke-test-large-read.sh): builds pcloudcc with ASAN,
starts the daemon, streams a large file (>=50 MB) from the FUSE mount
to /dev/null to force multi-range psync_request_range_t allocation and
teardown via psync_pagecache_free_request, then scans the ASAN log for
any bad-free reports.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add unit test for psync_task_free refcount fix (#377)
Adds tests/unit-tests/test_ptask_free.c to verify all code paths of the
psync_task_free fix from #377: single-owner free, last-ref destroy,
non-last-ref decrement, READY task signaling, and lock-before-refcnt
ordering. All 5 tests pass. Also adds compiled binary to .gitignore.
* Refactor test_ptask_free to link production code via --wrap
Extract psync_task_free + psync_task_destroy (and their static helpers
psync_task_dec_refcnt, psync_task_entry) from ptask.c into a new
separately-compilable unit pclsync/ptask_free.c. Add
pclsync/ptask_free_internal.h to expose the internal struct layout
(struct psync_task_manager_t_ / struct psync_task_t_) for test use
without pulling in ptask.c's heavyweight transitive dependencies.
Rewrite tests/unit-tests/test_ptask_free.c to:
- Include ptask_free_internal.h instead of duplicating structs inline
- Call the real psync_task_free() rather than a local replica
- Intercept pthread_mutex_lock/unlock and pmem_free via --wrap linker
flags to observe lock discipline and detect destroy invocations
Update the Makefile test_ptask_free target to link pclsync/ptask_free.c
and pass the required --wrap flags. Production build unchanged: ptask_free.o
is picked up automatically by the existing wildcard COBJ rule.
* Implement Tasks #3, #4, #5: tree tests, pfstasks tree layer, DB harness
Task #3 — Unit tests for ptree and pintervaltree
tests/unit-tests/test_ptree.c: 8 tests covering single-node insert,
in-order traversal after arbitrary and reverse inserts, BST lookup,
leaf/root/all-node deletion, and ptree_for_each visitation.
tests/unit-tests/test_pintervaltree.c: 18 tests covering single add,
non-overlapping, overlapping/adjacent/contained/spanning merges,
chain merge, remove middle split, remove exact/left/right/spanning,
cut_end, first_interval_containing_or_after, and free(NULL).
Task #4 — Extract pfstasks tree layer
pclsync/pfstasks_tree.h + pclsync/pfstasks_tree.c: pure tree layer
(zero psql calls) extracted from pfstasks.c — pfs_task_search_tree,
pfs_task_walk_tree (static helpers), pfs_task_insert_into_tree,
pfs_task_find_mkdir/rmdir/creat/unlink,
pfs_task_find_mkdir_by_folderid, pfs_task_find_creat_by_fileid.
pclsync/pfstasks.c: #includes pfstasks_tree.h; all moved functions
removed; all callers unchanged.
tests/unit-tests/test_pfstasks_tree.c: 13 tests using direct tree
construction (no DB) to verify find-by-name, taskid discrimination,
find-by-numeric-id, and empty-folder edge cases.
Task #5 — psql in-memory harness + pfstasks DB tests
tests/helpers/psql_test_helpers.h + .c: lightweight harness that
opens :memory: via sqlite3_open, enables PRAGMA foreign_keys=ON, and
applies the full PSYNC_DATABASE_STRUCTURE schema. Exposes
psql_test_db(), psql_test_exec(), psql_test_insert_fstask(),
psql_test_count_fstask/fstaskdepend(). No dependency on psql.c.
tests/unit-tests/test_pfstasks_db.c: 10 tests verifying schema
creation, fstask insertion/query, fstaskdepend insertion, CASCADE
DELETE propagation, FK enforcement, rmdir-blocking SQL pattern,
creat-after-unlink sequencing, and open/close idempotence.
All 11 new tests pass; production build clean.
* Fix P1 review findings in pfstasks_db test and helpers
1. Check psql_test_exec() return values in test_cascade_delete() and
test_creat_after_unlink() consistently with test_fstaskdepend_insert().
2. Remove dead dep_cnt variable and (void)dep_cnt suppressor from
test_creat_after_unlink().
3. Change SQLITE_STATIC → SQLITE_TRANSIENT for text1 binding in
psql_test_insert_fstask() to avoid dangling-pointer footgun on
future reuse.
* Fix ASAN/LSAN failures: ppath_home stack-use-after-scope + intentional leak
pclsync/ppath.c: Move buff[4096] to function scope in ppath_home() so
the pointer stored in dir via result->pw_dir remains live through the
putil_strdup(dir) call. Previously buff went out of scope at the if-block
close, causing a stack-use-after-scope ASAN report on every call that fell
through the getpwuid_r path.
tests/unit-tests/test_ptools_errptr.c: run_unfixed() intentionally leaks
errPtr to demonstrate the pre-fix bug. Wrap the allocation with
LSAN_DISABLE() / LSAN_ENABLE() so LSAN does not abort the process at exit
before stdio flushes, which was causing a non-zero exit code. The guard
uses nested #ifdef/__has_feature to remain compatible with both GCC
(__SANITIZE_ADDRESS__) and Clang (__has_feature(address_sanitizer))
without triggering "missing binary operator" errors on GCC.
* Implement Tasks #11 and #12: plocks stress test + pfsupload send tests
Task #11 — plocks.c stress test (test_plocks.c)
7 tests: basic rdlock/wrlock round-trip, recursive TLS counting (same
thread acquires rdlock N times; unlock only releases on final decrement),
upgrade under contention (N readers + towrlock; barrier-synchronized),
writer starvation prevention (sustained reader load; writer acquires
within 500ms), N-reader + M-writer counter-integrity stress test (ASAN),
and try-variant contention (trywrlock fails when another thread holds
rdlock). TSAN note documented: custom lock internals require ASAN-only
when ThreadSanitizer annotations are absent.
Task #12 — pfsupload send-function tests (pfsupload_send.c/h + test_pfsupload.c)
Extract psync_send_task_mkdir and psync_send_task_rmdir from pfsupload.c
into pclsync/pfsupload_send.c as non-static pfsupload_send_mkdir/rmdir.
Expose fsupload_task_t struct via pclsync/pfsupload_send.h. Add
__attribute__((weak)) get_urls() as an injectable URL seam for large-
upload paths. pfsupload.c updated to include pfsupload_send.h and use
the renamed functions in its dispatch table; pfsupload_send.o is
automatically picked up by the production wildcard build.
5 tests: mkdir (non-encrypted) command + folderid param, mkdir
(encrypted) key param present, rmdir command + sfolderid, API error path
(papi_send failure → -1), get_urls() weak override. Uses
--wrap=papi_send to intercept API calls and socketpair() to provide a
valid psock_t without real network I/O.
* P2 cleanup: comments, make_fake_api stack alloc, find_str_param fix
1. test_plocks.c: add comment to test_upgrade_under_contention clarifying
it verifies towrlock completion and holding_wrlock; notes that concurrent
exclusivity is covered by test_stress().
2. test_pfsupload.c / make_fake_api: replace static-local psock_t with
caller-supplied stack allocation (out parameter) to eliminate the
multiple-calls-per-test footgun.
3. test_pfsupload.c / find_str_param: replace ternary
`paramnamelen == strlen ? paramname : ""` with explicit length check +
strncmp, matching the cleaner pattern used in find_num_param.
4. Makefile: add comment next to -Wl,--wrap=papi_send noting it redirects
papi_send to __wrap_papi_send and is GNU ld only (not macOS Apple ld).
* Implement Tasks #19 and #20: ppagecache + pfs helper extraction
Task #19 — ppagecache.c decomposition (ppagecache_helpers.c/h)
ppagecache_compute_page_priority(usecnt): pure function returning the
LRU eviction tier (0–4) that matches the five pagecache_entry_cmp_*
sort comparators in ppagecache.c (thresholds 2/4/8/16).
ppagecache_verify_crc(data, size, stored_crc): wraps pcrc32c_compute
and compares; returns 0 on match, -1 on mismatch.
ppagecache_get_download_urls(fileid, hash, nout): __attribute__((weak))
URL-injection seam; default returns NULL (falls through to real API).
ppagecache.c updated to include ppagecache_helpers.h.
test_ppagecache.c: 10 tests covering tier boundary conditions (0/1/2/3/4
including UINT32_MAX), CRC match, single-bit flip, wrong stored CRC,
zero-length buffer, and weak URL override.
Task #20 — pfs.c helper extraction (pfs_helpers.c/h)
pfs_row_to_folder_stat(row, stbuf): converts psql folder row → struct
stat; uses pfs_task_get_folder_tasks_rdlocked for in-memory mtime.
pfs_row_to_file_stat(row, stbuf, flags): converts psql file row → stat;
encrypted path calls pfs_crpt_plain_size.
pfs_mkdir_to_folder_stat(mk, stbuf): converts in-memory mkdir task →
stat (no SQL).
pfs_apply_task_overlay(stbuf, folder, name, flags): applies pending
mkdir/rmdir/unlink/creat-new overlays from the in-memory task queue;
returns 1/2/-1/0.
pfs_stat_uid/gid: exported globals; pfs.c syncs them from myuid/mygid.
pfs.c updated to #include pfs_helpers.h and sync the uid/gid globals.
pfsfolder.c: pfs_fldr_resolve_path decorated __attribute__((weak)) so
tests can inject fake path resolution without a FUSE mount or psql.
test_pfs_helpers.c: 7 tests covering folder/file stat field correctness,
overlay NULL/mkdir/rmdir/no-match cases, and weak path override; uses
--wrap for pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, and
ptimer_time to stay SQL/crypto/timer-free.
Production build clean; make check exits 0.
---------
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add unit test for psync_task_free refcount fix (#377)
Adds tests/unit-tests/test_ptask_free.c to verify all code paths of the
psync_task_free fix from #377: single-owner free, last-ref destroy,
non-last-ref decrement, READY task signaling, and lock-before-refcnt
ordering. All 5 tests pass. Also adds compiled binary to .gitignore.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Refactor test_ptask_free to link production code via --wrap
Extract psync_task_free + psync_task_destroy (and their static helpers
psync_task_dec_refcnt, psync_task_entry) from ptask.c into a new
separately-compilable unit pclsync/ptask_free.c. Add
pclsync/ptask_free_internal.h to expose the internal struct layout
(struct psync_task_manager_t_ / struct psync_task_t_) for test use
without pulling in ptask.c's heavyweight transitive dependencies.
Rewrite tests/unit-tests/test_ptask_free.c to:
- Include ptask_free_internal.h instead of duplicating structs inline
- Call the real psync_task_free() rather than a local replica
- Intercept pthread_mutex_lock/unlock and pmem_free via --wrap linker
flags to observe lock discipline and detect destroy invocations
Update the Makefile test_ptask_free target to link pclsync/ptask_free.c
and pass the required --wrap flags. Production build unchanged: ptask_free.o
is picked up automatically by the existing wildcard COBJ rule.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Implement Tasks #3, #4, #5: tree tests, pfstasks tree layer, DB harness
Task #3 — Unit tests for ptree and pintervaltree
tests/unit-tests/test_ptree.c: 8 tests covering single-node insert,
in-order traversal after arbitrary and reverse inserts, BST lookup,
leaf/root/all-node deletion, and ptree_for_each visitation.
tests/unit-tests/test_pintervaltree.c: 18 tests covering single add,
non-overlapping, overlapping/adjacent/contained/spanning merges,
chain merge, remove middle split, remove exact/left/right/spanning,
cut_end, first_interval_containing_or_after, and free(NULL).
Task #4 — Extract pfstasks tree layer
pclsync/pfstasks_tree.h + pclsync/pfstasks_tree.c: pure tree layer
(zero psql calls) extracted from pfstasks.c — pfs_task_search_tree,
pfs_task_walk_tree (static helpers), pfs_task_insert_into_tree,
pfs_task_find_mkdir/rmdir/creat/unlink,
pfs_task_find_mkdir_by_folderid, pfs_task_find_creat_by_fileid.
pclsync/pfstasks.c: #includes pfstasks_tree.h; all moved functions
removed; all callers unchanged.
tests/unit-tests/test_pfstasks_tree.c: 13 tests using direct tree
construction (no DB) to verify find-by-name, taskid discrimination,
find-by-numeric-id, and empty-folder edge cases.
Task #5 — psql in-memory harness + pfstasks DB tests
tests/helpers/psql_test_helpers.h + .c: lightweight harness that
opens :memory: via sqlite3_open, enables PRAGMA foreign_keys=ON, and
applies the full PSYNC_DATABASE_STRUCTURE schema. Exposes
psql_test_db(), psql_test_exec(), psql_test_insert_fstask(),
psql_test_count_fstask/fstaskdepend(). No dependency on psql.c.
tests/unit-tests/test_pfstasks_db.c: 10 tests verifying schema
creation, fstask insertion/query, fstaskdepend insertion, CASCADE
DELETE propagation, FK enforcement, rmdir-blocking SQL pattern,
creat-after-unlink sequencing, and open/close idempotence.
All 11 new tests pass; production build clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix P1 review findings in pfstasks_db test and helpers
1. Check psql_test_exec() return values in test_cascade_delete() and
test_creat_after_unlink() consistently with test_fstaskdepend_insert().
2. Remove dead dep_cnt variable and (void)dep_cnt suppressor from
test_creat_after_unlink().
3. Change SQLITE_STATIC → SQLITE_TRANSIENT for text1 binding in
psql_test_insert_fstask() to avoid dangling-pointer footgun on
future reuse.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix ASAN/LSAN failures: ppath_home stack-use-after-scope + intentional leak
pclsync/ppath.c: Move buff[4096] to function scope in ppath_home() so
the pointer stored in dir via result->pw_dir remains live through the
putil_strdup(dir) call. Previously buff went out of scope at the if-block
close, causing a stack-use-after-scope ASAN report on every call that fell
through the getpwuid_r path.
tests/unit-tests/test_ptools_errptr.c: run_unfixed() intentionally leaks
errPtr to demonstrate the pre-fix bug. Wrap the allocation with
LSAN_DISABLE() / LSAN_ENABLE() so LSAN does not abort the process at exit
before stdio flushes, which was causing a non-zero exit code. The guard
uses nested #ifdef/__has_feature to remain compatible with both GCC
(__SANITIZE_ADDRESS__) and Clang (__has_feature(address_sanitizer))
without triggering "missing binary operator" errors on GCC.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Implement Tasks #11 and #12: plocks stress test + pfsupload send tests
Task #11 — plocks.c stress test (test_plocks.c)
7 tests: basic rdlock/wrlock round-trip, recursive TLS counting (same
thread acquires rdlock N times; unlock only releases on final decrement),
upgrade under contention (N readers + towrlock; barrier-synchronized),
writer starvation prevention (sustained reader load; writer acquires
within 500ms), N-reader + M-writer counter-integrity stress test (ASAN),
and try-variant contention (trywrlock fails when another thread holds
rdlock). TSAN note documented: custom lock internals require ASAN-only
when ThreadSanitizer annotations are absent.
Task #12 — pfsupload send-function tests (pfsupload_send.c/h + test_pfsupload.c)
Extract psync_send_task_mkdir and psync_send_task_rmdir from pfsupload.c
into pclsync/pfsupload_send.c as non-static pfsupload_send_mkdir/rmdir.
Expose fsupload_task_t struct via pclsync/pfsupload_send.h. Add
__attribute__((weak)) get_urls() as an injectable URL seam for large-
upload paths. pfsupload.c updated to include pfsupload_send.h and use
the renamed functions in its dispatch table; pfsupload_send.o is
automatically picked up by the production wildcard build.
5 tests: mkdir (non-encrypted) command + folderid param, mkdir
(encrypted) key param present, rmdir command + sfolderid, API error path
(papi_send failure → -1), get_urls() weak override. Uses
--wrap=papi_send to intercept API calls and socketpair() to provide a
valid psock_t without real network I/O.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* P2 cleanup: comments, make_fake_api stack alloc, find_str_param fix
1. test_plocks.c: add comment to test_upgrade_under_contention clarifying
it verifies towrlock completion and holding_wrlock; notes that concurrent
exclusivity is covered by test_stress().
2. test_pfsupload.c / make_fake_api: replace static-local psock_t with
caller-supplied stack allocation (out parameter) to eliminate the
multiple-calls-per-test footgun.
3. test_pfsupload.c / find_str_param: replace ternary
`paramnamelen == strlen ? paramname : ""` with explicit length check +
strncmp, matching the cleaner pattern used in find_num_param.
4. Makefile: add comment next to -Wl,--wrap=papi_send noting it redirects
papi_send to __wrap_papi_send and is GNU ld only (not macOS Apple ld).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Implement Tasks #19 and #20: ppagecache + pfs helper extraction
Task #19 — ppagecache.c decomposition (ppagecache_helpers.c/h)
ppagecache_compute_page_priority(usecnt): pure function returning the
LRU eviction tier (0–4) that matches the five pagecache_entry_cmp_*
sort comparators in ppagecache.c (thresholds 2/4/8/16).
ppagecache_verify_crc(data, size, stored_crc): wraps pcrc32c_compute
and compares; returns 0 on match, -1 on mismatch.
ppagecache_get_download_urls(fileid, hash, nout): __attribute__((weak))
URL-injection seam; default returns NULL (falls through to real API).
ppagecache.c updated to include ppagecache_helpers.h.
test_ppagecache.c: 10 tests covering tier boundary conditions (0/1/2/3/4
including UINT32_MAX), CRC match, single-bit flip, wrong stored CRC,
zero-length buffer, and weak URL override.
Task #20 — pfs.c helper extraction (pfs_helpers.c/h)
pfs_row_to_folder_stat(row, stbuf): converts psql folder row → struct
stat; uses pfs_task_get_folder_tasks_rdlocked for in-memory mtime.
pfs_row_to_file_stat(row, stbuf, flags): converts psql file row → stat;
encrypted path calls pfs_crpt_plain_size.
pfs_mkdir_to_folder_stat(mk, stbuf): converts in-memory mkdir task →
stat (no SQL).
pfs_apply_task_overlay(stbuf, folder, name, flags): applies pending
mkdir/rmdir/unlink/creat-new overlays from the in-memory task queue;
returns 1/2/-1/0.
pfs_stat_uid/gid: exported globals; pfs.c syncs them from myuid/mygid.
pfs.c updated to #include pfs_helpers.h and sync the uid/gid globals.
pfsfolder.c: pfs_fldr_resolve_path decorated __attribute__((weak)) so
tests can inject fake path resolution without a FUSE mount or psql.
test_pfs_helpers.c: 7 tests covering folder/file stat field correctness,
overlay NULL/mkdir/rmdir/no-match cases, and weak path override; uses
--wrap for pfs_task_get_folder_tasks_rdlocked, pfs_crpt_plain_size, and
ptimer_time to stay SQL/crypto/timer-free.
Production build clean; make check exits 0.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add unit test for psync_task_free refcount fix (#377)
Adds tests/unit-tests/test_ptask_free.c to verify all code paths of the
psync_task_free fix from #377: single-owner free, last-ref destroy,
non-last-ref decrement, READY task signaling, and lock-before-refcnt
ordering. All 5 tests pass. Also adds compiled binary to .gitignore.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Refactor test_ptask_free to link production code via --wrap
Extract psync_task_free + psync_task_destroy (and their static helpers
psync_task_dec_refcnt, psync_task_entry) from ptask.c into a new
separately-compilable unit pclsync/ptask_free.c. Add
pclsync/ptask_free_internal.h to expose the internal struct layout
(struct psync_task_manager_t_ / struct psync_task_t_) for test use
without pulling in ptask.c's heavyweight transitive dependencies.
Rewrite tests/unit-tests/test_ptask_free.c to:
- Include ptask_free_internal.h instead of duplicating structs inline
- Call the real psync_task_free() rather than a local replica
- Intercept pthread_mutex_lock/unlock and pmem_free via --wrap linker
flags to observe lock discipline and detect destroy invocations
Update the Makefile test_ptask_free target to link pclsync/ptask_free.c
and pass the required --wrap flags. Production build unchanged: ptask_free.o
is picked up automatically by the existing wildcard COBJ rule.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Implement Tasks #3, #4, #5: tree tests, pfstasks tree layer, DB harness
Task #3 — Unit tests for ptree and pintervaltree
tests/unit-tests/test_ptree.c: 8 tests covering single-node insert,
in-order traversal after arbitrary and reverse inserts, BST lookup,
leaf/root/all-node deletion, and ptree_for_each visitation.
tests/unit-tests/test_pintervaltree.c: 18 tests covering single add,
non-overlapping, overlapping/adjacent/contained/spanning merges,
chain merge, remove middle split, remove exact/left/right/spanning,
cut_end, first_interval_containing_or_after, and free(NULL).
Task #4 — Extract pfstasks tree layer
pclsync/pfstasks_tree.h + pclsync/pfstasks_tree.c: pure tree layer
(zero psql calls) extracted from pfstasks.c — pfs_task_search_tree,
pfs_task_walk_tree (static helpers), pfs_task_insert_into_tree,
pfs_task_find_mkdir/rmdir/creat/unlink,
pfs_task_find_mkdir_by_folderid, pfs_task_find_creat_by_fileid.
pclsync/pfstasks.c: #includes pfstasks_tree.h; all moved functions
removed; all callers unchanged.
tests/unit-tests/test_pfstasks_tree.c: 13 tests using direct tree
construction (no DB) to verify find-by-name, taskid discrimination,
find-by-numeric-id, and empty-folder edge cases.
Task #5 — psql in-memory harness + pfstasks DB tests
tests/helpers/psql_test_helpers.h + .c: lightweight harness that
opens :memory: via sqlite3_open, enables PRAGMA foreign_keys=ON, and
applies the full PSYNC_DATABASE_STRUCTURE schema. Exposes
psql_test_db(), psql_test_exec(), psql_test_insert_fstask(),
psql_test_count_fstask/fstaskdepend(). No dependency on psql.c.
tests/unit-tests/test_pfstasks_db.c: 10 tests verifying schema
creation, fstask insertion/query, fstaskdepend insertion, CASCADE
DELETE propagation, FK enforcement, rmdir-blocking SQL pattern,
creat-after-unlink sequencing, and open/close idempotence.
All 11 new tests pass; production build clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix P1 review findings in pfstasks_db test and helpers
1. Check psql_test_exec() return values in test_cascade_delete() and
test_creat_after_unlink() consistently with test_fstaskdepend_insert().
2. Remove dead dep_cnt variable and (void)dep_cnt suppressor from
test_creat_after_unlink().
3. Change SQLITE_STATIC → SQLITE_TRANSIENT for text1 binding in
psql_test_insert_fstask() to avoid dangling-pointer footgun on
future reuse.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix ASAN/LSAN failures: ppath_home stack-use-after-scope + intentional leak
pclsync/ppath.c: Move buff[4096] to function scope in ppath_home() so
the pointer stored in dir via result->pw_dir remains live through the
putil_strdup(dir) call. Previously buff went out of scope at the if-block
close, causing a stack-use-after-scope ASAN report on every call that fell
through the getpwuid_r path.
tests/unit-tests/test_ptools_errptr.c: run_unfixed() intentionally leaks
errPtr to demonstrate the pre-fix bug. Wrap the allocation with
LSAN_DISABLE() / LSAN_ENABLE() so LSAN does not abort the process at exit
before stdio flushes, which was causing a non-zero exit code. The guard
uses nested #ifdef/__has_feature to remain compatible with both GCC
(__SANITIZE_ADDRESS__) and Clang (__has_feature(address_sanitizer))
without triggering "missing binary operator" errors on GCC.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Implement Tasks #11 and #12: plocks stress test + pfsupload send tests
Task #11 — plocks.c stress test (test_plocks.c)
7 tests: basic rdlock/wrlock round-trip, recursive TLS counting (same
thread acquires rdlock N times; unlock only releases on final decrement),
upgrade under contention (N readers + towrlock; barrier-synchronized),
writer starvation prevention (sustained reader load; writer acquires
within 500ms), N-reader + M-writer counter-integrity stress test (ASAN),
and try-variant contention (trywrlock fails when another thread holds
rdlock). TSAN note documented: custom lock internals require ASAN-only
when ThreadSanitizer annotations are absent.
Task #12 — pfsupload send-function tests (pfsupload_send.c/h + test_pfsupload.c)
Extract psync_send_task_mkdir and psync_send_task_rmdir from pfsupload.c
into pclsync/pfsupload_send.c as non-static pfsupload_send_mkdir/rmdir.
Expose fsupload_task_t struct via pclsync/pfsupload_send.h. Add
__attribute__((weak)) get_urls() as an injectable URL seam for large-
upload paths. pfsupload.c updated to include pfsupload_send.h and use
the renamed functions in its dispatch table; pfsupload_send.o is
automatically picked up by the production wildcard build.
5 tests: mkdir (non-encrypted) command + folderid param, mkdir
(encrypted) key param present, rmdir command + sfolderid, API error path
(papi_send failure → -1), get_urls() weak override. Uses
--wrap=papi_send to intercept API calls and socketpair() to provide a
valid psock_t without real network I/O.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* P2 cleanup: comments, make_fake_api stack alloc, find_str_param fix
1. test_plocks.c: add comment to test_upgrade_under_contention clarifying
it verifies towrlock completion and holding_wrlock; notes that concurrent
exclusivity is covered by test_stress().
2. test_pfsupload.c / make_fake_api: replace static-local psock_t with
caller-supplied stack allocation (out parameter) to eliminate the
multiple-calls-per-test footgun.
3. test_pfsupload.c / find_str_param: replace ternary
`paramnamelen == strlen ? paramname : ""` with explicit length check +
strncmp, matching the cleaner pattern used in find_num_param.
4. Makefile: add comment next to -Wl,--wrap=papi_send noting it redirects
papi_send to __wrap_papi_send and is GNU ld only (not macOS Apple ld).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add unit test for psync_task_free refcount fix (#377)
Adds tests/unit-tests/test_ptask_free.c to verify all code paths of the
psync_task_free fix from #377: single-owner free, last-ref destroy,
non-last-ref decrement, READY task signaling, and lock-before-refcnt
ordering. All 5 tests pass. Also adds compiled binary to .gitignore.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Add GitHub Actions CI workflow for unit tests
Triggers on push/PR to automated-testing branch. Installs cmake and
build-essential, builds all test targets via cmake, and runs ctest
--output-on-failure. Fails workflow on any test failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Replace cmake CI with make tests/check targets
Adds tests and check targets to Makefile — no cmake required.
Each test binary is built with the correct flags (pthread, -lrt,
--wrap linker flags for prun/ptools_errptr). CI workflow installs
only build-essential, runs make tests then make check; exits non-zero
on any failure. All 8 test suites pass locally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix CI: install libfuse3-dev so Makefile parses on Ubuntu
detect_fuse.sh runs at Makefile parse time; without fuse headers the
$(error) fires before any target runs. Adding libfuse3-dev unblocks
make tests (test binaries themselves don't link fuse).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix makefile
* Add automated testing infrastructure
- Update CI workflow to run unit tests and build verification
- Add Makefile targets for test compilation and execution
- Implement unit tests for pdbg_path, prun, and read_response
- Add test stubs for pCloud API mocking
- Add test binaries for pfs_lock_ordering and signal_safety verification
* Add missing dependencies to CI workflow
Install libfuse-dev and libssl-dev required for build
* Add test job to c-cpp.yml workflow
Include unit test execution in C/C++ workflow
* Add missing stubs to test_stubs.c
Complete stub implementations for all required pCloud API functions
* Fix stub signatures to match headers
Correct function signatures for pCloud API stubs
* Fix psql_* stub signatures
Correct all psql function signatures to match headers
* Fix stub implementations and Makefile
Update stub functions and build configuration
* Link real utility files instead of stubbing
Update Makefile to use actual implementation files for utilities
* Complete test framework with all 41 tests passing
- Makefile: Add test rules with real dependencies
- tests/stubs/test_stubs.c: Minimal stubs for external APIs
- tests/stubs/test_stubs_cpp.c: Stubs for C++ test
- pclsync/putil.c: Add null check in putil_strdup
- pclsync/pdbg.c: Add recursion guard in pdbg_printf
* Remove duplicate ci.yml workflow
Consolidate CI configuration into c-cpp.yml
* Remove compiled test binaries from git
- Remove test_pfs_lock_ordering and test_signal_safety binaries
- Add tests/test_* to .gitignore to prevent future commits
---------
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* Migrate to FUSE 3.x API
- Update FUSE_USE_VERSION from 26 to 30
- Replace fuse_mount/fuse_unmount with fuse_session_mount/fuse_session_unmount
- Update fuse_new() to take args first, remove channel parameter
- Add flags parameter to readdir handler (enum fuse_readdir_flags)
- Add flags parameter to rename handler (for renameat2 support)
- Update filler function calls to include FUSE_FILL_DIR_PLUS flag
- Merge ftruncate into truncate handler (FUSE 3 combines them)
- Update fuse_loop_mt to fuse_loop_mt_31 with loop config
- Link against libfuse3 instead of libfuse
- Remove fuse_chan usage (deprecated in FUSE 3)
Addresses #342
* Remove deprecated FUSE options nonempty and hard_remove
- nonempty: Removed in FUSE 3.0, mounting on non-empty directories is now default behavior
- hard_remove: High-level API option not available in FUSE 3, immediate unlink is standard
These options are no longer recognized by FUSE 3 and cause mount errors.
* Fix init handler signature for FUSE 3
FUSE 3 init handler requires struct fuse_config* parameter.
This parameter provides access to high-level API configuration
options that can be modified during initialization.
Without this parameter, the init handler signature is incompatible
with FUSE 3, causing filesystem operations to fail.
* Add FUSE 2/3 backward compatibility
Automatically detects and builds against either FUSE 2 or FUSE 3:
- detect_fuse.sh script checks for fuse3 or fuse pkg-config
- Makefile dynamically sets FUSE_USE_VERSION, CFLAGS, and LDFLAGS
- Conditional compilation (#if FUSE_USE_VERSION >= 30) for API differences:
* init handler signature (fuse_config parameter)
* readdir handler signature (flags parameter)
* rename handler signature (flags parameter)
* truncate handler signature (fuse_file_info parameter)
* filler function calls (flags parameter)
* fuse_loop_mt vs fuse_loop_mt_31
* mount/unmount sequence (fuse_session_mount vs fuse_mount)
* ftruncate operations struct member
Maintains full compatibility with both FUSE 2.x (Debian Bookworm) and
FUSE 3.x (Debian Testing+) without requiring separate code branches.
* Add FORCE_FUSE build option and fix hardcoded version
- Remove hardcoded FUSE_USE_VERSION defines from source files
- Add FORCE_FUSE=2 or FORCE_FUSE=3 Makefile option to override detection
- Fix FUSE 2 channel tracking (psync_fuse_channel variable)
- Properly store and use channel in mount/unmount for FUSE 2
Usage:
make # Auto-detect (prefers FUSE 3)
make FORCE_FUSE=2 # Force FUSE 2 build
make FORCE_FUSE=3 # Force FUSE 3 build
Verified both FUSE 2 and FUSE 3 builds work correctly.
* Fix CI/CD: Add fallback for systems without pkg-config
- detect_fuse.sh now checks for pkg-config availability first
- Falls back to checking for header files in standard locations
- Makefile provides default CFLAGS when pkg-config unavailable
- Fixes build on minimal CI/CD environments
Fallback locations checked:
- /usr/include/fuse3/fuse.h (FUSE 3)
- /usr/local/include/fuse3/fuse.h (FUSE 3)
- /usr/include/fuse/fuse.h (FUSE 2)
- /usr/local/include/fuse/fuse.h (FUSE 2)
* Enhance FUSE detection for non-standard locations
Multi-stage detection strategy:
1. Try pkg-config (fastest, most reliable)
2. Search common header locations:
- /usr/include, /usr/local/include
- /opt/local/include, /opt/include
3. Compiler test as last resort:
- Attempts to preprocess #include <fuse.h>
- Tests both FUSE 3 and FUSE 2 versions
This handles:
- Minimal CI/CD environments (no pkg-config)
- Non-standard install locations (Homebrew, custom builds)
- Distro-specific paths (BSD, macOS, custom Linux)
The compiler test ensures detection works even when headers
are in unusual locations that gcc can find via its search paths.
* Support FUSE 3.12+ API
- Use fuse_loop_mt_312 for FUSE 3.12+, fuse_loop_mt_31 for 3.0-3.11
- Conditionally add fuse_file_info param to getattr/chmod/chown/utimens
- Replace fuse_session_mount/unmount with fuse_mount/unmount
- Tested with FUSE 3.18 on Debian forky
* Fix pcl-1ib: correct FUSE 3.x handler signatures and loop_mt call
- Replace FUSE_MINOR_VERSION >= 12 checks with FUSE_USE_VERSION >= 30
for getattr/chmod/chown/utimens handler signatures; FUSE 3.x always
requires the extra struct fuse_file_info* parameter — the wrong
signature caused type mismatches and "Transport endpoint is not
connected" failures
- Replace non-existent fuse_loop_mt_312 extern with correct conditional:
FUSE_USE_VERSION >= 32 uses fuse_loop_mt(f, &config), else
fuse_loop_mt_31(f, clone_fd)
- Fix inverted return logic in is_fuse3_installed_on_system(): was
returning 0 on success (when fusermount3 found), now correctly
returns non-zero
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Remove unused is_fuse3_installed_on_system() from pfs.c
The function was static and had no remaining call sites after the
-ononempty/-ohard_remove args block was cleaned up, producing a
-Wunused-function warning.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Restore shutdown_requested definition removed during rebase conflict resolution
The variable is declared extern in pfs.h and referenced in both
pfs.c (psync_signal_handler) and control_tools.cpp; the definition
must exist in pfs.c.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix pfs_creat: missing canmodify assignment causes EACCES on write
pfs_creat called pfs_create_file() but never set of->canmodify, leaving
it zero-initialized. pfs_write and pfs_ftruncate both check canmodify
and return -EACCES if false, making all writes to newly created files
fail with Permission denied.
All five open paths in pfs_open set canmodify correctly (added in #326),
but pfs_creat was missed. Fix by applying the same pattern immediately
after psql_unlock(), while fpath is still in scope.
Caught during FUSE 2/3 regression testing (Phase 2 write test).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Extend CI matrix: add FUSE 2, FUSE 3 legacy/current/forky build jobs
Replaces single build job with four jobs covering all tested FUSE configs:
- fuse2: debian:trixie + libfuse-dev (FUSE_USE_VERSION=26)
- fuse3-legacy: ubuntu:20.04 + libfuse3-dev 3.9.x (fuse_loop_mt_31 path)
- fuse3-current: ubuntu:22.04 + libfuse3-dev 3.12.x
- fuse3-forky: debian:forky + libfuse3-dev 3.18.1
Each job verifies correct symbol linkage after build. Runtime mount tests
are outside CI scope (require privileged containers with /dev/fuse).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* CI: replace Ubuntu 20.04/22.04 with ubuntu:24.04 for FUSE3 job
ubuntu:20.04 has old compiler (g++-9) and mbedTLS 2.x.
ubuntu:22.04 ships mbedTLS 2.28.x; incompatible with our mbedTLS 3.x-only code.
ubuntu:24.04 has mbedTLS 3.x and libfuse3 3.16.x, covering the 3.12-3.17 range.
FUSE3 < 3.12 is verified locally but not representable in CI without fighting
old toolchains.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* CI: add archlinux:latest job for rolling-release compatibility
Arch Linux is a rolling distro shipping latest fuse3, mbedtls 3.x, and
gcc — useful as an early-warning canary for future API breakage. Also
directly covers the maintained Arch package.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* CI: add fedora:41 and fedora:latest jobs with mbedtls3-devel
Fedora ships both mbedtls 2.x and 3.x; explicitly install mbedtls3-devel
to match our mbedTLS 3.x-only requirement. fedora:41 pins a known-good
release; fedora:latest tracks current as a rolling canary alongside Arch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* README: add compatibility matrix with CI badge references
Documents the 6-distro build matrix tested in CI, mbedTLS 3.x requirement,
and notes on manual runtime mount testing coverage.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Fix CI package names; clean up README compatibility matrix
- Fedora 41: mbedtls3-devel -> mbedtls3.6-devel (correct versioned name)
- Fedora latest (43): mbedtls3-devel -> mbedtls-devel (now ships 3.6.5)
- Ubuntu 24.04: replaced with debian:trixie FORCE_FUSE=3 (Ubuntu has no
mbedTLS 3.x in standard repos; trixie has both libfuse3-dev and mbedTLS 3.x)
- README: remove CI badge links from compatibility matrix
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
The weak/strong symbol approach requires that psql.c undefs the debug
macros to define function implementations. This means functions in psql.c
call psql_lock()/psql_unlock() as functions, not macros.
In debug builds, these must be strong overrides that call the _do_
variants to properly maintain lockctr/rdlockctr. Without these overrides,
the weak stubs are used which don't update counters, causing assertion
failures in psql_unlock().
Added strong overrides in psql_debug.c for:
- psql_lock() -> psql_do_lock(__FILE__, __LINE__)
- psql_rdlock() -> psql_do_rdlock(__FILE__, __LINE__)
- psql_statement() (calls psql_do_lock directly)
Also made psql_statement, psql_start, and all query/prepare functions
weak in psql.c to allow debug overrides.
Fixes#138
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
- Use pkg-config to dynamically detect mbedtls variants (mbedtls3.x, mbedtls)
- Fallback to /usr/local if pkg-config unavailable
- Tested on Fedora 41 (mbedtls3.6), Arch Linux (mbedtls 3.6.5), Debian Bookworm, and Slackware
- Eliminates need for hardcoded version-specific paths
Co-authored-by: Levi Neely <lkn@darkstar.example.net>
* pcommands.h for command ids
* remove compiler compat junk
* transient and unnecessary includes resolved
* unnecessary goto
* function naming and whatnot
* use readline for better command line UX
* add readline to CI workflow
* Revert "migrate mbedtls to 3.x (#89)"
This reverts commit f87367211d.
* prevent multiple execution of psync_fs_do_sotp and use atexit in sig hnd
* remove debug.[ch] in favor of plibs debug
* eliminate pcompat
* Decompose namespaces and other cleanup
* add missing unistd.h to psys.c (#113)
* remove useless comment
* code cleanup and removal
- remove useless wrapper functions
- remove obfuscating typedefs
- code reorganization and reformatting
- function renames for clarification
* prepare pcompat.h for decomposition
* remove dead function psync_rebuild_icons, reorganize pfsstatic.c headers
* separate pdevice from pcompat
* reorg function prototypes, remove more compat stuff
* renamed mutex for brevity
* remove dead debug code; plibs version is more sophisticated
* remove dead code, stale comments
* moved psync_delete_cached_crypto_keys to psynclib
* forgot to remove the function prototype from pdiff.h
* reorganized functions by accessibility, inlined useless psync_cache_contacts function
* remove unused header
* ppath object decomposed from pcompat
* ppath namespace
* ppath struct renames
* ppath final cleanup
* refactor, remove compatibility code
* format
* prun decomposed from pcompat for thread mgmt
* slim down prun.c
* remove useless comments
* move pagesize to pmemlock, pmemlock namespace
* pmem decomposed from pcompat, pmem namespace
* decompose psys from pcompat
* fix psys function names, remove more compatibility junk
* prand decomposed from pcompat
* psock decomposed from pcompat
* types renamed for ns consistency, brevity
* cleanup psock, remove unused proxy code
* remove unused headers psock
* putil for utility macros and functions
* forgot to move some psock constants
* move time macro to psys
* eliminate pcompat completely; new pfile namespace
* cleanup
- moved thread name definition
- explicit externs in implementation instead of headers for now
- better extern name
* migrate mbedtls to 3.x
* update github c-cpp workflow to use trixie
* update README and container-build
- default image and tag for container-build is now debian:trixie
- README for debian bookworm users
* final readme tweaks
* second attempt to update ci build workflow
* third attempt on ci build workflow
* fix deprecated calls in pssl.c
* list_sync_folders seems to work
* list_sync_folders breaks with sync folders in db; message size problem?
* fixed sync folders list
* fix double free and buffer overflow
* wip: add and remove sync folder; need to fix request->value in instance_thread
* remove comment
* removed pclsync_lib_c.{h,cpp}
* cleanup get_answer_to_request
* update comment
* changed reply struct to type response containing payload and msg
* cleanup
* refactored overlay_client, and poverlay; added docs
* better naming
* sock path in poverlay_protocol.h
* clarify comment
* refactor psync_overlay_get_response
* clarifications
* response_messages work with and without payload
* removed commented code
* syncadd works, but i need to fix the error handling
* remove sync folder
* remove commented code, change printfs to debug or simply remove
* removed unnecessary print
* removed more prints
* removed dead var
* update error handling in add_sync_folder, fix memleak
* simplified finalize
* openssl is a dependency
* openssl is not a real dpendency
* static link by default
* wip: build containers
* rm comments
* build containers, fix DEADLYSIGNAL on run with no args
* debug tools in deb/ubuntu, fix#44
* no slack container :(
* ensure P_LINUX_OS always defined
* error check fuse unmount ops
* fsanitize=thread temporarily
* undo
* clang static analysis
* fix dead assignment
* makefile quality of life stuff
* make install and uninstall; ignore scan results
* addressed dead assignments from static analysis
* debug and release builds possible
* gcc still default compiler
* debug symbols on debug build only
* fixed 2 of the 3 memory leaks
* fixed more memleaks
* more leaks
* another one
* BUGHUNT mode to look for more mem leaks
* merge poverlay_linux and pclsync
* reorg source tree, link system mbedtls
* remove cmakelists.txt
* update ci workflow
* -Werror to -Wall
* libpcloudcc_so in makefile
* single makefile works
* removed extra makefiles
* tagged mbedtls lib with version
* added source tree for mbedtls 2.1.14
* pcloudcc compiles with upgraded library mbedtls-2.x
* working on a segfault
* Revert "pcloudcc compiles with upgraded library mbedtls-2.x"
This reverts commit 8ccdb71ab8.
* build with debugging symbols
* trying a different approach; upgrading pupload.c
* updated mbedtls headers, using compat header
* all but pssl now using new headers
* updated pssl.h dependencies
* mbedtls2 rename.pl on all but pssl.{c,h}
* removed more commented code
* removed mbedtls-1.x from cmakelists
* program compiles with mbedtls 2.x, but remains offline.. why?
* removed mbedtls-1.x from source tree
* fixed handshake, now peer cert returning null
* mbedtls upgrade to 2.x completed!
* compat header removed; startcrypto no longer working
* fix start_crypto handler; need to check ret == 0
* fix linking problem on first login
* fixed startcrypto!
Levi Neely
Jonathan Senkerik
V10lator