Skip to content

Hide the loadable extension's internal symbols - #340

Open
MayCXC wants to merge 1 commit into
asg017:mainfrom
MayCXC:loadable-hidden-visibility
Open

MayCXC wants to merge 1 commit into
asg017:mainfrom
MayCXC:loadable-hidden-visibility

Conversation

@MayCXC

@MayCXC MayCXC commented Oct 23, 2026 •

Copy link
Copy Markdown

dist/vec0.so exports every non-static function in sqlite-vec.c, and sqlite3_api with them: on Linux arm64, nm -D --defined-only dist/vec0.so lists 111 symbols, among them array_append, bitmap_new, the diskann_* helpers and vec0Filter_fullscan. SQLite opens an extension with RTLD_GLOBAL, so the calls inside one library can bind to another library's exported function of the same name. With two builds of sqlite-vec in one process, the second build runs the first build's code:

import sqlite3

# a: make loadable; b: the same source built with -DSQLITE_VEC_ENABLE_RESCORE=0
for path in ["a/vec0", "b/vec0"]:
    db = sqlite3.connect(":memory:")
    db.enable_load_extension(True)
    db.load_extension(path)
    db.execute(
        "create virtual table v using vec0("
        "embedding float[8] indexed by rescore(quantizer=bit))"
    )

Loaded alone, b fails with vec0 constructor error: could not parse vector column 'embedding float[8] indexed by rescore(quantizer=bit)'. Loaded after a, the process crashes with SIGSEGV on that statement.

This PR compiles with -fvisibility=hidden, as vlasky/sqlite-vec@e8f156b does for vlasky's fork (it says it was inspired by #149, which its author closed without a review) and as photostructure/sqlite-vec's Makefile does, which has that commit in its history. SQLITE_VEC_API gets default visibility so sqlite3_vec_init stays exported; nm -D then lists it alone, and with this PR the reproduction fails on b the way b fails when loaded alone. One difference from vlasky's header: the SQLITE_VEC_STATIC branch uses __attribute__((visibility("default"))) only with GCC and Clang outside Windows, so MSVC can still compile a SQLITE_VEC_STATIC build and MinGW does not warn about an ignored attribute. A Windows DLL already exports only its __declspec(dllexport) functions, so Windows builds are unchanged.

site/compiling.md's manual Linux and macOS commands pass the flag too, with a sentence on why; the documented Linux command, built from this branch, exports sqlite3_vec_init alone and loads.

Test. test_exports_only_entry_point opens dist/vec0 with ctypes and checks that sqlite3_vec_init resolves and sqlite3_api, bitmap_new and vec0Filter_fullscan do not. On main it fails with AssertionError: sqlite3_api. make test-loadable passes (240 passed, 66 skipped), and the build adds no warnings to main's 12.

This merges cleanly with #293, #304 and #305, which also touch the Makefile or the tests.

The loadable extension exported every non-static function and sqlite3_api,
111 symbols in all. SQLite opens an extension with RTLD_GLOBAL, so with two
builds of sqlite-vec loaded into one process, the second build's vec0 calls
bound to the first build's functions: loading a build with the rescore index
and then one built with SQLITE_VEC_ENABLE_RESCORE=0 crashed the process when
the second connection created a table.

Compile with -fvisibility=hidden, as vlasky/sqlite-vec@e8f156b does, and give
SQLITE_VEC_API default visibility so sqlite3_vec_init stays exported. The
static branch uses the attribute only with GCC and Clang outside Windows, so
MSVC still compiles SQLITE_VEC_STATIC builds.
MayCXC added a commit to MayCXC/sqlite-vec that referenced this pull request Oct 23, 2026
mceachen pushed a commit to photostructure/sqlite-vec that referenced this pull request Oct 24, 2026
…VC in static builds

With -DSQLITE_VEC_STATIC the template defines SQLITE_VEC_API as
__attribute__((visibility("default"))) on every compiler. MSVC has no
__attribute__, so a static build with it stops at the header's declaration
of sqlite3_vec_init(); with MSVC 19.44 on windows-2022:

    cl /c /W4 /Ivendor/ /O2 /DSQLITE_CORE /DSQLITE_VEC_STATIC sqlite-vec.c

    sqlite-vec.h(33): error C2143: syntax error: missing ')' before '('
    sqlite-vec.h(33): error C2091: function returns function

The loadable build, which CI compiles with cl.exe, takes the _WIN32 branch
and is unaffected.

The static branch now applies the attribute only under GCC and Clang outside
Windows, the guard asg017#340 uses, and defines SQLITE_VEC_API
empty otherwise. With GCC and Clang the static objects are byte-identical
before and after. sqlite-vec.h regenerates from the template at release, as
it does for the template's other changes.
vlasky pushed a commit to vlasky/sqlite-vec that referenced this pull request Oct 28, 2026
With -DSQLITE_VEC_STATIC the template defines SQLITE_VEC_API as
__attribute__((visibility("default"))) on every compiler. MSVC has no
__attribute__, so a static build with it stops at the header's declaration
of sqlite3_vec_init(); with MSVC 19.44 on windows-2022:

    cl /c /W4 /Ivendor/ /O2 /DSQLITE_CORE /DSQLITE_VEC_STATIC sqlite-vec.c

    sqlite-vec.h(34): error C2143: syntax error: missing ')' before '('
    sqlite-vec.h(34): error C2091: function returns function

The loadable build, which CI compiles with cl.exe, takes the _WIN32 branch
and is unaffected.

The static branch now applies the attribute only under GCC and Clang outside
Windows, the guard asg017#340 uses, and defines SQLITE_VEC_API
empty otherwise. With GCC and Clang the static objects are byte-identical
before and after. sqlite-vec.h regenerates from the template with the next
version bump, as it does for the template's other changes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant