Skip to content

fix(config): preserve distance in exported collection schemas - #2240

Open
feiiiiii5 wants to merge 3 commits into
weaviate:mainfrom
feiiiiii5:codex/config-distance-serialization
Open

feiiiiii5 wants to merge 3 commits into
weaviate:mainfrom
feiiiiii5:codex/config-distance-serialization

Conversation

@feiiiiii5

Copy link
Copy Markdown
Contributor

Reading a collection config stores the wire distance field in .distance_metric. Exporting it currently produces distanceMetric, so both create_from_config() and create_from_dict(config.to_dict()) post the wrong field name.

Map the field back to distance in the concrete vector-index serializers, including Dynamic and its nested HNSW/Flat configs. The Python property and enum remain unchanged.

The regression follows real public HTTP GET → config parsing → export → HTTP POST. It covers HNSW, Flat, HFresh and Dynamic; legacy/named vectors; both creation methods; and an original-wire-schema control.

Validation on Python 3.12.8:

  • Unchanged 7266681a: 16 failures, each after its raw-schema control succeeds. Fixed: 16 passed, independently repeated.
  • .venv/bin/python -m pytest test -q --tb=short --disable-warnings: 564 passed, 1 skipped.
  • .venv/bin/python -m pytest mock_tests -q --tb=short --disable-warnings: 87 passed.
  • Ruff format/check, Flake8 with project plugins, Pyright 1.1.399 for the library and new test, and diff checks pass.

The first full mock run timed out in an existing authentication test's /meta request; that unchanged test passed alone and the complete rerun passed. The timeout's cause was not established. Live Weaviate integration and other Python/platform matrices were not run. No dependency/configuration changes.

@orca-security-eu orca-security-eu Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

@weaviate-git-bot

Copy link
Copy Markdown

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Forum?

@feiiiiii5

Copy link
Copy Markdown
Contributor Author

I agree to the Contributor License Agreement.

Comment thread mock_tests/test_config_distance.py Outdated


@pytest.mark.parametrize("from_config", [False, True], ids=["from_dict", "from_config"])
def test_exported_config_preserves_distance_in_create_request(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an integration test would prove this better than the mock, since the mock checks the POST body, not that the server picks up distance. test_create_export_and_recreate_named_vectors and test_collection_hfresh_export_and_reimport already recreate a collection from its export and compare configs, but both use cosine, the server default, so they pass on main. Could we give those a non-default distance like VectorDistances.DOT so they fail on main, cover hnsw and dynamic the same way, and drop this file? A new parametrized integration test over the four index types works too.

how I checked

On main, to_dict() writes "distanceMetric": "dot" for a dot-product index, and your mock test fails there with that body. The server's hnsw and flat config parsers only read distance and fall back to cosine, so the recreated collection reports cosine. With a dot-product flat index, conf2.vector_config == conf.vector_config in test_create_export_and_recreate_named_vectors would then fail on main and pass with this PR. I haven't run that against a live server.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: removed the mock test and extended test_create_export_and_recreate_named_vectors to flat, hnsw and dynamic named vectors with DOT, plus test_collection_hfresh_export_and_reimport with DOT. Details and the main-failure reasoning are in the PR comment; CI is the first live run.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flat and hnsw look good, but I think dynamic and hfresh don't test a non-default distance yet:

  • dynamic fails at collections.create in test_create_export_and_recreate_named_vectors. It needs async indexing, which only the 8090 server has. Could we drop dynamic there?
  • hfresh's test_collection_hfresh_export_and_reimport still uses cosine and passes on main. Your DOT went into test_collection_config_hfresh, which now fails at create, as HFresh only takes cosine or l2-squared. Could we revert that and use VectorDistances.L2_SQUARED in the export test?
how I checked

The client fixture connects to 8080, and only ci/docker-compose-async.yml (port 8090) sets ASYNC_INDEXING. Without it the server rejects the create with "the dynamic index can only be created when async indexing is enabled", on main and with your fix alike. The 8090 server has no contextionary module, so the whole test can't move there, but collection_factory(..., ports=(8090, 50061)) works for a dynamic-only test.

HFresh's config validation only allows cosine or l2-squared, on Weaviate 1.36 through 1.40 and on the server's main branch. test_collection_config_hfresh also still asserts COSINE after the create.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done:

  • Dropped dynamic from test_create_export_and_recreate_named_vectors.
  • Reverted test_collection_config_hfresh back to VectorDistances.COSINE.
  • Updated test_collection_hfresh_export_and_reimport to use VectorDistances.L2_SQUARED.

Comment thread mock_tests/test_config_distance.py Outdated
if posted["vectorIndexType"] == "dynamic":
for child in ["hnsw", "flat"]:
assert index[child].get("distance") == "dot", index[child]
assert "distanceMetric" not in index[child]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think an integration test would prove this better than the mock, since the mock checks the POST body, not that the server picks up distance. test_create_export_and_recreate_named_vectors and test_collection_hfresh_export_and_reimport already recreate a collection from its export and compare configs, but both use cosine, the server default, so they pass on main. Could we give those a non-default distance like VectorDistances.DOT so they fail on main, cover hnsw and dynamic the same way, and drop this file?

@feiiiiii5

Copy link
Copy Markdown
Contributor Author

Done. Dropped the mock file and gave the existing export/reimport tests a non-default DOT distance:

  • test_create_export_and_recreate_named_vectors now has flat, hnsw and dynamic named vectors with distance_metric=VectorDistances.DOT.
  • test_collection_hfresh_export_and_reimport uses DOT instead of the server default COSINE.

On main, to_dict() writes distanceMetric, so the recreated collection falls back to cosine and the vector_config / config == new assertions should fail; with the fix the exported distance is preserved. I did not run the integration suite locally because there is no Weaviate server here, so CI is the first live run. Does this cover the four index types you wanted?

I also removed the mock test rather than keeping both, as suggested.

Comment thread mock_tests/test_config_distance.py Outdated


@pytest.mark.parametrize("from_config", [False, True], ids=["from_dict", "from_config"])
def test_exported_config_preserves_distance_in_create_request(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flat and hnsw look good, but I think dynamic and hfresh don't test a non-default distance yet:

  • dynamic fails at collections.create in test_create_export_and_recreate_named_vectors. It needs async indexing, which only the 8090 server has. Could we drop dynamic there?
  • hfresh's test_collection_hfresh_export_and_reimport still uses cosine and passes on main. Your DOT went into test_collection_config_hfresh, which now fails at create, as HFresh only takes cosine or l2-squared. Could we revert that and use VectorDistances.L2_SQUARED in the export test?
how I checked

The client fixture connects to 8080, and only ci/docker-compose-async.yml (port 8090) sets ASYNC_INDEXING. Without it the server rejects the create with "the dynamic index can only be created when async indexing is enabled", on main and with your fix alike. The 8090 server has no contextionary module, so the whole test can't move there, but collection_factory(..., ports=(8090, 50061)) works for a dynamic-only test.

HFresh's config validation only allows cosine or l2-squared, on Weaviate 1.36 through 1.40 and on the server's main branch. test_collection_config_hfresh also still asserts COSINE after the create.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants