Weaviate Migration Guide
This version adds several new features like flexible embedding options, overwrite and append+dedup sync modes. When upgrading from prior versions on this connector, a one-time migration of existing connections is required. This is done to align the behavior of vector database destinations in Airbyte. The following changes are included:
Due to a change of the configuration structure, it's necessary to reconfigure existing destinations with the same information (e.g. credentials).
It's no longer possible to configure id
fields in the destination. Instead, the destination will generate a UUID for each Weaviate object. The id
for each record is stored in the _ab_record_id
property and can be used to identify Weaviate objects by Airbyte record.
It's not possible anymore to configure separate vector fields per stream. To load embedding vectors from the records itself, the embedding method From Field
can be used and configured with a single field name that has to be available in records from all streams. If your records contain multiple vector fields, you need to configure separate destinations and connections to configure separate vector field names.