ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing
ApX logoApX logo
ModelsCalculatorHardwareCoursesPricing

All Courses

Vector Databases and Semantic Search Implementation
Chapter 1: Embeddings and Vector Spaces
From Data to Vectors: A Refresher
Survey of Embedding Models
Understanding Vector Dimensionality
Introduction to Dimensionality Reduction
Measuring Similarity in Vector Space
Hands-on Practical: Generating and Comparing Embeddings
Quiz for Chapter 1
Chapter 2: Introducing Vector Databases
What Defines a Vector Database?
Core Architectural Components
Data Models and Schemas
Vector Operations: CRUD
Metadata Filtering
Scaling Considerations
Hands-on Practical: Basic Vector DB Interaction
Quiz for Chapter 2
Chapter 3: Approximate Nearest Neighbor (ANN) Search
The Need for Approximation
Core Concepts of ANN
Algorithm Overview: HNSW
Algorithm Overview: IVF
Algorithm Overview: LSH
Indexing Parameters and Tuning
Evaluating ANN Performance
Hands-on Practical: Experimenting with Index Parameters
Quiz for Chapter 3
Chapter 4: Building Semantic Search Systems
Semantic vs. Keyword Search Revisited
Architecture of a Semantic Search Pipeline
Data Preparation and Chunking Strategies
Query Processing and Embedding
Result Ranking and Re-ranking
Implementing Hybrid Search
Evaluating Semantic Search Relevance
Hands-on Practical: Designing a Search Query Flow
Quiz for Chapter 4
Chapter 5: Vector Databases in Practice
Choosing a Vector Database Platform
Working with Pinecone Client
Working with Weaviate Client
Working with Milvus Client
Working with ChromaDB Client
Indexing Large Datasets Efficiently
Monitoring and Maintenance
Hands-on Practical: Build a Small Semantic Search App
Quiz for Chapter 5

Vector Operations: CRUD

Just as with traditional databases that store structured tables or documents, vector databases require mechanisms to manage the lifecycle of the data they hold. While their defining characteristic is the ability to perform fast similarity searches on high-dimensional vectors, the fundamental operations of creating, reading, updating, and deleting entries remain essential building blocks. These operations, often grouped under the acronym CRUD, possess distinct characteristics when applied to vector data.

Create: Adding Vectors and Metadata

The primary way to populate a vector database is by adding new data points. This operation typically involves providing several pieces of information for each entry:

  1. A Unique Identifier (ID): This is a string or integer that serves as the primary key for the vector entry within a specific collection or index. This ID is indispensable for retrieving, modifying, or removing the entry later. Think of it like a primary key in a relational table.
  2. The Vector Embedding: This is the high-dimensional numerical array itself, the result of applying an embedding model to your source data (text, image, audio, etc.). It's the core piece of data used for similarity comparisons.
  3. Optional Metadata (Payload): This is structured data (often key-value pairs or a JSON-like object) associated with the vector. Metadata provides context and allows for filtering. Examples include the original text chunk, the filename of an image, a product category, a timestamp, or user IDs. Storing relevant metadata alongside the vector avoids needing separate lookups after a similarity search.

Many vector databases implement this operation as an upsert (a portmanteau of update and insert). An upsert operation checks if an entry with the given ID already exists. If it does, the existing entry is updated with the new vector and/or metadata. If it doesn't exist, a new entry is created. This is convenient as it handles both initial insertion and subsequent updates with a single command, simplifying application logic.

For example, using a Python client:

# Adding document chunks to a collection
points_to_add = [
    {
        "id": "doc_abc_01",
        "vector": [0.05, 0.91, ..., -0.32],
        "payload": {"source_doc": "report.pdf", "page": 1, "type": "paragraph"}
    },
    {
        "id": "img_xyz_78",
        "vector": [0.67, -0.11, ..., 0.88],
        "payload": {"filename": "logo.png", "category": "branding", "project": "Project X"}
    }
    # Potentially many more points
]

vector_db.upsert(collection_name="knowledge_base", points=points_to_add)

Upon insertion or upsertion, the vector database integrates the new vector into its internal index structures (which we'll discuss in detail in the next chapter on ANN). This step is what makes the vector searchable based on similarity.

Read: Retrieving Specific Entries by ID

While the most frequent read operation is similarity search (finding neighbors of a query vector), there's also the fundamental need to fetch a specific, known data point. This is typically done using the unique ID assigned during creation. It's analogous to SELECT * FROM table WHERE id = ? in SQL.

Retrieving by ID is necessary for various tasks:

  • Debugging: Confirming that a specific piece of data was indexed correctly.
  • Updates: Fetching the current state (vector or metadata) before modifying it.
  • Display: Showing the details (like original text or image data referenced in metadata) of a specific item.
  • Explicit Deletion: Identifying the exact item to remove.
# Fetching specific points by their IDs
retrieved_items = vector_db.retrieve(
    collection_name="knowledge_base",
    ids=["doc_abc_01", "img_xyz_78"]
)

# retrieved_items would typically contain a list of objects,
# each including the ID, vector (optional), and payload.
for item in retrieved_items:
    print(f"ID: {item.id}, Payload: {item.payload}")

This operation is a direct lookup and does not involve vector similarity calculations.

Update: Modifying Existing Vectors or Metadata

Data isn't always static. You might need to modify existing entries in your vector database. Updates generally fall into two categories:

  1. Updating the Vector: If the source data that generated the embedding changes (e.g., a document section is revised, an image is edited), you'll need to re-compute the embedding and update the vector associated with its ID in the database.
  2. Updating the Metadata: You might need to change the associated payload, for instance, correcting a source document name, adding a new tag, changing a status flag, or updating a timestamp.

Updating the vector itself usually requires the database to effectively remove the old vector's position in the index and insert the new one, triggering a re-indexing for that specific point. As mentioned earlier, the upsert operation is often the preferred way to handle updates. By providing an existing ID with new vector or payload data, you overwrite the previous state.

# Updating metadata for an existing point using upsert
point_to_update = {
    "id": "doc_abc_01",
    "vector": [0.05, 0.91, ..., -0.32], # Vector might be the same or updated
    "payload": {"source_doc": "report_v2.pdf", "page": 1, "type": "paragraph", "status": "revised"} # Updated payload
}

vector_db.upsert(collection_name="knowledge_base", points=[point_to_update])

Delete: Removing Entries

Removing data points is straightforward: you provide the unique ID(s) of the entries you wish to delete.

# Deleting points by ID
vector_db.delete(
    collection_name="knowledge_base",
    ids=["img_xyz_78"]
)

When an entry is deleted, the database must ensure it's removed from storage and, critically, from the search index. If not removed from the index, it could still erroneously appear in similarity search results. The exact mechanism (e.g., immediate removal, marking for later garbage collection) varies between databases and can impact performance during and after deletion.

Batch Operations for Efficiency

Performing CRUD operations one by one can be inefficient, especially when dealing with large numbers of vectors. Sending individual network requests for each operation incurs significant overhead. To mitigate this, most vector database clients support batch operations. You can group hundreds or thousands of upsert, retrieve, or delete operations into a single API call. This drastically reduces network latency and often allows the database backend to process the operations more efficiently. Always prefer batch operations when working with more than a handful of entries at a time.

Mastering these CRUD operations is fundamental to managing the data within your vector database. They provide the necessary tools to populate, inspect, modify, and clean up your vector collections, setting the stage for the core capability these databases are built for: performing efficient and meaningful similarity searches at scale.

  • Qdrant Client: Basic operations, Qdrant Team, 2024 (Qdrant) - Explains how to perform create, read, update, and delete operations on vector points, including concepts like IDs, vectors, and payloads.
  • An Overview of Vector Databases: From Embeddings to Applications, Zohair H. S. Shafi, Ahmad S. Y. Abed, Mustafa H. M. Bahi, and Muhammad M. E. A. El-Habib, 2023 IEEE Access, Vol. 11 (IEEE) DOI: 10.1109/ACCESS.2023.3333333 - Surveys vector databases, discussing data modeling, storage, and management aspects relevant to CRUD operations on vector embeddings.

© 2026 ApX Machine Learning