Create Partitioned Table

Hippo supports partitioned table, and users can specify partitioning column during table creation.

When inserting data, Hippo will store data in the light of the partition in which data is divided
When querying data, Hippo will perform partition pruning according to query conditions

curl -u shiva:shiva -XPUT 'localhost:8902/hippo/v1/{table}?pretty' -H 'Content-Type: application/json' -d'{
  "settings": {
    "number_of_shards" : 1,
    "number_of_replicas" : 1
  },
  "schema": {
    "auto_id": false,
    "fields": [
      {
        "name": "book_id",
        "is_primary_key": true,
        "data_type": "int64"
      },
      {
        "name": "word_count",
        "is_primary_key": false,
        "data_type": "int64"
      },
      {
        "name": "book_type",
        "is_primary_key": false,
        "data_type": "string"
      },
      {
        "name": "book_intro",
        "data_type": "float_vector",
        "is_primary_key": false,
        "type_params": {
          "dimension" : 2
        }
      }
    ]
  },
  "partition_schema": {
    "type": "single_value", // range|single_value
    "columns": ["book_type"],
    "partitions" : [
      {
        "name": "p1",
        "table_name": "p_p1",
        "bound": [
          {
            "field": "book_type",
            "value": "a1"
          }
        ]
      },
      {
        "name": "p2",
        "table_name": "p_p2",
        "bound": [
          {
            "field": "book_type",
            "value": "a2"
          }
        ]
      }
    ]
  }
}';

Result:

{
    "acknowledged" : true
}

Parameter description:

Parameters	Description	Options
partition_schema	Table is partitioned
type (partition_schema)	Partitioning type	range / single_value
columns (partition_schema)	Partitioning column
partitions (partition_schema)	Pre-create partition during table creation, not required
name (partitions)	Partition name
table_name (partition_schema)	Sub-table name
bound (partition_schema)	Partition value

Table 23 Create Partitioned Table (Restful API)

Add Partition

Users can perform the command listed below to add partition.

curl -u shiva:shiva -XPUT 'localhost:8902/hippo/v1/{table}/_partitions?pretty' -H 'Content-Type: application/json' -d'{
  "partitions" : [
    {
      "name": "p3",
      "table_name": "p_p3",
      "bound": [
        {
          "field": "book_type",
          "value": "a3"
        }
      ]
    },
    {
      "name": "p4",
      "table_name": "p_p4",
      "bound": [
        {
          "field": "book_type",
          "value": "a4"
        }
      ]
    }
  ]
}';

Result:

{
  "acknowledged" : true
}

Drop Partition

curl -u shiva:shiva -XDELETE 'localhost:8902/hippo/v1/{table}/_partitions?pretty' -H 'Content-Type: application/json' -d'{
  "partition_names" : ["p1","p2"]
}';

Result:

{
  "acknowledged" : true
}

Calculate Distance between Vectors

For vector search, scalar search, count operation, Hippo will perform partition pruning automatically in the light of different conditions. For calculating distance between vectors, primary key and partition value are both required for target vector.

curl -u shiva:shiva -XGET 'localhost:8902/hippo/v1/_distance?pretty' -H 'Content-Type: application/json' -d'{
  "vectors_left" : {
    "database_name" : "default",
    "table_name" : "table",
    "field" : "book_intro",
    "primary_keys" : [
      {
        "field_name": "book_id",
        "field": [1,2,3,4,5]
      },
      {
        "field_name": "book_type", // book_type为数据的分区值
        "field": ["a1","a1","a1","a1","a1"]
      }
    ]
  },
  "vectors_right" : {
    "float_vectors" : [[1,2], [3,4]]
  },
  "params" : {
    "metric_type": "cosine"
  }
}';