Unlike vectors, which have both magnitude and direction, scalars have only magnitude. Similar to traditional database, users can build indexes on scalar fields to speed up operations on scalars.
curl -u shiva:shiva -XPUT 'localhost:8902/hippo/v1/{table}/_create_scalar_index?database_name={database_name}&pretty' -H 'Content-Type: application/json' -d'{
"index_name" : "index",
"field_names" : ["word_count"]
}';
Result:
{
"acknowledged" : true
}
Parameter description:
| Parameters | Description | Required |
|---|---|---|
| table | Table name, such as "book" created in this example | Yes |
| database_name | Database where the table is located | No, defaults to "default" database |
| index_name | Scalar index name | Yes |
| filed_names | Field containing scalar index. Hippo supports single index and composite index now. | Yes |
Create Array Index
Hippo supports creating array containing scalars only. With index created on array column, the performance of hybrid search can be improved. This chapter introduces the array type and how to create array index in Hippo.
curl -u shiva:shiva -XPUT 'localhost:8902/hippo/v1/book?database_name={database_name}&pretty' -H 'Content-Type: application/json' -d'{
"settings": {
"number_of_shards" : 1,
"number_of_replicas" : 1
},
"schema": {
"fields": [
{
"name": "book_id",
"is_primary_key": true,
"data_type": "int64"
},
{
"name": "word_count",
"is_primary_key": false,
"data_type": "int64"
},
{
"name": "attributes",
"is_primary_key": false,
"data_type": "array",
"type_params": {
"element_data_type" : "string"
}
},
{
"name": "book_intro",
"data_type": "float_vector",
"is_primary_key": false,
"type_params": {
"dimension" : 2
}
}
]
}
}';
The example shown above creates a String array column called “attributes” for table “book”, and the related parameters are listed below:
| Parameters | Description | Options |
|---|---|---|
| element_data_type (type) | Data type of elements stored in array |
Table 32 Data Type of An Array in Hippo
Hippo supports creating index on array column, and the API used for array column during index creation is the same as the one used for other columns. However, there are several points users should pay attention to:
- The array index must be single column index.
- Only array columns containing elements with Int or String data type can be indexed.
- Currently array index only supports conditions with “=” or “in” operators.
curl -u shiva:shiva -XPUT 'localhost:8902/hippo/v1/book/_create_scalar_index?database_name={database_name}&pretty' -H 'Content-Type: application/json' -d'{
"index_name" : "array_index",
"field_names" : ["attributes"]
}';
Result:
{
"acknowledged" : true
}
The insert operation is similar to:
curl -u shiva:shiva -XPUT 'localhost:8902/hippo/v1/book/_bulk?database_name={database_name}&pretty' -H'Content-Type: application/json' -d'{
"fields_data": [
{
"field_name": "book_id",
"field": [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100
]
},
{
"field_name": "word_count",
"field": [1000,2000,3000,4000,5000,6000,7000,8000,9000,10000,11000,12000,13000,14000,15000,16000,17000,18000,19000,20000,21000,22000,23000,24000,25000,26000,27000,28000,29000,30000,31000,32000,33000,34000,35000,36000,37000,38000,39000,40000,41000,42000,43000,44000,45000,46000,47000,48000,49000,50000,51000,52000,53000,54000,55000,56000,57000,58000,59000,60000,61000,62000,63000,64000,65000,66000,67000,68000,69000,70000,71000,72000,73000,74000,75000,76000,77000,78000,79000,80000,81000,82000,83000,84000,85000,86000,87000,88000,89000,90000,91000,92000,93000,94000,95000,96000,97000,98000,99000,100000
]
},
{
"field_name": "attributes",
"field": [["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0","attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr0", "attr1", "attr2"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3","attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr3", "attr4", "attr5"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr6", "attr7", "attr8"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10","attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"],["attr9", "attr10", "attr11"]
]
},
{
"field_name": "book_intro",
"field": [[1,1],[2,1],[3,1],[4,1],[5,1],[6,1],[7,1],[8,1],[9,1],[10,1],[11,1],[12,1],[13,1],[14,1],[15,1],[16,1],[17,1],[18,1],[19,1],[20,1],[21,1],[22,1],[23,1],[24,1],[25,1],[26,1],[27,1],[28,1],[29,1],[30,1],[31,1],[32,1],[33,1],[34,1],[35,1],[36,1],[37,1],[38,1],[39,1],[40,1],[41,1],[42,1],[43,1],[44,1],[45,1],[46,1],[47,1],[48,1],[49,1],[50,1],[51,1],[52,1],[53,1],[54,1],[55,1],[56,1],[57,1],[58,1],[59,1],[60,1],[61,1],[62,1],[63,1],[64,1],[65,1],[66,1],[67,1],[68,1],[69,1],[70,1],[71,1],[72,1],[73,1],[74,1],[75,1],[76,1],[77,1],[78,1],[79,1],[80,1],[81,1],[82,1],[83,1],[84,1],[85,1],[86,1],[87,1],[88,1],[89,1],[90,1],[91,1],[92,1],[93,1],[94,1],[95,1],[96,1],[97,1],[98,1],[99,1],[100,1]
]
}
],
"num_rows": 100
}';
After creating vector index, users can perform hybrid search on array or vector data. For more details, please refer to Chapter Hybrid Search Using Array and Vector Field.