refactor: New Client Implementation (#52)

qdrant · Sep 9, 2024 · e52189d · e52189d
1 parent 6fb15a4
commit e52189d
Show file tree

Hide file tree

Showing 47 changed files with 6,299 additions and 2,491 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -1,10 +1,10 @@
-name: Go CI
+name: Go Tests
 
 on:
   push:
-    branches: [ master, dev ]
+    branches:
+      - master
   pull_request:
-    branches: [ '**' ]
 
 jobs:
   build:
@@ -19,6 +19,9 @@ jobs:
         with:
           go-version-file: ./go.mod
 
+      - name: Get
+        run: go get ./...
+
       - name: Build
         run: go build ./...
 

diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml
@@ -0,0 +1,23 @@
+name: golangci-lint
+on:
+  push:
+    branches:
+      - master
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  golangci:
+    name: lint
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-go@v5
+        with:
+          go-version: stable
+      - name: golangci-lint
+        uses: golangci/golangci-lint-action@v6
+        with:
+          version: v1.60
diff --git a/.golangci.yml b/.golangci.yml
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,104 @@
+## Report Issues on GitHub [Issues](https://github.com/qdrant/go-client/issues)
+
+We track public bugs and feature requests using GitHub issues. Please report by [opening a new issue](https://github.com/qdrant/go-client/issues/new).
+
+**Effective Bug Reports** should include:
+
+- A clear summary or background
+- Steps to reproduce the issue
+  - Be as specific as possible
+  - Include sample code when possible
+- What you expect to happen
+- What happened
+- Additional notes (e.g., why you think the issue occurs or solutions you’ve tried that didn’t work)
+
+## Contributing Code
+
+Follow these steps before submitting a pull request:
+
+### Building the Project
+
+```bash
+go build ./...
+```
+
+This will download all dependencies and compile the project.
+
+### Running Tests
+
+All test files are in the `qdrant_test` directory and use [Testcontainers Go](https://golang.testcontainers.org/) for integration tests.
+
+Run the following command to execute the test suites:
+
+```bash
+go test -v ./...
+```
+
+This command pulls a Qdrant Docker image to run integration tests. Ensure Docker is running.
+
+### Formatting and Linting
+
+Ensure your code is free from warnings and follows project standards.
+
+The project uses [Gofmt](https://go.dev/blog/gofmt) for formatting and [golangci-lint](https://github.com/golangci/golangci-lint) for linting.
+
+To format your code:
+
+```bash
+gofmt -s -w .
+```
+
+To lint your code:
+
+```bash
+golangci-lint run
+```
+
+### Preparing for a New Release
+
+The client uses generated stubs from upstream Qdrant proto definitions, which are downloaded from [qdrant/qdrant](https://github.com/qdrant/qdrant/tree/master/lib/api/src/grpc/proto).
+
+#### Steps:
+
+1. Download and generate the latest client stubs by running the following command from the project root:
+
+```bash
+BRANCH=dev sh tools/sync_proto.sh
+```
+
+2. Update the test image value in [`qdrant_test/image_test.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant_test/image_test.go) to `qdrant/qdrant:dev`.
+
+3. Remove the gRPC server definitions from the auto-generated code. 
+
+There is currently [no way](https://github.com/golang/protobuf/issues/373) to skip generating Go server definitions.
+
+You’ll need to manually delete them from [`snapshots_service_grpc.pb.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/snapshots_service_grpc.pb.go), [`points_service_grpc.pb.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/points_service.pb.go), and [`collections_service_grpc.pb.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/collections_service_grpc.pb.go).
+
+Remove lines starting from comments like `// CollectionsServer is the server API for Collections service.` until the end of the file. [Here’s an example commit](https://github.com/qdrant/go-client/commit/6d04e31bb2acccf54f964a634df8930533642892).
+
+4. Implement new Qdrant methods in [`points.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/points.go), [`collections.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/collections.go), or [`qdrant.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/qdrant.go) as needed.
+
+5. If there are any new `oneOf` properties in the proto definitions, add helper constructors in [`oneof_factory.go`](https://github.com/qdrant/go-client/blob/new-client/qdrant/oneof_factory.go) following the existing patterns.
+
+6. Submit your pull request and get those approvals.
+
+### Releasing a New Version
+
+Once the new Qdrant version is live:
+
+1. Run the following command:
+
+```bash
+BRANCH=master sh tools/sync_proto.sh
+```
+
+2. Update the test image value in `qdrant_test/image_test.go` to `qdrant/qdrant:NEW_VERSION`.
+
+3. Merge the pull request.
+
+4. Push a new Git tag to publish the version:
+
+```bash
+git tag v1.11.0
+git push --tags
+```
diff --git a/README.md b/README.md
@@ -1,75 +1,135 @@
-# Golang Qdrant client
+<p align="center">
+  <img height="120" src="https://github.com/user-attachments/assets/a69a26dd-ecfb-46d5-bbe1-3c0b9f0c430a" alt="Qdrant-Go">
+</p>
 
-Go client for Qdrant vector search engine
+<p align="center">
+    <b>Go client for the <a href="https://github.com/qdrant/qdrant">Qdrant</a> vector search engine.</b>
+</p>
 
-## Install
+<p align="center">
+    <a href="https://pkg.go.dev/github.com/qdrant/go-client"><img src="https://img.shields.io/badge/Docs-godoc-success" alt="Godoc"></a>
+    <a href="https://github.com/qdrant/go-client/actions/workflows/ci.yml"><img src="https://github.com/qdrant/go-client/actions/workflows/ci.yml/badge.svg?branch=master" alt="Tests"></a>
+    <a href="https://github.com/qdrant/go-client/blob/master/LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-success" alt="Apache 2.0 License"></a>
+    <a href="https://qdrant.to/discord"><img src="https://img.shields.io/badge/Discord-Qdrant-5865F2.svg?logo=discord" alt="Discord"></a>
+    <a href="https://qdrant.to/roadmap"><img src="https://img.shields.io/badge/Roadmap-2024-bc1439.svg" alt="Roadmap 2024"></a>
+</p>
+
+Go client library with handy utilities for interfacing with [Qdrant](https://qdrant.tech/).
+
+## 📥 Installation
 
 ```bash
-go get github.com/qdrant/go-client
+go get -u github.com/qdrant/go-client
 ```
 
-## Usage
+## 📖 Documentation
 
-Run Qdrant with enabled gRPC interface:
+- Usage examples are available throughout the [Qdrant documentation](https://qdrant.tech/documentation/quick-start/) and [API Reference](https://api.qdrant.tech/).
+- [Godoc Reference](https://pkg.go.dev/github.com/qdrant/go-client)
 
-```bash
-# With env variable
-docker run -p 6333:6333 -p 6334:6334 \
-    -e QDRANT__SERVICE__GRPC_PORT="6334" \
-    qdrant/qdrant
-```
+## 🔌 Getting started
+
+### Creating a client
+
+A client can be instantiated with
 
-Or by updating the configuration file:
+```go
+import "github.com/qdrant/go-client/qdrant"
 
-```yaml
-service:
-  grpc_port: 6334
+client, err := qdrant.NewClient(&qdrant.Config{
+  Host: "localhost",
+  Port: 6334,
+})
 ```
 
-More info about gRPC in [documentation](https://qdrant.tech/documentation/quick-start/#grpc).
+Which creates a client that will connect to Qdrant on <http://localhost:6334>.
 
-### Making requests
+Internally, the high-level client uses a low-level gRPC client to interact with
+Qdrant. `qdrant.Config` provides additional options to control how the gRPC
+client is configured. The following example configures API key authentication with TLS:
 
 ```go
-package main
+import "github.com/qdrant/go-client/qdrant"
+
+client, err := qdrant.NewClient(&qdrant.Config{
+	Host:   "xyz-example.eu-central.aws.cloud.qdrant.io",
+	Port:   6334,
+	APIKey: "<paste-your-api-key-here>",
+	UseTLS: true,  // uses default config with minimum TLS version set to 1.3
+	// TLSConfig: &tls.Config{...},
+	// GrpcOptions: []grpc.DialOption{},
+})
+```
+
+### Working with collections
 
+Once a client has been created, create a new collection
+
+```go
 import (
 	"context"
-	"flag"
-	"log"
-	"time"
 
-	pb "github.com/qdrant/go-client/qdrant"
-	"google.golang.org/grpc"
-	"google.golang.org/grpc/credentials/insecure"
+	"github.com/qdrant/go-client/qdrant"
 )
 
-var (
-	addr = flag.String("addr", "localhost:6334", "the address to connect to")
-)
+client.CreateCollection(context.Background(), &qdrant.CreateCollection{
+	CollectionName: "{collection_name}",
+	VectorsConfig: qdrant.NewVectorsConfig(&qdrant.VectorParams{
+		Size:     4,
+		Distance: qdrant.Distance_Cosine,
+	}),
+})
+```
+
+Insert vectors into the collection
+
+```go
+operationInfo, err := client.Upsert(context.Background(), &qdrant.UpsertPoints{
+	CollectionName: "{collection_name}",
+	Points: []*qdrant.PointStruct{
+		{
+			Id:      qdrant.NewIDNum(1),
+			Vectors: qdrant.NewVectors(0.05, 0.61, 0.76, 0.74),
+			Payload: qdrant.NewValueMap(map[string]any{"city": "London"}),
+		},
+		{
+			Id:      qdrant.NewIDNum(2),
+			Vectors: qdrant.NewVectors(0.19, 0.81, 0.75, 0.11),
+			Payload: qdrant.NewValueMap(map[string]any{"age": 32}),
+		},
+		{
+			Id:      qdrant.NewIDNum(3),
+			Vectors: qdrant.NewVectors(0.36, 0.55, 0.47, 0.94),
+			Payload: qdrant.NewValueMap(map[string]any{"vegan": true}),
+		},
+	},
+})
+```
+
+Search for similar vectors
 
-func main() {
-	flag.Parse()
-	// Set up a connection to the server.
-	conn, err := grpc.Dial(*addr, grpc.WithTransportCredentials(insecure.NewCredentials()))
-	if err != nil {
-		log.Fatalf("did not connect: %v", err)
-	}
-	defer conn.Close()
-
-	collections_client := pb.NewCollectionsClient(conn)
-
-	// Contact the server and print out its response.
-	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
-	defer cancel()
-	r, err := collections_client.List(ctx, &pb.ListCollectionsRequest{})
-	if err != nil {
-		log.Fatalf("could not get collections: %v", err)
-	}
-	log.Printf("List of collections: %s", r.GetCollections())
-}
+```go
+searchResult, err := client.Query(context.Background(), &qdrant.QueryPoints{
+	CollectionName: "{collection_name}",
+	Query:          qdrant.NewQuery(0.2, 0.1, 0.9, 0.7),
+})
+```
+
+Search for similar vectors with filtering condition
+
+```go
+searchResult, err := client.Query(context.Background(), &qdrant.QueryPoints{
+	CollectionName: "test_collection",
+	Query:          qdrant.NewQuery(0.2, 0.1, 0.9, 0.7),
+	Filter: &qdrant.Filter{
+		Must: []*qdrant.Condition{
+			qdrant.NewMatch("city", "London"),
+		},
+	},
+	WithPayload: qdrant.NewWithPayload(true),
+})
 ```
 
-> For authenticated request (using API KEY and TLS) to Qdrant Cloud, please refer to the [authenticated](https://github.com/qdrant/go-client/tree/master/examples/authentication/main.go) example.
+## ⚖️ LICENSE
 
-A full example for uploading, searching and filtering can be found in the [`examples`](https://github.com/qdrant/go-client/tree/master/examples) directory.
+Apache 2.0 © [2024](https://github.com/qdrant/go-client/blob/master/LICENSE)