Skip to content

Commit

Permalink
Merge pull request #1043 from gofr-dev/release/v1.21.0
Browse files Browse the repository at this point in the history
Release v1.21.0
  • Loading branch information
aryanmehrotra authored Sep 23, 2024
2 parents 343d135 + d6e1181 commit 8c27cc1
Show file tree
Hide file tree
Showing 43 changed files with 2,958 additions and 192 deletions.
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ docker run --name gofr-mysql -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=t
docker run --name gofr-redis -p 2002:6379 -d redis:7.0.5
docker run --name gofr-zipkin -d -p 2005:9411 openzipkin/zipkin:2
docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack
docker run --name cassandra-node -d -p 9042:9042 -v cassandra_data:/var/lib/cassandra cassandra:latest
docker run --name gofr-pgsql -d -e POSTGRES_DB=customers -e POSTGRES_PASSWORD=root123 -p 2006:5432 postgres:15.1
docker run --name gofr-mssql -d -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=reallyStrongPwd123' -p 2007:1433 mcr.microsoft.com/azure-sql-edge
docker run --name kafka-1 -p 9092:9092 \
Expand Down
78 changes: 76 additions & 2 deletions docs/advanced-guide/handling-data-migrations/page.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Handling Data Migrations

Suppose you manually make changes to your database, and now it's your responsibility to inform other developers to execute them. Additionally, you need to keep track of which changes should be applied to production machines in the next deployment.
GoFr supports data migrations for MySQL, Postgres and Redis which allows altering the state of a database, be it adding a new column to existing table or modifying the data type of existing column or adding constraints to an existing table, setting and removing keys etc.
GoFr supports data migrations for MySQL, Postgres, Redis, Clickhouse & Cassandra which allows altering the state of a database, be it adding a new column to existing table or modifying the data type of existing column or adding constraints to an existing table, setting and removing keys etc.

## Usage

Expand Down Expand Up @@ -156,4 +156,78 @@ Where,
**Method** : It contains the method(UP/DOWN) in which migration ran.
(For now only method UP is supported)

> ##### Check out the example to add and run migrations in GoFr: [Visit GitHub](https://github.com/gofr-dev/gofr/blob/main/examples/using-migrations/main.go)
### Migrations in Cassandra

`GoFr` provides support for migrations in Cassandra but does not guarantee atomicity for individual Data Manipulation Language (DML) commands. To achieve atomicity during migrations, users can leverage batch operations using the `NewBatch`, `BatchQuery`, and `ExecuteBatch` methods. These methods allow multiple queries to be executed as a single atomic operation.

Alternatively, users can construct their batch queries using the `BEGIN BATCH` and `APPLY BATCH` statements to ensure that all the commands within the batch are executed successfully or not at all. This is particularly useful for complex migrations involving multiple inserts, updates, or schema changes in a single transaction-like operation.

When using batch operations, consider using a `LoggedBatch` for atomicity or an `UnloggedBatch` for improved performance where atomicity isn't required. This approach provides a way to maintain data consistency during complex migrations.

> Note: The following example assumes that user has already created the `KEYSPACE` in cassandra. A `KEYSPACE` in Cassandra is a container for tables that defines data replication settings across the cluster.

```go
package migrations

import (
"gofr.dev/pkg/gofr/migration"
)

const (
createTableCassandra = `CREATE TABLE IF NOT EXISTS employee (
id int PRIMARY KEY,
name text,
gender text,
number text
);`

addCassandraRecords = `BEGIN BATCH
INSERT INTO employee (id, name, gender, number) VALUES (1, 'Alison', 'F', '1234567980');
INSERT INTO employee (id, name, gender, number) VALUES (2, 'Alice', 'F', '9876543210');
APPLY BATCH;
`

employeeDataCassandra = `INSERT INTO employee (id, name, gender, number) VALUES (?, ?, ?, ?);`
)

func createTableEmployeeCassandra() migration.Migrate {
return migration.Migrate{
UP: func(d migration.Datasource) error {
// Execute the create table statement
if err := d.Cassandra.Exec(createTableCassandra); err != nil {
return err
}

// Batch processes can also be executed in Exec as follows:
if err := d.Cassandra.Exec(addCassandraRecords); err != nil {
return err
}

// Create a new batch operation
batchName := "employeeBatch"
if err := d.Cassandra.NewBatch(batchName, 0); err != nil { // 0 for LoggedBatch
return err
}

// Add multiple queries to the batch
if err := d.Cassandra.BatchQuery(batchName, employeeDataCassandra, 1, "Harry", "M", "1234567980"); err != nil {
return err
}

if err := d.Cassandra.BatchQuery(batchName, employeeDataCassandra, 2, "John", "M", "9876543210"); err != nil {
return err
}

// Execute the batch operation
if err := d.Cassandra.ExecuteBatch(batchName); err != nil {
return err
}

return nil
},
}
}
```

> ##### Check out the example to add and run migrations in GoFr: [Visit GitHub](https://github.com/gofr-dev/gofr/blob/main/examples/using-migrations/main.go)
136 changes: 131 additions & 5 deletions docs/advanced-guide/injecting-databases-drivers/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,8 +181,6 @@ type Cassandra interface {

BatchQuery(stmt string, values ...any) error

ExecuteBatch() error

NewBatch(name string, batchType int) error

CassandraBatch
Expand Down Expand Up @@ -214,7 +212,8 @@ type Person struct {
ID int `json:"id,omitempty"`
Name string `json:"name"`
Age int `json:"age"`
State string `json:"state"`
// db tag specifies the actual column name in the database
State string `json:"state" db:"location"`
}

func main() {
Expand All @@ -240,7 +239,7 @@ func main() {
return nil, err
}

err = c.Cassandra.Exec(`INSERT INTO persons(id, name, age, state) VALUES(?, ?, ?, ?)`,
err = c.Cassandra.Exec(`INSERT INTO persons(id, name, age, location) VALUES(?, ?, ?, ?)`,
person.ID, person.Name, person.Age, person.State)
if err != nil {
return nil, err
Expand All @@ -252,14 +251,141 @@ func main() {
app.GET("/user", func(c *gofr.Context) (interface{}, error) {
persons := make([]Person, 0)

err := c.Cassandra.Query(&persons, `SELECT id, name, age, state FROM persons`)
err := c.Cassandra.Query(&persons, `SELECT id, name, age, location FROM persons`)

return persons, err
})

app.Run()
}
```
## DGraph
GoFr supports injecting Dgraph with an interface that defines the necessary methods for interacting with the Dgraph
database. Any driver that implements the following interface can be added using the app.AddDgraph() method.

```go
// Dgraph defines the methods for interacting with a Dgraph database.
type Dgraph interface {
// Query executes a read-only query in the Dgraph database and returns the result.
Query(ctx context.Context, query string) (interface{}, error)

// QueryWithVars executes a read-only query with variables in the Dgraph database.
QueryWithVars(ctx context.Context, query string, vars map[string]string) (interface{}, error)

// Mutate executes a write operation (mutation) in the Dgraph database and returns the result.
Mutate(ctx context.Context, mu interface{}) (interface{}, error)

// Alter applies schema or other changes to the Dgraph database.
Alter(ctx context.Context, op interface{}) error

// NewTxn creates a new transaction (read-write) for interacting with the Dgraph database.
NewTxn() interface{}

// NewReadOnlyTxn creates a new read-only transaction for querying the Dgraph database.
NewReadOnlyTxn() interface{}

// HealthChecker checks the health of the Dgraph instance.
HealthChecker
}
```

Users can easily inject a driver that supports this interface, allowing for flexibility without compromising usability.
This structure supports both queries and mutations in Dgraph.

### Example

```go
package main

import (
"encoding/json"
"fmt"

"github.com/dgraph-io/dgo/v210/protos/api"

"gofr.dev/pkg/gofr"
"gofr.dev/pkg/gofr/datasource/dgraph"
)

func main() {
// Create a new application
app := gofr.New()

db := dgraph.New(dgraph.Config{
Host: "localhost",
Port: "8080",
})

// Connect to Dgraph running on localhost:9080
app.AddDgraph(db)

// Add routes for Dgraph operations
app.POST("/dgraph", DGraphInsertHandler)
app.GET("/dgraph", DGraphQueryHandler)

// Run the application
app.Run()
}

// DGraphInsertHandler handles POST requests to insert data into Dgraph
func DGraphInsertHandler(c *gofr.Context) (interface{}, error) {
// Example mutation data to insert into Dgraph
mutationData := `
{
"set": [
{
"name": "GoFr Dev"
},
{
"name": "James Doe"
}
]
}
`

// Create an api.Mutation object
mutation := &api.Mutation{
SetJson: []byte(mutationData), // Set the JSON payload
CommitNow: true, // Auto-commit the transaction
}

// Run the mutation in Dgraph
response, err := c.DGraph.Mutate(c, mutation)
if err != nil {
return nil, err
}

return response, nil
}

// DGraphQueryHandler handles GET requests to fetch data from Dgraph
func DGraphQueryHandler(c *gofr.Context) (interface{}, error) {
// A simple query to fetch all persons with a name in Dgraph
response, err := c.DGraph.Query(c, "{ persons(func: has(name)) { uid name } }")
if err != nil {
return nil, err
}

// Cast response to *api.Response (the correct type returned by Dgraph Query)
resp, ok := response.(*api.Response)
if !ok {
return nil, fmt.Errorf("unexpected response type")
}

// Parse the response JSON
var result map[string]interface{}
err = json.Unmarshal(resp.Json, &result)
if err != nil {
return nil, err
}

return result, nil
}

```




## Solr
GoFr supports injecting Solr database that supports the following interface. Any driver that implements the interface can be added
Expand Down
18 changes: 18 additions & 0 deletions docs/references/context/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,24 @@ parts of the request.
ctx.Bind(&p)
// the Bind() method will map the incoming request to variable p
```

- `Binding multipart-form data`
- To bind multipart-form data, you can use the Bind method similarly. The struct fields should be tagged appropriately
to map the form fields to the struct fields.

```go
type Data struct {
Name string `form:"name"`

Compressed file.Zip `file:"upload"`

FileHeader *multipart.FileHeader `file:"file_upload"`
}
```

- The `form` tag is used to bind non-file fields.
- The `file` tag is used to bind file fields. If the tag is not present, the field name is used as the key.


- `HostName()` - to access the host name for the incoming request
```go
Expand Down
2 changes: 1 addition & 1 deletion examples/using-file-bind/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ it to the fields of the struct. GoFr currently supports zip file type and also b
type Data struct {
Compressed file.Zip `file:"upload"`

FileHeader *multipart.FileHeader `file:"a"`
FileHeader *multipart.FileHeader `file:"file_upload"`
}

func Handler (c *gofr.Context) (interface{}, error) {
Expand Down
2 changes: 1 addition & 1 deletion examples/using-file-bind/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ type Data struct {

// The FileHeader determines the generic file format that we can get
// from the multipart form that gets parsed by the incoming HTTP request
FileHeader *multipart.FileHeader `file:"a"`
FileHeader *multipart.FileHeader `file:"file_upload"`
}

func UploadHandler(c *gofr.Context) (interface{}, error) {
Expand Down
2 changes: 1 addition & 1 deletion examples/using-file-bind/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ func generateMultiPartBody(t *testing.T) (*bytes.Buffer, string) {
t.Fatalf("Failed to write file to form: %v", err)
}

fileHeader, err := writer.CreateFormFile("a", "hello.txt")
fileHeader, err := writer.CreateFormFile("file_upload", "hello.txt")
if err != nil {
t.Fatalf("Failed to create form file: %v", err)
}
Expand Down
33 changes: 17 additions & 16 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -24,25 +24,25 @@ require (
github.com/redis/go-redis/v9 v9.6.1
github.com/segmentio/kafka-go v0.4.47
github.com/stretchr/testify v1.9.0
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.53.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0
go.opentelemetry.io/otel v1.29.0
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.55.0
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.55.0
go.opentelemetry.io/otel v1.30.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.29.0
go.opentelemetry.io/otel/exporters/prometheus v0.51.0
go.opentelemetry.io/otel/exporters/zipkin v1.29.0
go.opentelemetry.io/otel/metric v1.29.0
go.opentelemetry.io/otel/sdk v1.29.0
go.opentelemetry.io/otel/sdk/metric v1.29.0
go.opentelemetry.io/otel/trace v1.29.0
go.opentelemetry.io/otel/exporters/prometheus v0.52.0
go.opentelemetry.io/otel/exporters/zipkin v1.30.0
go.opentelemetry.io/otel/metric v1.30.0
go.opentelemetry.io/otel/sdk v1.30.0
go.opentelemetry.io/otel/sdk/metric v1.30.0
go.opentelemetry.io/otel/trace v1.30.0
go.uber.org/mock v0.4.0
golang.org/x/oauth2 v0.23.0
golang.org/x/sync v0.8.0
golang.org/x/term v0.24.0
golang.org/x/text v0.18.0
google.golang.org/api v0.195.0
google.golang.org/grpc v1.66.1
google.golang.org/api v0.197.0
google.golang.org/grpc v1.66.2
google.golang.org/protobuf v1.34.2
modernc.org/sqlite v1.33.0
modernc.org/sqlite v1.33.1
)

require (
Expand All @@ -65,7 +65,7 @@ require (
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/s2a-go v0.1.8 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.3 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect
github.com/googleapis/gax-go/v2 v2.13.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.22.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
Expand All @@ -77,7 +77,7 @@ require (
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.55.0 // indirect
github.com/prometheus/common v0.59.1 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/redis/go-redis/extra/rediscmd/v9 v9.0.5 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
Expand All @@ -88,15 +88,16 @@ require (
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.54.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.29.0 // indirect
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
golang.org/x/crypto v0.26.0 // indirect
golang.org/x/net v0.28.0 // indirect
golang.org/x/crypto v0.27.0 // indirect
golang.org/x/net v0.29.0 // indirect
golang.org/x/sys v0.25.0 // indirect
golang.org/x/time v0.6.0 // indirect
google.golang.org/genproto v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240827150818-7e3bb234dfed // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
Expand Down
Loading

0 comments on commit 8c27cc1

Please sign in to comment.