Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

api: header and parameter names. #1900

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5e797e5
Update table path param to be a query param
KyleAMathews Oct 4, 2024
8544646
update elixir code
KyleAMathews Oct 4, 2024
95e87a0
Fix plug tests
msfstef Oct 23, 2024
511cea2
Fix field name for root table
msfstef Oct 23, 2024
c03eab7
Move seconds utility to utilities for cleaner code
msfstef Oct 23, 2024
148194e
Fix TS client tests
msfstef Oct 23, 2024
14a1ed6
Fix react tests
msfstef Oct 23, 2024
6986247
Add changesets
msfstef Oct 23, 2024
ad15684
Fix integration tests
msfstef Oct 23, 2024
8aa53be
Add small wait to check if it reduces flakes
msfstef Oct 23, 2024
e63f5a1
Fix types for examples
msfstef Oct 23, 2024
1e2887d
Slight fixes for docs etc
msfstef Oct 23, 2024
56d3bb9
Only use table instead of root_table in public docs
KyleAMathews Oct 23, 2024
79b5fe9
Update examples
KyleAMathews Oct 23, 2024
e2c2e48
Fix integration test
msfstef Oct 29, 2024
8bf442f
Fix remix type issue
msfstef Oct 29, 2024
a5ed58f
chore: Refactor shape_id -> shape_handle for clearer name
thruflo Oct 29, 2024
b67d29f
chore: Refactor shape_id -> shape_handle for clearer name
KyleAMathews Oct 4, 2024
e94c528
Updates
KyleAMathews Oct 7, 2024
52ff0dd
Fix formatting
KyleAMathews Oct 7, 2024
1f0c0ea
Fix more references to shape id
msfstef Oct 23, 2024
2bb8446
Remove crashing integration test
msfstef Oct 23, 2024
e8f0c60
api: update header names and query params.
thruflo Oct 29, 2024
1219a56
renaming: fix and format.
thruflo Oct 29, 2024
ba325fd
integration-tests: fix header name in crash-recovery.lux
thruflo Oct 29, 2024
02857ab
Update pnpm lcok
msfstef Oct 31, 2024
8261b25
Another full update of pnpm
msfstef Oct 31, 2024
60888b4
Fix types and tests
msfstef Oct 31, 2024
7486f2d
examples: lint changes to auth page/tsx.
thruflo Oct 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/hot-lions-smell.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@electric-sql/client": minor
"@electric-sql/react": minor
---

All `Shape` interfaces (`ShapeStream`, `Shape`, `useShape`) now require `table` as an additional configuration parameter, and the shape API endpoint url only needs to point to `/v1/shape`.
5 changes: 5 additions & 0 deletions .changeset/tender-pens-cheer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@core/sync-service": minor
---

[BREAKING] All shape API endpoints now accept `table` as a query parameter rather than a path parameter, so `/v1/shape/foo?offset=-1` now becomes `/v1/shape?table=foo&offset=-1`.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,5 @@ shape-data.json
test-dbs
tsconfig.tsbuildinfo
wal
shapes
.sst
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ docker compose -f .support/docker-compose.yml up
You can then use the [HTTP API](https://electric-sql.com/docs/api/http) to sync data from your Postgres. For example, to start syncing the whole `foo` table:

```sh
curl -i 'http://localhost:3000/v1/shape/foo?offset=-1'
curl -i 'http://localhost:3000/v1/shape?table=foo&offset=-1'
```

Or use one of the clients or integrations, such as the [`useShape`](https://electric-sql.com/docs/api/integrations/react) React hook:
Expand All @@ -69,7 +69,8 @@ import { useShape } from '@electric-sql/react'

function Component() {
const { data } = useShape({
url: `http://localhost:3000/v1/shape/foo`,
url: `http://localhost:3000/v1/shape`,
table: `foo`,
where: `title LIKE 'foo%'`,
})

Expand Down
11 changes: 5 additions & 6 deletions examples/auth/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,17 @@ const usersShape = (): ShapeStreamOptions => {
const queryParams = new URLSearchParams(window.location.search)
const org_id = queryParams.get(`org_id`)
return {
url: new URL(
`/shape-proxy/users?org_id=${org_id}`,
window.location.origin
).href,
url: new URL(`/shape-proxy?org_id=${org_id}`, window.location.origin)
.href,
table: `users`,
headers: {
Authorization: org_id || ``,
},
}
} else {
return {
url: new URL(`https://not-sure-how-this-works.com/shape-proxy/items`)
.href,
url: new URL(`https://not-sure-how-this-works.com/shape-proxy`).href,
table: `items`,
}
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
export async function GET(
request: Request,
{ params }: { params: { table: string } }
) {
export async function GET(request: Request) {
const url = new URL(request.url)
const { table } = params

// Constuct the upstream URL
const originUrl = new URL(`http://localhost:3000/v1/shape/${table}`)
const originUrl = new URL(`http://localhost:3000/v1/shape`)
url.searchParams.forEach((value, key) => {
originUrl.searchParams.set(key, value)
})
Expand Down
4 changes: 2 additions & 2 deletions examples/bash-client/bash-client.bash
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/bin/bash

# URL to download the JSON file from (without the output parameter)
BASE_URL="http://localhost:3000/v1/shape/todos"
BASE_URL="http://localhost:3000/v1/shape?table=todos"

# Directory to store individual JSON files
OFFSET_DIR="./json_files"
Expand Down Expand Up @@ -78,7 +78,7 @@ process_json() {

# Main loop to poll for updates every second
while true; do
url="$BASE_URL?offset=$LATEST_OFFSET"
url="$BASE_URL&offset=$LATEST_OFFSET"
echo $url

LATEST_OFFSET=$(process_json "$url" "shape-data.json")
Expand Down
3 changes: 2 additions & 1 deletion examples/basic-example/src/Example.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ const baseUrl = import.meta.env.ELECTRIC_URL ?? `http://localhost:3000`

export const Example = () => {
const { data: items } = useShape<Item>({
url: `${baseUrl}/v1/shape/items`,
url: `${baseUrl}/v1/shape`,
table: `items`
})

/*
Expand Down
1 change: 1 addition & 0 deletions examples/linearlite/.env.prod
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
DATABASE_URL=postgresql://neondb_owner:nM9OBhJAr6wv@ep-curly-truth-a43bi79a.us-east-1.aws.neon.tech/neondb?sslmode=require
3 changes: 2 additions & 1 deletion examples/linearlite/src/pages/Issue/Comments.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ export interface CommentsProps {
function Comments(commentProps: CommentsProps) {
const [newCommentBody, setNewCommentBody] = useState<string>(``)
const allComments = useShape({
url: `${baseUrl}/v1/shape/comment`,
url: `${baseUrl}/v1/shape`,
table: `comment`,
})! as Comment[]

const comments = allComments.data.filter(
Expand Down
3 changes: 2 additions & 1 deletion examples/linearlite/src/shapes.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { baseUrl } from './electric'

export const issueShape = {
url: `${baseUrl}/v1/shape/issue`,
url: `${baseUrl}/v1/shape`,
table: `issue`,
}
7 changes: 4 additions & 3 deletions examples/nextjs-example/app/page.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,13 @@ import { matchStream } from "./match-stream"
const itemShape = () => {
if (typeof window !== `undefined`) {
return {
url: new URL(`/shape-proxy/items`, window?.location.origin).href,
url: new URL(`/shape-proxy`, window?.location.origin).href,
table: `items`,
}
} else {
return {
url: new URL(`https://not-sure-how-this-works.com/shape-proxy/items`)
.href,
url: new URL(`https://not-sure-how-this-works.com/shape-proxy`).href,
table: `items`,
}
}
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
export async function GET(
request: Request,
{ params }: { params: { table: string } }
) {
export async function GET(request: Request) {
const url = new URL(request.url)
const { table } = params
const originUrl = new URL(`http://localhost:3000/v1/shape/${table}`)
const originUrl = new URL(`http://localhost:3000/v1/shape`)
url.searchParams.forEach((value, key) => {
originUrl.searchParams.set(key, value)
})
Expand Down
3 changes: 2 additions & 1 deletion examples/redis-sync/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,8 @@ client.connect().then(async () => {
const updateKeyScriptSha1 = await client.SCRIPT_LOAD(script)

const itemsStream = new ShapeStream({
url: `http://localhost:3000/v1/shape/items`,
url: `http://localhost:3000/v1/shape`,
table: `items`,
})
itemsStream.subscribe(async (messages: Message[]) => {
// Begin a Redis transaction
Expand Down
3 changes: 2 additions & 1 deletion examples/remix-basic/app/routes/_index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ import { matchStream } from "../match-stream"

const itemShape = () => {
return {
url: new URL(`/shape-proxy/items`, window.location.origin).href,
url: new URL(`/shape-proxy`, window.location.origin).href,
table: `items`,
}
}

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
import type { LoaderFunctionArgs } from "@remix-run/node"

export async function loader({ params, request }: LoaderFunctionArgs) {
export async function loader({ request }: LoaderFunctionArgs) {
const url = new URL(request.url)
const { table } = params
const originUrl = new URL(`http://localhost:3000/v1/shape/${table}`)
const originUrl = new URL(`http://localhost:3000/v1/shape`)
url.searchParams.forEach((value, key) => {
originUrl.searchParams.set(key, value)
})
Expand Down
13 changes: 7 additions & 6 deletions examples/tanstack-example/src/Example.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@ const baseUrl = import.meta.env.ELECTRIC_URL ?? `http://localhost:3000`
const baseApiUrl = `http://localhost:3001`

const itemShape = () => ({
url: new URL(`/v1/shape/items`, baseUrl).href,
url: new URL(`/v1/shape`, baseUrl).href,
table: `items`
})

async function createItem(newId: string) {
Expand Down Expand Up @@ -43,11 +44,11 @@ async function clearItems(numItems: number) {
const findUpdatePromise =
numItems > 0
? matchStream({
stream: itemsStream,
operations: [`delete`],
// First delete will match
matchFn: () => true,
})
stream: itemsStream,
operations: [`delete`],
// First delete will match
matchFn: () => true,
})
: Promise.resolve()

// Delete all items
Expand Down
3 changes: 2 additions & 1 deletion examples/todo-app/src/routes/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ type ToDo = {

export default function Index() {
const { data: todos } = useShape<ToDo>({
url: `http://localhost:3000/v1/shape/todos`,
url: `http://localhost:3000/v1/shape`,
table: `todos`,
})
todos.sort((a, b) => a.created_at - b.created_at)
console.log({ todos })
Expand Down
12 changes: 6 additions & 6 deletions integration-tests/tests/crash-recovery.lux
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,11 @@
# Initialize a shape and collect the offset
[shell client]
# strip ANSI codes from response for easier matching
!curl -v -X GET http://localhost:3000/v1/shape/items?offset=-1
?electric-shape-id: ([\d-]+)
[local shape_id=$1]
?electric-chunk-last-offset: ([\w\d_]+)
[local last_offset=$1]
!curl -v -X GET "http://localhost:3000/v1/shape?table=items&offset=-1"
?electric-handle: ([\d-]+)
[local handle=$1]
?electric-offset: ([\w\d_]+)
[local offset=$1]

## Terminate electric
[shell electric]
Expand All @@ -58,7 +58,7 @@

# Client should be able to continue same shape
[shell client]
!curl -v -X GET "http://localhost:3000/v1/shape/items?offset=$last_offset&shape_id=$shape_id"
!curl -v -X GET "http://localhost:3000/v1/shape?table=items&handle=$handle&offset=$offset"
??HTTP/1.1 200 OK

[cleanup]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@

## Initialize a couple of shapes so that Electric starts processing transactions from Postgres
[shell client]
!curl -i http://localhost:3000/v1/shape/roots?offset=-1
!curl -i "http://localhost:3000/v1/shape?table=roots&offset=-1"
??200 OK

!curl -i http://localhost:3000/v1/shape/leaves?offset=-1
!curl -i "http://localhost:3000/v1/shape?table=leaves&offset=-1"
??200 OK

## Commit enough new transactions for shape storage to hit the simulated failure.
Expand Down
3 changes: 2 additions & 1 deletion packages/react-hooks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ import { useShape } from "@electric-sql/react"

export default function MyComponent () {
const { isLoading, data } = useShape({
url: "http://my-api.com/shape/foo",
url: "http://my-api.com/shape",
table: `foo`,
})

if (isLoading) {
Expand Down
3 changes: 3 additions & 0 deletions packages/react-hooks/test/react-hooks.test-d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { Row } from 'packages/typescript-client/dist'
describe(`useShape`, () => {
it(`should infer correct return type when no selector is provided`, () => {
const shape = useShape({
table: ``,
url: ``,
})

Expand All @@ -20,6 +21,7 @@ describe(`useShape`, () => {

it(`should infer correct return type when a selector is provided`, () => {
const shape = useShape({
table: ``,
url: ``,
selector: (_value: UseShapeResult) => {
return {
Expand All @@ -36,6 +38,7 @@ describe(`useShape`, () => {

it(`should raise a type error if type argument does not equal inferred return type`, () => {
const shape = useShape<Row, number>({
table: ``,
url: ``,
// @ts-expect-error - should have type mismatch, because doesn't match the declared `Number` type
selector: (_value: UseShapeResult) => {
Expand Down
Loading
Loading