Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #82]Add doc and shell script #81

Merged
merged 1 commit into from
Feb 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ target/
*.iml
*.ipr
*.gz
*.zip


### NetBeans ###
Expand Down
87 changes: 83 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,92 @@
## Local Deploy
* Install RocketMQ 5.0, [quick start](https://rocketmq.apache.org/zh/docs/quickStart/02quickstart);
* Clone and install RocketMQ Streams 1.1.1-SNAPSHOT(develop branch in [RocketMQ Streams](https://github.com/apache/rocketmq-streams) repository)
* Compile antlr4 to source code:
## RSQLDB

The database build for stream processing.


## Overview

RSQLDB is a database for stream processing build on the of RocketMQ. It is distributed, highly available,scalable. SQL can be used to define a stream processing task, and
RSQLDB parse it into a stream-processing task. RSQLDB offers the fellow core features:

- Restful API - create, query, stop, stream-processing tasks;
- Standard SQL - describe the stream processing task with standard sql;
- Materialized views - incremental calculation on the of stream;


## Deploy

### Run RocketMQ 5.0 locally

Steps are as follows:
- **Install Java**

- **Download RocketMQ**

- **Start NameServer**

- **Start Broker**

More details can be obtained at [quick start](https://rocketmq.apache.org/zh/docs/quickStart/02quickstart);


### Run RSQLDB

#### From source code:

- Git clone and compile
```shell
git clone git@github.com:alibaba/rsqldb.git

#compile antlr4 file to source code:
mvn clean compile -DskipTests
```
* Run the entrance method:
```java
com.alibaba.rsqldb.rest.Application
````

#### From distribution package
- Download distribution package
- Unzip package
```shell
unzip rsqldb-distribution.zip
```
- Start rsqldb
```shell
cd rsqldb && sh bin/start.sh
```

## Use Cases and Examples

### Filter

```shell
select *
from sourceTable where age>20 and name like '%mack';
```


### Join

```shell
SELECT Websites.name as `count`, Websites.url as url, SUM(access_log.count) AS nums
FROM access_log
WHERE access_log.`count` > 100
INNER JOIN Websites ON access_log.site_id=Websites.id and access_log.url=Websites.url
```

### Window

```sql
select
TUMBLE_START(ts, INTERVAL '5' SECOND) AS window_start,
TUMBLE_END(ts, INTERVAL '5' SECOND) AS window_end,
position AS position,
sum(num) AS sumNum
from sourceTable
where num > 5
group by TUMBLE(ts, INTERVAL '5' SECOND), position
having sum(num) < 20;
```

- More examples can be found [here](docs/sql_example.md).
48 changes: 48 additions & 0 deletions distribution/bin/start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/bin/sh

binDir=$(cd `dirname $0`;pwd)
echo "binDir=$binDir"

cd $binDir/..
homeDir=$(pwd)
echo "homeDir=$homeDir"
cd $homeDir

confPath=$homeDir/conf/rsqldb.conf
if [ ! -z "$1" ]; then
confPath=$1
fi
echo "confPath=$confPath"

if [ ! -d log ]; then
mkdir log
fi

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${BASE_DIR}/server/*

JVM_CONFIG="-Xms512m -Xmx512m -Xmn128m"

JAVA_OPTIONS=${JAVA_OPTIONS:-}

JVM_OPTS=()
if [ ! -z "${JAVA_OPTIONS}" ]; then
JVM_OPTS+=("${JAVA_OPTIONS}")
fi

if [ ! -z "${JVM_CONFIG}" ]; then
JVM_OPTS+=("${JVM_CONFIG}")
fi

JVM_OPTS="${JVM_OPTS} -cp ${CLASSPATH}"

nohup $JAVA ${JVM_OPTS} -jar $homeDir/server/rsqldb.jar $confPath > /dev/null 2>&1 &

echo "start server success."

16 changes: 16 additions & 0 deletions distribution/bin/stop.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/bash

function_error_exit ()
{
echo "ERROR: $1 !!"
exit 1
}

stop()
{
STR=`ps -ef | grep java | grep -- "rsqldb.jar"`
[ -z "$STR" ] && return
kill `ps -ef | grep java | grep -- "rsqldb.jar" | awk '{print $2}'`
}

stop
8 changes: 7 additions & 1 deletion distribution/distribution.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,20 @@ limitations under the License.
<assembly>
<id>distribution</id>
<formats>
<format>tar.gz</format>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<fileMode>0775</fileMode>
<directory>conf</directory>
<outputDirectory>/conf</outputDirectory>
</fileSet>

<fileSet>
<fileMode>0775</fileMode>
<directory>bin</directory>
<outputDirectory>/bin</outputDirectory>
</fileSet>
</fileSets>
<files>
<file>
Expand Down
2 changes: 1 addition & 1 deletion distribution/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
</goals>
<configuration>
<target>
<copy file="${project.build.directory}/rsqldb-distribution.tar.gz" tofile="${project.basedir}/rsqldb-distribution.tar.gz" overwrite="true" />
<copy file="${project.build.directory}/rsqldb-distribution.zip" tofile="${project.basedir}/rsqldb-distribution.zip" overwrite="true" />
</target>
</configuration>
</execution>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,8 @@ protected RStream<JsonNode> join(RStream<JsonNode> leftStream, RStream<JsonNode>
}

return result;
}).apply((value1, value2) -> {
}).window(WindowBuilder.tumblingWindow(Time.seconds(10)))
.apply((value1, value2) -> {
ObjectNode objectNode = JsonNodeFactory.instance.objectNode();
//新建临时表

Expand Down