Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage: Move benchwrapper directory into 'tests/perf/'. #9246

Merged
merged 4 commits into from
Sep 18, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 0 additions & 21 deletions storage/test_utils/README.md

This file was deleted.

195 changes: 0 additions & 195 deletions storage/test_utils/storage_pb2.py

This file was deleted.

63 changes: 0 additions & 63 deletions storage/test_utils/storage_pb2_grpc.py

This file was deleted.

21 changes: 21 additions & 0 deletions storage/tests/perf/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# storage benchwrapp

main.py is a gRPC wrapper around the storage library for benchmarking purposes.

## Running

```bash
$ export STORAGE_EMULATOR_HOST=http://localhost:8080
$ pip install grpcio
$ cd storage
$ pip install -e . # install google.cloud.storage locally
$ cd tests/perf
$ python3 benchwrapper.py --port 8081
```

## Re-generating protos

```bash
$ pip install grpcio-tools
$ python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. *.proto
```
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,21 @@

parser = argparse.ArgumentParser()

if os.environ.get('STORAGE_EMULATOR_HOST') == None:
sys.exit('This benchmarking server only works when connected to an emulator. Please set STORAGE_EMULATOR_HOST.')
if os.environ.get("STORAGE_EMULATOR_HOST") is None:
sys.exit(
"This benchmarking server only works when connected to an emulator. Please set STORAGE_EMULATOR_HOST."
)

parser.add_argument('--port', help='The port to run on.')
parser.add_argument("--port", help="The port to run on.")

args = parser.parse_args()

if args.port == None:
sys.exit('Usage: python3 main.py --port 8081')
if args.port is None:
sys.exit("Usage: python3 main.py --port 8081")

client = storage.Client()


class StorageBenchWrapperServicer(storage_pb2_grpc.StorageBenchWrapperServicer):
def Write(self, request, context):
# TODO(deklerk): implement this
Expand All @@ -35,14 +38,17 @@ def Read(self, request, context):
blob.download_as_string()
return storage_pb2.EmptyResponse()


server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
storage_pb2_grpc.add_StorageBenchWrapperServicer_to_server(StorageBenchWrapperServicer(), server)
storage_pb2_grpc.add_StorageBenchWrapperServicer_to_server(
StorageBenchWrapperServicer(), server
)

print('listening on localhost:'+args.port)
server.add_insecure_port('[::]:'+args.port)
print("listening on localhost:" + args.port)
server.add_insecure_port("[::]:" + args.port)
server.start()
try:
while True:
time.sleep(_ONE_DAY_IN_SECONDS)
except KeyboardInterrupt:
server.stop(0)
server.stop(0)
File renamed without changes.
Loading