Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lambda support #14

Merged
merged 27 commits into from
Feb 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0086bcf
Integrated new email template in cli
rupal-bq Jan 24, 2023
14a0dbb
Update help test
rupal-bq Jan 24, 2023
cfc7aff
Delete temp img in for error case
rupal-bq Jan 24, 2023
991ae6f
use const for emailTemplateImageBuffer
rupal-bq Jan 24, 2023
6594bb5
embed opensearch logo
rupal-bq Jan 25, 2023
0695340
Accept text file for email body and add logic to convert text to html
rupal-bq Jan 26, 2023
546cffd
Increase wait time after dom modification
rupal-bq Jan 26, 2023
114e9e5
Add dockerfile
rupal-bq Jan 27, 2023
d523ef4
Updates required for working with lambda
rupal-bq Jan 31, 2023
e794048
Merge branch 'main' into docker-image
rupal-bq Jan 31, 2023
ade0f50
Change path to fix test failure
rupal-bq Jan 31, 2023
e4c0394
Add ora spinner back
rupal-bq Jan 31, 2023
87d6b53
nit
rupal-bq Jan 31, 2023
5fb125c
Add promise for sending email, Update Dockerfile with tgz install
rupal-bq Feb 1, 2023
f591d34
nit: log
rupal-bq Feb 1, 2023
1df443c
Update doc
rupal-bq Feb 1, 2023
3435a6b
Address PR comments
rupal-bq Feb 1, 2023
ba961d0
Update Dockerfile
rupal-bq Feb 1, 2023
7e211ad
nit: remove ls
rupal-bq Feb 1, 2023
6fa7429
Move Dockerfile to example doc
rupal-bq Feb 1, 2023
4ce55be
Add Dockerfile for building from source
rupal-bq Feb 1, 2023
df689dd
Nit: remove ls from Dockerfile
rupal-bq Feb 1, 2023
14609f7
Adding apt-get remove -y google-chrome-stable
rupal-bq Feb 1, 2023
e172c7a
Merge branch 'main' into docker-image
rupal-bq Feb 2, 2023
dcd6b40
Update example docs
rupal-bq Feb 2, 2023
94687e2
nit
rupal-bq Feb 2, 2023
be08114
Update docs/examples/lambda_container_image.md
rupal-bq Feb 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ The node version "^12.20.0 || >=14" is required.
2. Run `yarn` inside `reporting-cli/src`
3. You can run the below commands inside `reporting-cli/src`
```
node index.js --url <url>
node cli.js --url <url>
```
For additional command line parameter options
```
node index.js -h
node cli.js -h
```
4. Alternatively, you can use npm install to run this command from any directory.
```
Expand All @@ -31,7 +31,7 @@ The node version "^12.20.0 || >=14" is required.

To uninstall, use
```
npm uninstall -g opensearch-reporting-cli
npm uninstall -g @opensearch-project/reporting-cli
```

### Running Tests
Expand All @@ -48,4 +48,4 @@ curl -XPOST -u 'admin:admin' 'http://localhost:5601/api/sample_data/ecommerce' -
curl -XPOST -u 'admin:admin' 'http://localhost:5601/api/sample_data/logs' -H 'osd-xsrf:true' -H 'securitytenant: global'
curl -XPOST -u 'admin:admin' 'http://localhost:5601/api/sample_data/flights' -H 'osd-xsrf:true' -H 'securitytenant: admin_tenant'
```
Run `yarn test` inside `reporting-cli`.
Run `yarn test` inside `reporting-cli`.
85 changes: 85 additions & 0 deletions docs/examples/lambda_container_image.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
# Scheduling Reports with Lambda

Opensearch reporting CLI can be used with Lambda for scheduling email reports.

### Prerequisites

- AMD64 Systems
- Docker

### Creating Lambda container image

1. Create a Dockerfile. Following Dockerfile can be used to create a container image for v1.0.0.
```dockerfile
# Define function directory
ARG FUNCTION_DIR="/function"

# Base image of the docker container
FROM node:lts-slim as build-image

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# AWS Lambda runtime dependencies
RUN apt-get update && \
apt-get install -y \
g++ \
make \
unzip \
libcurl4-openssl-dev \
autoconf \
automake \
libtool \
cmake \
python3 \
libkrb5-dev \
curl

# Copy function code
WORKDIR ${FUNCTION_DIR}
RUN curl -LJO https://artifacts.opensearch.org/reporting-cli/opensearch-reporting-cli-1.0.0.tgz
RUN tar -xzf opensearch-reporting-cli-1.0.0.tgz
RUN mv package/* .
RUN npm install && npm install aws-lambda-ric

# Build Stage 2: Copy Build Stage 1 files in to Stage 2. Install chromium dependencies and chromium.
FROM node:lts-slim
# Include global arg in this stage of the build
ARG FUNCTION_DIR
# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}
# Copy in the build image dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

# Install latest chrome dev package and fonts to support major charsets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer installs, work.
RUN apt-get update \
&& apt-get install -y wget gnupg \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
--no-install-recommends \
&& apt-get remove -y google-chrome-stable \
&& rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["/usr/local/bin/npx", "aws-lambda-ric"]

ENV HOME="/tmp"
CMD [ "/function/src/index.handler" ]
```

2. Run the build command in from the directory where Dockerfile exists.

```
docker build -t opensearch-reporting-cli .
joshuali925 marked this conversation as resolved.
Show resolved Hide resolved
```
### Use Lambda container image

1. Push the Docker image to [Amazon ECR](https://docs.aws.amazon.com/AmazonECR/latest/userguide/getting-started-console.html)

2. Create a Lambda Function. Select previously created container image and architecture x86_64'.

3. Update Lambda timeout to 5 min and memory size to at least 1024MB.


71 changes: 71 additions & 0 deletions docs/examples/lambda_container_image_from_source.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
## Create Lambda container image from source code

1. Checkout this package from version control
```
git clone git@github.com:opensearch-project/reporting-cli.git
cd reporting-cli
```
2. Add the following Dockerfile inside `reporting-cli`.
```dockerfile
# Define function directory
ARG FUNCTION_DIR="/function"

# Base image of the docker container
FROM node:lts-slim as build-image

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# AWS Lambda runtime dependencies
RUN apt-get update && \
apt-get install -y \
g++ \
make \
unzip \
libcurl4-openssl-dev \
autoconf \
automake \
libtool \
cmake \
python3 \
libkrb5-dev

# Copy function code
COPY package.json src/ ${FUNCTION_DIR}/
WORKDIR ${FUNCTION_DIR}
RUN npm install && npm install aws-lambda-ric

# Build Stage 2: Copy Build Stage 1 files in to Stage 2. Install chromium dependencies and chromium.
FROM node:lts-slim

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}

# Copy in the build image dependencies
COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR}

# Install latest chrome dev package and fonts to support major charsets (Chinese, Japanese, Arabic, Hebrew, Thai and a few others)
# Note: this installs the necessary libs to make the bundled version of Chromium that Puppeteer installs, work.
RUN apt-get update \
&& apt-get install -y wget gnupg \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
--no-install-recommends \
&& apt-get remove -y google-chrome-stable \
&& rm -rf /var/lib/apt/lists/*

ENTRYPOINT ["/usr/local/bin/npx", "aws-lambda-ric"]

ENV HOME="/tmp"
CMD [ "/function/index.handler" ]

```
3. Run the build command.
```
docker build -t opensearch-reporting-cli .
```
7 changes: 3 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
{
"name": "@opensearch-project/reporting-cli",
"description": "Opensearch Reporting CLI to download and email OpenSearch Dashboards reports.",
"main": "index.js",
"type": "module",
"main": "./src/index.js",
"homepage": "https://www.opensearch.org/",
"version": "1.0.0",
"keywords": [
Expand Down Expand Up @@ -33,11 +32,11 @@
"lodash": "^4.17.21",
"nodemailer": "^6.8.0",
"nodemailer-express-handlebars": "^5.0.0",
"ora": "^6.1.2",
"ora": "^5.4.1",
"puppeteer": "^18.2.0"
},
"bin": {
"opensearch-reporting-cli": "./src/index.js"
"opensearch-reporting-cli": "./src/cli.js"
},
"devDependencies": {
"jest": "^29.3.1"
Expand Down
51 changes: 40 additions & 11 deletions src/arguments.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,16 @@
* SPDX-License-Identifier: Apache-2.0
*/

import { program, Option } from 'commander';
import { exit } from 'process';
import ora from 'ora';
import fs from 'fs';
import { AUTH, CLI_COMMAND_NAME, DEFAULT_AUTH, DEFAULT_FILENAME, DEFAULT_FORMAT, DEFAULT_MIN_HEIGHT, DEFAULT_TENANT, DEFAULT_WIDTH, ENV_VAR, FORMAT, TRANSPORT_TYPE, DEFAULT_EMAIL_SUBJECT, DEFAULT_EMAIL_NOTE } from './constants.js';
import dotenv from "dotenv";
const { program, Option } = require('commander');
const { exit } = require('process');
const fs = require('fs');
const ora = require('ora');
const { AUTH, CLI_COMMAND_NAME, DEFAULT_AUTH, DEFAULT_FILENAME, DEFAULT_FORMAT, DEFAULT_MIN_HEIGHT, DEFAULT_TENANT, DEFAULT_WIDTH, ENV_VAR, FORMAT, TRANSPORT_TYPE, DEFAULT_EMAIL_SUBJECT, DEFAULT_EMAIL_NOTE } = require('./constants.js');
const dotenv = require("dotenv");
dotenv.config();
const spinner = ora('');

const spinner = ora();

export async function getCommandArguments() {
async function getCommandArguments() {

program
.name(CLI_COMMAND_NAME)
Expand Down Expand Up @@ -73,6 +72,27 @@ Note: The tenant in the url has the higher priority than tenant value provided a
return getOptions(options);
}

async function getEventArguments(event) {
if (event.auth === undefined)
event['auth'] = DEFAULT_AUTH;
if (event.tenant === undefined)
event['tenant'] = DEFAULT_TENANT;
if (event.format === undefined)
event['format'] = DEFAULT_FORMAT;
if (event.width === undefined)
event['width'] = DEFAULT_WIDTH;
if (event.height === undefined)
event['height'] = DEFAULT_MIN_HEIGHT;
if (event.filename === undefined)
event['filename'] = DEFAULT_FILENAME;
if (event.subject === undefined)
event['subject'] = DEFAULT_EMAIL_SUBJECT;
if (event.note === undefined)
event['note'] = DEFAULT_EMAIL_NOTE;

return getOptions(event);
}

function getOptions(options) {
var commandOptions = {
url: null,
Expand All @@ -94,7 +114,8 @@ function getOptions(options) {
smtppassword: null,
subject: null,
time: null,
note: null
note: null,
emailbody: null
}

// Set url.
Expand Down Expand Up @@ -152,6 +173,9 @@ function getOptions(options) {
? `${commandOptions.filename}-${commandOptions.time.toISOString().replace(/:/g, '-')}.${commandOptions.format}`
: `${commandOptions.filename}.${commandOptions.format}`;

// Set name for email body report image
commandOptions.emailbody = `email-body-${commandOptions.time.toISOString().replace(/:/g, '-')}.png`

// Set width and height of the window
commandOptions.width = Number(options.width);
commandOptions.height = Number(options.height);
Expand Down Expand Up @@ -180,7 +204,7 @@ function getOptions(options) {
}
commandOptions.note = getHtml(commandOptions.note);

spinner.succeed('Fetched argument values')
spinner.succeed('Fetched argument values');
return commandOptions;
}

Expand All @@ -194,4 +218,9 @@ function getHtml(text) {
.replace(/\t/g, " ")
.replace(/ /g, "&#8203;&nbsp;&#8203;")
.replace(/\r\n|\r|\n/g, "<br />");
}

module.exports = {
getCommandArguments,
getEventArguments
}
9 changes: 9 additions & 0 deletions src/cli.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/usr/bin/env node
/*
* Copyright OpenSearch Contributors
* SPDX-License-Identifier: Apache-2.0
*/

const run = require('./run.js');

run(undefined);
Loading