Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport][2.x] [Dev] enable opensearch snapshot for Darwin (#3537) #3548

Merged
merged 2 commits into from
Mar 8, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- [Data] Make the newly created configurations get added to beginning of the `aggConfig` array when using `createAggConfig` ([#3160](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3160))
- [Optimizer] Increase the amount of time an optimizer worker is provided to exit before throwing an error ([#3193](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3193))
- [Multiple DataSource] Add support for SigV4 authentication ([#3058](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3058))
- Make build scripts find and use the latest version of Node.js that satisfies `engines.node` ([#3467](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3467))
- [Multiple DataSource] Refactor test connection to support SigV4 auth type ([#3456](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/3456))
- [Darwin] Add support for Darwin for running OpenSearch snapshots with `yarn opensearch snapshot` ([#3537](https://github.com/opensearch-project/OpenSearch-Dashboards/pull/3537))

### 🐛 Bug Fixes

Expand Down
6 changes: 3 additions & 3 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ $ yarn osd clean

OpenSearch Dashboards requires a running version of OpenSearch to connect to. In a separate terminal you can run the latest snapshot built using:

_(Linux and Windows only - for MacOS, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
_(Linux, Windows, Darwin (MacOS) only - for others, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
```bash
$ yarn opensearch snapshot
```
Expand Down Expand Up @@ -184,7 +184,7 @@ By default, the snapshot command will run [a minimal distribution of OpenSearch]

If you would like to run OpenSearch with a particular plugin installed on the cluster snapshot, pass the `--P` flag after `yarn opensearch snapshot`. You can use the flag multiple times to install multiple plugins. The argument value can be a URL to the plugin's zip file, maven coordinates of the plugin, or a local zip file path (use `file://` followed by the absolute or relative path, in that case). For example:

_(Linux and Windows only - for MacOS, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
_(Linux, Windows, Darwin (MacOS) only - for others, you'll need to [run OpenSearch from a tarball](#alternative---run-opensearch-from-tarball) instead)_
```bash
$ yarn opensearch snapshot --P https://repo1.maven.org/maven2/org/opensearch/plugin/opensearch-test-plugin/2.4.0.0/opensearch-test-plugin-2.4.0.0.zip
```
Expand Down Expand Up @@ -214,7 +214,7 @@ $ yarn opensearch snapshot --version 2.2.0 -E cluster.name=test -E path.data=/tm

### Alternative - Run OpenSearch from tarball

OpenSearch does not yet create artifacts for MacOS, so you'll need to download, install, and run the tarball instead. (You'll also need Java installed and the `JAVA_HOME` environmental variable set - see [OpenSearch developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#install-prerequisites) for details).
If you would like to run OpenSearch from the tarball, you'll need to download the minimal distribution, install it, and then run the executable. (You'll also need Java installed and the `JAVA_HOME` environmental variable set - see [OpenSearch developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#install-prerequisites) for details).

1. Download the latest minimal distribution of OpenSearch from [the downloads page](https://opensearch.org/downloads.html#minimal). Note the version and replace in commands below.
2. Unzip the `tar.gz` file: `tar -xvf opensearch-<OpenSearch-version>-linux-x64.tar.gz`
Expand Down
5 changes: 3 additions & 2 deletions packages/osd-opensearch/src/artifact.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ const { createHash } = require('crypto');
const path = require('path');

const asyncPipeline = promisify(pipeline);
const SUPPORTED_PLATFORMS = ['linux', 'windows', 'darwin'];
const DAILY_SNAPSHOTS_BASE_URL = 'https://artifacts.opensearch.org/snapshots/core/opensearch';
// TODO: [RENAMEME] currently do not have an existing replacement
// issue: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/475
Expand Down Expand Up @@ -188,8 +189,8 @@ async function getArtifactSpecForSnapshotFromUrl(urlVersion, log) {
const arch = process.arch === 'arm64' ? 'arm64' : 'x64';
const extension = process.platform === 'win32' ? 'zip' : 'tar.gz';

if (platform !== 'linux' && platform !== 'windows') {
throw createCliError(`Snapshots are only available for Linux and Windows`);
if (!SUPPORTED_PLATFORMS.includes(platform)) {
throw createCliError(`Snapshots are only available for Linux, Windows, and Darwin`);
}

const latestUrl = `${DAILY_SNAPSHOTS_BASE_URL}/${desiredVersion}-SNAPSHOT`;
Expand Down
42 changes: 39 additions & 3 deletions packages/osd-opensearch/src/artifact.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -163,17 +163,17 @@ describe('Artifact', () => {
});
});

it('should throw when on a non-Linux or non-Windows platform', async () => {
it('should throw when on a non-Linux, non-Windows, non-Darwin platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'darwin',
value: 'android',
},
arch: {
value: ORIGINAL_ARCHITECTURE,
},
});
await expect(Artifact.getSnapshot('default', 'INVALID_PLATFORM', log)).rejects.toThrow(
'Snapshots are only available for Linux'
'Snapshots are only available for Linux, Windows, and Darwin'
);
});

Expand All @@ -189,6 +189,42 @@ describe('Artifact', () => {
mockFetch(MOCKS.multipleArch[0]);
artifactTest();
});

it('should not throw when on a Linux platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'linux',
},
arch: {
value: 'x64',
},
});
artifactTest();
});

it('should not throw when on a Windows platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'win32',
},
arch: {
value: 'x64',
},
});
artifactTest();
});

it('should not throw when on a Darwin platform', async () => {
Object.defineProperties(process, {
platform: {
value: 'darwin',
},
arch: {
value: 'x64',
},
});
artifactTest();
});
});

describe('with custom snapshot manifest URL', () => {
Expand Down