sidebar_label | sidebar_position | slug |
---|---|---|
How to Setup Object Storage |
4 |
/how_to_setup_object_storage |
As you can learn from JuiceFS Technical Architecture, JuiceFS is a distributed file system with data and metadata stored separately. JuiceFS uses object storage as the main data storage and uses databases such as Redis, PostgreSQL and MySQL as metadata storage.
When creating a JuiceFS file system, there are following options to set up the storage:
--storage
: Specify the type of storage to be used by the file system, e.g.--storage s3
--bucket
: Specify the storage access address, e.g.--bucket https://myjuicefs.s3.us-east-2.amazonaws.com
--access-key
and--secret-key
: Specify the authentication information when accessing the storage
For example, the following command uses Amazon S3 object storage to create a file system:
$ juicefs format --storage s3 \
--bucket https://myjuicefs.s3.us-east-2.amazonaws.com \
--access-key abcdefghijklmn \
--secret-key nmlkjihgfedAcBdEfg \
redis://192.168.1.6/1 \
myjfs
Some query string parameters are available for bucket URLs, which can be appended to the option --bucket
of the commands format
and mount
. For example, to skip certificate verification for https requests, you can append tls-insecure-skip-verify
parameter to a bucket URL like https://myjuicefs.s3.us-east-2.amazonaws.com/?tls-insecure-skip-verify=true
In general, object storages are authenticated with Access Key ID
and Access Key Secret
. For JuiceFS file system, they are provided by options --access-key
and --secret-key
(or AK, SK for short).
It is more secure to pass credentials via environment variables ACCESS_KEY
and SECRET_KEY
instead of explicitly specifying the options --access-key
and --secret-key
in the command line when creating a filesystem, e.g.,
$ export ACCESS_KEY=abcdefghijklmn
$ export SECRET_KEY=nmlkjihgfedAcBdEfg
$ juicefs format --storage s3 \
--bucket https://myjuicefs.s3.us-east-2.amazonaws.com \
redis://192.168.1.6/1 \
myjfs
Public clouds typically allow users to create IAM (Identity and Access Management) roles, such as AWS IAM role or Alibaba Cloud RAM role, which can be assigned to VM instances. If the cloud server instance already has read and write access to the object storage, there is no need to specify --access-key
and --secret-key
.
Typically, object storage services provide a unified URL for access, but the cloud platform usually provides both internal and external endpoints. For example, the platform cloud services that meet the criteria will automatically resolve requests to the internal endpoint of the object storage. This offers you a lower latency, and internal network traffic is free.
Some cloud computing platforms also distinguish between internal and public networks, but instead of providing a unified access URL, they provide separate internal Endpoint and public Endpoint addresses.
JuiceFS also provides flexible support for this object storage service that distinguishes between internal and public addresses. For scenarios where the same file system is shared, the object storage is accessed through internal Endpoint on the servers that meet the criteria, and other computers are accessed through public Endpoint, which can be used as follows:
- When creating a file system: It is recommended to use internal Endpoint address for
--bucket
- When mounting a file system: For clients that do not satisfy the internal line, you can specify a public Endpoint address to
--bucket
.
Creating a file system using an internal Endpoint ensures better performance and lower latency, and for clients that cannot be accessed through an internal address, you can specify a public Endpoint to mount with the option --bucket
.
Object storage usually supports multiple storage classes, such as standard storage, infrequent access storage, and archive storage. When creating an object storage bucket you can choose an appropriate storage class according to your actual needs, or automatically convert the storage class of existing objects through lifecycle management. Storage classes that support real-time access to data (e.g. standard storage and infrequent access storage) can be used as the underlying JuiceFS data store, while those that require thawing for access in advance (e.g. archive storage) cannot.
:::note When using certain storage classes (such as infrequent access), there are minimum bill units, and additional charges may be incurred for reading data. Please refer to the user manual of the object storage you are using for details. :::
If the network environment where the client is located is affected by firewall policies or other factors that require access to external object storage services through a proxy, the corresponding proxy settings are different for different operating systems. Please refer to the corresponding user manual for settings.
On Linux, for example, the proxy can be set by creating http_proxy
and https_proxy
environment variables.
$ export http_proxy=http://localhost:8035/
$ export https_proxy=http://localhost:8035/
$ juicefs format \
--storage s3 \
... \
myjfs
If you wish to use a storage system that is not listed, feel free to submit a requirement issue.
Name | Value |
---|---|
Amazon S3 | s3 |
Google Cloud Storage | gs |
Azure Blob Storage | wasb |
Backblaze B2 | b2 |
IBM Cloud Object Storage | ibmcos |
Oracle Cloud Object Storage | s3 |
Scaleway Object Storage | scw |
DigitalOcean Spaces | space |
Wasabi | wasabi |
Storj DCS | s3 |
Vultr Object Storage | s3 |
Alibaba Cloud OSS | oss |
Tencent Cloud COS | cos |
Huawei Cloud OBS | obs |
Baidu Object Storage | bos |
Kingsoft KS3 | ks3 |
NetEase Object Storage | nos |
QingStor | qingstor |
Qiniu Object Storage | qiniu |
Sina Cloud Storage | scs |
CTYun OOS | oos |
ECloud Object Storage | eos |
UCloud US3 | ufile |
Ceph RADOS | ceph |
Ceph RGW | s3 |
Swift | swift |
MinIO | minio |
WebDAV | webdav |
HDFS | hdfs |
Apache Ozone | s3 |
Redis | redis |
TiKV | tikv |
Local disk | file |
S3 supports two styles of endpoint URI: virtual hosted-style and path-style. The difference is:
- Virtual-hosted-style:
https://<bucket>.s3.<region>.amazonaws.com
- Path-style:
https://s3.<region>.amazonaws.com/<bucket>
The <region>
should be replaced with specific region code, e.g. the region code of US East (N. Virginia) is us-east-1
. All the available region codes can be found here.
:::note
For AWS users in China, you need add .cn
to the host, i.e. amazonaws.com.cn
, and check this document for region code.
:::
:::note
If the S3 bucket has public access (anonymous access is supported), please set --access-key
to anonymous
.
:::
Versions prior to JuiceFS v0.12 only support the virtual hosting type, v0.12 and later versions support both styles. For example,
# virtual hosted-style
$ juicefs format \
--storage s3 \
--bucket https://<bucket>.s3.<region>.amazonaws.com \
... \
myjfs
# path-style
$ juicefs format \
--storage s3 \
--bucket https://s3.<region>.amazonaws.com/<bucket> \
... \
myjfs
You can also set --storage
to s3
to connect to S3-compatible object storage, e.g.
# virtual hosted-style
$ juicefs format \
--storage s3 \
--bucket https://<bucket>.<endpoint> \
... \
myjfs
# path-style
$ juicefs format \
--storage s3 \
--bucket https://<endpoint>/<bucket> \
... \
myjfs
:::tip
The format of the option --bucket
for all S3 compatible object storage services is https://<bucket>.<endpoint>
or https://<endpoint>/<bucket>
. The default region
is us-east-1
. When a different region
is required, it can be set manually via the environment variable AWS_REGION
or AWS_DEFAULT_REGION
.
:::
Google Cloud uses IAM to manage permissions for accessing resources. Through authorizing service accounts, you can have a fine-grained control of the access rights of cloud servers and object storage.
For cloud servers and object storage that belong to the same service account, as long as the account grants access to the relevant resources, there is no need to provide authentication information when creating a JuiceFS file system, and the cloud platform will automatically complete authentication.
For cases where you want to access the object storage from outside the Google Cloud Platform, for example, to create a JuiceFS file system on your local computer using Google Cloud Storage, you need to configure authentication information. Since Google Cloud Storage does not use Access Key ID
and Access Key Secret
, but rather the JSON key file
of the service account to authenticate the identity.
Please refer to "Authentication as a service account" to create JSON key file
for the service account and download it to the local computer, and define the path to the key file via the environment variable GOOGLE_APPLICATION_ CREDENTIALS
, e.g.
export GOOGLE_APPLICATION_CREDENTIALS="$HOME/service-account-file.json"
You can write the command to create environment variables to ~/.bashrc
or ~/.profile
and have the shell set it automatically every time you start.
Once you have configured the environment variables for passing key information, the commands to create a file system locally and on Google Cloud Server are identical. For example,
$ juicefs format \
--storage gs \
--bucket <bucket> \
... \
myjfs
As you can see, there is no need to include authentication information in the command, and the client will authenticate the access to the object storage through the JSON key file set in the previous environment variable. Also, since the bucket name is globally unique, when creating a file system, you only need to specify the bucket name in the option --bucket
.
In addition to providing authorization information through the options --access-key
and --secret-key
, you could also create a connection string and set the environment variable AZURE_STORAGE_CONNECTION_STRING
. For example:
# Use connection string
$ export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;AccountName=XXX;AccountKey=XXX;EndpointSuffix=core.windows.net"
$ juicefs format \
--storage wasb \
--bucket https://<container> \
... \
myjfs
:::note
For Azure users in China, the value of EndpointSuffix
is core.chinacloudapi.cn
.
:::
To use Backblaze B2 as a data storage for JuiceFS, you need to create application key first. Application Key ID and Application Key corresponds to Access key
and Secret key
, respectively.
Backblaze B2 supports two access interfaces: the B2 native API and the S3-compatible API.
The storage type should be set to b2
, and only the bucket name needs to be set in the option --bucket
. For example:
$ juicefs format \
--storage b2 \
--bucket <bucket> \
--access-key <application-key-ID> \
--secret-key <application-key> \
... \
myjfs
The storage type should be set to s3
, and the full bucket address in the option bucket
needs to be specified. For example:
$ juicefs format \
--storage s3 \
--bucket https://s3.eu-central-003.backblazeb2.com/<bucket> \
--access-key <application-key-ID> \
--secret-key <application-key> \
... \
myjfs
When creating JuiceFS file system using IBM Cloud Object Storage, you first need to create an API key and an instance ID. The "API key" and "instance ID" are the equivalent of access key and secret key, respectively.
IBM Cloud Object Storage provides multiple endpoints for each region, depending on your network (e.g. public or private). Thus, please choose an appropriate endpoint. For example:
$ juicefs format \
--storage ibmcos \
--bucket https://<bucket>.<endpoint> \
--access-key <API-key> \
--secret-key <instance-ID> \
... \
myjfs
Oracle Cloud Object Storage supports S3 compatible access. Please refer to official documentation for more information.
The endpoint
format for this object storage is: ${namespace}.compat.objectstorage.${region}.oraclecloud.com
, for example:
$ juicefs format \
--storage s3 \
--bucket https://<bucket>.<endpoint> \
--access-key <your-access-key> \
--secret-key <your-sceret-key> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.s3.<region>.scw.cloud
. Remember to replace <region>
with specific region code, e.g. the region code of "Amsterdam, The Netherlands" is nl-ams
. All available region codes can be found here. For example:
$ juicefs format \
--storage scw \
--bucket https://<bucket>.s3.<region>.scw.cloud \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<space-name>.<region>.digitaloceanspaces.com
. Please replace <region>
with specific region code, e.g. nyc3
. All available region codes can be found here. For example:
$ juicefs format \
--storage space \
--bucket https://<space-name>.<region>.digitaloceanspaces.com \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.s3.<region>.wasabisys.com
, replace <region>
with specific region code, e.g. the region code of US East 1 (N. Virginia) is us-east-1
. All available region codes can be found here. For example:
$ juicefs format \
--storage wasabi \
--bucket https://<bucket>.s3.<region>.wasabisys.com \
... \
myjfs
:::note For users in Tokyo (ap-northeast-1) region, please refer to this document to learn how to get appropriate endpoint URI.*** :::
Please refer to this document to learn how to create access key and secret key.
Storj DCS is an S3-compatible storage, using s3
for option --storage
. The setting format of the option --bucket
is https://gateway.<region>.storjshare.io/<bucket>
, and please replace <region>
with the corresponding region code you need. There are currently three available regions: us1
, ap1
and eu1
. For example:
$ juicefs format \
--storage s3 \
--bucket https://gateway.<region>.storjshare.io/<bucket> \
--access-key <your-access-key> \
--secret-key <your-sceret-key> \
... \
myjfs
Vultr Object Storage is an S3-compatible storage, using s3
for --storage
option. The format of the option --bucket
is https://<bucket>.<region>.vultrobjects.com/
. For example:
$ juicefs format \
--storage s3 \
--bucket https://<bucket>.ewr1.vultrobjects.com/ \
--access-key <your-access-key> \
--secret-key <your-sceret-key> \
... \
myjfs
Please find the access and secret keys for object storage in the customer portal.
Please follow this document to learn how to get access key and secret key. If you have already created RAM role and assigned it to a VM instance, you could omit the options --access-key
and --secret-key
.
Alibaba Cloud also supports using Security Token Service (STS) to authorize temporary access to OSS. If you wanna use STS, you should omit the options --access-key
and --secret-key
and set environment variables ALICLOUD_ACCESS_KEY_ID
, ALICLOUD_ACCESS_KEY_SECRET
and SECURITY_TOKEN
instead, for example:
# Use Security Token Service (STS)
$ export ALICLOUD_ACCESS_KEY_ID=XXX
$ export ALICLOUD_ACCESS_KEY_SECRET=XXX
$ export SECURITY_TOKEN=XXX
$ juicefs format \
--storage oss \
--bucket https://<bucket>.<endpoint> \
... \
myjfs
OSS provides multiple endpoints for each region, depending on your network (e.g. public or internal network). Please choose an appropriate endpoint.
If you are creating a file system on AliCloud's server, you can specify the bucket name directly in the option --bucket
. For example.
# Running within Alibaba Cloud
$ juicefs format \
--storage oss \
--bucket <bucket> \
... \
myjfs
The naming rule of bucket in Tencent Cloud is <bucket>-<APPID>
, so you must append APPID
to the bucket name. Please follow this document to learn how to get APPID
.
The full format of --bucket
option is https://<bucket>-<APPID>.cos.<region>.myqcloud.com
, and please replace <region>
with specific region code. E.g. the region code of Shanghai is ap-shanghai
. You could find all available region codes here. For example:
$ juicefs format \
--storage cos \
--bucket https://<bucket>-<APPID>.cos.<region>.myqcloud.com \
... \
myjfs
If you are creating a file system on Tencent Cloud's server, you can specify the bucket name directly in the option --bucket
. For example.
# Running within Tencent Cloud
$ juicefs format \
--storage cos \
--bucket <bucket>-<APPID> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.obs.<region>.myhuaweicloud.com
, and please replace <region>
with specific region code. E.g. the region code of Beijing 1 is cn-north-1
. You could find all available region codes here. For example:
$ juicefs format \
--storage obs \
--bucket https://<bucket>.obs.<region>.myhuaweicloud.com \
... \
myjfs
If you are creating a file system on Huawei Cloud's server, you can specify the bucket name directly in the option --bucket
. For example,
# Running within Huawei Cloud
$ juicefs format \
--storage obs \
--bucket <bucket> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.<region>.bcebos.com
, and please replace <region>
with specific region code. E.g. the region code of Beijing is bj
. You could find all available region codes here. For example:
$ juicefs format \
--storage bos \
--bucket https://<bucket>.<region>.bcebos.com \
... \
myjfs
If you are creating a file system on Baidu Cloud's server, you can specify the bucket name directly in the option --bucket
. For example,
# Running within Baidu Cloud
$ juicefs format \
--storage bos \
--bucket <bucket> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
KS3 provides multiple endpoints for each region, depending on your network (e.g. public or internal). Please choose an appropriate endpoint. For example:
$ juicefs format \
--storage ks3 \
--bucket https://<bucket>.<endpoint> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.<endpoint>
, and please replace <endpoint>
with specific value, e.g. mtmss.com
. You could find all available endpoints here. For example:
$ juicefs format \
--storage mss \
--bucket https://<bucket>.<endpoint> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
NOS provides multiple endpoints for each region, depending on your network (e.g. public or internal). Please choose an appropriate endpoint. For example:
$ juicefs format \
--storage nos \
--bucket https://<bucket>.<endpoint> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.<region>.qingstor.com
, replace <region>
with specific region code. E.g. the region code of Beijing 3-A is pek3a
. You could find all available region codes here. For example:
$ juicefs format \
--storage qingstor \
--bucket https://<bucket>.<region>.qingstor.com \
... \
myjfs
:::note
The format of --bucket
option for all QingStor compatible object storage services is http://<bucket>.<endpoint>
.
:::
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.s3-<region>.qiniucs.com
, replace <region>
with specific region code. E.g. the region code of China East is cn-east-1
. You could find all available region codes here. For example:
$ juicefs format \
--storage qiniu \
--bucket https://<bucket>.s3-<region>.qiniucs.com \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.stor.sinaapp.com
. For example:
$ juicefs format \
--storage scs \
--bucket https://<bucket>.stor.sinaapp.com \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
The --bucket
option format is https://<bucket>.oss-<region>.ctyunapi.cn
, replace <region>
with specific region code. E.g. the region code of Chengdu is sccd
. You could find all available region codes here. For example:
$ juicefs format \
--storage oos \
--bucket https://<bucket>.oss-<region>.ctyunapi.cn \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
ECloud Object Storage provides multiple endpoints for each region, depending on your network (e.g. public or internal). Please choose an appropriate endpoint. For example:
$ juicefs format \
--storage eos \
--bucket https://<bucket>.<endpoint> \
... \
myjfs
Please follow this document to learn how to get access key and secret key.
US3 (formerly UFile) provides multiple endpoints for each region, depending on your network (e.g. public or internal). Please choose an appropriate endpoint. For example:
$ juicefs format \
--storage ufile \
--bucket https://<bucket>.<endpoint> \
... \
myjfs
:::note The minimum version of Ceph supported by JuiceFS is Luminous (v12.2.*). Please make sure your version of Ceph meets the requirements. :::
The Ceph Storage Cluster has a messaging layer protocol that enables clients to interact with a Ceph Monitor and a Ceph OSD Daemon. The librados
API enables you to interact with the two types of daemons:
- The Ceph Monitor, which maintains a master copy of the cluster map.
- The Ceph OSD Daemon (OSD), which stores data as objects on a storage node.
JuiceFS supports the use of native Ceph APIs based on librados
. You need to install librados
library and build juicefs
binary separately.
First, install librados
:
:::note
It is recommended to use librados
that matches your Ceph version. For example, if Ceph version is Octopus (v15.2.*), then it is recommended to use librados
v15.2.*. Some Linux distributions (e.g. CentOS 7) may come with a lower version of librados
, so if you fail to compile JuiceFS, try to download a higher version of the package.
:::
# Debian based system
$ sudo apt-get install librados-dev
# RPM based system
$ sudo yum install librados2-devel
Then compile JuiceFS for Ceph (make sure you have Go 1.17+ and GCC 5.4+ installed):
$ make juicefs.ceph
The --bucket
option format is ceph://<pool-name>
. A pool is logical partition for storing objects. You may need first creating a pool. The value of --access-key
option is Ceph cluster name, the default cluster name is ceph
. The value of --secret-key
option is Ceph client user name, the default user name is client.admin
.
For connecting to Ceph Monitor, librados
reads Ceph configuration file by searching default locations and the first found will be used. The locations are:
CEPH_CONF
environment variable/etc/ceph/ceph.conf
~/.ceph/config
ceph.conf
in the current working directory
The example command is:
$ juicefs.ceph format \
--storage ceph \
--bucket ceph://<pool-name> \
--access-key <cluster-name> \
--secret-key <user-name> \
... \
myjfs
Ceph Object Gateway is an object storage interface built on top of librados
to provide applications with a RESTful gateway to Ceph Storage Clusters. Ceph Object Gateway supports S3-compatible interface, so we could set --storage
to s3
directly.
The --bucket
option format is http://<bucket>.<endpoint>
(virtual hosted-style). For example:
$ juicefs format \
--storage s3 \
--bucket http://<bucket>.<endpoint> \
... \
myjfs
OpenStack Swift is a distributed object storage system designed to scale from a single machine to thousands of servers. Swift is optimized for multi-tenancy and high concurrency. Swift is ideal for backups, web and mobile content, and any other unstructured data that can grow without bound.
The --bucket
option format is http://<container>.<endpoint>
. A container defines a namespace for objects.
Currently, JuiceFS only supports Swift V1 authentication.
The value of --access-key
option is username. The value of --secret-key
option is password. For example:
$ juicefs format \
--storage swift \
--bucket http://<container>.<endpoint> \
--access-key <username> \
--secret-key <password> \
... \
myjfs
MinIO is an open source lightweight object storage, compatible with Amazon S3 API.
It is easy to run a MinIO object storage instance locally using Docker. For example, the following command sets and maps port 9900
for the console with -console-address ":9900"
and also maps the data path for the MinIO object storage to the minio-data
folder in the current directory, which can be modified if needed.
$ sudo docker run -d --name minio \
-p 9000:9000 \
-p 9900:9900 \
-e "MINIO_ROOT_USER=minioadmin" \
-e "MINIO_ROOT_PASSWORD=minioadmin" \
-v $PWD/minio-data:/data \
--restart unless-stopped \
minio/minio server /data --console-address ":9900"
It is accessed using the following address:
- MinIO UI: http://127.0.0.1:9900
- MinIO API: http://127.0.0.1:9000
The initial Access Key and Secret Key of the object storage are both minioadmin
.
When using MinIO as data storage for JuiceFS, set the option --storage
to minio
.
$ juicefs format \
--storage minio \
--bucket http://127.0.0.1:9000/<bucket> \
--access-key minioadmin \
--secret-key minioadmin \
... \
myjfs
:::note
Currently, JuiceFS only supports path-style MinIO URI addresses, e.g., http://127.0.0.1:9000/myjfs
.
:::
WebDAV is an extension of the Hypertext Transfer Protocol (HTTP) that facilitates collaborative editing and management of documents stored on the WWW server among users. From JuiceFS v0.15+, JuiceFS can use a storage that speaks WebDAV as a data storage.
You need to set --storage
to webdav
, and --bucket
to the endpoint of WebDAV. If basic authorization is enabled, username and password should be provided as --access-key
and --secret-key
, for example:
$ juicefs format \
--storage webdav \
--bucket http://<endpoint>/ \
--access-key <username> \
--secret-key <password> \
... \
myjfs
HDFS is the file system for Hadoop, which can be used as the object storage for JuiceFS.
When HDFS is used, --access-key
can be used to specify the username
, and hdfs
is usually the default superuser. For example:
$ juicefs format \
--storage hdfs \
--bucket namenode1:8020 \
--access-key hdfs \
... \
myjfs
When --access-key
is not specified on formatting, JuiceFS will use the current user of juicefs mount
or Hadoop SDK to access HDFS. It will hang and fail with IO error eventually, if the current user don't have enough permission to read/write the blocks in HDFS.
JuiceFS will try to load configurations for HDFS client based on $HADOOP_CONF_DIR
or $HADOOP_HOME
. If an empty value is provided to --bucket
, the default HDFS found in Hadoop configurations will be used.
For HA cluster, the addresses of NameNodes can be specified together like this: --bucket=namenode1:port,namenode2:port
.
Apache Ozone is a scalable, redundant, and distributed object storage for Hadoop. It supports S3-compatible interface, so we could set --storage
to s3
directly.
$ juicefs format \
--storage s3 \
--bucket http://<endpoint>/<bucket>\
--access-key <your-access-key> \
--secret-key <your-sceret-key> \
... \
myjfs
Redis can be used as both metadata storage for JuiceFS and as data storage, but when using Redis as a data storage, it is recommended not to store large-scale data.
The --bucket
option format is redis://<host>:<port>/<db>
. The value of --access-key
option is username. The value of --secret-key
option is password. For example:
$ juicefs format \
--storage redis \
--bucket redis://<host>:<port>/<db> \
--access-key <username> \
--secret-key <password> \
... \
myjfs
TiKV is a highly scalable, low latency, and easy to use key-value database. It provides both raw and ACID-compliant transactional key-value API.
TiKV can be used as both metadata storage and data storage for JuiceFS.
The --bucket
option format is <host>:<port>,<host>:<port>,<host>:<port>
, and <host>
is the address of Placement Driver (PD). The options --access-key
and --secret-key
have no effect and can be omitted. For example:
$ juicefs format \
--storage tikv \
--bucket "<host>:<port>,<host>:<port>,<host>:<port>" \
... \
myjfs
If you need to enable TLS, you can set the TLS configuration item by adding the query parameter after the Bucket-URL. Currently supported configuration items:
name | value |
---|---|
ca | CA root certificate, used to connect TiKV/PD with tls |
cert | certificate file path, used to connect TiKV/PD with tls |
key | private key file path, used to connect TiKV/PD with tls |
verify-cn | verify component caller's identity, reference link |
example:
$ juicefs format \
--storage tikv \
--bucket "<host>:<port>,<host>:<port>,<host>:<port>?ca=/path/to/ca.pem&cert=/path/to/tikv-server.pem&key=/path/to/tikv-server-key.pem&verify-cn=CN1,CN2" \
... \
myjfs
When creating JuiceFS storage, if no storage type is specified, the local disk will be used to store data by default. The default storage path for root user is /var/jfs
, and ~/.juicefs/local
is for ordinary users.
For example, using the local Redis database and local disk to create a JuiceFS storage named test
:
$ juicefs format redis://localhost:6379/1 test
Local storage is usually only used to help users understand how JuiceFS works and to give users an experience on the basic features of JuiceFS. The created JuiceFS storage cannot be mounted by other clients within the network and can only be used on a single machine.