Skip to content

Commit

Permalink
docs: Adjust doc comment locations (vectordotdev#17154)
Browse files Browse the repository at this point in the history
  • Loading branch information
spencergilbert authored Apr 14, 2023
1 parent 10fce65 commit 730c938
Show file tree
Hide file tree
Showing 2 changed files with 31 additions and 33 deletions.
32 changes: 0 additions & 32 deletions src/sinks/webhdfs/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,38 +21,6 @@ use crate::{
};

/// Configuration for the `webhdfs` sink.
///
/// The Hadoop Distributed File System (HDFS) is a distributed file system
/// designed to run on commodity hardware. HDFS consists of a namenode and a
/// datanode. We will send rpc to namenode to know which datanode to send
/// and receive data to. Also, HDFS will rebalance data across the cluster
/// to make sure each file has enough redundancy.
///
/// ```txt
/// ┌───────────────┐
/// │ Data Node 2 │
/// └───────────────┘
/// ▲
/// ┌───────────────┐ │ ┌───────────────┐
/// │ Data Node 1 │◄──────────┼───────────►│ Data Node 3 │
/// └───────────────┘ │ └───────────────┘
/// ┌───────┴───────┐
/// │ Name Node │
/// └───────────────┘
/// ▲
/// │
/// ┌──────┴─────┐
/// │ Vector │
/// └────────────┘
/// ```
///
/// WebHDFS will connect to the HTTP RESTful API of HDFS.
///
/// For more information, please refer to:
///
/// - [HDFS Users Guide](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html)
/// - [WebHDFS REST API](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html)
/// - [opendal::services::webhdfs](https://docs.rs/opendal/latest/opendal/services/struct.Webhdfs.html)
#[configurable_component(sink("webhdfs"))]
#[derive(Clone, Debug)]
#[serde(deny_unknown_fields)]
Expand Down
32 changes: 31 additions & 1 deletion src/sinks/webhdfs/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,36 @@
//! `webhdfs` sink.
//!
//! This sink will send it's output to WEBHDFS.
//! The Hadoop Distributed File System (HDFS) is a distributed file system
//! designed to run on commodity hardware. HDFS consists of a namenode and a
//! datanode. We will send rpc to namenode to know which datanode to send
//! and receive data to. Also, HDFS will rebalance data across the cluster
//! to make sure each file has enough redundancy.
//!
//! ```txt
//! ┌───────────────┐
//! │ Data Node 2 │
//! └───────────────┘
//! ▲
//! ┌───────────────┐ │ ┌───────────────┐
//! │ Data Node 1 │◄──────────┼───────────►│ Data Node 3 │
//! └───────────────┘ │ └───────────────┘
//! ┌───────┴───────┐
//! │ Name Node │
//! └───────────────┘
//! ▲
//! │
//! ┌──────┴─────┐
//! │ Vector │
//! └────────────┘
//! ```
//!
//! WebHDFS will connect to the HTTP RESTful API of HDFS.
//!
//! For more information, please refer to:
//!
//! - [HDFS Users Guide](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html)
//! - [WebHDFS REST API](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html)
//! - [opendal::services::webhdfs](https://docs.rs/opendal/latest/opendal/services/struct.Webhdfs.html)
//!
//! `webhdfs` is an OpenDal based services. This mod itself only provide
//! config to build an [`crate::sinks::opendal_common::OpenDalSink`]. All real implement are powered by
Expand Down

0 comments on commit 730c938

Please sign in to comment.