Skip to content

Commit

Permalink
feat: Implement Iceberg values (#20)
Browse files Browse the repository at this point in the history
* implement values

* improve getters

* fix clippy warnings

* fix clippy warnings

* change into bytebuf to from

* add license header

* Use Long instead of LongInt

Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>

* use more general error kind

* use Naivetime

* use naivedate

* use naivedatetime

* fix clippy warnings

* use uuid

* use orderedfloat

* fix clippy warnings

* fix tests

* use datatime utz

* fix docs

* rename value to literal

* introduce primitive literal

* remove length from fixed

* serialize json via serde_json value

* remove derive serialize/deserialize

* implement From Literal for ByteBuf

* implement From Literal for JsonValue

* fix From Literal for JsonValue

* implement struct

* fix clippy warnings

* add avro tests for some types

* fix clippy warnings

* fix nested field

* fix nested field

* implement list test

* implement map test

* fix error

* fix clippy warnings

* change timestamps to int/long

* convert nulls to None

* add tests for null

* null test for struct

* fix clippy warning

* convert json null to option

---------

Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
  • Loading branch information
JanKaul and liurenjie1024 authored Aug 9, 2023
1 parent c497121 commit 4ee05b4
Show file tree
Hide file tree
Showing 5 changed files with 999 additions and 0 deletions.
5 changes: 5 additions & 0 deletions crates/iceberg/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ serde_json = "^1.0"
serde_derive = "^1.0"
anyhow = "1.0.72"
once_cell = "1"
rust_decimal = "1.31.0"
chrono = "0.4"
uuid = "1.4.1"
ordered-float = "3.7.0"
bitvec = "1.0.1"

[dev-dependencies]
pretty_assertions = "1.4.0"
24 changes: 24 additions & 0 deletions crates/iceberg/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,30 @@ define_from_err!(
"handling invalid utf-8 characters"
);

define_from_err!(
std::array::TryFromSliceError,
ErrorKind::DataInvalid,
"failed to convert byte slive to array"
);

define_from_err!(
std::num::TryFromIntError,
ErrorKind::DataInvalid,
"failed to convert integer"
);

define_from_err!(
chrono::ParseError,
ErrorKind::DataInvalid,
"Failed to parse string to date or time"
);

define_from_err!(
uuid::Error,
ErrorKind::DataInvalid,
"Failed to convert between uuid und iceberg value"
);

#[cfg(test)]
mod tests {
use anyhow::anyhow;
Expand Down
5 changes: 5 additions & 0 deletions crates/iceberg/src/spec/datatypes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ use ::serde::de::{MapAccess, Visitor};
use serde::de::{Error, IntoDeserializer};
use serde::{de, Deserialize, Deserializer, Serialize, Serializer};
use std::cell::OnceCell;
use std::slice::Iter;
use std::{collections::HashMap, fmt, ops::Index};

/// Field name for list type.
Expand Down Expand Up @@ -293,6 +294,10 @@ impl StructType {
.get(&field_id)
.copied()
}
/// Returns an iteratorr over the struct fields
pub fn iter(&self) -> Iter<NestedField> {
self.fields.iter()
}
}

impl PartialEq for StructType {
Expand Down
1 change: 1 addition & 0 deletions crates/iceberg/src/spec/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,4 @@
pub mod datatypes;
pub mod schema;
pub mod values;
Loading

0 comments on commit 4ee05b4

Please sign in to comment.