Skip to content

Commit

Permalink
Merge branch 'master' into in_place_ntoh
Browse files Browse the repository at this point in the history
  • Loading branch information
Moelf authored Oct 10, 2022
2 parents 4942049 + 8b16c8f commit d275941
Show file tree
Hide file tree
Showing 30 changed files with 1,315 additions and 232 deletions.
45 changes: 45 additions & 0 deletions .github/workflows/CompatHelper.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: CompatHelper
on:
schedule:
- cron: 0 0 * * *
workflow_dispatch:
permissions:
contents: write
pull-requests: write
jobs:
CompatHelper:
runs-on: ubuntu-latest
steps:
- name: Check if Julia is already available in the PATH
id: julia_in_path
run: which julia
continue-on-error: true
- name: Install Julia, but only if it is not already available in the PATH
uses: julia-actions/setup-julia@v1
with:
version: '1'
# arch: ${{ runner.arch }}
if: steps.julia_in_path.outcome != 'success'
- name: "Add the General registry via Git"
run: |
import Pkg
ENV["JULIA_PKG_SERVER"] = ""
Pkg.Registry.add("General")
shell: julia --color=yes {0}
- name: "Install CompatHelper"
run: |
import Pkg
name = "CompatHelper"
uuid = "aa819f21-2bde-4658-8897-bab36330d9b7"
version = "3"
Pkg.add(; name, uuid, version)
shell: julia --color=yes {0}
- name: "Run CompatHelper"
run: |
import CompatHelper
CompatHelper.main()
shell: julia --color=yes {0}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
COMPATHELPER_PRIV: ${{ secrets.DOCUMENTER_KEY }}
# COMPATHELPER_PRIV: ${{ secrets.COMPATHELPER_PRIV }}
5 changes: 5 additions & 0 deletions .github/workflows/TagBot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ on:
types:
- created
workflow_dispatch:
inputs:
lookback:
default: 3
permissions:
contents: write
jobs:
TagBot:
if: github.event_name == 'workflow_dispatch' || github.actor == 'JuliaTagBot'
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,7 @@ jobs:
matrix:
version:
- '1.6'
- '1.5'
- '1.4'
- '1.3' # Replace this with the minimum Julia version that your package supports. E.g. if your package requires Julia 1.5 or higher, change this to '1.5'.
- '1.7'
- '1' # Leave this line unchanged. '1' will automatically expand to the latest stable 1.x release of Julia.
- 'nightly'
os:
Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/draft-pdf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
on: [push]

jobs:
paper:
runs-on: ubuntu-latest
name: Paper Draft
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Build draft PDF
uses: openjournals/openjournals-draft-action@master
with:
journal: joss
# This should be the path to the paper within your repo.
paper-path: paper/paper.md
- name: Upload
uses: actions/upload-artifact@v1
with:
name: paper
# This is the output path where Pandoc will write the compiled
# PDF. Note, this should be the same directory as the input
# paper.md
path: paper/paper.pdf
13 changes: 13 additions & 0 deletions CITATION.bib
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
@article{
UnROOT-2022,
doi = {10.21105/joss.04452},
url = {https://doi.org/10.21105/joss.04452},
year = {2022},
publisher = {The Open Journal},
volume = {7},
number = {76},
pages = {4452},
author = {Tamás Gál and Jerry (Jiahong) Ling and Nick Amin},
title = {UnROOT: an I/O library for the CERN ROOT file format written in Julia},
journal = {Journal of Open Source Software}
}
18 changes: 18 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
cff-version: 1.2.0
message: "Cite this paper whenever you use this package"
authors:
- family-names: "Gál"
given-names: "Tamás"
orcid: "https://orcid.org/0000-0001-7821-8673"
- family-names: "Ling"
given-names: "Jerry"
orcid: "https://orcid.org/0000-0002-3359-0380"
- family-names: "Amin"
given-names: "Nick"
orcid: "https://orcid.org/0000-0003-2560-0013"
title: "UnROOT: an I/O library for the CERN ROOT file format written in Julia"
version: "v1"
license: "MIT"
doi: "10.21105/joss.04452"
date-released: 2021-10-11
url: "https://github.com/JuliaHEP/UnROOT.jl/"
6 changes: 6 additions & 0 deletions CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
UnROOT.jl Standards
=========================

We abide by the larger Julia Communities Code of Conduct (CoC). You can find that CoC listed here: https://julialang.org/community/standards/

If you have a conflict or concern that requires resolution, please contact the [Julia Community Stewards](https://julialang.org/community/stewards/).
39 changes: 39 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Contribute to UnROOT.jl

To make your first PR to this repo:

1. Have basic understanding of **Git**. The tutorial [Making a first Julia pull request](https://kshyatt.github.io/post/firstjuliapr/) could be helpful for learning both git and how to contribute to the Julia language projects.
2. Set up your local environment. We recommend use `Revise.jl` workflow.
3. Familiarise yourself with the source code. See [Source code organization](#source-code-organization).
4. Make changes & test them & submit PR.

## Contribution example ideas

### Core functionality
1. Parsing more ROOT types
2. Implement writing `.root` files

#### Help Wanted Issues
One of the best ways to contribute is by looking at issues labelled [help wanted](https://github.com/JuliaHEP/UnROOT.jl/labels/help%20wanted). These issues are not always beginner-friendly. However, you are welcome to [ask clarifying questions](#get-help) or just browse
help wanted issues to see if there is anything that seems interesting to help with.

### Write tutorials
We can always use more tutorial on how to use UnROOT.jl efficiently and with other visualization or statistics tools in Julia for doing
HEP.

## Contribution guidelines
- We use the GitHub issue page for any bug filing or feature request, feel free to use them.
- For usage related discussion, feel free to use [HEP tag on Julia discourse](https://discourse.julialang.org/tag/hep) or join
our [mailist](https://groups.google.com/g/julia-hep).

### source code organization

The following table shows how the code is organized:

| Directory | Contents |
| ------------- | ------------- |
| docs | Documentation|
| paper | JOSS paper |
| src | Source code |
| test | Test suites |
| test/samples | .root files for tests |
21 changes: 12 additions & 9 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,47 +1,50 @@
name = "UnROOT"
uuid = "3cd96dde-e98d-4713-81e9-a4a1b0235ce9"
authors = ["Tamas Gal", "Jerry Ling", "Johannes Schumann", "Nick Amin"]
version = "0.8.1"
version = "0.8.16"

[deps]
AbstractTrees = "1520ce14-60c1-5f80-bbc7-55ef81b5835c"
ArraysOfArrays = "65a8f2f4-9b39-5baf-92e2-a9cc46fdf018"
CodecLz4 = "5ba52731-8f18-5e0d-9241-30f10d1ec561"
CodecXz = "ba30903b-d9e8-5048-a5ec-d1f5b0d4b47b"
CodecZstd = "6b39b394-51ab-5f42-8807-6242bab2b4c2"
HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
IterTools = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
LRUCache = "8ac3fa9e-de4c-5943-b1dc-09c6b5f20637"
LazyArrays = "5078a376-72f3-5289-bfd5-ec5146d43c02"
LibDeflate = "9255714d-24a7-4b30-8ea3-d46a97f7e13b"
LorentzVectors = "3f54b04b-17fc-5cd4-9758-90c048d965e3"
Memoization = "6fafb56a-5788-4b4e-91ca-c0cea6611c73"
Mixers = "2a8e4939-dab8-5edc-8f64-72a8776f13de"
Mmap = "a63ad114-7e13-5084-954f-fe012c677804"
Parameters = "d96e819e-fc66-5662-9728-84c9c7592b0a"
PrettyTables = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
TypedTables = "9d95f2ec-7b3d-5a63-8d20-e2491e220bb9"
xrootdgo_jll = "9d84c17e-11f2-50ef-8cc9-e9701362097f"

[compat]
AbstractTrees = "^0.3.0"
ArraysOfArrays = "^0.5.3"
AbstractTrees = "^0.3.0, 0.4"
ArraysOfArrays = "^0.5.3, ^0.6"
CodecLz4 = "^0.3.0, ^0.4.0"
CodecXz = "^0.6.0, ^0.7.0"
CodecZstd = "^0.6.0, ^0.7.0"
HTTP = "^0.9.7, 1"
IterTools = "^1"
LRUCache = "^1.3.0"
LazyArrays = "^0.15, ^0.21, ^0.22, ^1"
LazyArrays = "^0.21, ^0.22, ^1"
LibDeflate = "^0.4.1"
LorentzVectors = "^0.4.0"
Memoization = "^0.1.10"
Mixers = "^0.1.0"
Parameters = "^0.12.0"
Polyester = "^0.5.3"
PrettyTables = "^1.2.0"
PrettyTables = "2"
StaticArrays = "^0.12.0, ^1"
Tables = "^1.0.0"
TypedTables = "^1.0.0"
julia = "^1.3"
julia = "^1.6"
xrootdgo_jll = "^0.31.1"

[extras]
InteractiveUtils = "b77e0a4c-d291-57a0-90e8-8db25a27a240"
Expand All @@ -52,4 +55,4 @@ Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
ThreadsX = "ac1d9e8a-700a-412c-b207-f0111f4b6c0d"

[targets]
test = ["Test", "Pkg", "ThreadsX", "MD5", "InteractiveUtils"]
test = ["Test", "Pkg", "ThreadsX", "MD5", "InteractiveUtils", "Polyester"]
55 changes: 37 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,23 @@
<!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section -->
[![All Contributors](https://img.shields.io/badge/all_contributors-7-orange.svg?style=flat-square)](#contributors-)
<!-- ALL-CONTRIBUTORS-BADGE:END -->
[![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://tamasgal.github.io/UnROOT.jl/dev)
[![Build Status](https://github.com/tamasgal/UnROOT.jl/workflows/CI/badge.svg)](https://github.com/tamasgal/UnROOT.jl/actions)
[![Codecov](https://codecov.io/gh/tamasgal/UnROOT.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/tamasgal/UnROOT.jl)
[![JOSS](https://joss.theoj.org/papers/bab42b0c60f9dc7ef3b8d6460bc7229c/status.svg)](https://joss.theoj.org/papers/bab42b0c60f9dc7ef3b8d6460bc7229c)
[![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://juliahep.github.io/UnROOT.jl/dev)
[![Build Status](https://github.com/JuliaHEP/UnROOT.jl/workflows/CI/badge.svg)](https://github.com/JuliaHEP/UnROOT.jl/actions)
[![Codecov](https://codecov.io/gh/JuliaHEP/UnROOT.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/JuliaHEP/UnROOT.jl)

UnROOT.jl is a reader for the [CERN ROOT](https://root.cern) file format
written entirely in Julia, without any dependence on ROOT or Python.

## Quick Start (see [docs](https://tamasgal.github.io/UnROOT.jl/dev/) for more)
## Installation Guide
1. Download the latest [Julia release](https://julialang.org/downloads/)
2. Open up Julia REPL (hit `]` once to enter Pkg mode, hit backspace to exit it)
```julia
julia>]
(v1.8) pkg> add UnROOT
```
## Quick Start (see [docs](https://JuliaHEP.github.io/UnROOT.jl/dev/) for more)

```julia
julia> using UnROOT

Expand Down Expand Up @@ -51,23 +60,32 @@ julia> Threads.@threads for event in mytree # multi-threading
```

Only one basket per branch will be cached so you don't have to worry about running out of RAM.
At the same time, `event` inside the for-loop is not materialized until a field is accessed. If your event
is fairly small or you need all of them anyway, you can `collect(event)` first inside the loop.
At the same time, `event` inside the for-loop is not materialized until a field is accessed. This means you should avoid double-access,
see [performance tips](https://juliahep.github.io/UnROOT.jl/dev/performancetips/#Don't-%22double-access%22)

XRootD is also supported, depending on the protocol:
- the "url" has to start with `http://` or `https://`:
- (1.6+ only) or the "url" has to start with `root://` and have another `//` to separate server and file path
```julia
julia> r = @time ROOTFile("https://scikit-hep.org/uproot3/examples/Zmumu.root")
0.034877 seconds (5.13 k allocations: 533.125 KiB)
ROOTFile with 1 entry and 18 streamers.

julia> r = ROOTFile("root://eospublic.cern.ch//eos/root-eos/cms_opendata_2012_nanoaod/Run2012B_DoubleMuParked.root")
ROOTFile with 1 entry and 19 streamers.
```

## Branch of custom struct

We provide an experimental interface for hooking up UnROOT with your custom types
that only takes 2 steps, as explained [in the docs](https://tamasgal.github.io/UnROOT.jl/dev/advanced/custom_branch/).
that only takes 2 steps, as explained [in the docs](https://JuliaHEP.github.io/UnROOT.jl/dev/advanced/custom_branch/).
As a show case for this functionality, the `TLorentzVector` support in UnROOT is implemented
with the said plug-in system.

## Main challenges
- ROOT data is generally stored as big endian and is a
self-descriptive format, i.e. so-called streamers are stored in the files
which describe the actual structure of the data in the corresponding branches.
These streamers are read during runtime and need to be used to generate
Julia structs and `unpack` methods on the fly.
- Performance is very important for a low level I/O library.
## Support & Contributiing
- Use Github issues for any bug reporting or feature request; feel free to make PRs,
bug fixing, feature tuning, quality of life, docs, examples etc.
- See `CONTRIBUTING.md` for more information and recommended workflows in contributing to this package.


## TODOs
Expand Down Expand Up @@ -98,6 +116,7 @@ of inspiration and information for reverse engineering the ROOT binary
structures.

## Behind the scene

<details><summary>Some additional debug output: </summary>
<p>

Expand Down Expand Up @@ -190,10 +209,10 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
<!-- markdownlint-disable -->
<table>
<tr>
<td align="center"><a href="http://www.tamasgal.com"><img src="https://avatars.githubusercontent.com/u/1730350?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Tamas Gal</b></sub></a><br /><a href="https://github.com/tamasgal/UnROOT.jl/commits?author=tamasgal" title="Code">💻</a> <a href="https://github.com/tamasgal/UnROOT.jl/commits?author=tamasgal" title="Documentation">📖</a> <a href="#infra-tamasgal" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a> <a href="#data-tamasgal" title="Data">🔣</a> <a href="https://github.com/tamasgal/UnROOT.jl/commits?author=tamasgal" title="Tests">⚠️</a></td>
<td align="center"><a href="https://github.com/Moelf"><img src="https://avatars.githubusercontent.com/u/5306213?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Jerry Ling</b></sub></a><br /><a href="https://github.com/tamasgal/UnROOT.jl/commits?author=Moelf" title="Code">💻</a> <a href="https://github.com/tamasgal/UnROOT.jl/commits?author=Moelf" title="Tests">⚠️</a> <a href="#data-Moelf" title="Data">🔣</a> <a href="https://github.com/tamasgal/UnROOT.jl/commits?author=Moelf" title="Documentation">📖</a></td>
<td align="center"><a href="https://github.com/8me"><img src="https://avatars.githubusercontent.com/u/17862090?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Johannes Schumann</b></sub></a><br /><a href="https://github.com/tamasgal/UnROOT.jl/commits?author=8me" title="Code">💻</a> <a href="https://github.com/tamasgal/UnROOT.jl/commits?author=8me" title="Tests">⚠️</a> <a href="#data-8me" title="Data">🔣</a></td>
<td align="center"><a href="https://github.com/aminnj"><img src="https://avatars.githubusercontent.com/u/5760027?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Nick Amin</b></sub></a><br /><a href="https://github.com/tamasgal/UnROOT.jl/commits?author=aminnj" title="Code">💻</a> <a href="https://github.com/tamasgal/UnROOT.jl/commits?author=aminnj" title="Tests">⚠️</a> <a href="#data-aminnj" title="Data">🔣</a></td>
<td align="center"><a href="http://www.tamasgal.com"><img src="https://avatars.githubusercontent.com/u/1730350?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Tamas Gal</b></sub></a><br /><a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=tamasgal" title="Code">💻</a> <a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=tamasgal" title="Documentation">📖</a> <a href="#infra-tamasgal" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a> <a href="#data-tamasgal" title="Data">🔣</a> <a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=tamasgal" title="Tests">⚠️</a></td>
<td align="center"><a href="https://github.com/Moelf"><img src="https://avatars.githubusercontent.com/u/5306213?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Jerry Ling</b></sub></a><br /><a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=Moelf" title="Code">💻</a> <a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=Moelf" title="Tests">⚠️</a> <a href="#data-Moelf" title="Data">🔣</a> <a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=Moelf" title="Documentation">📖</a></td>
<td align="center"><a href="https://github.com/8me"><img src="https://avatars.githubusercontent.com/u/17862090?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Johannes Schumann</b></sub></a><br /><a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=8me" title="Code">💻</a> <a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=8me" title="Tests">⚠️</a> <a href="#data-8me" title="Data">🔣</a></td>
<td align="center"><a href="https://github.com/aminnj"><img src="https://avatars.githubusercontent.com/u/5760027?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Nick Amin</b></sub></a><br /><a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=aminnj" title="Code">💻</a> <a href="https://github.com/JuliaHEP/UnROOT.jl/commits?author=aminnj" title="Tests">⚠️</a> <a href="#data-aminnj" title="Data">🔣</a></td>
<td align="center"><a href="https://giordano.github.io"><img src="https://avatars.githubusercontent.com/u/765740?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Mosè Giordano</b></sub></a><br /><a href="#infra-giordano" title="Infrastructure (Hosting, Build-Tools, etc)">🚇</a></td>
<td align="center"><a href="https://github.com/oschulz"><img src="https://avatars.githubusercontent.com/u/546147?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Oliver Schulz</b></sub></a><br /><a href="#ideas-oschulz" title="Ideas, Planning, & Feedback">🤔</a></td>
<td align="center"><a href="https://github.com/mmikhasenko"><img src="https://avatars.githubusercontent.com/u/22725744?v=4?s=100" width="100px;" alt=""/><br /><sub><b>Misha Mikhasenko</b></sub></a><br /><a href="#data-mmikhasenko" title="Data">🔣</a></td>
Expand Down
5 changes: 3 additions & 2 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ makedocs(;
),
pages=[
"Introduction" => "index.md",
"Example Usage" => "exampleusage.md",
"Performance Tips" => "performancetips.md",
"Advanced Usage" => [
"Parse Custom Branch" => "advanced/custom_branch.md",
Expand All @@ -16,11 +17,11 @@ makedocs(;
"For Contributors" => "devdocs.md",
"APIs" => "internalapis.md",
],
repo="https://github.com/tamasgal/UnROOT.jl/blob/{commit}{path}#L{line}",
repo="https://github.com/JuliaHEP/UnROOT.jl/blob/{commit}{path}#L{line}",
sitename="UnROOT.jl",
authors="Tamas Gal and contributors",
)

deploydocs(;
repo="github.com/tamasgal/UnROOT.jl",
repo="github.com/JuliaHEP/UnROOT.jl",
)
Loading

0 comments on commit d275941

Please sign in to comment.