-
Notifications
You must be signed in to change notification settings - Fork 20
/
README.Rmd
97 lines (63 loc) · 4.52 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
# tidyRSS <a href='https://github.com/RobertMyles/tidyrss/'><img src='man/figures/logo.png' align="right" height="139" /></a>
```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "README-"
)
```
[](https://cran.r-project.org/package=tidyRSS)
[](https://CRAN.R-project.org/package=tidyRSS)
[](https://CRAN.R-project.org/package=tidyRSS)
[](https://app.codecov.io/gh/RobertMyles/tidyRSS?branch=master)
tidyRSS is a package for extracting data from [RSS feeds](https://en.wikipedia.org/wiki/RSS), including Atom feeds and JSON feeds. For geo-type feeds, see the section on changes in version 2 below, or jump directly to [tidygeoRSS](https://github.com/RobertMyles/tidygeoRSS), which is designed for that purpose.
It is easy to use as it only has one function, `tidyfeed()`, which takes five arguments:
- the url of the feed;
- a logical flag for whether you want the feed returned as a tibble or a list containing two tibbles;
- a logical flag for whether you want HTML tags removed from columns in the dataframe;
- a config list that is passed off to [`httr::GET()`](https://httr.r-lib.org/reference/config.html);
- and a `parse_dates` argument, a logical flag, which will attempt to parse dates if `TRUE` (see below).
If `parse_dates` is `TRUE`, `tidyfeed()` will attempt to parse dates using the [anytime](https://github.com/eddelbuettel/anytime) package. Note that this removes some lower-level control that you may wish to retain over how dates are parsed. See [this issue](https://github.com/RobertMyles/tidyRSS/issues/37) for an example.
## Installation
It can be installed directly from [CRAN](https://cran.r-project.org/) with:
```{r, eval=FALSE}
install.packages("tidyRSS")
```
The development version can be installed from GitHub with the [remotes](https://github.com/r-lib/remotes) package:
```{r, eval = F}
remotes::install_github("robertmyles/tidyrss")
```
## Usage
Here is how you can get the contents of the [R Journal](https://journal.r-project.org/):
```{r, eval = FALSE}
library(tidyRSS)
tidyfeed("http://journal.r-project.org/rss.atom")
```
## Changes in version 2.0.0
The biggest change in version 2 is that tidyRSS no longer attempts to parse geo-type feeds into [sf](https://github.com/r-spatial/sf/) tibbles. This functionality has been moved to [tidygeoRSS](https://github.com/RobertMyles/tidygeoRSS).
## Issues
XML feeds can be finicky things, if you find one that doesn't work with `tidyfeed()`, feel free to create an [issue](https://github.com/robertmyles/tidyrss/issues) with the url of the feed that you are trying. Pull Requests are welcome if you'd like to try and fix it yourself. For older RSS feeds, some fields will almost never be 'clean', that is, they will contain things like newlines (`\n`) or extra quote marks. Cleaning these in a generic way is more or less impossible so I suggest you use [stringr](https://github.com/tidyverse/stringr), [strex](https://rorynolan.github.io/strex/) and/or tools from base R such as gsub to clean these. This will mainly affect the `item_description` column of a parsed RSS feed, and will not often affect Atom feeds (and should never be a problem with JSON).
## Related
There are two other related packages that I'm aware of:
- [rss](https://github.com/noahhl/r-does-rss)
- [feedeR](https://github.com/DataWookie/feedeR)
In comparison to feedeR, tidyRSS returns more information from the RSS feed (if it exists), and development on rss seems to have stopped some time ago.
# Other
For the schemas used to develop the parsers in this package, see:
- RSS: [https://validator.w3.org/feed/docs/rss2.html](https://validator.w3.org/feed/docs/rss2.html)
- Atom: [https://validator.w3.org/feed/docs/atom.html](https://validator.w3.org/feed/docs/atom.html)
- JSON: [https://jsonfeed.org/version/1](https://jsonfeed.org/version/1)
I've implemented most of the items in the schemas above. The following are not yet implemented:
**Atom meta info:**
- contributor, generator, logo, subtitle
**Rss meta info:**
- cloud
- image
- textInput
- skipHours
- skipDays