-
Notifications
You must be signed in to change notification settings - Fork 8
/
README.Rmd
204 lines (157 loc) · 8.99 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
---
output: github_document
editor_options:
markdown:
wrap: 80
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# ***bdc*** <a href='https://github.com/brunobrr/bdc'><img src="https://raw.githubusercontent.com/brunobrr/bdc/master/man/figures/logo.png" align="right" width="155"/></a>
## **A toolkit for standardizing, integrating, and cleaning biodiversity data**
<!-- badges: start -->
[](https://CRAN.R-project.org/package=bdc)
[](https://cranlogs.r-pkg.org:443/badges/grand-total/bdc)
<!-- [](https://cranlogs.r-pkg.org:443/badges/bdc) -->
[](https://github.com/brunobrr/bdc/actions/workflows/R-CMD-check.yaml)
[](https://app.codecov.io/gh/brunobrr/bdc)
[](https://doi.org/10.5281/zenodo.6450390)
[-lightgrey.svg?style=flat)](http://www.gnu.org/licenses/gpl-3.0.html)
<!-- badges: end -->
#### **Overview**
Handle biodiversity data from several different sources is not an easy task.
Here, we present the **B**iodiversity **D**ata **C**leaning (*bdc*), an R
package to address quality issues and improve the fitness-for-use of biodiversity
datasets. *bdc* contains functions to harmonize and integrate data from
different sources following common standards and protocols, and implements
various tests and tools to flag, document, clean, and correct taxonomic,
spatial, and temporal data.
Compared to other available R packages, the main strengths of the *bdc* package
are that it brings together available tools – and a series of new ones – to
assess the quality of different dimensions of biodiversity data into a single
and flexible toolkit. The functions can be applied to a multitude of taxonomic
groups, datasets (including regional or local repositories), countries, or
worldwide.
#### **Structure of *bdc***
The *bdc* toolkit is organized in thematic modules related to different
biodiversity dimensions.
--------------------------------------------------------------------------------
> :warning: The modules illustrated, and **functions** within, **were linked to
> form** a proposed reproducible **workflow** (see
> [**vignettes**](https://brunobrr.github.io/bdc/)). However, all functions
> **can also be executed independently**.
--------------------------------------------------------------------------------
#### 
<br/>
#### 1. [**Merge databases**](https://brunobrr.github.io/bdc/articles/integrate_datasets.html)
Standardization and integration of different datasets into a standard database.
- `bdc_standardize_datasets()` Standardization and integration of different
datasets into a new dataset with column names following Darwin Core
terminology
#### 2. [**Pre-filter**](https://brunobrr.github.io/bdc/articles/prefilter.html)
Flagging and removal of invalid or non-interpretable information, followed by
data amendments (e.g., correct transposed coordinates and standardize country
names).
- `bdc_scientificName_empty()` Identification of records lacking names or with
names not interpretable
- `bdc_coordinates_empty()` Identification of records lacking information on
latitude or longitude
- `bdc_coordinates_outOfRange()` Identification of records with out-of-range
coordinates (latitude \> 90 or -90; longitude \>180 or -180)
- `bdc_basisOfRecords_notStandard()` Identification of records from doubtful
sources (e.g., fossil or machine observation) impossible to interpret and
not compatible with Darwin Core recommended vocabulary
- `bdc_country_from_coordinates()` Derive country name from valid geographic
coordinates
- `bdc_country_standardized()` Standardization of country names and retrieve
country code
- `bdc_coordinates_transposed()` Identification of records with potentially
transposed latitude and longitude
- `bdc_coordinates_country_inconsistent()` Identification of coordinates in
other countries or far from a specified distance from the coast of a
reference country (i.e., in the ocean)
- `bdc_coordinates_from_locality()` Identification of records lacking
coordinates but with a detailed description of the locality associate with
records from which coordinates can be derived
#### 3. [**Taxonomy**](https://brunobrr.github.io/bdc/articles/taxonomy.html)
Cleaning, parsing, and harmonization of scientific names against multiple
taxonomic references.
- `bdc_clean_names()` Name-checking routines to clean and split a taxonomic
name into its binomial and authority components
- `bdc_query_names_taxadb()` Harmonization of scientific names by correcting
spelling errors and converting nomenclatural synonyms to currently accepted
names.
- `bdc_filter_out_names()` Function used to filter out records according to
their taxonomic status present in the column "notes". For example, to filter
only valid accepted names categorized as "accepted"
#### 4. [**Space**](https://brunobrr.github.io/bdc/articles/space.html)
Flagging of erroneous, suspicious, and low-precision geographic coordinates.
- `bdc_coordinates_precision()` Identification of records with a coordinate
precision below a specified number of decimal places
- `clean_coordinates()` (From *CoordinateCleaner* package and part of the
data-cleaning workflow). Identification of potentially problematic
geographic coordinates based on geographic gazetteers and metadata. Include
tests for flagging records: around country capitals or country or province
centroids, duplicated, with equal coordinates, around biodiversity
institutions, within urban areas, plain zeros in the coordinates, and
suspect geographic outliers
#### 5. [**Time**](https://brunobrr.github.io/bdc/articles/time.html)
Flagging and, whenever possible, correction of inconsistent collection date.
- `bdc_eventDate_empty()` Identification of records lacking information on
event date (i.e., when a record was collected or observed)
- `bdc_year_outOfRange()` Identification of records with illegitimate or
potentially imprecise collecting year. The year provided can be out-of-range
(e.g., in the future) or collected before a specified year supplied by the
user (e.g., 1900)
- `bdc_year_from_eventDate()` This function extracts four-digit year from
unambiguously interpretable collecting dates
#### [**Other functions**](https://brunobrr.github.io/bdc/reference/index.html)
Aim to facilitate the **documentation, visualization, and interpretation** of
results of data quality tests the package contains functions for documenting the
results of the data-cleaning tests, including functions for saving i) records
needing further inspection, ii) figures, and iii) data-quality reports.
- `bdc_create_report()` Creation of data-quality reports documenting the
results of data-quality tests and the taxonomic harmonization process
- `bdc_create_figures()` Creation of figures (i.e., bar plots and maps)
reporting the results of data-quality tests
- `bdc_filter_out_flags()` Removal of columns containing the results of data
quality tests (i.e., column starting with ".") or other columns specified
- `bdc_quickmap()` Creation of a map of points using ggplot2. Helpful in
inspecting the results of data-cleaning tests
- `bdc_summary_col()` This function creates or updates the column summarizing
the results of data quality tests (i.e., the column ".summary")
#### **Installation**
```{r eval=FALSE}
install.packages("bdc")
library(bdc)
```
or the development version from [GitHub](https://github.com/brunobrr/bdc) using:
```{r, message=FALSE, warning=FALSE,echo=TRUE,eval=FALSE}
install.packages("remotes")
remotes::install_github("brunobrr/bdc")
```
Load the package with:
```{r, message=FALSE, warning=FALSE,echo=TRUE,eval=TRUE}
library(bdc)
```
#### **Package website**
See *bdc* package website (<https://brunobrr.github.io/bdc/>) for detailed
explanation on each module.
#### **Getting help**
> If you encounter a clear bug, please file an issue
> [**here**](https://github.com/brunobrr/bdc/issues). For questions or
> suggestion, please send us a email (ribeiro.brr\@gmail.com).
#### **Citation**
Ribeiro, BR; Velazco, SJE; Guidoni-Martins, K; Tessarolo, G; Jardim, Lucas;
Bachman, SP; Loyola, R (2022). bdc: A toolkit for standardizing, integrating,
and cleaning biodiversity data. Methods in Ecology and Evolution.
[doi.org/10.1111/2041-210X.13868](https://doi.org/10.1111/2041-210X.13868)