Skip to contents

Staging data functions

Source: R/stagingData.R
stagingData.Rd

Low level functions for handling registry staging data at Rapporteket. As such, these functions does not provide staging data management per se. Proper management, e.g. staging data updates and fallback logic must therefore be established within each registry that take staging data into use.

Usage

listStagingData(registryName, dir = Sys.getenv("R_RAP_CONFIG_PATH"))

mtimeStagingData(registryName, dir = Sys.getenv("R_RAP_CONFIG_PATH"))

saveStagingData(
  registryName,
  dataName,
  data,
  dir = Sys.getenv("R_RAP_CONFIG_PATH")
)

loadStagingData(registryName, dataName, dir = Sys.getenv("R_RAP_CONFIG_PATH"))

deleteStagingData(
  registryName,
  dataName,
  dir = Sys.getenv("R_RAP_CONFIG_PATH")
)

cleanStagingData(eolAge, dryRun = TRUE)

Arguments

registryName

Character string providing the registry name.

dir

Character string providing the path to where the staging data directory resides in case of storage as files. Default value is Sys.getenv("R_RAP_CONFIG_PATH").

dataName

Character string providing the data set name.

data

A data object such as a data.frame to be stored as dataName.

eolAge

Numeric providing the staging data end-of-life age in seconds. Based on the current time and the time of storage staging files older than eolAge will be identified as subject for removal.

dryRun

Logical defining if function is to be run in dry (none destructive) mode.

Value

  • listStagingData() returns a character vector of staging data sets for the given registry (registryName).

  • mtimeStagingData() returns a staging data set named POSIXct vector of modification times for the given registry (registryName).

  • saveStagingData() when successful returns the data object (data), invisibly. If saving fails a warning is issued and the function returns FALSE.

  • loadStagingData() returns the data object corresponding to the name given upon saving (dataName). If the requested data set for loading does not exist the function returns FALSE.

  • deleteStagingData() returns TRUE if the data set was deleted and FALSE if not.

  • cleanStagingData() returns a list of data sets (to be) removed.

  • rapbase:::pathStagingData() is an internal helper function and returns a character string with the path to the staging directory of registryName. If its parent directory (dir) does not exists an error is returned.

Details

Staging data can be stored as files or as binary large objects in a database and method of choice is defined by the rapbase configuration. Regardless of storage method a per registry symmetric encryption of storage content is enforced. Keys used for encryption are generated from existing database credentials. Therefore, please note that removing or changing such credentials will render any historic staging data inaccessible.

cleanStagingData() globally removes all staging data with store date prior to the end-of-life age provided. This is a vastly destructive function that should be used with great care.

Examples

## Prep test data
registryName <- "rapbase"
dataName <- "testData"
data <- mtcars
dir <- tempdir()

## Save data for staging
saveStagingData(registryName, dataName, data, dir)

## List data currently in staging
listStagingData(registryName, dir)
#> [1] "testData"

## Retrieve data set from staging and compare to outset
stagedData <- loadStagingData(registryName, dataName, dir)
identical(data, stagedData)
#> [1] TRUE

## Get modification time for staging file(s)
mtimeStagingData(registryName, dir)
#>                  testData 
#> "2023-11-17 07:59:04 UTC"