Source an R script as a Slurm job

This function sources R scripts using Slurm by creating a batch script file and submitting it via sbatch.

sourceSlurm(
  file,
  job_name = NULL,
  tmp_path = opts_slurmR$get_tmp_path(),
  rscript_opt = list(vanilla = TRUE),
  plan = "submit",
  ...
)

slurmr_cmd(
  cmd_path,
  cmd_name = "slurmr",
  add_alias = TRUE,
  bashrc_path = "~/.bashrc"
)

Arguments

file

Character. Path to the R script to source using Slurm.

job_name

Character. Name of the job to be passed to Slurm.

tmp_path

Character. Path to the directory where all the data (including scripts) will be stored. Notice that this path must be accessible by all the nodes in the network (See opts_slurmR).

rscript_opt

List. Options to be passed to Rscript.

plan

A character scalar. (See the_plan).

...

Further options passed to sbatch.

cmd_path

Character scalar. Path (directory) where to put the command function. This is usually your home directory.

cmd_name

Character scalar. Name of the command (of the file).

add_alias, bashrc_path

Logical scalar and character scalar. When add_alias=TRUE it will modify (or create, if non-existent) the .bashrc file to add an alias of the same name of cmd_name. The path to .bashrc can be specified via the bashrc_path option.

Value

In the case of sourceSlurm, Whatever sbatch returns.

The function slurmr_cmd returns invisible().

Details

sourceSlurm checks for flags that may be included in the Slurm job file. If the R script starts with #!/bin/ or similar, then #SBATCH flags will be read from the R script and added to the Slurm job file.

The function slurmr_cmd writes a simple command that works as a wrapper of sourceSlurm. In particular, from command line, if the user wants to source an R script using sourceSlurm, we can either:

$ Rscript -e "slurmR::sourceSlurm('path/to/the/script.R', plan = 'submit')"

Or, after calling slurmr_cmd from within R, do the following instead

$ ./slurmr path/to/the/script.R

And, if you used the option add_alias = TRUE, then, after restarting bash, you can run R scripts with Slurm as follows:

$ slurmr path/to/the/script.R

The main side effect of this function is that it creates a file named cmd_name in the directory specified by cmd_path, and, if add_alias = TRUE. it will create (if not found) or modify (if found) the .bashrc file adding a line with an alias. For more information on .bashrc see here.

Examples

# In this example we will be sourcing an R script that also has #SBATCH
# flags. Here are the contents
file <- system.file("example.R", package="slurmR")

cat(readLines(file), sep="\n")
#> #!/bin/sh
#> #SBATCH --account=lc_ggv
#> #SBATCH --partition=scavenge
#> #SBATCH --time=01:00:00
#> #SBATCH --mem-per-cpu=4G
#> #SBATCH --job-name=Waiting
#> Sys.sleep(10)
#> message("done.")
#> 
# #!/bin/sh
# #SBATCH --account=lc_ggv
# #SBATCH --time=01:00:00
# #SBATCH --mem-per-cpu=4G
# #SBATCH --job-name=Waiting
# Sys.sleep(10)
# message("done.")

# We can directly submit this R script as a job by calling `sourceSlurm`.
# (of course you need Slurm to do this!)
if (FALSE) {
sourceSlurm(file)
}

# The function will create a bash script that is used later to be submitted to
# the queue using `sbatch`. The resulting file looks something like this
# #!/bin/sh
# #SBATCH --job-name=Waiting
# #SBATCH --output=/home/vegayon/Documents/slurmR/Waiting.out
# #SBATCH --account=lc_ggv
# #SBATCH --time=01:00:00
# #SBATCH --mem-per-cpu=4G
# /usr/lib/R/bin/Rscript --vanilla /usr/local/lib/R/site-library/slurmR/example.R