Code

As theorists we often write complex computer code to generate the results which we present in our papers. We’re making that code openly available so that others can use it in their own research, or simply check that what we’re doing is correct. Here, you’ll find the code, frameworks and documentation we write to support our research.

Output	status Field	Description
currRealMem	VmRSS	Resident set size
peakRealMem	VmHWM	Peak resident set size ("high water mark")
currVirtMem	VmSize	Virtual memory size
peakVirtMem	VmPeak	Peak virtual memory size

Linux memory profiler

Output to Mathematica
Convert Python structures to Mathematica structures

Get the code here.

The module

from mmaformatter import get_mma, save_as_mma

provides functions for converting Python floats, complex numbers, lists, tuples, sets and dictionaries to their corresponding Mathematica form, e.g. scientific notation, associations. An equivalent module for C can be found here.

One can pass any nested structure of arrays, tuples, sets, dictionaries (of primitives ints, floats, comple numbers, bools and strings) to get_mma to get strings of their Mathematica form.

For example

>>> get_mma(-1/999) '-1.00100*10^-03' >>> get_mma(1/(3+2j), precision=2) '2.31*10^-01-1.54*10^-01I' >>> get_mma([1, 1/2, 3j]) '{1, 5.00000*10^-01, 0+3I}'

and here printing the output (for clarity)

>>> get_mma({1:2, "cat":"dog"})

<| 1 -> 2, "cat" -> "dog" |>

>>> get_mma( {"Tyson":(0,0,7), "nest":{"birds":4}}, keep_symbols=True)

<| Tyson -> {0, 0, 7}, nest -> <| birds -> 4 |> |>

The function save_as_mma can take the same structures and write their Mathematica form to file, to be read by Mathematica.

For example,

data = [1.013, 10.331, .59102, 5.0, 6.0] struc = {"patient":"Tyson", "trial":112, "conductivity":data} save_as_mma(struc, "assoc.txt", precision=2)

produces a file "assoc.txt" with contents

<| "trial" -> 112, "patient" -> "Tyson", "conductivity" -> {1.01*10^+00, 1.03*10^+01, 5.91*10^-01, 5, 6} |>

which can then be read into Mathematica via

SetDirectory @ NotebookDirectory[]; assoc = Get["assoc.txt"] ListLinePlot @ assoc["conductivity"]

which produces the very-terrific graph

Both get_mma and save_as_mma accept optional arguments

key_order: use explicit ordering in the Mathematica association for the keys in the passed dictionary

keep_symbols: whether python strings should be converted to Mathematica symbols, else be enquoted to become strings

precision: the number of decimal digits in the scientific notation format of floats

keep_ints: whether integers should be kept formatted as such, else converted to scientific notation (at the precision supplied); applies to the real and imaginary components of complex numbers too

view raw Py_MMA_Formatter.md hosted with by GitHub

Output	status Field	Description
currRealMem	VmRSS	Resident set size
peakRealMem	VmHWM	Peak resident set size ("high water mark")
currVirtMem	VmSize	Virtual memory size
peakVirtMem	VmPeak	Peak virtual memory size

SLURM param sweeper

Get the code here.

This python library generates SLURM submission scripts which launch multiple jobs to 'sweep' a given set of parameters; that is, a job is run for every possible configuration of the params.

For example, you might have a script which accepts parameters a, b and c, which you call (passing a=1, b=2, c=3) like

myscript.sh 1 2 3

inside a SLURM submission script.

Say that a can be any value in [1, 2, 3], while b is in [4, 5, 6] and c is in [7, 8, 9]. How can you run myscript.sh once for every possible configuration of a, b and c? Creating multiple submission scripts which each contain one of

myscript.sh 1 4 7
myscript.sh 1 4 8
myscript.sh 1 4 9
myscript.sh 1 5 7
...
myscript.sh 3 6 9

becomes impractible for many parameters. While you can use the SLURM field --array=0-26 to launch multiple jobs in a single script, writing bash code to map $SLURM_ARRAY_TASK_ID to a unique a,b,c configuration can be tedious. param_sweeper.py creates a single submission script which does this mapping for you.

example

param_sweeper.py exposes get_script and save_script which accept a list of SLURM fields (e.g. job_name) and a dictionary where each key is a parameter name and each item is a list of values the parameter can take.

For example,

from param_sweeper import save_script

fields = {
    'memory': 8,
    'job_name': 'abc_sweep',
    'time_h': 1
}

params = {
    'a':[1, 2, 3],
    'b':[4, 5, 6],
    'c':range(7,10)
}

save_script('mysubmit.sh', fields, params)

creates a file mysubmit.sh containing

#!/bin/env bash

#SBATCH --array=0-26
#SBATCH --job-name=abc_sweep
#SBATCH --mem=8GB
#SBATCH --time=0-1:0:0

a_values=( 1 2 3 )
b_values=( 4 5 6 )
c_values=($( seq 7 1 10 ))

trial=${SLURM_ARRAY_TASK_ID}
a=${a_values[$(( trial % ${#a_values[@]} ))]}
trial=$(( trial / ${#a_values[@]} ))
b=${b_values[$(( trial % ${#b_values[@]} ))]}
trial=$(( trial / ${#b_values[@]} ))
c=${c_values[$(( trial % ${#c_values[@]} ))]}

## use ${a}, ${b}, ${c} below

which, after adding myscript.sh $a $b $c to the bottom, can be submitted to SLURM via sbatch mysubmit.sh.

customisation

param_sweeper.py contains a constant TEMPLATE which should be edited to add/remove custom code in the generated submission script. For example, the template

TEMPLATE = '''

#!/bin/env bash

#SBATCH --array=0-{num_jobs}
#SBATCH --job-name={job_name}
#SBATCH --output={output}
#SBATCH --mem={memory}{memory_unit}
#SBATCH --time={time_d}-{time_h}:{time_m}:{time_s}
#SBATCH --nodes={num_nodes}
#SBATCH --cpus-per-task={num_cpus}
#SBATCH --reservation={reserve}

{param_arr_init}

trial=${{SLURM_ARRAY_TASK_ID}}
{param_val_assign}

source ../../prep.sh
export OMP_NUM_THREADS={num_cpus}
export OMP_PROC_BIND=spread

## use {param_list} below
myscript.sh $a $b

'''.strip()

when combined with the default fields

DEFAULT_SLURM_FIELDS = {
    'memory': 64,
    'memory_unit': 'GB',
    'num_nodes': 1,
    'num_cpus': 16,
    'time_d': 0, 'time_h': 0, 'time_m': 0, 'time_s': 0,
    'reserve': 'nqit',
    'job_name': 'myjob',
    'output': 'output.txt'
}

after calling

script = get_script({}, {'a':range(100), 'b':range(100)})
print(script)

produces

#!/bin/env bash

#SBATCH --array=0-9999
#SBATCH --job-name=myjob
#SBATCH --output=output.txt
#SBATCH --mem=64GB
#SBATCH --time=0-0:0:0
#SBATCH --nodes=1
#SBATCH --cpus-per-task=16
#SBATCH --reservation=nqit

a_values=($( seq 0 1 100 ))
b_values=($( seq 0 1 100 ))

trial=${SLURM_ARRAY_TASK_ID}
a=${a_values[$(( trial % ${#a_values[@]} ))]}
trial=$(( trial / ${#a_values[@]} ))
b=${b_values[$(( trial % ${#b_values[@]} ))]}

source ../../prep.sh
export OMP_NUM_THREADS=16
export OMP_PROC_BIND=spread

## use ${a}, ${b} below
myscript.sh $a $b

arguments

Both get_script (returns a string) and save_script (writes to file) accept an optional argument param_order which explicitly specifies in what order to sweep the given parameters. E.g.

get_script(
    {}, 
    {'a':range(10), 'b':range(10), 'c':range(10)},
    param_order=['c','a','b']
)

produces a submission script which contains

c_values=($( seq 0 1 10 ))
a_values=($( seq 0 1 10 ))
b_values=($( seq 0 1 10 ))

trial=${SLURM_ARRAY_TASK_ID}
c=${c_values[$(( trial % ${#c_values[@]} ))]}
trial=$(( trial / ${#c_values[@]} ))
a=${a_values[$(( trial % ${#a_values[@]} ))]}
trial=$(( trial / ${#a_values[@]} ))
b=${b_values[$(( trial % ${#b_values[@]} ))]}

and which would launch jobs where c (the first param in param_order) is iterated the fastest:

myscript.sh 0 0 0
myscript.sh 0 0 1
myscript.sh 0 0 2
...
myscript.sh 1 0 0
myscript.sh 1 0 1
...
...
myscript.sh 0 1 0
...

view raw Py_SLURM_param_sweeper.md hosted with

by GitHub

WordPress Tabs

4 Comments

Tyson Jones · January 10, 2018 at 6:15 pm

Feel free to comment below (using a WordPress account) to ask questions, point out errors or suggest changes, or otherwise email me at

firstname.lastname@materials.ox.ac.uk

Simon Benjamin · April 17, 2018 at 2:38 pm

Instructions worked like a charm, thanks Tyson!
Two minor things:
1. The in-page links from the contents to the item don’t seem to work in Chrome.
2. For the examples, maybe add “gsl_matrix_get(myMatr, 1,2);” after populating the matrix, to show users that they can get the number out with …_get

Simon Benjamin · April 21, 2018 at 12:39 am

Something for the GSL Guide under “construct matrices and vectors”… It really useful to stress that the matrices and vectors are NOT zero initialised, so they are fully of random junk when instantiated (albeit the random junk is quite likely to be a zero). Programmer must take care to set every value.
I forgot this and spent ages chasing a bug.

Simon Benjamin · April 21, 2018 at 5:07 pm

Extra: I see that things like gsl_permutation_calloc (i.e. ending in calloc not alloc) will initialise to zero for you.

Measure memory of a Linux process in C

Generate Mathematica Associations in C

Measure memory of a Linux process in Python

Convert Python structures to Mathematica structures

SLURM param sweeper

example

customisation

arguments

Delay an enqueued job from running

Requeue and immediately delay running jobs

Installation

load a C11 compiler and anaconda

prepare an anaconda environment

check pip

clear any previous attempts to build projectq

install pybind11

build ProjectQ

check installation was successful

possible errors

Usage

In the frontend

In a job

ARCUS-B:

ARCHER:

GSL

Installation

OSX

Ubuntu

ARCUS-B

Compilation

OSX

Ubuntu

ARCUS-B

Usage

import

functions

construct matrices and vectors

multiply matrices and vectors

solve linear equations

numerical errors

Connecting

Usage

1. Setup passphrase-free SSH

Windows

Linux & MacOS

2. Create the remote kernel

Windows

Linux & MacOS

3. Connect to the remote kernel

4 Comments

Tyson Jones · January 10, 2018 at 6:15 pm

Simon Benjamin · April 17, 2018 at 2:38 pm

Simon Benjamin · April 21, 2018 at 12:39 am

Simon Benjamin · April 21, 2018 at 5:07 pm

Leave a Reply Cancel reply