LAMMPS – Lennard-Jonesium

To get acquainted with Hippolyta, use LAMMPS to model an fcc vacancy in Lennard-Jonesium.¹

Load the LAMMPS module

Start by loading the LAMMPS module and it’s dependencies:

module load lammps-all

In addition to telling your shell where to find the LAMMPS executable, loading the LAMMPS module sets two environment variables: LAMMPS_POTENTIALS and LAMMPS_EXAMPLES. The LAMMPS_POTENTIALS variable tells LAMMPS where to look for the system versions of interatomic potentials to use. The LAMMPS_EXAMPLES variable holds the location of the example simulation scripts distributed with the LAMMPS source code, along with a few example scripts developed by Hippolyta users.

brian@hippolyta > ls ${LAMMPS_EXAMPLES}/
accelerate  COUPLE    ELASTIC      indent   min        peptide  reax     USER
ASPHERE     crack     ELASTIC_T    KAPPA    msst       peri     rigid    vashishta
balance     deposit   ellipse      kim      nanotubes  pour     shear    VISCOSITY
body        DIFFUSE   fccVacancy   MC       nb3b       prd      snap     voronoi
colloid     dipole    flow         meam     neb        python   srd
comb        dreiding  friction     melt     nemd       qeq      streitz
coreshell   eim       hugoniostat  micelle  obstacle   README   tad

Obtain the example files

Copy the fccVacancy example from the ${LAMMPS_EXAMPLES} directory into your ~/data directory to proceed with the example:

cp -r ${LAMMPS_EXAMPLES}/fccVacancy ~/data/
cd ~/data/fccVacancy

The fccVacancy example consists of three files: a LAMMPS input script in.fccVacancy, a job submission script submit_vacancy.sh, and a README file. The LAMMPS script in.fccVacancy contains the LAMMPS commands for the molecular statics simulation.

Submit a LAMMPS script to slurm

The usual way of running LAMMPS is by entering lammps -in in.lammps_script at the command prompt. Similarly, parallel LAMMPS jobs are invoked by entering mpirun -np 4 lammps -in in.lammps_script, which in this case will launch LAMMPS with 4 processes. Invoking LAMMPS this way on Hippolyta will launch heavy LAMMPS processes on the shared head node.

Instead, we want to use slurm to run LAMMPS on the compute nodes. This is typically accomplished by submitting a job submission script using the sbatch command. A job submission script is just a shell script with special comments (beginning with #SBATCH) for passing options to sbatch:

#!/bin/bash
#SBATCH --job-name=vacancy-example
#SBATCH --output=vacancy-example.stdout
#SBATCH --ntasks 4
#SBATCH --time=1:00
#SBATCH --partition=test


# lammps input script as the first argument
#   if supplied
if [ -z "$1" ]
  then
    LAMMPS_SCRIPT="in.fccVacancy"
  else
    LAMMPS_SCRIPT=${1}
fi

mpirun lammps -in ${LAMMPS_SCRIPT}

This example script asks slurm for four processors (for one minute of wall-clock time) on the test partition to run a job named lammps-example. Any output that would normally be printed to the screen is saved to the file lammps-example.stdout. Finally, the job script runs LAMMPS in the usual way. Note that explicitly specifying the -np option to mpirun is unnecessary, since the number of processors has already been requested in an #SBATCH directive.

The #SBATCH directives included in the example script are just a few of the possible directives. You can read more in the online documentation, or by view the sbatch software manual (run man sbatch).

Submit this job script to slurm by entering

sbatch submit_vacancy.sh in.fccVacancy

at the command prompt. sbatch will respond with a job id number, which you can check the status of using the squeue command.

brian@hippolyta > sbatch submit_lammps.sh in.fccVacancy
Submitted batch job 11223
brian@hippolyta > squeue
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
              1837     batch     iso1   philip PD       0:00      1 (PartitionTimeLimit)
             11223      test lammps-e    brian  R       0:01      1 cacophony31

After your job has finished running, you can check that all of the expected LAMMPS output files (dump files, log files, etc) are present:

brian@hippolyta > ls
dump.imperfect  in.fccVacancy  README             vacancy-example.stdout
dump.perfect    log.lammps     submit_vacancy.sh

If you want to analyze the LAMMPS output on your own machine, you can copy the output files using the scp (secure copy) utility. scp works similarly to the cp command, but can copy files over ssh if you specify the address of a remote computer.

From a terminal window on your client machine:

scp -r me@hippolyta.materials.cmu.edu:data/fccVacancy ./

If you’ve set up an ssh config file, you can save some typing:

scp -r hippolyta:data/fccVacancy ./