Development Environment For Parallel Programming

Tool Chain

• gcc
• make
• mpicc/mpiexec
• OpenMP

Operating System Dependent Installation

Linux

• install gcc and make sudo apt install build-essential
• install mpich using sudo apt install mpich

SSH server

• gcc and make pre-installed
• enable MPI using command module load mpi/openmpi-x86_64

Windows WSL (Windows subsystem for linux)

• install Ubuntu 22.04 LTS from the Microsoft Store
• Run it for the first time and set user name and password for Ubuntu
• install gcc and make sudo apt install build-essential
• install mpich using sudo apt install mpich

macOS

Overall, macOS is not recommended. One of the reason is that the OpenMP and MPI library under macOS is different from the one under Linux unless you compile your own from source.

• Install Xcode command line tool (may have compatibility issue)

  By installing the Xcode command-line tool, you get the gcc command that is actually an alias to the clang command. The clang command is the invocation command of the LLVM C compiler rather than the GNU C compiler. It works in most cases but need special handling for packages like OpenMP, etc.

  • install xcode command line tool sudo xcode-select --install
  • use homebrew to install mpich
    • You need to install homebrew first
    • brew install mpich
  • special handling to install OpenMP
    • install libomp brew install libomp
    • configure as needed

• Install the real GCC

  • install gcc brew install gcc

  • alias gcc

    The command gcc and g++ are usually linked to clang and clang++ when Xcode command-line tools are installed. You need to change it back to point to the real gcc and g++.

    In your .bash_profile file, add the following lines (change the suffix 12 to the version number you have):

    alias gcc='gcc-12'
    alias g++='g++-12'
    

    Source the .bash_profile or rerun your shell.

  • install MPI

    • add the following to the .bash_profile (change the suffix 12 to the version number you have):

      export HOMEBREW_CC=gcc-12
      export HOMEBREW_CXX=g++-12
      

    • brew install mpich --build-from-source will take long time to compile

  • OpenMP is built-in in the gcc package

    • compile with -pthread flag

External HPC/Cloud

• Google Colab (https://colab.research.google.com/)
  • free for everyone
  • access to GPU
• access-ci.org
  • NSF Funded
  • access to real cluster
  • may apply GPU node
  • proposal required

Collect Hardware Info

• Comprehensive CPU information
  • Linux shell: lscpu; cat /proc/cpuinfo
  • macOS shell: sysctl -a machdep.cpu
• Architecture
  • shell: uname -m
• Important information
  • Number of cores
  • Architectures
    • x86_64 - Intel/AMD x86
    • aarch64 - ARM
  • CPU features
    • SIMD features: MMX, SSE, AVX, AVX2, AVX512, etc.

Programming language

• C
  • used in all demonstrations
  • recommend to learn by reference search
• C++
  • not officially supported in this class but you can choose to use it
  • more features/libraries
  • can call any C library
  • heterogeneous compilation with C libraries using gcc
  • object-oriented