************************************************
Development Environment For Parallel Programming
************************************************

Tool Chain
==========
+ gcc
+ make
+ mpicc/mpiexec
+ OpenMP

Operating System Dependent Installation
=======================================
Linux
-----
* install gcc and make ``sudo apt install build-essential``
* install mpich using ``sudo apt install mpich``

SSH server
----------
* gcc and make pre-installed
* enable MPI using command ``module load mpi/openmpi-x86_64``

Windows WSL (Windows subsystem for linux)
-----------------------------------------
* install Ubuntu 22.04 LTS from the Microsoft Store
* Run it for the first time and set user name and password for Ubuntu
* install gcc and make ``sudo apt install build-essential``
* install mpich using ``sudo apt install mpich``

Mac OS
------
Overall, Mac OS is not recommended. One of the reason is that the OpenMP and
MPI library under Mac OS is different from the one under Linux unless you
compile your own from source.

+ Install XCode command line tool (may have compatibility issue)

  By installing the XCode command-line tool, you get the ``gcc`` command that
  is actually an alias to the ``clang`` command. The ``clang`` command is the
  invocation command of the LLVM C compiler rather than the GNU C compiler. It
  works in most cases but need special handling for packages like OpenMP, etc.

  * install xcode command line tool  ``sudo xcode-select --install``
  * use ``homebrew`` to install mpich

    - You need to install homebrew first
    - ``brew install mpich``

  * special handling to install OpenMP

    - install libomp ``brew install libomp``
    - configure as needed

+ Install the real GCC

  * install gcc ``brew install gcc``
  * alias gcc

    The command ``gcc`` and ``g++`` are usually linked to ``clang`` and
    ``clang++`` when XCode command-line tools are installed. You need to change
    it back to point to the real ``gcc`` and ``g++``.

    In your ``.bash_profile`` file, add the following lines (change the suffix
    12 to the version number you have)::

      alias gcc='gcc-12'
      alias g++='g++-12'

    Source the ``.bash_profile`` or rerun your shell.

  * install MPI

    - add the following to the ``.bash_profile`` (change the suffix
      12 to the version number you have)::

        export HOMEBREW_CC=gcc-12
        export HOMEBREW_CXX=g++-12

    - ``brew install mpich --build-from-source`` will take long time to compile

  * OpenMP is built-in in the gcc package

    - compile with ``-pthread`` flag

External HPC/Cloud
------------------
* Google Colab (https://colab.research.google.com/)

  - free for everyone
  - access to GPU

* access-ci.org

  - NSF Funded
  - access to real cluster
  - may apply GPU node
  - proposal required

Collect Hardware Info
=====================
+ Comprehensive CPU information

  * Linux shell: ``lscpu``; ``cat /proc/cpuinfo``
  * Mac OS shell: ``sysctl -a machdep.cpu``

+ Architecture

  * shell: ``uname -m``

+ Important information

  * Number of cores
  * Architectures

    - x86_64 - Intel/AMD x86
    - aarch64 - ARM

  * CPU features

    - SIMD features: MMX, SSE, AVX, AVX2, AVX512, etc.

Programming language
====================
+ C

  * used in all demonstrations
  * recommend to learn by reference search

+ C++

  * not officially supported in this class but you can choose to use it
  * more features/libraries
  * can call any C library
  * heterogeneous compilation with C libraries using gcc
  * object-oriented