/ BLOG

DevEnv Setup

  • For purpose of setting local development environment on a new SSD storage, followed instructions below. Post for later use.
  1. Document Enable NVIDIA CUDA on WSL
    • Install stable version of Windows 11
    • Enable WSL, install Ubuntu(20.04.3 LTS)
      On Windows Settings app, select Check for updates in the Windows Update section and get the latest kernel(5.10.43.3 or higher)
      To check the version, run wsl cat /proc/version command in Powershell.
    • Install the GPU driver
      Download and install the NVIDIA CUDA enabled driver for WSL
      (Studio version: 511.65-desktop-win10-win11-64bit-international-nsd-dch-whql)
  2. Install Docker Desktop app on Windows
    • Run:
      docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
    • Result: 
        > Windowed mode
  > Simulation data stored in video memory
  > Single precision floating point simulation
  > 1 Devices used for simulation
  GPU Device 0: "Ampere" with compute capability 8.6

  > Compute 8.6 CUDA device: [NVIDIA GeForce RTX 3070]
  47104 bodies, total time for 10 iterations: 40.275 ms
  = 550.910 billion interactions per second
  = 11018.199 single-precision GFLOP/s at 20 flops per interaction
  3. Setting Docker image for TensorFlow-GPU
    • Pull the latest TensorFlow-GPU image docker run -it --gpus all tensorflow/tensorflow:latest-gpu
    • Install Anaconda on user:root(ref: blog) 
        # update and install prerequisites
  apt-get update
  apt-get install wget
  # get proper version of anaconda3
  wget https://repo.anaconda.com/archive/Anaconda3-2021.11-Linux-x86_64.sh 
  sh Anaconda3-2021.11-Linux-x86_64.sh
  exec bash
  # create anaconda environment and install libraries(for stability)
  conda create -n !env_name pip python=3.7
  conda activate !env_name
  pip install tensorflow-gpu
  pip install ipykernel
  python -m ipykernel install --user --name !env_name --display-name !dispaly_name
  pip install jupyter
  # escape with Ctrl + p, Ctrl + q
  docker commit -m "!message" !container_id !image_name:tag
    • (Optional) Install TensorFlow Object Detection API 
        apt-get install git
  git clone --depth 1 https://github.com/tensorflow/models
  cd models/research/
  apt install -y protobuf-compiler
  # found a symlink err, fixed with running:
  # ln -s /usr/lib/x86_64-linux-gnu/libcuda.so.1 /usr/local/cuda/lib64/libcuda.so
  # and rerun: apt install -y protobuf-compiler
  protoc object_detection/protos/*.proto --python_out=.
  cd models/research/
  # install Object Detection API
  cp object_detection/packages/tf2/setup.py .
  python -m pip install --use-feature=2020-resolver .
  # run test
  python object_detection/builders/model_builder_tf2_test.py
  # rm -rf models (if desired)
    • (Optional) Install JupyterLab Extensions and enable TensorBoard within Jupyterlab-Docker container
      Stable versions worked on my local environment 
        curl -sL https://deb.nodesource.com/setup_12.x | bash -
  apt-get install -y nodejs
  node --version # check: v12.22.10
  npm --version # check: 6.14.16
  pip install jupyterlab==2.3.2 
  pip install git+https://github.com/cliffwoolley/jupyter_tensorboard.git 
  pip install tensorboard==2.2
  jupyter labextension install jupyterlab_tensorboard

      Commit and run container with any open port for JupyterLab
      e.g.

        docker run --rm -it --gpus all -p 4000:4000 !image_name:tag
  conda activate !env_name
  jupyter lab --ip='0.0.0.0' --port=4000 --no-browser --allow-root

      On your Windows, open localhost:4000 with browser

  4. Setting Docker image for Jekyll blog
    • Get latest Ubuntu image and install packages 
        docker run --rm -it -p 4000:4000 ubuntu
  apt-get update
  apt-get install git
  apt-get install vim ruby-full build-essential zlib1g-dev -y

  echo '# Install Ruby Gems to ~/gems' >> ~/.bashrc
  echo 'export GEM_HOME="$HOME/gems"' >> ~/.bashrc
  echo 'export PATH="$HOME/gems/bin:$PATH"' >> ~/.bashrc
  source ~/.bashrc
  gem install jekyll bundler
		
  jekyll -v # 4.2.1
  mkdir -p /root/blog_home
  echo 'export BLOG_HOME="/root/blog_home"' >> ~/.bashrc
  echo '# Start jekyll' >> ~/.bashrc
  source ~/.bashrc
  cd $BLOG_HOME # Get any jekyll blog template here
  rm Gemfile.lock # if needed
  bundle install
  bundle exec jekyll serve --host 0.0.0.0 -p 4000