Installing Open AI Gym in a Windows Environment

Lesson 1

1.  Setup your Visual Studio settings (from Visual Studio Installer).

2.  Add to your system PATH - C:\Program Files(x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\Extensions\Microsoft\Python\Miniconda\Miniconda3-x64\Scripts

Afterwards your PATH environment variable should look like similar to this.You can have other things in your environment variables, just make sure what's here are also there in the order listed (use move up/move down buttons to move things around).  Check your user path as well as your system path and make sure there isn't any reference to other python directories.

  1. Install C++/WinRT from here.
  2. open a command prompt as administrator
  3. conda create -n gym python=3 pip
  4. activate gym
  5. Check pip version by typing pip --version
  6. If it's 19.1.1 or higher skip step 7
  7. python -m pip install --upgrade pip
  8. pip install gym
  9. pip install pylint
  10. conda install pystan
  11. conda install git
  12. conda install swig
  13. pip3 install Box2D --no-cache-dir
  14. pip3 install gym[all]   (*ignore any errors)
  15. pip install pyglet==1.2.4
  16. pip install gym[box2d]
  17. pip install git+
  18. Install Xming (start when finished)
  19. Install Visual Studio Code, add Python extension
  20. Run Visual Studio Code
  21. Put this code into a new file and save as a python file ( somewhere:
import gym

env = gym.make('CartPole-v0')

for i_episode in range(20):
	observation = env.reset()
	for t in range(100):
		action = env.action_space.sample()
		observation, reward, done, info = env.step(action)
		if done:
			print("Episode finished after {} timesteps".format(t+1))

22.  At bottom left of Visual Code screen, click on the Python version listed and then select the gym environment interpreter from the list of environments like in this video.

23. hit Ctrl ` to open up the terminal at bottom of screen.

24. type set DISPLAY=:0

25. Right click anywhere in the code and select "Run Python File in Terminal"

A white screen should pop up with a cart trying to balance a pole and a bunch of numbers will scroll by in the terminal area.  If you end up with a list of import errors, continue on to step 26.

26.  Uninstall Git for Windows.

27.  Reinstall Git for Windows with following options:

  • Choosing default editor used by Git - select "Use Visual Studio Code as Git's default editor"
  • Adjusting your path environment - leave Git from command line and also from 3rd party software checked (Recommended option)
  • Choosing HTTPS transport backend - leave Use the OpenSSL library checked
  • Configuring line ending conversions - leave checkout windows-style checked
  • Configuring the terminal emulator to use with Git Bash - change to Use Windows' default console window
  • Leave everything else as defaults.

28.  Retry steps 20-25.  

If you see the cart trying to balance a pole - congratulations - your environment is setup.