Installing Open AI Gym in a Windows Environment

Lesson 1

1.  Ensure Visual Studio 2017 is installed/Python 3.5+ (conda - Anaconda)

2.  Open command prompt:  conda create -n gym python=3 pip

3. activate gym

4. pip install gym

5. conda install pystan

6. conda install git

7. pip install git+

8. conda install swig

9. pip install Box2D

10.  pip install gym[all]   (*ignore any errors)

11. pip install pyglet==1.2.4

12. pip install gym[box2d]

13.  Install Xming  - download from

14. Install Visual Studio Code, add Python extension

15.  Run Visual Studio Code

16. Put this code into a new file:

import gym

env = gym.make('CartPole-v0')

for i_episode in range(20):

observation = env.reset()

for t in range(100):



action = env.action_space.sample()

observation, reward, done, info = env.step(action)

if done:

print("Episode finished after {} timesteps".format(t+1))



17.  Save it as a python file somewhere.

18.  At bottom left of Visual Code screen, click on the Python version listed and then select the gym environment interpreter from the list of environments.

19. hit Ctrl ` to open up the terminal at bottom of screen.

20. type set DISPLAY=:0

19. Right click anywhere in the code and select "Run Python File in Terminal"

A white screen should pop up with a cart trying to balance a pole and a bunch of numbers will scroll by in the terminal area.