Using highway‑env with OpenAI Gym for Reinforcement Learning: Installation, Configuration, and DQN Training

The article introduces gym and the highway‑env package as a lightweight reinforcement‑learning environment for autonomous driving, describing six built‑in scenarios such as highway‑v0, merge‑v0, and parking‑v0.

Installation is performed via pip install gym and pip install --user git+https://github.com/eleurent/highway-env . After installation, the environment can be instantiated with env = gym.make('highway‑v0') and rendered.

The environment provides three observation modes: Kinematics (a V×F matrix of vehicle features), Grayscale Image (W×H pixel map), and Occupancy Grid (W×H×F tensor). The article shows how to configure a Kinematics observation with a JSON‑like config dictionary specifying vehicle count, selected features, feature ranges, and other parameters.

Action space includes five discrete meta‑actions (LANE_LEFT, IDLE, LANE_RIGHT, FASTER, SLOWER) defined in ACTIONS_ALL . The default reward function is illustrated with an image and noted to be modifiable only in the source code.

For the learning algorithm, a simple DQN network is defined in PyTorch with two linear layers (35→35 and 35→5). The DQN class manages a replay buffer, epsilon‑greedy action selection, and periodic learning updates. Key hyper‑parameters such as GAMMA = 0.9 , LR = 0.01 , and BATCH_SIZE = 80 are listed.

A training loop repeatedly resets the environment, selects actions, steps the simulator, stores transitions, and calls dqn.learn() every 99 steps. After each episode, episode time, reward, and collision information are recorded, and every 40 training steps the average metrics are plotted using matplotlib .

Result visualizations show decreasing collision rates, increasing episode duration, and improving average reward as training progresses. The author concludes that highway‑env offers a more abstract and convenient platform than CARLA for end‑to‑end RL research, though it provides limited control over low‑level vehicle dynamics.