When I first tried Q-learning, I was surprised how much time the agent needed to spend in the exploration phase before it was able to make meaningful progress. So I decided to make a very detailed simulation of a Q-learning agent in a very simple environment to help build intuition on how the agent learns. The environment I chose is called NChain.
I made the simulation at Hack Lodge intending to use it in an educational Youtube video, but never got around to making the video. If I ever circle back to it, I'll write a more detailed explanation of what's going on in †he simulation here.