Week 6: ML with Python

[BekahHW]

Getting Started

The goal this week is to get to "Conclusion" lesson from freeCodeCamp's ML with python and check-in here.

Learning Resources

• https://towardsdatascience.com/q-learning-algorithm-from-explanation-to-implementation-cdbeda2ea187
• https://www.datacamp.com/tutorial/introduction-q-learning-beginner-tutorial
• https://www.youtube.com/watch?v=iKdlKYG78j4

Check-in

1. What is Q-Learning? What is reinforcement learning?
2. What are some pros and cons of Q-Learning?
3. What do you know today that you didn't when you started?
4. What are some key terms or ideas from these lessons?
5. What do you need to be clarified or what questions do you have?
6. What are you excited to learn more about?

[BekahHW]

1. What is Q-Learning? What is reinforcement learning? Algorithm to implement reinforcement learning. An agent navigates states and action and maximizes reward. Matrix of actions and states.
2. What are some pros and cons of Q-Learning?
3. What do you know today that you didn't when you started? Learning more about using Q-tables.
4. What are some key terms or ideas from these lessons? agent, state, action, reward, observation space
5. What do you need to be clarified or what questions do you have? I'm still not clear on how this transitions from what we've been doing before and how to make decisions about what to use.
6. What are you excited to learn more about?

[paceaux]

The Collab for Reinforcement learning:

https://colab.research.google.com/drive/1IlrlS3bB8t1Gd5Pogol4MIwUxlAjhWOQ?usp=sharing

[BekahHW]

thank you!

[dominicduffin1]

Y'all may have noticed I haven't really been posting in these discussions, unfortunately the time I was hoping to devote to this cohort ended up being swallowed up so I had to drop out from following along. I'm still excited to learn more about this stuff and I hope to get back to the content someday..

[paceaux]

1. What's Q-Learning? It's a kind of reinforcement learning. RL is a way to simulate motivations and incentives as they relate to learning.
2. Pros and cons? Not sure. This seems to be a useful way to discover or evaluate human behaviors?
3. What I know now that I didn't before? that this exists? The problem though is that it's all about maximizing reward. You have to strike the right balance of risk vs. reward to learn something. You need enough reward to solicit change, but not so much that it gives an unnatural result.
4. Key terms/ideas:
   • environment: the thing we're trying to solve/do
   • agent: the explorer within the environment
   • state: the status of the agent
   • action: the thing the agent does in the environment
   • reward: what the agent is trying to maximize while in the environemnt
   • local maxima: the maximum amount the agent "earns"
5. Excited to learn more about? I would actually want to try this with language learning. First language acquisition is a kind of reinforcement learning so I wonder if this could be used to model how babies or non-natives acquire language.

[rilhia]

This took me back a few years to my uni days. I used to love software agents. Give SARSA a go, if you haven't already. I took a look at this tutorial (https://www.datacamp.com/tutorial/introduction-q-learning-beginner-tutorial) and had a play with it. It should be noted that using the OpenAI FrozenLake environment creates extra problems for Q Learning. Since falling into a hole ends the session with no reward at all, every time your agent doesn't reach the goal, it essentially wastes the training episode. This makes it really inefficient to get your agent to learn the environment. So I created a "walled" environment. It is not possible for the agent to step into a hole. If it attempts to, it uses a step and stays where it is. This way, the agent can reach the end (given enough steps for the training episode) and the reward for reaching the end can be distributed across the whole route. It is SO MUCH more efficient to train your agent in this way.

Prior to changing this, an 8x8 map with 2 rows with only 1 frozen square in each, was difficult to get the agent learning a route. After the modification to the environment I was able to get a 150x150 environment, with a significant number of holes, to be learnt reasonably quickly. I've attached a GIF of the map and the agent navigating it. Oh, I also added diagonal movement to make it more interesting. You will need to zoom in to see the agent moving.