Introduction to Week 1
This week we will cover foundational topics critical to understanding how generative AI works. You do not need to learn the details of the math involved! Your goal is to develop an understanding of key concepts. This first week will likely be the most difficult of the course. Please stay with me! I promise it will be worth it!
Weekly Activity: Part 1
For the first part of this week’s activity, watch the following five videos. This will take approximately 1.5 hours, so leave yourself plenty of time. These videos will introduce you to key concepts associated with neural networks. The StatQuest videos use simple examples and have a lighthearted tone. The 3Blue1Brown videos assume that you’re intellectually curious about the topic and dig into slightly more substantive examples. Repetition is your friend here, so watch all five videos even though they cover the same topics. Again, you can ignore the specifics of the math. You’re just trying to develop a solid conceptual understanding of a few key ideas. Listen for information that will help you answer the following questions:
- What are neural networks?
- What are nodes in a neural network?
- What are connections (or “edges”) in a neural network?
- What are hidden layers in a neural network?
- What are activation functions in a neural network?
- What are weights in a neural network?
- What are biases in a neural network?
- What are parameters in a neural network? (Hint: We’ve already mentioned them above.)
- What is the purpose of gradient descent in a neural network?
- What is the purpose of backpropagation in a neural network?
Statquest Videos
Neural Networks Pt. 1: Inside the Black Box
Neural Networks Pt. 2: Backpropagation Main Ideas
(You can ignore the videos it suggests you watch first.)
3Blue1Brown Videos
But what is a neural network? | Chapter 1, Deep learning
Gradient descent, how neural networks learn | Chapter 2, Deep learning
What is backpropagation really doing? | Chapter 3, Deep learning
Weekly Activity: Part 2
Next you’ll review the 10 key concepts with ChatGPT or Bing Chat.
- If you have a ChatGPT Pro account, you can use that. Make sure you select GPT-4 at the top left as the model you’re using.
- Otherwise, you can use Bing Chat / Copilot. This requires you to use the Edge web browser. When asked to choose a conversation style, choose “More Creative.” That will get you working with GPT-4.
Start by asking ChatGPT / Bing Chat questions about any of the concepts you feel like you’re not understanding yet. Asking it to “explain it to my like I’m 5” can be a great help here!
Then copy and paste the prompt below into ChatGPT / Bing Chat and answer the questions to complete the review. When you are finished, copy and paste the entire review conversation into Assignment 1: Neural Networks in Canvas. This assignment is due by 11:59pm Mountain on Thursday, Jan 11.
I've just finished studying about neural networks. I'm not trying to master the mathematics, I'm just trying to get a conceptual understanding of the key concepts. Give me a quiz where you ask me the following questions:
- What are neural networks?
- What are nodes in a neural network?
- What are connections (or "edges") in a neural network?
- What are hidden layers in a neural network?
- What are activation functions in a neural network?
- What are weights in a neural network?
- What are biases in a neural network?
- What are the parameters in a neural network?
- What is the purpose of gradient descent in a neural network?
- What is the purpose of backpropagation in a neural network?
Ask me one question at a time and wait for my answer. After each answer, give me feedback on my answer and explain anything it seems like I don't understand. Then ask if I'd like additional information on that question. When I indicate I'm finished, ask me the next question.
Don’t worry – I’m not expecting you to master all the key concepts in this first week. But I am expecting you to actively engage in this activity and give your best effort. This activity will be graded all or nothing – if you upload a review transcript that shows you engaged in the entire review process, you will get all 10 points.
Weekly Activity: Part 3
Finally, complete this week’s pre-class survey (should take < 2 minutes). This assignment is due by 9:00am Mountain on Friday, Jan 12.