Live-Sessions on Tools in Data Science

Mon, 01 Jan 0001 00:00:00 +0000

Live Session - 2#

Duration: 0h 29m

Here’s an FAQ-style summary of the Tools in Data Science (TDS) live tutorial:

Tools in Data Science (TDS) Live Tutorial FAQ#

Q1: I’m having trouble hearing; your voice is buzzing and echoing.

A1: Apologies for the audio issue; this is the first time we’ve encountered this. I’ve adjusted my microphone, and hopefully, it’s clearer now. Please let me know if it’s still buzzing.

Mon, 01 Jan 0001 00:00:00 +0000

01 - Intro to Web scraping and HTML#

Duration: 1h 33m

Here is an FAQ for the TDS live tutorial:

Q1: What will be covered in this tutorial?

A1: This introductory session will cover the basics of web scraping and HTML. We will explore how websites are structured, how to retrieve specific information from them, and the potential applications of these techniques.

Mon, 01 Jan 0001 00:00:00 +0000

03 - Intermediate web scraping use of cookies#

Duration: 2h 19m

This is a summary of the Q&A from a web scraping tutorial on Tools in Data Science (TDS).

Q1: Why are we wasting so much time on web scraping when we can just find information manually with our eyes?

A1: Web scraping is useful for several reasons:

Dynamic Data: Websites with constantly changing data (like stock market prices or real-time world population) can be scraped to get up-to-date information efficiently.
Large Datasets: When dealing with huge amounts of data (thousands or millions of rows daily, as in research data or large tables), manual entry is impractical and time-consuming. A script can gather this data in minutes, saving hours of effort.
Automation: Once a script is written, it can be run daily to collect new data without manual intervention, saving repetitive work.
Data Structuring: Scraping allows you to extract data and organize it into structured formats (like tables or CSV files) for further analysis, which is difficult to do manually from web pages.

Q2: In real-world practical applications, do we open the inspect page, identify tags, and then write Python code accordingly?

Mon, 01 Jan 0001 00:00:00 +0000

02 - Fundamentals of web scraping with urllib and BeautifulSoup#

Duration: 1h 37m

Here’s an FAQ based on the live tutorial, incorporating student questions and succinct answers:

Q1: What tools do I need for web scraping and how do I set up my Python environment?

A1: You’ll need a web browser (any browser works, but I often use Wikipedia as a reference) and a Python environment. While Jupyter Notebook is an option, I recommend using Google Colab. It’s an online Python notebook that processes everything on Google’s servers, so it doesn’t burden your local computer’s CPU.

Mon, 01 Jan 0001 00:00:00 +0000

04 - XML intro and scraping#

Duration: 2h 3m

This live tutorial, “Tools in Data Science (TDS)”, covers XML scraping and data handling. Here’s an FAQ summary:

Q1: What is XML, and how does it compare to HTML and JSON?

A1: XML stands for Extensible Markup Language. It’s similar to HTML in using tags, but unlike HTML, XML tags are not predefined. You can create your own tags in XML (e.g., <note>, <to>). This makes XML less about displaying content (like HTML) and more about giving structure to data for storage and transfer. Compared to JSON, XML can be less compact, but both are used for data structuring.

Mon, 01 Jan 0001 00:00:00 +0000

05 ROE revision session jan 24#

Duration: 2h 56m

Here’s an FAQ-style transcription of the live tutorial, addressing common questions and providing concise answers.

ROE Exam Preparation FAQ#

Q1: What are the general rules and format of the ROE exam?

A1: The ROE exam is not proctored, meaning you are allowed to use any resources, including friends, ChatGPT, or even data scientists in your family. It’s completely open book. The exam duration is 45 minutes, as specified in your email. It’s structured like a timed graded assignment; the portal opens at a specific time (e.g., 8:00 AM) and closes precisely at the end (e.g., 8:45 AM). Questions will be multiple-choice (MCQ) or Numerical Answer Type (NAT). You will not be required to upload files, but you may need to download several files via provided links during the exam.

Mon, 01 Jan 0001 00:00:00 +0000

TA session : PYQ Dec 23 discussion#

Duration: 1h 12m

This is an FAQ summary of a live tutorial.

Q1: What is this session about?

A1: This session is about discussing previous year question papers. Specifically, we’ll be reviewing the December 2023 question paper.

Q2: What was different about the December 2023 exam paper compared to previous ones?

A2: Historically, exams had 60 questions and were 1.5 hours long. The December 2023 paper was different, with only 21 questions. These questions were also very conceptual and a bit tough.

Mon, 01 Jan 0001 00:00:00 +0000

TA session : PYQ April 23 discussion part - 1#

Duration: 1h 44m

Here’s an FAQ summary of the live tutorial, focusing on student questions and concise answers:

Q1: What’s the main difference in the question paper format we’re discussing today (April 2023) compared to yesterday’s (December 2023)?

A1: Yesterday’s paper had only 21 highly conceptual questions. Today’s paper has 62 questions, which are mostly theoretical. If you know the answer, you know it.

Mon, 01 Jan 0001 00:00:00 +0000

TA session : PYQ April 23 discussion part - 2#

Duration: 2h 9m

Here’s an FAQ based on the live tutorial, structured to address your points:

Q1: What’s the best way to approach learning TDS content?

A1: To approach TDS content effectively, I recommend revising topics, watching lectures, and actively making notes, especially focusing on library names, tools, and their functions. Self-study on your own is also very beneficial.

Mon, 01 Jan 0001 00:00:00 +0000

Duration: 1h 2m

Here’s an FAQ-style transcription of the Tools in Data Science (TDS) live tutorial:

Q1: What’s the correct code to extract scheduled airline details from a webpage, given table[0] and table[1] as options?

A1: Without knowing the specific structure of the webpage we’re scraping (e.g., from Wikipedia, which can have many tables), it’s impossible to definitively choose. Both table[0] (the first table) and table[1] (the second table) refer to valid tables. In an exam setting where the source isn’t provided, both options might be considered correct due to ambiguity. However, typically, the main table of interest is often table[0].

Mon, 01 Jan 0001 00:00:00 +0000

TA session 13 June : Introduction and week 1,2 recap#

Duration: 2h 2m

Here’s an FAQ based on the live tutorial:

Q1: What are your names and roles?

A1: I’m Amit, and I’m a Teaching Assistant (TA) for the TDS course, this being my third term. Carton is also a TA; he was a student last term. Please call us by our names, Amit and Carton, not “sir,” as we are all essentially students here.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 16 June : Introduction to HTML and web scraping#

Duration: 2h 7m

Here’s a summary of the TDS live tutorial structured as an FAQ:

TDS Live Tutorial FAQ: Web Scraping Basics#

Q1: What will we cover in this session?

A1: We’ll start from scratch with web scraping. This includes creating a very simple website, understanding how to read and interpret the structure of a website (even complex ones like Wikipedia), and the fundamental concepts of web scraping. We’ll also cover a special topic: how to scrape websites that require authentication or login using “cookies.” We’ll practice on real-world websites like Wikipedia. If time permits, we’ll delve into more advanced topics.

Mon, 01 Jan 0001 00:00:00 +0000

A session 23 June : Web scraping with cookies and understanding JSON#

Duration: 1h 57m

Here’s an FAQ based on the live tutorial:

Q1: I’m trying to open a specific badge page on the Discourse platform. It works fine in my regular browser, but when I open it in an Incognito window, it just shows a login page. Why is this happening?

Mon, 01 Jan 0001 00:00:00 +0000

TA session 20 June : Week 2 concepts ,API, JSON, PDF scraping and GA discussion#

Duration: 2h 32m

Here’s an FAQ summary of the live tutorial:

Q1: I’m trying to scrape Amazon product pages, but I can only extract data from the first page. How can I navigate to the second page and extract products from there too? (0:00)

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 27 June : Processing files with Pandas, Open Refine and Week 3 Graded Assignment#

Duration: 1h 51m

Here’s an FAQ-style summary of the tutorial, incorporating your requirements:

Tools in Data Science (TDS) Live Tutorial: Week 3 Graded Assignment FAQ#

Q1: What’s the plan for today’s session?

A1: Today, we’ll focus on the Week 3 content, primarily the graded assignment. I’ll quickly go over some initial topics to ensure everyone is on the same page.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 30 June : More complex webscraping#

Duration: 2h 5m

Here’s an FAQ based on the provided transcript:

Q1: What is the main problem we are trying to solve with web scraping today?

A1: We’ll be given a web page, which acts as an index. From this page, you’ll find some initial information, like user names. These names are actually links to other individual pages. Your task is to visit each of those linked pages, scrape more specific information from them, and then consolidate all this extracted data into a single file for potential visualization. It’s a slightly more complex, real-world web scraping scenario.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 4 July: More Pandas, Parquet processing, Forecast.ETS, Database Processing and Week 4 GA#

Duration: 1h 48m

Here’s the FAQ-style transcription of the TDS live tutorial:

Q1: What is the agenda for this session?

A1: Today’s session will cover topics from week 4, go through the graded assignment, and troubleshoot any issues you encountered.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 7 July: Project Discussion, and Excel Filters, Slope Regression#

Duration: 1h 51m

Here’s an FAQ summary of the TDS tutorial:

Q1: Quiz 2 just finished, and I haven’t had a chance to look at the project yet. Could you give a brief overview of the project now, and then perhaps a more detailed one later when more students are available?

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 11 July: LLM APi Setup and Use#

Duration: 2h 48m

Here’s an FAQ based on the provided transcript, summarizing the student questions and instructor’s answers.

Q1: What is the content for today’s session?

A1: Today’s session is on Week 5 content. We’ll be discussing the first two videos in depth.

Q2: I haven’t watched the Week 5 lectures yet. Should I have watched them already?

Mon, 01 Jan 0001 00:00:00 +0000

“TA session 12 July : week 5 , LLM API and embedding”#

Duration: 1h 54m

Here’s an FAQ based on the TDS tutorial:

Tools in Data Science (TDS) Live Tutorial: LLM Techniques#

This FAQ summarizes a live tutorial on advanced Large Language Model (LLM) techniques, including aspect-based sentiment, LLM extraction, JSON schema, and function calling.

Q1: What’s the main goal of today’s session compared to yesterday’s?

Mon, 01 Jan 0001 00:00:00 +0000

TA session 14 July : ROE intro and strategy#

Duration: 1h 15m

Here’s a detailed FAQ based on the provided transcript, summarizing student questions and instructor answers in a conversational style.

Tools in Data Science (TDS) - ROE FAQ#

Q1: What exactly is ROE? Is it an open-book exam?

A1: ROE stands for “Review of Everything”. It’s an open-book exam, which means you can use any resources you want. You can ask a friend to sit with you, use ChatGPT, Google, or anything else you find helpful.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 18 July: The best ROE Survival Strategies, Tricks and Tips: Part 1#

Duration: 2h 12m

Here’s an FAQ based on the provided transcript:

Tools in Data Science (TDS) Live Tutorial FAQ#

Q1: What is the agenda for this session?

A1: This session will delve deeper into some of the key topics Amit outlined in the last session, focusing on essential skills you’ll need for ROE. I won’t cover every single detail, but I’ll provide enough depth to help you succeed.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 19 July: ROE Survival : Part 2#

Duration: 2h 19m

Here’s an FAQ summary of the TDS live tutorial:

Q1: What are the arrangements for the ROE zip file, and how do I access it?

A1: The zip file will be made available via email and possibly a drive link ahead of the ROE start time. It’s password-protected and contains several files/datasets. You won’t be able to open it until ROE officially begins (8 PM Indian Standard Time tomorrow). The password will be delivered to you via the portal during the exam. Once you have the password, you’ll need to unzip the file to access the data.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 19 August: Streamlit, Heroku#

Duration: 1h 47m

Here’s an FAQ summary of the live tutorial:

Q1: What is Streamlit and why should I use it for deploying applications? (0:16, 2:03, 2:45)

A1: Streamlit is an application that allows you to quickly and easily create and deploy interactive web applications. If you’ve developed an application and want customers to use and interact with it, Streamlit offers a “happy middle ground” compared to other methods. You avoid the complexities of building a full front-end with technologies like Flask, HTML, or CSS, which is a significant undertaking. It’s very quick and easy to deploy your apps.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 22 August: Mock and End Term, Revision: Modules 1 to 3#

Duration: 2h 6m

Here’s an FAQ-style summary of the live tutorial:

TDS Live Tutorial: End Term Preparation FAQ#

Q1: What is the main agenda for today’s session?

A1: Today, I’ll be giving you a mock quiz for the final exam. After you attempt it, I’ll explain how it was designed, which might be useful for your other courses as well. If time permits, I’ll also provide a quick overview of the important topics and terms from the first three modules of TDS.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 25th August: Revision: Modules 4 to 6#

Duration: 1h 51m

Here’s an FAQ-style transcription of the TDS live tutorial:

Q1: Will you share the model you mentioned in the last session for generating mocks? (0:39, 1:20)

A1: I haven’t had a chance to discuss that with Anand yet (who built it at his own expense). I’m not sure if it’s ready to be released to all students. I’ll find out first if it’s possible. If not, the alternative of uploading your own data to create a model would require a lot of information and tokens, which might be an issue.

Mon, 01 Jan 0001 00:00:00 +0000

TA Session 29th August: Revision: Modules 7 to 9, CSS selectors, Javascript#

Duration: 1h 25m

Here’s an FAQ summary of the tutorial:

Q1: What should I expect from the TDS end-term exam?

A1: The end-term exam will have between 40 to 60 questions, which should be fairly quick to answer with no complex math involved. I suggest completing the TDS paper first, as you can breeze through it rapidly. Don’t leave any questions unanswered; try to mark something for every question, as they will be either MCQ or MSQ format. You should be able to finish in about 30-35 minutes, leaving you plenty of time for other subjects.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 09 29 19 49 IST – Recording#

Duration: 1h 40m

Here’s an FAQ based on the provided TDS live tutorial:

Q1: How do I know if my peer review is an outlier?

A1: You won’t know this directly. When you submit your review, the system automatically checks it against rubrics and other reviews. If your score is significantly different from the others for the same submission (e.g., you give a 0 when others give 80-90, or vice versa), it will be flagged as an outlier.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 10 06 19 54 IST – Recording#

Duration: 1h 35m

Here’s an FAQ based on the live tutorial:

Q1: My filtering code isn’t returning the correct answer even after I tried adjusting the date (increasing and decreasing it). Can you help me identify common filtering mistakes?

A1: This indicates a filtering issue in your code. Common mistakes we’ve observed include:

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 10 13 19 49 IST – Recording#

Duration: 2h 11m

Here’s an FAQ summary of the live tutorial:

Q1: What’s the plan for today’s session and future topics?

A1: Today, we’ll go through GA1, most of Week 2, and cover GA2. Many of your questions about tool usage will be answered. We won’t cover web scraping extensively today, but there are videos available that cover it in depth. For upcoming topics like ROE (Return on Investment), we’ll have extra sessions, some led by me and some by other instructors.

Mon, 01 Jan 0001 00:00:00 +0000

TAs Session - 2024 10 17 20:09:44#

Duration: 1h 50m

Here’s an FAQ based on the provided transcript:

Q1: What’s the main problem we’re trying to solve today?

A1: We want to extract specific information from a website, like how many students or instructors have received a particular badge, or details about the users themselves. We can’t do these calculations directly in the browser.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024-10-20 19:50 IST – Recording#

Duration: 1h 59m

Here’s a transcription of the provided audio, presented as an FAQ:

Q1: What will be covered in today’s session?

A1: Today’s session will primarily focus on Week 3 content, particularly the Graded Assignment (GA). Additionally, I’ll address a few leftover items from previous sessions that we couldn’t solve or where I made a mistake, specifically comparing files and CSS selectors.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 10 24 19 50 IST – Recording 2#

Duration: 2h 41m

Here’s a summary of the live tutorial’s Q&A, formatted as an FAQ:

Q1: Are the project questions different for each student? A1: The questions themselves are generally the same, but the specific datasets you’ll work with (like city names and followers) will differ.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 10 27 19 55 IST – Recording#

Duration: 1h 45m

Here’s an FAQ summary of the TDS live tutorial:

Q1: What is the main focus of Week 4’s content?

A1: Week 4 is primarily about data analysis. We’ll be covering topics like Excel, Pandas, Parquet files, SQL, and database interactions, with a specific focus on the GA4 assignment.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 11 04 16 56 IST – Recording#

Duration: 1h 55m

Here’s an FAQ based on the TDS live tutorial:

Q1: What will be covered in this session?

A1: We will be covering the content from Week 5 and addressing any doubts or issues you may have regarding the course material or the project. Jivraj, the other TA, is also available to help.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS 2024 11 14 19 54#

Duration: 1h 52m

Here’s an FAQ-style summary of the live tutorial:

Q1: What will the ROE session cover, and when is it? A1: The ROE session on Thursday, November 14th, will cover skills required for ROE. ROE will comprise content from weeks 1-6. The ROE itself will be on a Sunday, from 1:00 PM (though the exact duration isn’t fixed yet). You’ll receive a link, so there’s no need to log in precisely at 1:00 PM.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 11 10 19 49 GMT+05 30 – Recording#

Duration: 1h 39m

Here’s an FAQ based on the live tutorial:

Q1: Why are my GA5, Question 5 answers not matching the provided options?

A1: This is happening because Open AI, which provides the vector representations for the words used in the assignment, made slight changes to their model after the assignment was generated. Since we’re dealing with very small numerical values, these minor tweaks can lead to discrepancies in your calculated answers compared to the provided options. This primarily affects a small group of students. We’ve informed operations, and they will decide on a resolution.

Mon, 01 Jan 0001 00:00:00 +0000

ROE Session TDS 2024 11 15 19 53#

Duration: 2h 0m

Here’s an FAQ-style transcription of the live tutorial:

Q1: What are the main steps involved in working with data in a Python notebook?

A1: The process typically involves four main steps:

Get the data: Obtain the data you need.
Clean up: Process the data to handle missing values, errors, or inconsistencies.
Transform: Manipulate the data into a more suitable format or structure for analysis.
Answer questions: Use the processed data to derive insights and answer specific questions.

Q2: What is required to get data into a Python notebook from a web source?

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 11 07 19 58 GMT+05 30 – How to hack Project 1 and possibly ROE, pdf scrapi#

Duration: 2h 14m

Here’s an FAQ-style transcription of the live tutorial:

Q1: How quickly can web scraping be done for data extraction tasks like Project 1?

Mon, 01 Jan 0001 00:00:00 +0000

ROE Session TDS 2024 11 16 19 52#

Duration: 1h 13m

Here’s an FAQ based on the live tutorial:

Q1: Can you explain the code logic for processing PDF files?

A1: The code loops through a folder, identifies files ending with .pdf, and then opens each PDF using the PDF Plumber library. It extracts the text or tables from each page.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS 2024 12 05 19 52 GMT+05 30 Recording#

Duration: 3h 0m

Here’s an FAQ based on the live tutorial:

Q1: Jivraj, can you give an update on GSA6 or internet access?

A1: I’m having trouble accessing the internet on my laptop. Regarding GSA6, I haven’t received an email, so I’m not sure if it’s updated or what the last information was. I can check if it’s updated.

Mon, 01 Jan 0001 00:00:00 +0000

Project 2 TA’s Session TDS 2024 12 05 19 52#

Duration: 3h 0m

Here’s an FAQ based on the live tutorial:

Q1: What is Project 2 about?

A1: Project 2 is about developing an innovative LLM-based application that can analyze a given dataset and convince another LLM of the quality of your code and its output.

Q2: What does “convincing the LLM” mean?

Mon, 01 Jan 0001 00:00:00 +0000

Mock QP - TA Session - TDS 2024 12 19 18 51#

Duration: 2h 25m

Here’s an FAQ summary of the live tutorial:

Q1: What is the purpose of this mock session? A1: This session is designed to go through questions that are related to what you can expect in your end-term exam.

Q2: Can you explain the RIGHT and MID functions in Excel? A2: Sure.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 15 Jan 2025#

Q1: Where can I find the uploaded documents?

A1: The uploaded documents are added as part of a YouTube playlist. You should be able to see them on YouTube.

Q2: How should I approach TDS?

A2: The best way to learn TDS is through practice. The course is designed to handle day-to-day data science problems. It introduces everyday development tools (from VS Code to npm) and provides hands-on experience solving problems. The graded assignments (GAs) are practical exercises designed to reinforce theoretical concepts. Focus on working through the examples and familiarizing yourself with the tools.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 16 Jan 2025#

Q1: What will be covered in this session?

A1: This session will cover concepts from Week 1 of the graded assignment in Tools in Data Science (TDS). The focus is on guiding students toward solutions, not providing exact answers. Week 2 will be covered in the next session.

Q2: Is it possible to reschedule the TDS session?

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 17 Jan 2025#

Q1: How to solve question number 7 (counting Wednesdays between two dates)?

A1: Use Excel. Input the two dates into two cells. Use a command to find the difference in days, then divide by 7 to get the number of weeks. Since there’s one Wednesday per week, that’s the number of Wednesdays. If the start date isn’t a Wednesday, a different approach is needed.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 20 Jan 2025#

Q1: What is an ML project, and how does it compare to the “fitting of six models” project?

A1: An ML (Machine Learning) project involves more factors than just fitting models. The “fitting of six models” project was a simplified version done for a friend’s certification course. It involved training six models on a pre-cleaned dataset with some hyperparameter tuning but minimal data preprocessing or cleaning.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 21 Jan 2025#

Q1: How much depth of knowledge is needed for the tools in the TDS course, specifically for GA1 and GA2?

A1: The goal is to give you sufficient proficiency to execute the purpose of the tools, not to make you an expert. The focus is on GA1 and GA2 because of the approaching deadline. Basic knowledge will suffice; how deep you go is up to you. The biggest challenge is usually reading and understanding documentation. These sessions are designed to help you overcome that hurdle.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 22 Jan 2025#

Q1: What are the essential base technologies to learn for the TDS course?

A1: Focus on these three areas: 1) Bash and shell scripting; 2) Git; 3) CSS, HTML, and JavaScript. Also, learn REST APIs. Other tools can be learned on the go once you have a basic understanding of these core technologies.

Q2: Regarding the tools and techniques needed for the TDS course, are Bash and shell scripting, Git, CSS, HTML, JavaScript, and REST APIs the only essential ones?

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 23 Jan 2025#

Q1: How can I compare two files using the command line?

A1: You can use the diff command in bash, or the git diff command if you’ve initialized a git repository. The diff command shows the differences between two files directly. git diff compares commits in a git repository.

Q2: I’m having trouble viewing my GA1. It’s showing zero ones. Can you help?

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 28 Jan 2025#

Q1: Can I ask a question about the graded assignment for week one?

A1: I’ll try to answer it at the end of the session if there’s time. We’ll also have extra sessions this week, and we’re hoping to release video solutions showing how to solve some of the assignments. But for now, let’s focus on today’s session.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 29 Jan 2025#

Q1: I am a student, and I joined the session without using my IITM ID. How do I rejoin with my IITM ID?

A1: You need to rejoin the session using your IITM ID. We don’t allow anyone to join without their IITM ID.

Q2: Today’s session is mostly about doubt clearing. I haven’t seen the GAs yet. Can I come tomorrow after I’ve solved them?

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 30 Jan 2025#

Q1: My screen is visible?

A1: Yes, your screen is visible.

Q2: I posted on Discourse and emailed you about a problem deploying a Flask application on Vercel. The Flask module is not getting downloaded. Can you show the vercel.json configuration file?

A2: The error likely stems from your requirements.txt file. Did you create a new virtual environment for this project, or did you install Flask on your global environment? When deploying Flask applications on Vercel, you need to keep a few things in mind. I’ll show you how to create a requirements.txt file without manually typing it.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 31 Jan 2025#

Q1: In today’s session, what will be covered?

A1: We will be discussing the function calling part of OpenAI. We will create a program that takes user prompts in English, identifies which function to call based on the prompt, and then calls that function.

Q2: What is the first step in the process?

A2: First, we import the requests library. Then, we’ll get the headers and URL for the request, and finally, we’ll create a JSON body for the request.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 01 Feb 2025#

Q1: Can I use Python notebooks instead of VS Code for this project?

A1: You can use Python notebooks for interactive development and debugging, but for a production environment, a script file (.py) is better because it allows for a defined order of execution, unlike notebooks where code cells may not necessarily branch from one section to another. Notebooks are great for sharing with others who may not understand the code, as you can add Markdown to explain each step.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 04 Feb 2025#

Q1: I’m having trouble with question 7 of the GA3 assignment. I’m trying to send a POST request, but it keeps showing “method not allowed”.

A1: When you access a URL from a browser, it sends a GET request. To send a POST request, you need an extension like Thunder Client or Postman. Thunder Client doesn’t require a login, but Postman does. However, Thunder Client is now a paid service.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 06 Feb 2025#

Q1: In this GA4 session, what’s new compared to previous GAs?

A1: The trajectory has moved from hard to easy. This GA is much easier and lighter than previous ones.

Q2: How do I extract a table from HTML using Google Sheets?

A2: This functionality is available in Google Sheets but not in Excel (unless you use plugins). The IMPORTHTML function will give you a #NAME? error in Excel.

Mon, 01 Jan 0001 00:00:00 +0000

Live Session: 07 Feb 2025#

Q1: How difficult are the TDS assignments?

A1: The first assignment was easy, the second was hard, the third was average, and the fourth seems easy. However, the instructor notes that the first assignment was only easy for those with a technical background; for most students, it was very difficult. The difficulty level of subsequent assignments will decrease.

Mon, 01 Jan 0001 00:00:00 +0000

2025-02-11 Week 5 - Session 1 - TDS Jan 25#

Duration: 3h 57m

Here’s the FAQ for the TDS live tutorial:

Q1: Will the deadline for GA5 be extended?

A1: Yes, we are currently discussing moving the GA5 deadline. We don’t have an exact date yet, but it definitely won’t be this upcoming weekend. This extension is to give you extra time to focus on Project 1 and also because we won’t be able to cover all of Week 5’s content this week.

Mon, 01 Jan 0001 00:00:00 +0000

2025-02-13 Week 5 - Session 3 - TDS Jan 25#

Duration: 3h 12m

Okay, let’s break down your questions about working with Docker, Python, and the task execution.

FAQ for Docker & Task Execution#

Q1: How do I make sure uvicorn and fastapi are installed, and where should I check?

A1: You need to make sure uvicorn and fastapi are listed in your requirements.txt file. This file typically contains all the Python packages your project depends on.

Mon, 01 Jan 0001 00:00:00 +0000

2025-02-18 Week 6 - Session 1 - TDS Jan 25#

Duration: 2h 8m

Here’s the FAQ for the provided audio:

Q1: Is the recording going?

A1: Yes, I guess the recording is up, and let’s get started.

Q2: What is this entire week about?

A2: This entire week is about data preprocessing and cleaning up data.

Q3: What will we be doing in this particular session?

Mon, 01 Jan 0001 00:00:00 +0000

2025-02-18 Week 6 - Session 3 - TDS Jan 25#

Duration: 1h 34m

Here’s an FAQ based on the live tutorial:

Q1: What is the most efficient tool for processing large JSON files, especially for tasks like checking if a key exists or how many times it appears?

A1: You can use batch processing tools like ijson. It’s the most efficient choice, particularly for large files, because it processes data in smaller batches rather than loading everything into memory at once.

Mon, 01 Jan 0001 00:00:00 +0000

TA’s Session TDS – 2024 11 10 19 49#

Duration: 1h 39m

Here’s an FAQ summary of the TDS live tutorial:

Q1: I just joined, could you please repeat the issue with GA5, Question 5?

A1: Some students’ answers for GA5, Question 5 won’t match the options provided. This is because OpenAI has made slight changes to the vector representations of some words/tokens in their model between when the assignment was generated and when you access it. This primarily affects a small subset of students, and you should pick the option that is closest to your calculated answer for Question 6. We’ve informed operations about this discrepancy and are waiting for their decision on how to handle it for Question 5.

Mon, 01 Jan 0001 00:00:00 +0000

2025-02-26 Week 7 - Session 2 - TDS Jan 25#

Duration: 5h 8m

Here’s an FAQ summary of the live tutorial:

Q1: How did you find the mock ROIs (yesterday’s and today’s)? Were they difficult or workable?

A1: The questions were solvable, but not if you tried to do them alone. For the best approach, I’d suggest pre-deciding who you’ll collaborate with, then assigning questions based on who masters which topics, and finally approaching the test.

Mon, 01 Jan 0001 00:00:00 +0000

2025-02-27 Week 7 - Session 3 - TDS Jan 25#

Duration: 2h 24m

Here’s an FAQ summary of the tutorial:

Q1: How can I access session recordings if they haven’t been processed by Google yet?

A1: You can access them earlier by going to your calendar, finding the specific date, and clicking on the live session link directly.

Q2: I tried extracting data from the PDF using Tabula, but it was a mess. What went wrong?

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-04 Week 8 - Session 2 - TDS Jan 25#

Duration: 0h 47m

Here’s an FAQ summary of the Tools in Data Science (TDS) live tutorial session:

Q1: What is the purpose of this live tutorial session?

A1: This session is primarily for addressing your queries, facilitating discussions, and gathering suggestions regarding the TDS program and projects. It’s not a knowledge-sharing session for this week but a dedicated space for question resolution and open dialogue.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-04 Week 8 - Session 3 - TDS Jan 25#

Duration: 1h 52m

Here’s an FAQ-style summary of the TDS live tutorial:

Q1: Many GA questions (like GA1 Q13, GA2 Q7 for GitHub CLI; GA2 Q8 for local LLM/Docker; GA2 Q6 for Python API deployment; FastAPI servers) require tools/privileges (sudo access, local execution) that Vercel doesn’t provide. How can I complete these tasks? A1: I acknowledge your concerns, which are genuine and have been raised on Discourse. We will discuss these internally with Anand Sir and provide clarification. For now, please proceed with other questions that do not require these specific functionalities.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-04 Week 8 - Session 1 - TDS Jan 25#

Duration: 1h 52m

Here’s an FAQ summary of the session:

Q1: I’m using a different operating system. Can you confirm if I’m audible?

A1: Yes, you are perfectly audible.

Q2: The “Project Book” section isn’t visible on the main TDS page sidebar. Will this be updated?

A2: You’re right, it’s not there. We’ll ensure that the “Project Book” section, along with the missing FAQs after February 5th, are added to the main page soon. They are currently available on a different page.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-11 Week 9 - Session 1 - TDS Jan 25#

Duration: 2h 11m

Here’s an FAQ based on the provided TDS tutorial transcript:

Tools in Data Science (TDS) Tutorial FAQ#

Q1: What will this session cover? A1: This session will clarify expectations around Project 2, specifically regarding the use of function calling and other related concepts. We’ll introduce you to the agentic coding workflow.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-13 Week 9 - Session 3 - TDS Jan 25#

Duration: 1h 58m

Here’s an FAQ-style transcription of the tutorial:

Q1: How do I link my academic status with GitHub Education?

A1: You need to add your academic email (like your IIT Madras email) to your GitHub account. For proof of academic status, you’ll need to upload a photo of your student ID card or an academic transcript. Make sure it’s your latest ID card. When prompted, specify your expected degree completion year (e.g., 2027 or 2028). What you mention here, GitHub generally extends for you.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-12 Week 9 - Session 2 - TDS Jan 25#

Duration: 1h 14m

Here’s an FAQ based on the live tutorial:

Q1: I’m currently working on Project 2. My progress is going well, but I’m stuck on some parts in Week 6. What specifically are these parts?

A1: That’s great to hear your progress! Regarding Week 6, you mentioned being stuck. Could you please check your notes and let me know the specific areas you’re facing issues with?

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-18 Week 10 - Session 1 - TDS Jan 25#

Duration: 2h 24m

Here’s an FAQ summary of the tutorial:

Q1: What are the main deployment options discussed for Project 2?

A1: You have several options:

Digital Ocean Droplet: Offers $100 in credits for 60 days, but requires a credit card or PayPal for setup.
Azure Education Pack: Provides $100 in credits for 12 months (renewable), and importantly, does not require a credit card for activation. This is ideal if you don’t have a credit or debit card.
AWS: Mentions 750 hours of compute for one year, which likely requires credit card details.
GitHub Student Developer Pack: Another resource that might offer various benefits for students.

Q2: How can I access and use the Azure Education Pack?

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-20 Week 10 - Session 3 - TDS Jan 25#

Duration: 1h 44m

Here’s the FAQ based on the provided transcript:

Q1: What’s the recommended workflow for Project 2?

A1: You’ll need to follow a specific pipeline. The most important recommendation is to collaborate. You’ve been given 5 Graded Assignments (GAs) with over 50 questions in total. It would be incredibly tedious for one person to create 50 functions. Form groups, where multiple people work on creating individual functions for each question, and then one person can be responsible for merging all those functions together.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-26 Week 11 - Session 2 - TDS Jan 25#

Duration: 1h 27m

Here’s an FAQ based on the tutorial:

Q1: I’m having trouble with some GA questions where even ChatGPT isn’t giving the correct answers. Will ChatGPT be able to solve all answers correctly 100% after I complete the project?

A1: The assignments and their solutions were created by Anand using ChatGPT. He’s very skilled in prompt engineering. While ChatGPT can be powerful, achieving 100% accurate results consistently requires significant prompt engineering skill.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-25 Week 11 - Session 1 - TDS Jan 25#

Duration: 1h 17m

Here’s the live tutorial transcribed into an FAQ format:

Q1: I’m having trouble getting the endpoint to work. It seems like the question should be sent in form data, not JSON, is that right?

A1: Yes, you’re absolutely correct! My apologies, that was the issue. Thank you for pointing that out.

Mon, 01 Jan 0001 00:00:00 +0000

2025-03-27 Week 11 - Session 3 - TDS Jan 25#

Duration: 2h 7m

Here’s an FAQ-style transcription of the provided live tutorial:

Q1: For Project 2, how can I run multiple ngrok tunnels with the free version?

A1: You can’t directly run multiple separate ngrok http 8000 commands. Instead, create a YAML file where you specify all the different ports your applications will use. Then, run a single ngrok command referencing this YAML file. This way, a single ngrok tunnel will listen on multiple ports, which is a good workaround for running different applications.

Mon, 01 Jan 0001 00:00:00 +0000

2025-04-08 End Term Session 1 - TDS Jan 25#

Duration: 7177.0Here’s an FAQ summary of the live tutorial:

Q1: How are bonus scores calculated for submissions, and when are they awarded for Project 2?

A1: Bonus scores are calculated automatically based on specific metrics and weights defined for your submission. For Project 2, bonuses are typically awarded if the distribution of scores is not a normal distribution. If the scores follow a normal distribution, it suggests the project was inherently doable, and therefore, no additional bonus will be provided.

Mon, 01 Jan 0001 00:00:00 +0000

2025-04-09 Live Session - Qs & As - TDS Jan 25#

Duration: 4792.0Here’s an FAQ based on the live tutorial:

Q1: What are bonus marks and how can I earn them?

A1: If you already completed GA7 (all submissions and 3 peer reviews by tomorrow’s deadline), you have an opportunity to earn up to 5 bonus marks. You do this by completing 5 additional peer reviews. Each review can earn you up to 1 bonus mark, which is added to your total T-score.

Mon, 01 Jan 0001 00:00:00 +0000

2025-05-10 Week 1 - Session 1: Introduction to TDS - TDS May 25#

Duration: 2h 29m

Here’s an FAQ summary of the TDS live tutorial:

Q1: What is Tools in Data Science (TDS), and how is the course structured?

A1: TDS covers difficult topics in data science. The course has 7 modules, typically spread over 9-10 weeks (not a strict weekly cadence), with 7 Graded Assignments (GAs) and 2 projects.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 11 Week 1 Session 2 Setup & Debugging TDS May 2025#

Duration: 11005.0Here’s an FAQ based on the provided transcript:

Q1: VirtualBox didn’t work when I tried to install it yesterday. What went wrong, and why can’t I follow yesterday’s installation instructions today?

A1: The issue likely stems from your computer’s architecture. I was using a Mac with an ARM architecture (M-series chip). VirtualBox doesn’t currently support ARM architecture, so it failed. This also applies to Windows ARM-based machines (like some Windows Surface models). If your machine is ARM-based, VirtualBox won’t work because it hasn’t been compiled for ARM architecture, and it’s missing necessary drivers.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 17 Week 1 Session 5 TDS May 2025#

Duration: 9444.0Here’s an FAQ-style summary of the discussion:

General Course Information#

Q1: What was covered in the last session?

A1: We covered the installation and download of WSL (Windows Subsystem for Linux) for Windows users, and then moved on to Git and GitHub, including concepts like pushing, committing, and using VS Code with Git.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 15 Week 1 Session 3 Linux, BASH, git & github TDS May 2025#

Duration: 11038.0Here’s an FAQ-style transcription of the live tutorial:

Q1: I’m seeing “this term is not recognized” when I type bash in my Windows command prompt, or I’m unsure where to type commands when following videos. How do I know where to run these commands?

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 16 Week 1 Session 4 Env Settings, Env Variables, git and github TDS May 2025#

Duration: 7958.0Here’s an FAQ summary of the live tutorial:

General Concepts & WSL Setup#

Q1: Can you briefly recap the “append” operation from the previous class?

A1: The “append” operation is about adding the output of a command to an existing file without overwriting its current content. This is useful for combining information or logging data sequentially.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 22 Week 2 Session 2 - Local Lamma, Ngrok, Google Authentication TDS May 2025#

Duration: 9651.0Here’s an FAQ based on the provided live tutorial transcript:

General Ollama Setup & Purpose#

Q1: Why are we learning about local LLM deployment with Ollama, especially if powerful hardware isn’t always available?

A1: We’re covering local LLM deployment with Ollama because it’s a key part of your assignment (Question 12). Beyond that, it addresses significant concerns for organizations, particularly around data privacy. Many companies don’t want their sensitive data leaving their internal networks to interact with external LLM APIs. Ollama allows them to host and run models internally, ensuring data stays within their control. While hardware can be a limitation now, models are constantly evolving to be smaller and more efficient, making local deployment increasingly feasible even on less powerful devices in the future.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 21 Week 2 Session 1 TDS May 2025#

Duration: 2h 31m

Here’s a summary of the live tutorial in an FAQ format:

Tools in Data Science (TDS) Live Tutorial FAQ#

Q1: How is the TDS course progressing (GA1, GA2)? Are there any common challenges students are facing?

A1: GA1 (General Assignment 1) is complete, and GA2 is currently in progress. I’ve noticed some of you are finding the coding aspects challenging, especially those without a prior coding background. This course will be particularly important for you. Regarding specific assignments, some feel that certain video content isn’t always directly relevant to the assignment tasks.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 23 Week 2 Session 2 - Github Actions, Docker - TDS May 2025#

Duration: 11854.0Here’s an FAQ based on the provided transcript:

Q1: I missed previous TDS sessions and have started Graded Assignment 2. I’ve completed 6-7 questions. How many hours a day should I spend over the next 2-3 days to cover the remaining topics for Graded Assignment 2?

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 24 Week 3 Session 1 - TDS May 2025#

Duration: 6430.0Here’s an FAQ summary of the live tutorial:

Q1: What is the main focus of Module 3 in this course?

A1: Module 3 is about interacting with a specialized API. If you’ve completed weeks 1 and 2, you’ll find it manageable as it builds upon those API concepts. Today’s session will focus more on conceptual understanding than extensive coding demonstrations.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 25 Week 3 Session 2 - TDS May 2025#

Duration: 5899.0Here’s an FAQ summary of the tutorial:

Q1: What topics will be covered in this session and why are they important?

A1: This session covers key topics from Module 3, which are crucial for Project 1. We’ll delve into embeddings (and multimodal embeddings), vector databases, RAG (Retrieval Augmented Generation) and hybrid RAG, and function calling. The session is mostly code-based, so I encourage you to follow along and replicate the results.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 29 Week 3 Session 4 - TDS May 2025#

Duration: 9863.0Here’s an FAQ summary of the tutorial:

OpenAI API & Function Calling: Frequently Asked Questions#

Q1: What will this tutorial cover?

A1: We’ll dive into practical applications of Tools in Data Science (TDS), specifically focusing on OpenAI API calls. The core topics include making HTTPX and API calls, working with Base64 embeddings, understanding vector databases and RAG (Retrieval Augmented Generation), exploring multimodal embeddings using Ginja AI, and implementing function calling. We’ll build a chatbot with conversation history and demonstrate extracting product details from images.

Mon, 01 Jan 0001 00:00:00 +0000

2025 05 28 Week 3 Session 3 - TDS May 2025#

Duration: 5440.0Here’s an FAQ based on the live tutorial transcript:

Q1: I’m repeatedly getting “Open API key expired” or similar errors, even after trying both AI Pipe and AI Proxy methods. It worked the first time, but not anymore.

A1: This issue was covered in detail during the Week 3, Session 1 live tutorial on May 24th. You can find the recording on YouTube, and it includes specific timestamps. The key is to ensure you’ve correctly set up your API tokens in the environment settings and replaced the URL and token accordingly. If you’re still facing issues after reviewing the tutorial and trying the suggested steps, please raise a specific question in our Discord channel or during a live session, detailing the exact error message. While it’s frustrating, troubleshooting these steps is a crucial part of building your coding background.

Mon, 01 Jan 0001 00:00:00 +0000

2025 06 10 Project 1 Session 1 - TDS May 2025#

Here is an FAQ-style transcription of the TDS live tutorial:

Q1: Where can I find the poll you mentioned for the project focus areas?

A1: You can find the poll in the bottom right corner of your screen, under the “meeting tools” icon. Please participate, as it helps us understand where you need more focus.

Mon, 01 Jan 0001 00:00:00 +0000

2025 06 11 Project 1 Session 2 - TDS May 2025#

Duration: 2h 59m

Here’s an FAQ based on the provided transcript of the TDS live tutorial:

Q1: I’m experiencing audio issues, and it sounds like yours might be breaking up too. Is my audio audible, and can I share my screen if needed?

A1: Yes, I can hear you, though there might be some audio issues on both our ends. For now, let’s proceed with the session.

Mon, 01 Jan 0001 00:00:00 +0000

2025 06 12 Project 1 Session 3 - TDS May 2025#

Here’s an FAQ based on the provided transcription:

Q1: What is this session about?

A1: This session is a live tutorial part of the “Tools in Data Science” (TDS) course. We’re creating an API endpoint that takes a question and an image as input and responds with an answer and its sources.

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ summary of the tutorial:

Q1: How do I find the API URL in my browser’s developer tools?

A1: When interacting with a webpage (like typing in a search bar), if the page updates without a full reload, it’s likely making an API call. To see these calls:

Right-click on the webpage and select “Inspect” to open the developer tools.
Go to the “Network” tab.
Clear any existing network logs using the “Clear” button (a circle with a diagonal line).
Perform the action on the webpage (e.g., type one character in a search bar) that triggers the dynamic update.
You’ll see new API calls appear in the Network tab. Click on a specific call by its name to view its details, including the “Headers” and “Response” tabs. The “Request URL” in the Headers tab is your API endpoint.

Q2: How do I handle special characters like spaces and commas in the API URL parameters?

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ-style transcription of your live tutorial session:

Q1: Can we change the Git repo after the deadline?

A1: No, that won’t be possible. The deadline has already been extended to the end of today. If you could modify the repository after the deadline, it would essentially mean the deadline is ignored. So, any changes you want to make should be done now.

Q2: Is there a chance to increase the AI pipe limit as I’ve reached it?

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ-style transcription of the live tutorial:

Q1: What does cut -c 1,2,4 do?

A1: cut -c is used to select specific characters from each line of a file. For example, if a line is “Hello”, cut -c 1,2,4 would extract ‘H’ (1st character), ’e’ (2nd character), and ’l’ (4th character). The cut command processes each line individually. It’s important to note that you cannot repeat character positions in the selection (e.g., 1,1 is not a valid argument for cut).

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ based on the provided transcript:

General Information & Access#

Q1: Where can I find Module 6? I don’t see it on the TDS dashboard.

A1: It’s available as a graded assignment (GA6). You can access it directly via a link on Discourse. You can also manually change the URL from a previous GA (e.g., GA5 to GA6) in your browser to access the content.

Q2: I don’t have the “Data Analysis” option in Excel. How do I enable it?

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ based on the provided transcription of the TDS live tutorial:

Q1: What did we cover in yesterday’s session?

A1: Most topics were covered except for SQL and DuckDB, which are the focus of this session.

Q2: What is the goal of this session?

A2: We’ll be focusing on how to connect to databases using SQL and DuckDB, especially when the database is hosted remotely and not on your local machine.

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ summary of the tutorial:

Tools in Data Science (TDS) Live Tutorial FAQ#

Q1: Where can I access the mock ROE (Review of Educational Exercises) files mentioned in the tutorial?

A1: The mock ROE files, including the PDF, are usually accessible from your TDS dashboard. If you’re having trouble accessing the mock ROE4.zip file directly from a link within the GitHub repository (e.g., it’s not clickable), it might be due to your PDF viewer blocking external links. Try downloading the PDF to your local system first, and then access the links.

Mon, 01 Jan 0001 00:00:00 +0000

Here is an FAQ summary of the Tools in Data Science (TDS) live tutorial:

Q1: Will there be additional support for Project 2?

A1: Yes, Professor Anand will host two dedicated sessions for Project 2, likely on a Monday and Wednesday from 6 PM to 8 PM. An official email with the finalized dates and times will be sent out, but these sessions are definitely happening next week to help you with any roadblocks, questions, or further guidance you might need.

Mon, 01 Jan 0001 00:00:00 +0000

2025 07 18 MOCK ROE session 4 - TDS May 2025#

Duration: 2h 7m

Here is the FAQ from the Tools in Data Science live tutorial.

General & Logistical Questions#

Q1: Will a recording of this session be available later?

A1: Yes, the recording will be uploaded for you to watch later.

Q2: What AI tool do you recommend for the exam: ChatGPT, Perplexity, or Copilot?

Mon, 01 Jan 0001 00:00:00 +0000

Here is the FAQ-style transcription of your TDS live tutorial:

Q1: Will there be more ROE (Return on Equity) sessions after the previous one?

A1: Yes, we have ROE sessions scheduled for both today and tomorrow. Today, I’ll be solving the September 2024 mock questions. Tomorrow, Djivraj will cover the January ROE questions. We’ve already solved three to four mocks in previous weeks.

Q2: I noticed some mock questions and answers on GitHub. Are these officially provided, and will future ones be more accurate?

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ based on the provided transcript of the TDS tutorial:

Q1: I have an ROE score discrepancy. My highest official score is 5, but after a server stop and computer reload, my most recent score became 4. Can my highest score be considered?

A1: That’s an administrative decision, and I don’t handle it directly. Please post your question on the discourse forum, and the relevant personnel will address it.

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ-style summary of the tutorial:

Q1: Is this session being recorded?

A1: Yes, it is being recorded.

Q2: Can you explain what you’re doing? I’m not able to follow. Are you sending the request to the LLM?

A2: I’m setting up a FastAPI server. I’ve added a post endpoint that takes a file as input to upload it to my application. Later, I’ll process it using an LLM. For testing purposes, I’m using Gemini to avoid consuming too many tokens.

Mon, 01 Jan 0001 00:00:00 +0000

Here’s an FAQ based on the live tutorial:

Q1: What issues did I face initially with the AI agent?

A1: You faced several challenges:

Token Exhaustion: Your AI agent quickly ran out of tokens, sometimes within 3-4 attempts.
Code Execution Failures: The agent would generate Python code, but instead of executing it, it would often just return the code itself or pipe back errors.
Gemini-Specific Problems: Gemini frequently generated incomplete code or code with syntax errors (e.g., missing triple quotes), making it unreliable.
Ineffective Error Feedback: Attempting to correct the agent by feeding back the errors didn’t work; it would just make different mistakes in subsequent attempts.
Lack of Generalizability: You expressed concern that without custom tooling, the agent struggles with diverse requests, requiring a lot of tokens for even simple tasks.

Q2: What is the proposed general strategy for web scraping and data extraction for analysis?

Mon, 01 Jan 0001 00:00:00 +0000

2025 08 04 project 2 - Q&A 2 TDS May 2025#

Here’s an FAQ based on the provided TDS live tutorial:

Q1: What is the main goal of Project 2?

A1: The goal is for you to learn how to effectively use Large Language Models (LLMs) like ChatGPT or Codex as coding engines for data science tasks. We’ll be generating sample questions for a “Data Analyst Agent” and refining the prompts to make them robust and comprehensive.

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 20 Week 1 - Session 1 TDS Sep 2025#

Duration: 2h 52m

Here’s an FAQ based on the provided transcript, summarizing student questions and instructor answers in a conversational style:

Q1: Where can I find the recorded video sessions for Tools in Data Science (TDS)? I’ve checked the C Portal and couldn’t find them.

A1: The entire course content, including recorded videos, is not hosted on the C Portal. Instead, it’s available on a separate site. The C Portal usually provides a link to this external site. All content will be available only via this specific site.

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 21 Week 1 - Session 2 TDS Sep 2025#

Duration: 2h 37m

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 22 Week 1 - Session 3 TDS Sep 2025#

Duration: 1h 58m

Here’s an FAQ-style transcription of the live tutorial:

Q1: What is a hypervisor and how does it relate to WSL?

A1: A hypervisor is a software that allows a single physical computer to host multiple virtual machines (VMs). It efficiently shares the underlying hardware resources (like CPU, RAM, storage) among these VMs.

Mon, 01 Jan 0001 00:00:00 +0000

In this tutorial, we will learn the following:

Setting up Git on your local machine
Creating a GitHub repository
Committing code to GitHub
Generating SSH keys, activating SSH agent, and using them with GitHub

Step 1: Install Git#

First, install Git on your machine.

Windows:#

Download from git-scm.com/download/win and run the installer.

macOS:#

brew install git

Linux:#

sudo apt-get install git

Step 2: Configure Git#

Set your name and email for commits:

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 28 week 2 - Session 2 TDS Sep 2025#

Duration: 2h 36m

Here’s an FAQ summary of the live tutorial:

Q1: Why should I use Podman/containers instead of just VS Code or a terminal for application development, especially for AI applications?

A1: You can definitely use VS Code or a terminal for coding, but Podman (and containers generally) offer isolated, lightweight environments. This helps manage complex applications by allowing you to run different parts (like a database and a front-end) in separate, self-contained units, making management easier. For AI applications, this means you can encapsulate specific models or services within their own pods.

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 29 Week 1 - Session 4 TDS Sep 2025#

Duration: 1h 38m

Here’s an FAQ summary of the live tutorial:

Q1: What is the main focus of this session?

A1: This session is about setting up basic tools for data science, primarily for Mac OS and Linux users, especially those who might have struggled with initial setup. We’ll cover essential tools like VS Code, Homebrew, UV, LMM, and GitHub CLI, and touch upon their core functionalities.

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 24 week 1 - Session 5 TDS Sep 2025#

Duration: 1h 41m

Here’s an FAQ based on the provided transcript, covering all questions and answers:

Q1: Can I complete Graded Assignment 1 (GA1) without watching the videos?

A1: Yes, you can complete most of the assignment just by following the instructions provided.

Q2: How difficult is the course, and does the teaching approach accommodate different learning paces and backgrounds?

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 27 week 2 - Session 1 TDS Sep 2025#

Duration: 2h 53m

Here’s an FAQ based on the tutorial:

Q1: Is the audio and screen sharing working correctly?

A1: Yes, everything is working fine, and I am audible.

Q2: What are the main challenges when setting up a development environment locally?

A2: Running code locally involves several prerequisites like installing Python and specific libraries (e.g., Flask, FastAPI). A major challenge is that these installations often depend on your operating system, leading to compatibility issues and complex setups.

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 03 week 2 - Session 4 TDS Sep 2025#

Duration: 2h 16m

Here’s an FAQ-style transcription of the live tutorial:

Q1: What exactly is a “workflow” in the context of this course, and what specific aspects should I focus on to avoid wasting time on irrelevant details?

A1: You’re right, “workflow” is a key topic. I understand that searching online can provide an overwhelming amount of information. To be efficient, you need to understand only what’s required for this course’s graded assignments (GA2, GA3, GA4, or the project). I will demonstrate the specific workflow needed today, and this session’s recording will be the official guide.

Mon, 01 Jan 0001 00:00:00 +0000

2025 09 30 week 2 - Session 3 TDS Sep 2025#

Duration: 1h 45m

Here’s an FAQ based on the live tutorial:

GitHub CodeSpaces & Actions FAQ#

Q1: What’s the difference between using CLI and a desktop application (like GitHub Desktop) to connect my local machine to GitHub? Does it matter which one I use for the course content?

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 04 Project 1 - Session 1 TDS Sep 2025#

Duration: 2h 24m

Here’s an FAQ summary of the live tutorial:

Tools in Data Science (TDS) Tutorial: Project 1 Walkthrough - FAQ#

This FAQ covers common questions and answers from a live tutorial session on Project 1 of Tools in Data Science (TDS), focusing on setting up a GitHub-integrated application.

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 05 Week 3 - Session 1 TDS Sep 2025#

Duration: 2h 6m

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 06 Week 3 - Session 2 TDS Sep 2025#

Duration: 2h 39m

Here’s an FAQ-style transcription of the live tutorial:

Q1: What are the main issues being addressed from previous sessions? A1: I’ll summarize the issues you’ve raised and how we’re handling them:

“Save” button not working due to heavy load: If this is the issue, our team lead, Anand sir, will look into it.
“Lock after correct answer” feedback: This feature, which prevents editing after a correct answer, is unlikely to be implemented. We aim to replicate the existing system in graded assignments (like the C-portal), which doesn’t have this “lock” functionality.
“Different questions should have different code provided save button”: This feedback is likely to be accepted and implemented.
Solutions after deadline: You can still access and complete solutions for questions even after the deadline has passed.
“Unexpected token” error: If you encounter this error, the first step is to log out and then log back in. If the issue persists, try logging in using an incognito browser window. If the problem continues, it might be a network issue on your end.
Login with student ID: It’s crucial to always log in with your official student mail ID. Using a non-student ID can lead to issues with submissions and marks.
Save button not working (no error shown): If you experienced this and submitted a discrepancy form with proof, it will be investigated.
Scores showing “absent” despite previous submission: This issue, where a previously submitted and scored assignment (like GA1) shows “absent,” will be rectified. There might be a date issue that needs to be resolved.
Disabled buttons for file selection: If you can’t select files due to disabled buttons (because the file has been “ended”), you can click on the item, navigate to its part/location, and then manually remove the disabled elements (buttons) to enable selection. This workaround applies to all questions for practice purposes.

Q2: I’m still getting the “unexpected token” error despite trying all the suggested solutions (logging out/in, incognito mode, multiple browsers). What should I do? A2: I understand your frustration. Since others aren’t facing the same persistent error and you’ve tried all browser-based solutions, it’s likely an issue on your side, possibly related to your network. I suggest you try accessing the portal using a different network, perhaps a mobile phone hotspot. For privacy, please hide your screen while logging in, and it’s a good practice to change your password after experiencing such issues.

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 08 Week 4 - Session 1 TDS Sep 2025#

Duration: 1h 0m

Here’s an FAQ summary of the TDS live tutorial:

Q1: What is the purpose of this session?

A1: This is a doubt-clearing and query session, not a learning session. Feel free to ask any questions related to grievances, specific General Assignments (GAs), or operational issues.

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 11 Project 1 - Session 2 TDS Sep 2025#

Duration: 1h 20m

Part 1#

Here’s an FAQ summary of the first part of the tutorial:

Q1: How do I test the deployment of my project?

A1: You can test it on the existing public test cases. The project page includes checks, so try to see if your deployment passes those tests.

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 22 Project 1 - Session 4 TDS Sep 2025#

Duration: 0h 59m

Here’s an FAQ summary of the live tutorial:

Q1: My API credits for the LLM script have run out. I was using OpenAI’s GPT-3.5 free tier, which initially worked, but now it’s out of credits. I tried other modern GPT versions and Hugging Face, but they require payment. I also tried the course’s AI Pipe token, but it didn’t work properly. What should I do?

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 15Project 1 - Session 3 TDS Sep 2025#

Duration: 1h 33m

Here’s an FAQ summary of the TDS live tutorial Q&A session:

Q1: I’ve finished my project and tested it locally. For submission, I need to provide a GitHub repo, API endpoint, and a secret code in the Google Form. What exactly should be in my GitHub repo, what is the secret code, and for the evaluator (instructor) step, will my test code also be required in the repo? I also saw a mention of a “demo trial” after submission, do I need to do anything for that? Finally, where can I find the Google Form?

Mon, 01 Jan 0001 00:00:00 +0000

2025 10 10 Week 4 - Session 2 TDS Sep 2025#

Duration: 2h 59m

Here’s an FAQ summary of the live tutorial:

Q1: What are the initial steps for submitting my project? A1: You’ll submit a Google Form including your API endpoint, GitHub repository URL, and a secret key. Our server will then make an initial JSON request to your deployed application (this marks the start of Round 1).

Live-Sessions on Tools in Data Science

Live Session - 2#

Tools in Data Science (TDS) Live Tutorial FAQ#

01 - Intro to Web scraping and HTML#

03 - Intermediate web scraping use of cookies#

02 - Fundamentals of web scraping with urllib and BeautifulSoup#

04 - XML intro and scraping#

05 ROE revision session jan 24#

ROE Exam Preparation FAQ#

TA session : PYQ Dec 23 discussion#

TA session : PYQ April 23 discussion part - 1#

TA session : PYQ April 23 discussion part - 2#

ET revision : Attribute related to Anonymization and autocorrelation plot#

TA session 13 June : Introduction and week 1,2 recap#

TA Session 16 June : Introduction to HTML and web scraping#

TDS Live Tutorial FAQ: Web Scraping Basics#

A session 23 June : Web scraping with cookies and understanding JSON#

TA session 20 June : Week 2 concepts ,API, JSON, PDF scraping and GA discussion#

TA Session 27 June : Processing files with Pandas, Open Refine and Week 3 Graded Assignment#

Tools in Data Science (TDS) Live Tutorial: Week 3 Graded Assignment FAQ#

TA Session 30 June : More complex webscraping#

TA Session 4 July: More Pandas, Parquet processing, Forecast.ETS, Database Processing and Week 4 GA#

TA Session 7 July: Project Discussion, and Excel Filters, Slope Regression#

TA Session 11 July: LLM APi Setup and Use#

“TA session 12 July : week 5 , LLM API and embedding”#

Tools in Data Science (TDS) Live Tutorial: LLM Techniques#

TA session 14 July : ROE intro and strategy#

Tools in Data Science (TDS) - ROE FAQ#

TA Session 18 July: The best ROE Survival Strategies, Tricks and Tips: Part 1#

Tools in Data Science (TDS) Live Tutorial FAQ#

TA Session 19 July: ROE Survival : Part 2#

TA Session 19 August: Streamlit, Heroku#

TA Session 22 August: Mock and End Term, Revision: Modules 1 to 3#

TDS Live Tutorial: End Term Preparation FAQ#

TA Session 25th August: Revision: Modules 4 to 6#

TA Session 29th August: Revision: Modules 7 to 9, CSS selectors, Javascript#

TA’s Session TDS – 2024 09 29 19 49 IST – Recording#

TA’s Session TDS – 2024 10 06 19 54 IST – Recording#

TA’s Session TDS – 2024 10 13 19 49 IST – Recording#

TAs Session - 2024 10 17 20:09:44#

TA’s Session TDS – 2024-10-20 19:50 IST – Recording#

TA’s Session TDS – 2024 10 24 19 50 IST – Recording 2#

TA’s Session TDS – 2024 10 27 19 55 IST – Recording#

TA’s Session TDS – 2024 11 04 16 56 IST – Recording#

TA’s Session TDS 2024 11 14 19 54#

TA’s Session TDS – 2024 11 10 19 49 GMT+05 30 – Recording#

ROE Session TDS 2024 11 15 19 53#

TA’s Session TDS – 2024 11 07 19 58 GMT+05 30 – How to hack Project 1 and possibly ROE, pdf scrapi#

ROE Session TDS 2024 11 16 19 52#

TA’s Session TDS 2024 12 05 19 52 GMT+05 30 Recording#

Project 2 TA’s Session TDS 2024 12 05 19 52#

Mock QP - TA Session - TDS 2024 12 19 18 51#

Live Session: 15 Jan 2025#

Live Session: 16 Jan 2025#

Live Session: 17 Jan 2025#

Live Session: 20 Jan 2025#

Live Session: 21 Jan 2025#

Live Session: 22 Jan 2025#

Live Session: 23 Jan 2025#

Live Session: 28 Jan 2025#

Live Session: 29 Jan 2025#

Live Session: 30 Jan 2025#

Live Session: 31 Jan 2025#

Live Session: 01 Feb 2025#

Live Session: 04 Feb 2025#

Live Session: 06 Feb 2025#

Live Session: 07 Feb 2025#

2025-02-11 Week 5 - Session 1 - TDS Jan 25#

2025-02-13 Week 5 - Session 3 - TDS Jan 25#

FAQ for Docker & Task Execution#

2025-02-18 Week 6 - Session 1 - TDS Jan 25#

2025-02-18 Week 6 - Session 3 - TDS Jan 25#

TA’s Session TDS – 2024 11 10 19 49#

2025-02-26 Week 7 - Session 2 - TDS Jan 25#

2025-02-27 Week 7 - Session 3 - TDS Jan 25#

2025-03-04 Week 8 - Session 2 - TDS Jan 25#

2025-03-04 Week 8 - Session 3 - TDS Jan 25#

2025-03-04 Week 8 - Session 1 - TDS Jan 25#

2025-03-11 Week 9 - Session 1 - TDS Jan 25#

Tools in Data Science (TDS) Tutorial FAQ#