How to Turn Audio and Video into Notes with AI

Meetings are inevitable, and you probably remember typing out an entire transcript. Fortunately, that’s a thing of the past, at least if you read this article to the end. In this article, we’ll zoom in on how to transcribe and summarize hours of lecture recordings or meeting audios simultaneously.

Nowadays, you can upload files to AI models, which often have advanced annotation tools. Take OpenAI, better known as ChatGPT, for example. Add a video file or audio recording and ask for a summary of the most important points. In the background, the model uses Whisper to provide you with a transcript. But leaving it solely to an transcription would be leaving you with an disadvantage, as there are more opporunities to be explored.

We’ll start this article by explaining how this process works and what the benefits can be. Then, we’ll discuss best practices for using ChatGPT to analyze these transcripts, with practical prompts so you can integrate everything into your study or meeting routine as effectively and easily as possible!

We are talking about applications of video or audio files here. But be aware that you should always check for possible copyright issues before uploading these files to an AI model. In addition, it is important to think in advance whether the file contains sensitive information about yourself or someone else. Think of names, addresses, etc., but also sensitive company information. The companies behind these AI models can store this information. Curious about tips on privacy? Then check out this article!

How to combine Whisper and ChatGPT?

Transcribing your lectures or meetings can be done using tools like Whisper and AssemblyAI. Using Whisper for transcription can be beneficial for you as a student or teacher. Whisper saves time by automating the transcription process, instead of manually transcribing lengthy audio recordings. It can be used in practice in the following ways:

1. Note-taking during lectures: You can use Whisper to transcribe online lectures, allowing you to focus on understanding the content instead of taking extensive notes. Teachers can also benefit from this feature when recording their lectures or presentations.
2. Improved documentation and follow-up during meetings: Transcribing meetings allows you to have a detailed and accurate written record of the meeting. This makes it easier for you to track action items, decisions, and key points discussed during the meeting. This can lead to efficient follow-up, ensuring that tasks and responsibilities are properly documented and assigned. It can also be beneficial for you if you missed a meeting and can easily read back the transcripts.

Using transcription tools like Whisper or AssemblyAI for lectures and meetings results in a plain text transcript, which, while accurately capturing spoken words, lacks deeper insights or analysis. This raw text is a factual record, but it doesn’t provide interpretations, summaries, or highlight key points. To bridge this gap, you can integrate ChatGPT with well written prompts. By feeding the transcript into ChatGPT with specific, targeted prompts, you can extract the relevant information from the transcript, such as identifying central themes, summarising, or getting a bullet point list of actions to be taken. This can help you revise lectures and meetings.

Prompts for getting valuable insights.

ChatGPT allows you to upload documents, which can include straight-up .mp3 or .mp4 files. You may have also taken the route of following an earlier article of ours by using Whisper. If so, you now have a .txt file with a full transcript of the lecture or meeting, like the one shown in the picture below.

You can open this .txt file and copy its content into ChatGPT. Otherwise, you can simply upload the video or audio file directly! In the examples below, we will refer to this copied text as [Transcript].

Prompt 1: Making a bullet-list of deliverables.
When you have uploaded a file to ChatGPT or pasted the text, you can add an overarching prompt, referred to here as a task. Why should you specify the task? ChatGPT is unaware of what it needs to do unless you explicitly specify it. If you don’t, it might assume the text belongs to the transcription itself.

In our example below, we have surrounded the transcript with backticks. These delimiters ensure that ChatGPT “sees” this as a separate part of the prompt. This way, it won’t be confused, great! Look at this article, for a more in depth approach in explaning how to structure several distinct parts of your prompt.

Below, an example of this prompt is shown for a meeting of the recruitment team at a university that discussed how to recruit new students.

You: "```Transcript```

Task: Make a bullet-list of the deliverables of this meeting."


ChatGPT: "The deliverables of this meeting include:

1. Create a set of personalized email templates to engage with prospective students, highlighting the university's unique selling points and programs.
2. Develop a social media content calendar with a schedule of posts, including engaging visuals and captions, to promote upcoming events and application deadlines.
3. Compile a frequently asked questions (FAQ) document for the admissions team, enabling them to provide quick and accurate responses to inquiries from prospective students.
4. Coordinate the production of a short video showcasing the university's campus, programs, and student life, suitable for sharing on the website and social media."


# Note: All output is fictional

While we were very. explicit with our format in the input. We can also ask ChatGPT to give the deliverables in a certain format. For example, if we want to make issues out of the deliverables, we can give ChatGPT a follow- up prompt like in the iframe below. It will then readily give all deliverables in code blocks when you prompt the correct way!

Prompt 2: Asking for a summary of the meeting or lecture.
Below an example of this prompt can be shown for a meeting of this website, asking ChatGPT to make a summary of one paragraph for this meeting.

You: "[Transcript] 

Task: Make a summary of the meeting of one paragraph"

ChatGPT: "During the meeting, the team discussed various topics and action points. They scheduled a hackathon-style meeting to work collaboratively and create content efficiently, emphasizing the importance of starting to write articles and content on WordPress. The team also explored potential collaboration with student associations, to expand their reach. Technical questions were addressed, and support was offered for using WordPress. The meeting concluded with plans for future engagements and events to promote Tilburg AI and its valuable content."

# Note: All output is fictional

Prompt 3: Asking for key learning points from a lecture
For a lecture, you want to have the key learning points, filtering out all unimportant talk and only keeping the main points. After this, you can even ask follow up questions based on these focal learning points, for example to create 10 practice questions.

You: "[Transcript] 

Task: Make a bullet-list of the key learning points from this lecture"

ChatGPT: "Certainly! Here's a bullet-list of key learning points from the lecture about economic games:

1. Definition of Economic Games: Economic games are experiments designed to study human behavior in economic decision-making scenarios. They often involve strategic interactions and real or hypothetical rewards.
2. Game Theory Basics: Game theory provides the framework for analyzing economic games. It involves studying the strategies and outcomes of players in various situations.
3. Types of Economic Games: Examples include the Prisoner's Dilemma, Ultimatum Game, and Trust Game. Each game simulates different economic and social scenarios.
4. Rationality and Nash Equilibrium: Players are assumed to be rational decision-makers, aiming to maximize their utility. Nash equilibrium is a key concept where no player can improve their outcome by changing their strategy unilaterally.
5.Cooperation and Competition: Economic games illustrate the tension between cooperation and competition. Factors like trust, reputation, and incentives influence players' decisions."

You: "Based on these key learning points, come up with four example questions to practice this material"

ChatGPT: "Certainly! Here are ten example questions to help you practice the material on economic games:

1. What is the primary purpose of economic games, and how do they differ from traditional economic modeling?
2. Define Nash equilibrium and explain its significance in the context of economic games.
3. Provide an example of a well-known economic game and describe the strategic decisions that players need to make in that game.
4. How do economic games help researchers explore social preferences, and what are some common social preferences studied in these games?
5. Give an example of an economic game that illustrates the tension between cooperation and competition. Describe how trust and incentives play a role in this game."

Prompting: As shown in this article we can get great insights from our lecture or meeting transcripts, especially when the quality of the transcript is high. It does however also depend on the quality of the prompts. In this article, we have given some example prompts for this specific use case, but the result always varies with every scenario. Having good prompts and understanding the different practices that are useful to follow when prompting ChatGPT is therefore always important.