The latest meta for using ChatGPT as the service has evolved and grown since its release nearly a year ago  ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌

	Forwarded this email? Subscribe here for more How to Use ChatGPT as a PM or Product Leader in 2024 The latest meta for using ChatGPT as the service has evolved and grown since its release nearly a year ago Feb 1   READ IN APP   AI won’t replace PMs. PMs who use AI will replace PMs who don’t. This post might save you more than any other I’ve sent to your inbox. It’s hard to overstate how important of an innovation ChatGPT is. Released just 1.5 years ago - November 2022 - it can feel like it’s been around forever. But, really, social media got over-inundated with posts about it last year. We all ended up with a bit of nausea just seeing, “10 prompts for ChatGPT.” The cringe social media posts didn’t mean the product got worse, though. ChatGPT Keeps Getting Better If anything - the Human Reinforcement Learning is driving ChatGPT to get better and better. I recently met with a VP of Product who said that he was “stunned” at how much better GPT-4 is than it is now compared to 10 months ago at its release date. Not only does ChatGPT keep getting better, it’s going to keep getting better: Up next is GPT-5, which Sam Altman described as his top priority. AGI doesn’t seem likely in 2024. Then, ASI (Artificial Super-intelligence) is within the horizon by 2031: This makes ChatGPT a key PM skill All these upcoming developments mean learning how to use ChatGPT well is like learning how to use Excel or SQL before everyone had to. It’s a huge career advantage. And unlike things like Excel or SQL, which have a huge learning curve, it has a medium learning curve - but very high usefulness for PMs: The reason ChatGPT gets a 5 rating above SQL and spreadsheets? It can help you do wizardry in your spreadsheets and SQL. But the meta keeps changing In video games like Fortnite, players talk about the “meta.” It’s the ideal way to use functionality that’s widely available to everyone in the game. The meta for ChatGPT keeps changing. OpenAI has an incredible velocity of shipping. So if you’ve been using it the same way as 10 months ago, you may have missed some significant changes. Introducing Carl Carl is a Senior PM at GoodRX. All the way back in May of last year, I collaborated with Carl on an Advanced Guide to Impact Sizing. You all loved the piece! So, we’re back with another post. We concocted the idea a few weeks back, and having been investigating the meta ever since. You can find Carl on X, LinkedIn, or in his newsletter. Today’s Post Words: 8,671 | Est. Reading Time: 40 minutes The New Meta The Top 10 Use Cases The Top 7 Mistakes People Make Including… What not to use ChatGPT for How to manage hallucinations How to avoid leaking company secrets The Extended Long-Tail of Use Cases and Prompts Alternate and Free Ways to Access ChatGPT Key Use Cases for Product Leaders Tips for Debugging The Future Upgrade to paid The New Meta There’s almost been innumerable changes in the last 10 months. On top of the neural networks responding to the user feedback, the team at OpenAI ships incredibly fast. The key changes - for users of ChatGPT Plus - that have happened with the product experience in the last 10 months are: Search and code interpreter have been built-in to to the core experience It is much better at understanding long context and mega-prompts The model is much less lazy and will do things you ask You can upload PDFs directly into ChatGPT It’s easy to talk with it via Voice What it means for you This has lots of implications for how you can use it. We’ll cover them in glorious detail throughout this piece. But a key one to highlight is: Short prompts are OUT Long prompts are IN There has been a bit of a Yo-Yo with ChatGPT. As the GPT-3 meta developed, mega-prompts became clearly meta. But, as GPT-4 was released, the model was not reliable for long prompts. This led to the proliferation of short prompts. That advice is old now. And mega-prompts are back. The Top 10 Use Cases There’s unlimited ways PMs can use ChatGPT - and we’ll cover more at the end. But these are the top 10 that should make you more productive today: Draft a PRD Write a SQL Query Apply a Framework Do a Design Critique Create Your Prototype Draft an Impact Sizing Respond to Email Faster Explain a Technical Topic Create a Discussion Guide Edit a Feature Results Writeup Let’s go through each in depth. Use Case 1 - Drafting a PRD Under the old model of prompting, you might just ask ChatGPT to write a PRD. But now, it really helps to get things ready for it. The Prompt This mega-prompt below is your template: # Instructions ## Act as a an expert writer of Product Requirements Documents. Make targeted assumptions to draft the type of PRD that a designer or engineer picking up would have absolutely no questions about. # Skills and Areas of Knowledge ## Possess a deep understanding of the product, and creating a product requirements document to be ready for engineering and design. # Next Steps ## Draft the PRD and share the questions you have to make it even better. Make best guesses where I don’t specify based on what I do specify. # Personality and style ## Act as someone who has a good strategy, UX sense, business sense, and is confident this feature will work. # PRD Template ## Problem  Describe the problem we are trying to solve in 1-2 sentences. Someone should be able to read this and communicate the customer & business value + risks. Highlight evidence or insights we you have to support this. ## High Level Approach Describe the rough shape of how we might tackle the problem. This should be a few sentences max. ## Narrative Share (hypothetical) stories to paint a picture of what life looks like for customers today. Describe common and edgy use cases to consider when designing the solution.  ## Goals Describe high-level goals, ideally in priority order and not too many. Include measurable (metrics) and immeasurable (feelings) goals. Keep it short and sweet ### Metrics Highlight specific, operationalized north star metric, secondary metrics, and guardrail metrics. Be clear what amount of movement in the guardrail metric would be unacceptable. ### Impact Sizing Model Create a model taking the known information to calculate impact size against the key identified metrics, as well as relevant bottom-line metrics like revenue/profit. State the estimated impact on the north star and bottom-line metrics. Show the calculation steps to get there. ## Non-goals List explicit areas we do not plan to address. Explain why they are not goals. These are as important and clarifying as the goals ## Solution Alignment  Draw the perimeter. Do not force others to identify scope. ## Key Features ### Plan of record List the features that shape the solution, Ideally in priority order. Think of this like drawing the perimeter of the solution space. Draw the boundaries so the team can focus on how to fill it in. Challenge the size to see if a smaller component can be shipped independently. ### Future considerations Optionally list features you are saving for later. These might inform how you build now ## Key Flows Show what the end-to-end experience will be for customers. Get detailed here screen by screen, not high level. Highlight what screens a user is on, what they click, and the result. ## Key Logic List rules to guide design and development. Address common scenarios and edge cases. Make it so an engineer feels everything is thought through. ## Launch Plan Define the various phases that will get this product to market, the purpose of each phase, and the criteria you must meet to move on to the next one. Put this into a table. Highlight whether it will be an A/B test and how long would be needed for stat sig results based on north star metric goal. Highlight risks and dependencies that can throw a wrench in timelines or progress (and ideally contingency plans). Create a table. ## Key Milestones Highlight the key developmental and design steps to go. Put this in a table. # PRD Known Information <Add what you already have> This template, adapted from our deep dive on PRDs and inspired by Kevin Yien, will get you a great output. It works better if you can also add some chicken scratch to each section and iterate with Chat GPT. Specific Example You can see the actual chat conversation here. In this example, we’re working at Gmail, and we’re working on a Superhuman-like messaging framework that only shows them e-mails to respond to soon. At the end of the PRD known information section, I’ve added details to the prompt of what we know so far. People prefer the Superhuman inbox, which is more selective, to Gmail's Primary inbox, which is more permissive and user-rules based. We know this because Gmail saw churn of 0.75% of users last quarter to Superhuman, the largest churn to competitor. We want to create a Superhuman style inbox and notification system that helps people only see what they need to respond to now, now. As a user, when I open Gmail into Primary I see way too many messages. Superhuman just shows me the one's I want to reply to. We want to reduce churn, increase people's satisfaction and NPS of Gmail, and ultimately increase MAU of Gmail beyond the 1.24B it is now We don't plan to redo the primary inbox. We need to figure this out in addition / enhancement / or toggle Several VPs are obsessed with the idea, but we have to make it work. The result is a pretty good PRD first draft. It takes your chicken scratch and gives you a first version to iterate upon. In particular, what’s valuable is it does quite a bit of work to get the barebones of a document going. Of course, the mistake would be to share this with the team. When looking at it, we immediately as PMs will several improvement areas. You have two options at this point: iterate on those with ChatGPT in conversation, or just replace them yourself. To give ChatGPT feedback, a numbered list works best. For example: Great stuff. Some feedback. 1. Primary goal should be reduced churn, secondary should be user satisfaction and NPS 2. And that should also flow through to metrics 3. We should specify what engagement with the feature metrics we are considering 4. It's unclear how this interacts with primary inbox and we should make that clear in key features and flows 5. It's unclear how users opt in and out and we should make that clear 6. It's unclear how the AI model is different so we should make that clear 7. Just assuming a 2% increase in MAU isn't as persuasive as giving a reeason and calculating it off the 1.24b we have. This leads to a nice improvement over the prior PRD: Both patterns work. But probably the ideal PRD workflow has become: Chicken scratch ChatGPT Draft to start Edit it (or with ChatGPT) Keep editing it as the PRD progresses Once you see what ChatGPT has done, you’re not working from the blank page of your PRD template. You have something to react to and improve. PRDs are one of the primary work products of PMs. Making them awesome is more important than doing them. ChatGPT can help the time to awesome. This post was too long for e-mail. You may want to head over to your browser: Continue Reading Online Use Case 2 - Write a SQL Query In the ideal scenario, we’d have access to all the data we need - and our friends in the analytics department can help with the rest. But in the reality of the PM day-to-day, there’s lots of data questions that come up, that you would benefit from being able to query yourself. A strong mastery over a tool like Snowflake can be a PM superpower. It certainly was for me at Epic Games, where I was able to bring data to every conversation and knew how to write SQL queries that would yield results in 1 minute - not 1 hour. But that took me 1000s of hours over years to achieve. Now, PMs can do that just with ChatGPT. The Prompt # Instructions Develop a SQL query. # Skills and Areas of Knowledge Proficiency in SQL, with a specific focus on fast-executing queries that look up a reasonable sample, use sub-queries when appropriate, use readable aliases, and have great notes. # Data and Tables Overview <Insert> # Query Crafting The query should: <Insert> # Result Expectation A comprehensive SQL query that provides a clear use of the data tables to result in the query craft, along with a brief explanation of each part of the query for better understanding and future tweaking. You basically want to a result that has: Great notes Great explanation And doesn’t get anything wrong So this mega-prompt gives you an easy copy-paste. Note: You do want to include information about your tables, in addition to your query goals, to get the best response. If you aren’t familiar enough with the data tables, you can also put: What types of data tables should I be looking for? Walk me through iteratively to get to the right tables. Example You can see the actual chat conversation here. Let’s say you’re a growth PM at Facebook. Your team just rolled out an A/B test on a revamped notification system aimed at increasing DAU. You want to check-in on the results. Here’s the type of information you’d want to add to the query. # Data and Tables Overview Access to 'user_activity' table with columns like 'user_id', 'session_id', 'session_length', 'activity_date', and 'notification_version' (1176a or 1177b). The 'user_segmentation' table includes 'user_id' and 'segment_type'. # Query Crafting The query should: Segment users based on the notification version they received. Sample because we have absolutely massive data with billions of users. Calculate the average session length and daily active usage for each segment. Compare these metrics between the two segments to determine the impact of the new notification system. The result is a nice query that uses CTEs and has all the explanation you could need to use them yourself. What’s especially helpful is the guidance at the end on sampling and dates: That’s what ChatGPT is the king of SQL advice. I spoke with a Senior PM who says he uses ChatGPT 2-3x per week for SQL. That sounds about right. You can also continue with ChatGPT to iterate on what further questions you have, or error messages you run into. The numbered pattern we mentioned earlier works well for this: Great. A couple things: 1. 1% sampling might be too big still 2. How can we add some other cool metrics to this 3. EG how do we cut the data by people who receive 1,2,or 3 notifications; and males vs females; young vs old The result is a query that will run fast and segments the data a bunch of ways. Those types of case when statements are a bear to write on your own. Even I usually end up running into an error or two while writing them, after decades of SQL. But with ChatGPT, they’re easy. And the best part: From there, you can just keep iterating for what you want. Using ChatGPT for SQL dramatically increases the volume of insights you can have, because it decreases the time to insight. Use Case 3 - Create Your Prototype One of the most “wow” things to do is go from napkin sketch to working prototype for user testing in 2 minutes. And ChatGPT can do that for you with this mega-prompt:... Keep reading with a 7-day free trial Subscribe to Product Growth to keep reading this post and get 7 days of free access to the full post archives. Start trial A subscription gets you: 100x/ Year Deep-Dives: 3K+ words designed to pay the subscription fee back in career earnings Full archive Many expense it with their company's learning budget   Like Comment Restack   © 2024 Aakash Gupta 548 Market Street PMB 72296, San Francisco, CA 94104 Unsubscribe