What does product sense look like in a startup? Insights from a CEO

Highlights

You’ve worked at a lot of big organizations like LinkedIn, Pinterest, and Soundcloud. What did product sense look like in those organizations?

Kristin: People at all different stages of their product career have asked me about product sense and how to develop it. And although metrics and data-driven decision-making are really emphasized, those are only showing you one piece of the puzzle. I think the other half is having a very strong sense of your users. It’s like a gut feeling. You need to be very in tune with the community of people you’re building for.

What are their biggest pain points? What are the challenges they're facing every day? That is how I would define product sense. Keeping a pulse on the market is also very important because to develop that intuition. You need to be familiar with where the landscape is changing.

The problems that your users face are changing over time. At the core, product sense is being in tune with your users’ deepest pain points and also having a sense of how to solve them.

You don't necessarily want to do everything your users say, but you want to listen and dig deep to understand their problems and then work with your team to find innovative and simple solutions to solve them.

How do you ensure your connection with the user stays deep over a long period of time? What tactics, for example, would you give a junior product manager?

Kristin: I’ll start with my first product role at Twitch. If I was talking to a junior PM, I would ask them to think about their first 30, 60, and 90 days on the job. Your goal is to understand your customers. Try to talk to them. At Twitch, for example, I set up many calls with developers, streamers and viewers. In the days before COVID, there were also conferences– TwitchCon, BlizzCon, etc.

Think about where your customers aggregate and go where they go. Whether it's online or in person,go there to listen and learn. You can get a great sense of product sense on Twitter, but take that with a grain of salt. Sometimes the loudest users are the ones praising you or giving their feedback on forums like Reddit.

But I do think it's really important to monitor all of these sources of feedback.If you’re just starting up, though, you might want to use a user research team. Sometimes you might not have this support. In one of my early roles, I did not get any support from user research because I think there were four people across the entire company.

They didn't have time. And so I did sort of my own user research just to get a pulse on the market and what people were looking for from an insights perspective. If you work in a B2B environment, you could also spend some time with customer-facing teams like business development or customer success. Maybe you even go into the help center and read Zendesk logs. This can help you to really get in tune with the problems of your user base.

When you’re bringing on a new team member, do you embed content like Zendesk logs as part of their onboarding process or do you let it naturally occur over time?

Kristin: For my teams, I like to use a blend. When we had someone new join us at SoundCloud, their onboarding packet would have a mix of past research, key takeaways, or recordings of different interviews with creators or listeners.

One thing that I had implemented at SoundCloud was a guest speaker series event where every month we'd host creators from different platforms. So I invited some Pinterest creators, we had Twitch streamers, musicians who use YouTube, and we had the SoundCloud DJ of the month.

All of this would be part of their onboarding experience. But in that onboarding, having these conversations live is also really important. But it doesn't just stop with onboarding.

Some great advice I received early on in my career is that it doesn’t matter if you’re the CPO or entry-level, you should be interacting with your customers in some way a couple times a month. At Pinterest, for example, our CEO would often have lunch with a “Pinner” on a bi-weekly cadence to keep that product pulse.

He would read and present to us letters that he received from our users, as well. I think that product sense is something you never stopped working on, no matter what level of product you are, no matter what maturity product is in.

Latif: In the B2C world, you may be able to call on the millions of users that are using your product. Whether it's for SoundCloud or LinkedIn or Pinterest.

But for the B2B companies out there– if you have 500 companies as customers, it can seem a little bit weird to ask for their time because they can be really busy. But it's so important. And I remember in 2017 when we brought on a very notable bank in Canada as a customer, they came into our office and we took a full 90 minutes and we actually asked them to present how they use our software.

What was their onboarding experience like? What changed for them in their organization? And there were so many insights in there. We had to spend 45 minutes on Q&A because every single person in the company was throwing up their hands.

It was the craziest experience. So that story really resonates with me. And I don't think you have a huge large user base to do that. You could do that at any company, at any size. And I think that's such a great example that you've provided of how people can continue to engage with the customer if they're not on the front lines or in product management.

Sometimes PMs will try to build something their users are talking about and it just doesn’t work. Can you talk about a time that feedback or product sense didn’t work out?

Kristin: I learned both good and bad things from the first product I launched. The biggest lesson learned was that you have to be careful about the loudest voices or maybe your highest paying customers. In B2B, for example, higher paying customers can sometimes drown out the other voices in the room. Or sometimes people will ask a lot for something and then you launch it but no one uses it.

I learned this at Twitch. If I could go back in time, when I launched Twitch insight, I wish that I would have only accessed the data through our portal and an easy spreadsheet download versus spinning up an API. The API had a ton of requirements and it was really difficult for us. Anytime we made a change to the API, we had to let people know. And we couldn't make a ton of changes at once.

At my first role, I spent a lot of time doing a limited preview with a selection of larger game developers, indie game developers, and international developers. That helped us catch a lot of mistakes and made sure the product was welcomed when it went to the market. Those developers then turned into big champions for us.

But I will say I felt so much pressure from the larger game developers and even from the BD team to push out the API. I did it for that first release. Then I also realized that of course they're going to use it, but at the end of the day, it was more important for them to have more of the right data than have access to the API so early.

So if I could go back in time, I would have pushed back even more strongly and said, okay, you can wait for the API. It's hard because sometimes you are in a tough position as a PM. If you're working on a B2B product, you need to keep your largest customers happy. Sometimes those are the people paying your bills.

It's kind of a push and pull, but I do wish I had negotiated a bit harder. It all worked out in the end, but things were slower than I would have liked them to be.

How do you strike a balance between being data-driven or data-informed?

Kristin: My background before PM was all data analytics and insights. So I have an interesting perspective on this. I would say typically it's a blend. When I work with our teams, we try to start out with a list of questions that we're trying to find the answers to.

We'll sit down and try to understand if we can find this out through data. Can we find this out through research? Do we need to even run an experiment? When we’re asking those questions we're trying to maximize our resources.

Data is very important for any team because it helps you understand how your product is being used. It's important to set hypotheses. I'm a big fan of OKRs to really understand and have visibility into what's happening with your product and your customers.

Data can often tell you where a fire is, but sometimes it can't tell you why. I think that's also where the qualitative piece comes in. Let's say if likes drop at Pinterest, we’ll do a data analysis to figure out what's happening
But sometimes there could be something else happening in our product that triggered that.

You’ll need to do a deeper investigation because data is going to tell your numbers but even then, your data's only as good as your tracking.
Sometimes you don't even track the right thing. So I think it's important to build from the bottom up. Make sure you're tracking the things you want to track before you launch. But I do think there's such power in just having a simple conversation with someone who might use the product.
That may tell you so much more and give you that product sense versus looking at all of your dashboards and spreadsheets.

Another big launch we had at Pinterest was Reactions. One thing we did before we even went to market was just show some really early prototypes to a couple of creators that I knew closely just to get their own reactions to.
And I think the insights from just a few key conversations were so powerful and that saved us so much time. No amount of data analysis could have shown us that insight.

How do you think about product sense in a smaller company versus larger organizations?

Kristin: In both big and small companies you almost need to prove yourself before getting more investment by finding that product market fit.
In a bigger company, you’re going to have a lot of historical data available. But Top.gg is a small team so we don’t have all that information.

Right now we are laying the data foundation, but there are some opportunities where we're looking beyond the market and we have to trust our gut and make that call to take the company to where we want to be in five years.
Sometimes that does mean you do take a hit on your initial metrics. But you have faith that you're going to build those back up later. I will say, this is a much different approach than what I've traditionally done at larger companies.

Sometimes at larger companies, you're in a state where it's so hard to move your monthly active users or weekly active users. And so, you're running a lot of experiments to get those small percentage incremental wins. But I think where Top.gg is right now, we're really in a 0.1 to one stage. And we're going to be making some bold moves to figure out the future of our company. I'll give an example.

Traditionally, at a larger company, if you're doing a redesign you would typically remove that from core product functionality changes. But I've been in situations where we’ve had resource limitations and couldn’t wait. We just had to go for it.

We have to launch the redesign and change the core product to help get us into the future.

Latif: That's amazing that there is such a correlation between the zero to one type projects or the zero to one atmospheres in a company like SoundCloud. I'm a huge fan of their product and what they've done, but having to make changes that large, to get the needle to move is important.

I suspect in some of those decisions you made or were part of– how did the decisions get made? For example, there could be someone who thinks the product should go in one way or another, and product sense at those levels could be just a bunch of people yelling at each other. And that’s just the reality of business, sometimes.

Have you seen a situation where information you've had gets thrown out the window when there's a large bet to be made on product direction? And what have you learned in terms of the realities of business around product sense and in that regard?

Kristin: One thing they did a fantastic job of at Pinterest was transparency and communication.

When I first joined Pinterest, I joined to build out the foundation of creators. And it was actually quite controversial at the time because there were a lot of debates around Pinterest’s positioning.

Do we want it to be more of an individual experience or a social experience? If you looked at Pinterest, at the time you didn’t see many kinds of people on the platform. It was more about the images. But bringing in creators and bringing them to the forefront was really what changed the dynamic.

So when I joined, it was almost controversial adding reactions to the platform. Pinterest wasn't really regarded, at least internally, as a social network. And they wanted more differentiation. I bring that up because while I was there the leadership and I was of course advocating for creators and community. Finally, leadership made the call to make it our focus.

And when this decision was made, our CEO and leadership were very transparent with the company on their thinking and why they pivoted the entire company to focus on creators and shopping. At the time, this was a big pivot for a big company.

The transparency paid off for Pinterest. They were just named number one platform for creators by AdWeek. I was so proud to be part of that foundation, but living through it was very controversial at the time.

I will contrast it to another example from SoundCloud. If you know the history of the company, we’d joke that SoundCloud was a startup with 12 or 13 years of tech debt. But the company is great. It’s been around for so long that it’s gone through several transformations. There was a previous back and forth between focusing on creators and then listeners, but that episode didn’t work out.

But now, they're really focused back on creators. One tough thing that many startups often have to make when you have limited resources is where are you going to invest in? What's going to fall off the table? And while I was there, one hot topic was trying to prioritize how much we want to lean into creators versus becoming like a digital service provider.

The latter meant focusing on building and selling creator tools. But there's a listener experience, too. Where are we going to make the call? This was the decision that I as a director of product has to make. One value I remember from Twitch and Amazon that I loved was “disagree and commit.”

When those changes are happening, as a product person you might not like it. But I do believe in having a culture where you can speak your mind. At the end of the day, leadership needs to make that call and say, “Hey, can we disagree and commit? And can we do this and go all in?”

One thing that was tough at SoundCloud was finding a focus.

There was so much we wanted to do. But we didn't have the resources to do 80% of that. Accepting that was difficult, we wasted a lot of cycles because of this denial. Whereas we could have done something more similar to Pinterest. Sometimes leadership does have to make the call and say, “Hey, let's focus here and let's move on.” Those are some of the learnings that I want to apply to Top.gg because we need to move fast and we have limited resources.

Latif: That’s the quality of great leaders. There could be difficult decisions and directions in the product, and nothing can set the stage where people can disagree and say, here's the direction we're going.

This means that the sensibilities around the product need to be in that direction. It creates a focus. It creates an alignment. So, sometimes product sense needs to actually be informed by leadership, especially if your whole company is surrounded by a singular product. Which in today's age seems to be a very common trend as opposed to a collection of them.

So, I really love those examples because sometimes when things are not working for a product manager, often it's about just understanding what the executives company's goals are and, or strategy, and that should come out with a bullet point sometimes around the product.

How have you transferred some of the product knowledge and senses into being a CEO now?

Kristin: One project that I always bring up for my time at Pinterest was a product called the Engagement Tab. This product actually came out of a hackathon that myself and an incredible engineer worked on outside of our regular jobs.

Product sense was such a key role in building this out. We took it from zero to one and it's one of the most loved products by creators today. It lets them respond to every single comment on their pin in one place on the platform.
Again, at the time that was a very controversial product because we weren't sure if we were going to invest in creators. So, working on that as a little side project and taking that to scale, taught me so much about what I want to bring to Top.gg

Part of that was having a really clear vision for it and also building directly with the community and with the creators themselves. That's one of the reasons I joined Top.gg, they have a really vibrant vibrant community that's heavily, heavily engaged. I think our Discord servers have almost 200,000 people. There’s generally 40,000 people online at any given time.
I think this applies actually to every product, but especially in the creator and passione economy– bring the community along and build for them. This is a value I’m living as a CEO.

You also need to be okay wearing many different hats over the years. At Pinterest, when our product analyst left I had to do product analytics until we hired someone new.

Or when something's not working you have to be willing to go dig through that spreadsheet or do something manually. That's something that I've brought to Top.gg and that's a big part of our culture. It doesn’t matter what level you are or what your function is.

We're all willing to get our hands dirty. That is a really successful trait I've seen in product managers.

About Our Guest – Kristin Chen

Big thank you to our guest, Kristin Chen, for joining us on this episode of Product to Product! To learn more about her product thinking, follow Kristin on LinkedIn.