Back to Blog

Using Customer and Non-Customer Feedback to Deliver Value at Torchlight


Carolyn Romano

“Innovate or die.” It’s one of many famous quotes attributed to management guru Peter Drucker. 3M, Apple, and many others serve as examples of innovation rescuing companies from the brink of extinction. But innovation shouldn’t just be a lifeline to a failing venture. It should be part of a company’s DNA that inspires and nurtures a culture and environment for creating products, processes, and business models that deliver new value.

Unfortunately, innovation too often gets swallowed by ongoing efforts to maintain existing product value. How do product leaders protect innovation, and why is innovation something that needs protecting? Who is responsible for innovation – if everyone is, no one is. These are some of the challenges we probe in our discussions with innovation leaders.

This week we sat down with Carolyn Romano, Head of Product and Content at Torchlight. Carolyn is also the Owner and Holistic Practitioner at BLISS Healing Arts, where she practices mind-body medicine, self-care planning, and skill-building for individuals, groups, and organizations.

It’s not all that uncommon for companies to have to communicate with multiple different constituent groups. In Torchlight’s case, they have customers who are licensing their software, and they have caregivers who are the actual users of the platform. To complicate things further, these caregivers don’t necessarily always self-identify as users/caregivers. And when they do, their use tends to be more episodic and sometimes in a moment of crisis! How do you then resolve that balance between communicating to each of those constituents in a manner that doesn’t feel confusing or at least targets them when they’re in the right frame of mind for interactions with you?

Part of the answer involves data – quantitative and qualitative data – from customers and from people who don’t even know what or who Torchlight is.

How seriously do they take this data and these interactions? Feedback from these “Product Councils” made it into their latest product release after sharing roughly seven iterations, wireframes, and prototypes with customers, users, and non-customers!

What to hear more customer feedback wisdom from Carolyn? Click on the show below to listen to the full podcast. And don’t forget to subscribe on iTunes or SoundCloud.

Follow The Dirt on Twitter

Listen to the Show


Podcast Transcript

Richard: What I want to start with is to ask you what’s on your to-do list? What are you working on right now?

Carolyn: Right now I am working on a few things with the product team, the first of which is around analytics. We are working on verifying that what we hope our users are doing is what we want them to be doing, that it’s meaningful for them and how do we measure meaning, usefulness, that sort of thing. We’ve been spending a lot of time looking at our data, looking at what’s on our product road map, making sure that aligns with this metric that we’re going after around the … Go ahead, sorry.

Richard: Sorry. Tell me a little bit more about how you guys treat data. Is this basic data analytics or does this stuff ultimately find its want into predictive analytics as well?

Carolyn: I think it’s a both and. We’re certainly looking at it to make sure that we’re seeing more engagement, more visits to the site, that sort of thing, but we also wanna use it in a more formative way, that if things are useful, why were they useful, if they weren’t, that sort of thing, what can we expect people to do. We’re learning some really interesting things about the users that we have, which probably doesn’t fit the typical mold of an active user. We have folks who generally use our product when they’re having some kind of crisis in caregiving, and so it’s kind of episodic use, which maybe others might say, well, you haven’t seen someone for a couple of months, so they’re not meaningfully engaged, but what we have come to learn is that as soon as another crisis comes up, they’re right back in. So, we’re kind of redefining some of what our typical metrics out there. It’s just been kind of interesting for us.

Richard: And does that mean when the average user is interacting with your product they’re coming to you with a crisis mindset as well? Do they associate you with crisis and you have to turn that dip into a positive experience?

Carolyn: That’s a really good question. I think some users do, and I think others users are quite relieved that we exist, and they know they can come to us. A, because we really try to make sure people understand that we’ve been there and we get it and we look at how things are supposed to be out there in the world of caregiving, whether you’re caring for an aging loved one or you have a child with a disability, saying, Okay, this is how it’s supposed to look like, what the laws say, but here’s what you may find, and it might not line up, and therefore here’s what you can do. We’ve really found a way to support users where they are to be able to take action on behalf of their children or their parents, that sort of thing.

Richard: So, you’re talking about taking the data that you have and analyzing that and analyzing that and trying to figure out how to make it a better or higher quality interaction. Why would you need to do that? Is it not obvious enough that the value that you provide is something that people need or is it something else?

Carolyn: Well, it’s a few things. Top of mind is that sometimes people don’t identify themselves as caregivers. Some of that is marketing messaging, but it also is how we market to users. We’ve got our customers who are writing the check for a license to use the product, and then we’ve got all our users. We’re looking at how are we making sure that we’re meeting the needs of both of those constituent groups.

Richard: Yeah, so there’s obviously some purpose behind analyzing the data. What’s the problem you’re trying to solve with that investigation, if you like?

Carolyn: Oh, sure. Well, making sure that we are helping people care for the most important people in their lives, making sure that they’re getting the information that they need, that they’re able to take some action, that they feel empowered to do something, to make the phone call they need to make or to talk with the school system or write the letter or show up at a meeting feeling prepared to make sure their kids’ needs are being met.

Richard: Right. You said there’s multiple constituents there. How do you then resolve that balance between communicating to each of those constituents in a manner that doesn’t feel confusing or at least targets them when they’re in the right frame of mind for interactions with you?

Carolyn: That’s a good question. A lot of that messaging that goes to our customers, that’s more front of the house. Marketing handles that. Our client success team handles that more than what our team does, so we’re more focused on the users who login to the platform. Although, we are having to always keep the marketing messages in mind around return on investment and productivity, retention, recruitment, all of those arguments for why this would be an important benefit to offer.

Richard: So, you’re ahead of product. You’re thinking mostly about what happens once they become a client or a user of the Torchlight platform. Beyond that, do you feel like you’re able to maybe influence the way that marketing talks about the product? For instance-

Carolyn: Absolutely. Yeah.

Richard: … it’s obvious that you would need to request a demo in this case and understand what the platform can do for you before you understand the value. How do you then translate that to something that somebody doesn’t need to interact with before or they need to be able to understand the value of before they even start interacting with it?

Carolyn: Right. Well, we do a lot in our nurturing campaigns to educate the marketplace around caregiving and how it impacts the workforce. The product team is heavily involved in the demos, or we’re available to be subject matter experts. We do have a significant amount of influence on the HR folks that we’re talking to who are considering purchasing, especially if you find someone who they’re in a current caregiving crisis of their own, they’re taking care of their mom with Alzheimer’s. You really make that connection. Or their child’s on an education plan in school and they need to get in there themselves to check things out.

So yeah, I would say that product does heavily influence how we’re messaging, and I also think front of the house stuff impacts how we’re developing as well because sales teams, they hear about the needs out there, so we’re trying to make sure we’re being responsive while still being integral.

Richard: Yeah. And as the product lead, as that feedback comes from the sales teams, from marketing, and from your customers, can you describe how that gets to you, how that percolates back to you and what do you do with that stuff?

Carolyn: We’re still a flat organization given our size, so a lot of those conversations, at the beginning I’m often involved in kind of a generative way for how those conversations end up taking place along with other people here. I can give you an example. Benefits integration is a really hot thing right now out in the world of employee benefits, so helping a company find easier ways to communicate everything that’s available and not duplicating services, that sort of thing. As that need was emerging and we were becoming more aware of it and from the sales team, they communicate that to us, and then we looked at what are ways that we can provide integration points, content that takes, okay, your company has an autism benefit, here’s our content on autism, and here’s how we can connect with that other benefit and make sure people are aware of it and can then go directly to it from our product. That would be one example.

Richard: Yeah. How big is your product team?

Carolyn: We are five.

Richard: And what does that look like from a functional point of view? What is the cross-functional …

Carolyn: Sure. We’ve got me. We’ve got Lenore, who’s our elder care practice leader. We’ve got Stephanie, who’s a content writer and more leaning towards the child practice side of things. We’ve got a UX/UI researcher, and then we’ve got someone who handles a lot of the day to day operational tasks around webinars and court requests and advising sessions, doing all of that detail, a lot of the data stuff. And then, we all do everything. We have our specialty or focus area, but we also all pitch in and swarm and make sure that we’re moving the rock forward.

Richard: Let’s talk about this from a cross-functional point of view. How are you guys working as a team to get stuff done, whether it’s the day to day updates and improvements to the product or whether it’s thinking about what’s coming down the pipeline in six months time?

Carolyn: Sure. We sprint. We do two week sprints right now, so based on what we’re doing, probably we go out a quarter, maybe six months. Beyond that, we know that things can change pretty quickly. We’re meeting daily. We’re checking in. I’m in charge of removing obstacles for people and liaising with the front of the house. If something crops up, like for instance that integration work, we know we need to leave a buffer. We follow a lot of the SCRUM recommendations to get work done. We also make sure that some of our performance measures are team driven so that people feel really good about helping each other out. It’s good for everyone if we are all meeting our goal.

A lot of communication, a lot of collaboration. We also periodically will do reviews of what we’ve been working on with the rest of the leadership team so that they’re aware of what’s coming down the pipeline.

Richard: Yeah, that was gonna be my next question. What is the review process? What does senior management or leadership look like?

Carolyn: About every two weeks, we have a like 30 minute meeting where we present to leadership teams, so sales, marketing, biz dev, our CEO, and we do a little show and tell. Here’s what we’re up to, here’s the new content areas we’ve been covering, here’s what’s coming down the pike. It gives them an opportunity to weigh in or let us know things that they’ve been hearing. That sort of thing. We try to keep things short and focused.

Richard: And how do you then do the work in reverse? How does the vision of the leadership find its way back into your day to day operational stuff?

Carolyn: Usually in sprint planning, and we have that fairly regularly. We’ll either do that, or we’ll extend a standup and say, okay, this is what sales is hearing right now. And we also have a weekly staff meeting where some of that stuff will also come up there. We use a lot of internal communication. We’re all big Slackers. But we work really hard. That’s also a really important way. We just converted to that, probably in the last year and a half, and it’s made a huge different in how quickly information is shared. Yeah.

Richard: Yeah, yeah. Do you use any other tools for communication or is it just the Slack?

Carolyn: We use Trello for sprint planning. We don’t need anything too sophisticated for that.

Richard: Right. You’re a small team. Right?

Carolyn: Yeah. You know, calendar, the basic kind of stuff. But we don’t do lot of emailing anymore, which is interesting. We used to be so heavy on the emails, and it’s just been separated out in a way that I think has been positive for us.

Richard: The balance between doing the day to day stuff and then thinking ahead, sprint planning, let’s talk about sprint planning that goes beyond just the next sprint. How do you think about the next 6 months or the next 12 months? Do you have a roadmap in mind or are you still early enough in that stage where that’s not a big concern to you?

Carolyn: No, we do have a roadmap for sure. As I mentioned, we kind of three to six months planners because of how quickly things tend to be changing in the caregiving space. Beginning of a quarter we’ll say, What kind of loose ends have we got from the quarter before. Typically, there are a few, and then, What are we jumping into. A lot of it depends heavily on what our users are saying, so we’ll do a review of our survey data, our evaluations, our interviews, our UX person will maybe launch a new set of interviews and then do some recommendations around new feature development. That’s kind of how we’ll plan how to prioritize our content, which believe or not changes. We know there’s a universe of content that we need to cover, but the priority will change, sometimes month to month. And even just the format of how we decide we’re gonna share it. We’re doing a lot more around multimedia, podcasts and things like that.

Richard: So, do you ever do the kind of qualitative research with your clients where you’re sitting with them and talking to them about fundamental problems, or is it mostly about the specifics of the product?

Carolyn: We do both. We did a new release back in June that grew out of just what are your caregiving challenges from people who didn’t even know what Torchlight was, and we redesigned our platform based on that interview process. And then we included those folks in our wireframing and iterating.

Richard: You’re doing ongoing research with them? You’re not just doing something up front and then taking it from there? You’re actually seeing how they respond to the feedback you’re integrating into the product?

Carolyn: Yes. And with the new release, I think we went through seven iterations that we put in front of people.

Richard: Wow. That’s awesome. And do they become somewhat of a … let’s call them for the lack of a better term a customer advocacy group? Do you use the same people-

Carolyn: We call them our product council with kind of rolling admission to it. Every time we have an opportunity, we invite folks to be a part of it. I do monthly webinars where I orient folks to the platform and always put out that invitation. I’d say we get anywhere from one to four people who then become part of it. Yeah, it’s been a really valuable way. And then we also know lots of people who are caregivers. Everyone is a caregiver at some point, so we also will tap those resources too, folks who haven’t really never seen the platform so that they’re not responding to that, that it’s just really fresh.

Richard: It sounds like you speak so calmly and maybe even authoritatively over this process. What is the biggest challenge for you then? What feels like something that is a little more out of control and needs more attention? What could benefit from a little bit more control, let’s just put it that way.

Carolyn: Well, bandwidth is always an issue for us with so much that we wanna cover. The juggling of okay we thought we were gonna do this and now we need to do that. Sometimes that feels a little bit … your adrenals get involved with that for sure. Most of us here have also been doing it for a while, so we kind of know. We’re very careful when we’re screening new hires that people understand that if you’re not flexible person, if you can’t pivot, if you can’t be okay with having to abandon something you’ve been working on that really isn’t quite meeting the needs anymore, then this might not be the place for you to work. And that’s served us really well. We really take time with our hires, even when we’re filling desperately needs … we need an engineer or whatever, we really try not to rush that process.

Richard: Yeah. Sure, that pays off in the long run, right?

Carolyn: Yeah, it really does.

Richard: Yeah. And for you personally as the leader of this group, is there a challenge that you feel you need to take on in the coming months?

Carolyn: Yeah, absolutely. I was just trying to pick which one. There’s plenty of those. You know, I think some of it is still where’s the balance point for me with being that bridge between front of the house, back of the house, navigating that. The demands can be immense. So balancing that with what we’ve got on the roadmap. It kind of heats up, especially as you get into Q3, Q4 and you’re trying to close out the year on a high note. I think that balance for me is always challenging.

I also think really trying to showcase my team, making sure I’m advocating for them, making sure that they’re feeling really good about working here is also top of mind for me, and am I advocating for them in the right way. I spend a lot of time on that because they’re great people and they’re working really hard and they’re doing excellent work, and that’s an amazing combination, and we don’t want to lose them. We have a lot of support from our CEO for sure, but when is the right time to bring things up. We have that kind of stuff.

Richard: Sounds like you’re the poster child for head of product here, I must say.

Carolyn: Well, you know what’s so interesting to me is that when I first talked to Adam about my growth here and came into this role, I didn’t exactly understand what it was, what it encompassed, and so when I was reading your book, it was so affirming for me. It was how you laid out that there’s so many ways people interpret it or it’s this emerging field, like all of that stuff. I was like, Oh, okay.

Richard: Yeah, there’s definitely no one right way to do this job, and that’s one of the challenges of writing a book is that I think readers of books like that sometimes want a silver bullet answer, and the truth is there’s not one answer, there are multiple answers, and you can get there through different paths, and that’s almost the message that becomes the … how should I say, the guiding light for a lot of leaders, that they feel relieved when they hear that there isn’t a single path, that they can find their own journey, and they can craft a good leadership experience for their team in their own way. It doesn’t have to be done in a generic way.

Carolyn: Yeah. That was so affirming for me, because you do question yourself. You’re in a startup, it’s high stakes, and are you doing it right? Is it enough? All of those questions. So to really have it kind of memorialized the way you did, there are many paths to enlightenment, right? It’s the same notion. There are many ways to do this, and you need to really pay attention to your team and your mission and yourself and your own biases, all of that stuff.

Richard: Yeah, the one criticism that we got on the book was a couple of people said that they felt like these lessons weren’t product leadership specific, and after they wrote generalized around leadership, and the answer to that is yes, of course. All leadership has some set of common experiences that go well beyond your domain or your protocol or even the time and era that you’re living in. I think leadership has consistencies well beyond the fashion of the day. It’s hard to write for a specific role when those things are true.

Carolyn: Well, absolutely. That’s almost a compliment that came out of a criticism, but it’s almost a compliment. It’s like in the education world when you say, okay, well good teaching for a child with a reading disability, for instance, is good teaching for everybody. Let’s look at how we can do it well for this population and have others reap the benefit. I don’t know if that’s a perfect analogy, but it’s what it reminded me of.

Richard: Yeah. That certainly is true. Well, Carolyn, thank you so much for your time. This has been a really interesting and insightful way to share your wisdom. We love your wisdom. Good luck with what you’re doing at Torchlight. If I see anything interesting, I’ll send it your way.

Carolyn: Okay. I really appreciate it. Thanks for your time as well.

Richard: You’re welcome.


Author Richard Banfield

As CEO, Richard leads Fresh Tilled Soil’s strategic vision. He’s a mentor at TechStars and BluePrintHealth, an advisor and lecturer at the Boston Startup School, and serves on the executive committees of TEDxBoston, the AdClub’s Edge Conference, and Boston Regional Entrepreneurship Week.

More posts from this author

Sign up to receive updates from our blog

What we do Expertise

From concept to design, we'll partner with your team to deliver amazing product and website experiences.

Recent Projects Work

See the results of our most recent digital product and website engagements.