Ep. 10: Adam DuVander, Principal at EveryDeveloper
Moesif’s Podcast Network: Providing actionable insights for API product managers and other API professionals.
Derric Gilling, Moesif’s CEO, is your host today.
Listen to the episode on SoundCloud above, watch it on YouTube or download it on Apple or Google.
Table of Contents
- 0:41Tell Stories to Engage
- 3:38Help Devs Do Their Jobs Better
- 5:33Show You Understand Devs’ Problems
- 7:18Use Non-Dev Messaging
- 13:23Move Beyond Hello World
- 16:41Measure Success
- 18:19Pinpoint Your Solution
- 20:20Talk to Your Users
- 22:16Allow Devs to Provide Feedback in Your Docs
- 25:05Docs are Only One Piece of DevEx
- 28:48For Great DevEx Focus on Getting Started Items
- 32:00Communicate Over All Channels
- 35:14Identify Usage Before Deprecating
- 38:33Streamline Your Guide
- 41:15Spark Dev Creativity Through Use Cases
Derric Gilling (Moesif): Welcome to Episode 10 of Moesif’s APIs over IPAs podcast network. I’m Derek Gilling your host today and the CEO of Moesif, the API analytics platform.
Joining me is Adam DuVander. After being a technical journalist for Wired and editor of Programmable Web, he brought a very interesting perspective to the provider side. He ran developer communications and marketing at Sendgrid and also Zapier. Funnily enough, Moesif itself is a Zapier customer, so love using that product. He also helps out with a lot of different companies and startups working on developer marketing and all things developer platforms. Happy to have you here today, Adam.
Adam DuVander: Yeah. Thanks for having me.
Tell Stories To Engage
As an accidental marketer, I discovered that the best way to engage a technical audience in your API was to tell a story on why what you have matters
Derric: Well, so just getting into things, would love to hear a little bit about your story, starting off with writing for Programmable Web to how you got into developer-first businesses.
Adam: Well, so firstly I’m a developer. I have a computer science degree and worked professionally as a Dev, so I had that background. But really all along I’ve been interested in teaching and sort of that “A-ha” moment that someone gets to when something really clicks for them. I was really doing that all along. The writing for Wired started as tech tutorials for Webmonkey. I was a Dev and thought “Hey, more Devs should know about this cool new technology or tool that I’m using”. Then at some point I realized that that was the bit I wanted to do.
I wrote some more for Wired and Webmonkey and then actually my last piece was “Mashups are dead, the web is alive”. This would have been in the late 2000s decade, really where everything was a mash up and I was saying, the future is that all of these APIs are just what the web will be. Of course mobile was pretty nascent. I’m pretty sure that programmable web found me from that post and it was sort of perfect timing. I was cocooning writing my first book on mapping APIs and so that was how I started writing for Programmable Web and then eventually joined full time as the first editor.
I built a team that found new APIs and reported on APIs. We followed all those trends. Really, part of the story, for me, of developer marketing too, is that I would receive all these PR pitches from companies with new APIs and they were so excited about their API, but weren’t really telling the story of why what they had mattered. And so when I left programmable web I realized there was this opportunity to help tell those stories. And so I call myself an accidental marketer, because really I went back to that sort of teaching and storytelling part that I was doing before just for fun as a Dev, because I realized that’s actually the best way for companies to engage a technical audience. So that’s how I made that transition, or those couple of transitions, from Dev to journalist to technical marketer.
Help Devs Do Their Job Better
Dev’s view traditional marketing skeptically — something that pulls them in another direction and wastes their time. Instead, give them something that’s going to help them do their job better and communicate like a dev.
Derric: Oh definitely. Love hearing that story, especially since it’s super hard to engage with a technical audience. What makes it so hard? Why is it so much harder than just traditional marketing?
Adam: You know, you could say that Devs have been burnt by tricky sales tactics before. I think that’s probably true. They have had their time wasted. But really the biggest thing it comes down to is that it’s a developer doing their job, and part of their job is to be skeptical and think about all of the edge cases that you might have. So a lot of sort of traditional marketing they see as gotchas that are going to keep them from reaching their goal, which is to build some great software and not get their time wasted, get pulled in other directions.
So I think that a lot of times marketing that is not oriented toward developers can kind of feel like it raises that skepticism and feels like “Oh, this is the stuff that’s wasted my time before, that has pulled me in other directions”. And so they aren’t drawn to it.
Contrast that with something that is going to help them do their job better and really comes from that place where they can tell that the one sending that message either has a similar background, or understands where they’re at. That makes all the difference in building that trust.
Show You Understand Devs’ Problems
Build trust with devs by showing that you’re familiar with their problems: “We know our stuff and here’s what we’ve seen to be successful”
Derric: Well that’s a really good point thinking about trust and turning that developer, that skeptic into something that’s more of an evangelist. Is there anything that a company can be doing to build that trust weather it’s around the product, marketing or messaging.
Adam: So I think companies who are building tools for developers often are started by developers, for sure have an understanding of those problems, those real developer problems that they are solving. So I think the more that you can share that you understand that, that alone will build trust, because you’re able to say “Look, we were there. We we get it. That’s why this exists.” I mean, companies are started on sort of those opinions, those beliefs, that something is harder than it should be, or the way that other companies are doing this is not the way that they think it should be done. So the more you can kind of share those points of view, you can get developers on board.
So part of it is sharing that you understand those problems, but then also being able to show that you know how to solve those problems too is another key piece. That’s why I keep coming back to education, because a developer can’t know everything about everything, as much as they might try, and so, if you’re able to show them “We know our stuff and here’s what we’ve seen to be successful”, you’ll gain a lot of trust and potentially customers.
Use Non-Dev Messaging
Your product might be used by non-devs: product, marketing, support or customer success, so use non-technical marketing and provide tools that a non-coder could use, such as a visual builder
Derric: That’s a really good point to think about educating the market and also it’s better to show than just tell, developers want to see how something works and what makes them do their job better. With that said, there’s a lot of companies out there, Zapier included, that have two different audiences: the developer audience and also you’ve got the business folks that are coming in and trying to figure out what is an API or how do even call it an API. Can you walk me through some of the challenges there and how do you actually speak to both audiences at the same time?
Adam: At Zapier there might have even been more audiences than that if you consider that there’s sort of the user side of Zapier. So, those actually building these automations for your own tools, the ones who pay for Zapier, there’s that audience. And that audience definitely had devs, I still have a paid Zapier account that I use quite a bit and I could code up some of the stuff that I do instead in Zaps, but for the most part that audience is not technical and doesn’t know how to make those connections between their tools. They might not even know that APIs are the stuff that’s making it happen.
Then you have the platform side, which is where I did most of my work at Zapier. So with the platform side you have a SaaS tool that has an API, you want to connect it to Zapier, integrate it there, to enable all of your users to be able to make those automations. And, my title had developer marketing in it at Zapier, but it took a bit of time there to realize that even that audience was less developery than we might have guessed. There were definitely developers who were building some of those integrations with the Zapier platform, but a lot of those folks were from product or from marketing in some cases, support, customer success, anyone in the company who cared about being able to say “Yes” to more of their customers, “Yes, we do that. Yes, we integrate with that”, was looking to come to Zapier for the platform side, to be able to enable their customers that way. And so there for sure, it was a lot less of the developer message.
There were certainly some API best practices, that I tried to share, that spoke more to that technical audience. And sometimes they did kind of go between the business and developer sides, but a lot of the messages were really about wanting to get your product in front of more of your users and expand your product’s ability to do the things that the users want it to do. So, I gave a talk actually about this, I have a post about it, that is “Your portal’s invisible audience”, and I really do believe that more APIs then you realize have an audience that is not what you and I might call a developer, not someone who’s going to write code all day. I think going through ways to be able to have them raise their hands, be able to discover that they’re there.
Easily Visualize & Fix API Issues with MoesifLearn More
In that post I gave an example of how we launched at Zapier a CLI for the platform, based on research from some of the top integrators who wanted a more developer-like experience. But, we realized over several months and changes to the developer platform that they were a minority amongst those who were starting out. That really people wanted a visual builder. So we were able to determine that based on watching the things that they cared about. And slowly making changes to be able to make that case that “Oh, we’ve invested in this really developer-heavy platform and that’s not going away, but we need another interface that speaks to the larger group that’s here”.
As much as I love developer audiences I think that there’s room for that in any API, especially in a developer-first sort of infrastructure type of API that maybe doesn’t support a SaaS tool. The differentiator I make there is that developer enabled is often SaaS, where you need a developer to help you use the API, as opposed to something like Twilio or Stripe where it takes a piece of your stack, and that becomes a more developer-first, developer-focused company. But I think for all of those there’s definitely room for that less technical audience if you’re able to look for that audience. But then at some level you also need to have all those things that a developer looks for. So it does put you down a road of looking in two places, so if you’re early enough you might be looking for the ones who are ready to more fully integrate.
Move Beyond Hello World
Determine that your API is successful by monitoring if it’s being used to make calls
Derric: It sounds like you did a lot of measurement to figure out how much are they getting activated with Zapier and what are they doing with the platform. Was there any type of feedback loop back into onboarding or other type of communication they get, and how did it differ depending on if they’re a developer or not a developer audience?
Adam: A lot of that was with the platform product team which I did work pretty closely with, but they drove that part. That would absolutely be looking at “do they create an app” (an app being the integration to the Zapier platform) and then how far did they get through that? In that case there are triggers and actions, so you can look and see if they’ve created some triggers and actions to enable their own users. And then, have those been used? Has a true call gone through against that? That was all available in log data and, at least at that point I’m sure they still collect quite a bit, but it was all in a system, that we could as employees, really dig into and put together the case for why we’re being more or less successful. And again a lot of that was with the product team.
And then, of course, you might imagine that on the user side there’s a lot of that as well about activation. And I think, even though that’s the user side there’s definitely an API lesson to be had for that, because Zapier, when it works, you don’t have to come back. And an API is sort of the same thing, right? If someone’s returning to your developer portal everyday there might actually be a problem and they haven’t reached the full integration. It’s similar in that way for the user of Zapier. So there it’s looking for the proof in the logs and Zapier even charges that way. So, has someone had a successful run of this integration, this automation that they set up, and how many of those have they had and are they continuing to have those over time? That’s the stuff that you can do also, as you’re looking at “Is someone using my API?”, again looking for those signs that it’s gone beyond that “Hello World”, they have had an initial success and they have moved the code to their machine or to their servers and have calls that are coming regularly enough that you at least know that they’re still testing things out.
Derric: That’s a really good point, especially since you’re not logging into a portal every day. If you’re integrating Stripe why should I? But then that also brings up a different challenge which is how do you actually measure that and create KPIs when you don’t want people to log into your portal? Does that mean the KPIs are different? How do you leverage that API data that you were mentioning earlier at Zapier?”
Adam: I think being able to recognize what are the signs of someone who has had success and then making sure that you can measure that. I mean this is great for Moesif, there’s a lot that’s within API logs that can tell a lot of that story for sure. And so I think looking for recent calls and then calls that a certain threshold, calls to particular endpoints with hopefully success. So if you’re seeing a lot of 401s/403s then someone’s still stuck in authentication and, certainly from my perspective as a content person, I immediately think “What’s in that getting started guide”, do we talk enough about what it takes to get authenticated, or do we just sort of hand wave through that. So that would be the sign that I would need to look there if you’re seeing that type of error.
Pinpoint Your Solution
The more specific your platform’s use case, the more success you’ll have with getting the right devs to integrate
I think you can see similar things that are going to be specific to your API that you know what it looks like for someone to have success connecting. Even if you’re brand new you can at least figure out what that would look like - to have API calls coming through and then making sure you’re seeing that and looking at it on a per user basis. It’s going to be different for different APIs. I was developer relations at a database API and lots of people came in and kicked the tires once and moved on because in a database you’re not going to rip out your whole database and replace it, you have to be in the right spot, in the right moment. We were an early startup, so we were casting that net probably a little wider than one might when they’re further along and kind of know the use cases that someone would use their API for. But, I would expect any database type of API to have fewer people making it through to success than something that’s much more pinpointed of a solution. I really do think that Twilio and Stripe, some of the things they have going for them, is that they have very specific use cases.
Talk To Your Users
To truly understand your platform user hop on a call or get an email exchange going to get qualitative data about where someone might be struggling
Derric: Definitely, line of business basically, it’s very pointed transactions that you’re providing or business functionality. You spoke of creating more content around how authentication works, when you identified that there might have been some struggles. Were there other ways that maybe you were able to change your marketing strategy based off of looking at activations or looking at how people adopted your platform?
Adam: One thing I didn’t mention is actually talking to people. So I know we got right into the sort of the analytics, but another piece was when people would sign up, this was at Zapier but also at Orchestrate which was a database as a service. It was: if someone’s just getting started, how can we hop on a call or get an email exchange going back and forth to get more qualitative data about where someone might be struggling. A lot of that is with the database as a service for sure, how can we get them to that “Hello World”, but at Zapier it was more of understanding what that audience was. A lot of it was internal communication. Many of the people listening here probably use APIs that exists within a company so there’s a lot of internal conversation that has to happen and helping other people see what you have learned. And so that was a big part of the project, to understand that Zapier platform user as well, plotting them on a developer background access along with willingness to write code and being able to make that case that there were a lot more folks who didn’t have that background and weren’t ready to spin up the CLI on their terminal.
Moesif: Ship Faster and with More ConfidenceLearn More
Allow Feedback in Docs
The capability for users to suggest edits in your docs presents another important way to communicate with your devs. When they make a suggestion, be sure to reply and fix the problem.
Derric: Definitely a good point on thinking about both qualitative and quantitative feedback. I know it’s so easy to get very buried in the analytics piece without talking with developers, they’re humans after all. But sometimes they’re not responsive to email or they want to be left alone. Were there any best practices that you found that you can share on the best ways to reach out to developers and have that dialogue go?
Adam: Yea, I think being open to multiple ways of communication is important. So we’ve mentioned a couple already here, which is reply to the email: on that list of things that developers get skeptical about is the “I just want to have a few minutes of your time on a video call”. Some are ready to make that leap, but not everyone, so being willing to have email as another one. Then I think something that everyone can do is that within documentation placing opportunities for feedback. Algolia I know has a sort of exclamation point on several places in every page, where when you click it you have an opportunity to fill in ”this isn’t working” sort of thing, at a minimum. At Zapier we had kind of a box at the bottom of every page. Many now have the kind of “edit this”, you can go and fork it in GitHub and put in the edit you think should go here. All of those different levels are a way to be able to get some kind of feedback. I think that the important piece there though, is that you have to actually look at it and then do something about it. At Zapier we won’t be surprised that we had had a Zap that would pipe that into a specific slack channel that all of the platform team was in. Having some way to be able to then reply if that’s attached to a particular user, but at least be able to take action, especially when someone notices a problem because that’s the “we’ve all been there in the documentation that is wrong place” and it’s just developer after developer who you’re upsetting. I mean that’s the quick path to keeping those developers away, is to continue to have inaccuracies in your docs.
Docs Are Only One Piece of DevEx
DevEx includes all interactions someone has with your company and products including docs, so look ofr ways to show up positively as a benefit to the developer experience
Derric: I laughed pretty hard because I know that keeping those docs up to date is always a challenge. In fact, here at Moesif we leverage an edit at GutHub button and our docs are all open source. We use Jekyll with an open source static generator, a great product. But, who actually owns docs in this case? Does that mean it’s a product team, is it marketing? Is there any good process to keep those up to date, because we’re always seeing that as a challenge with developer platforms?
Adam: As for who owns it, I often find myself talking to marketing teams and reminding them that they do need to care about the documentation. And many marketing teams do, but very few I would say own it. I think probably most often it is owned in product, but we also know that not every product team is going to see that as the core piece of the product. Larger teams will have documentation teams dedicated to that or individuals, but that of course is not always the case either.
I think if you are listening to this, then then you’re probably a good person to champion it within your own company, that at least recognizing that docs spread between multiple teams that might care about it and kind of at the very minimum, putting together an ad hoc group that cares about it. We definitely did that and had multiple calls at Zapier over that time frame of people from support, product, engineering and marketing who had a reason to want more people to have success with the docs.
Documentation also is just one piece of that overall developer experience and certainly the product side of the onboarding is a piece of that. I see a lot of the developer content as related to developer experience as well. Because even if it’s the first time that they’ve arrived at a piece of content, they found you because you’re writing about the developer problem that they’re experiencing. That initial experience is an experience and that is part of the whole journey that someone hopefully will go on with your API. It certainly starts before that, it starts with developers chatting in Discord rooms, about which APIs are easier or frustrating them at that moment. That sort of word of mouth is happening too. If you go really broad with what is developer experience, and I tend to go fairly broad in that definition of all interactions someone has with your company and with your products, I’m looking for ways to show up positively is a benefit to the developer experience, when they actually do get to the point of using your API.
Focus On Getting Started Items
To ensure a dev has a good experience focus on providing excellent getting started types of things, such as SDKs in popular languages and a complete reference thats up to date
Derric: I’m glad you just defined that. I was just about to ask what is developer experience and how you define it. With that said, it’s such an abstract thing, is there a developer experience team, or is that more of a mindset? How do you actually make sure you have a good developer experience and is there a way to measure it or is it not really measurable?
Adam: So, in terms of a team, I think that depends on how core the API is to the company. So, if the API is the primary product of a company, and more and more we’re seeing a lot of these, in that case developer experience should probably matter to the whole company on some level. In which case I’m not sure that a team makes sense. I’ve definitely seen developer experience teams. I think most often those teams are either a product team that is in charge of onboarding/dashboard experience or is a documentation team with a different name. I’ve seen those two. I’m sure that some companies may even lump those two together, though I think I’ve seen it more often in separate groups. So, I don’t know whether there needs to be a team with that name or not, but definitely there needs to be a team or multiple teams who are wanting to improve that developer experience, regardless of whether the name is on the group.
And so, certainly being able to measure something, we talked a little bit about some of the things you’d want to see to be able to say this is developer success. From a high level I have 13 criteria I look for to be able to say whether someone is in a place that would give a developer a good experience. Certainly there’s more to it, whether they’re actually successful. But things to look for include: do you have SDKs in popular languages, do you have a complete reference that is up to date, and then a lot of the sort of getting started type of things that I look for there. And so I do measure it, but it’s really meant to be high level, and I think the more granular measurement really gets down to the usage that we’ve already talked about, because that’s really where it answers whether it’s a good developer experience or not - “Did the developers have success?”. Did they get from finding you to not only that “Hello World”, but something that’s actually useful and pushes them forward towards solving their problem.
Communication Over All Channels
When changing an API use all channels to communicate the change including: email, dashboard notifications, sunset headers and even flicking the lights on and off
Derric: And then what’s next after they’re fully up and running? Sounds like they’re not kicking the tires anymore. Any good practices when it comes to developer communication?
Adam: Certainly if things are changing, all of your docs should be up to date and wonderful to get to developer success. But then if your API is changing a bunch and breaking things for the developer, even if you’re updating your docs every single time that happens, every time you make a change it’s always up to date, but what they built on before, if that’s breaking, then that’s certainly going to create a worse developer experience. So that sort of product change is something that has to be happening continually, but that means that you need to have those open lines of communication. That’s tough when a skeptical dev might sign up with a different email address that they don’t check very often, those sorts of things. But email can be just one of many ways to be able to get a hold of them. Certainly if someone’s logging into a dashboard you have that as a way to get a hold of them. We, even at Zapier, Ben Peter who’s the API master, would always be the one in charge of figuring out when there were deprecations or problems across what is now 3,000 APIs. I sure hope Ben has some help now, not just Ben working on it. But he wrote about some sort of deprecation best practices, even right down to putting some stuff in the header.
I know that Eric Wilde has a sunset header. Going through the steps of trying to communicate, then putting things into the header and even so much as doing a kind of flick the lights on and off kind of test, where you give them a chance to catch it actually going down, potentially breaking something, but then put it back up so that it’s not a complete firefight on their side. Ben wrote a couple of posts about those best practices. But certainly if you can start out communicating with a dev and set that expectation that you’re going to send helpful things via email and posting on your blog, then they’ll be more likely to want to have that as an ongoing communication channel. Which would then mean that you’d be more likely to be able to reach them when you do have to make changes like that.
Identify Usage Before Deprecating
If you have to deprecate, then analyze usage and success using Moesif to understand what would be impacted even down to a per-call basis
Derric: Yeah funnily enough, we just came across a few of those blog posts. We actually implement a lot of that functionality at Moesif itself, flicking the lights on and off or a deprecation header. How do you actually identify what to deprecate and is this something that you write about or how do you actually decide on this?
Adam: So from the complete ideal developer standpoint, you would never deprecate anything. I think if you look out there, Twilio might not have ever deprecated anything. I think they do a versioning based on the date, at least the last I had looked. So if you integrated at a particular time you have this endpoint and you can always upgrade to the latest, but they keep it going. That’s definitely very developer friendly, but you and I know that’s not always possible. There might be a system that you want to take down that is expensive in some way, in maintenance or in actual server costs, and so there’s a lot of reasons why you might want to make that choice. But definitely erring on the side of not updating is good. I think it’s the typical sort of product decisions where it’s “how much is this used?”. Understanding who would be impacted, and even understanding who would be impacted on a per-call basis is actually possible if you really have access to your logs. You know what endpoints people are calling. You know what data they’re receiving. So if you’re going to change the shape of data, you do have ways to be able to get at that if you have set things up to be able to access that.
Derric: You’ve just got to make sure it’s set up ahead of time right.
Adam: I think OpenAPIs as a data format for APIs is one way that we have now to at least have some expectations around what is within an API, in a way to talk about it and even programmatically test against. There’s a company called the Optic that will actually look against your live calls and do some of that stuff that I mentioned of sort of “hey the shape of this API call changed or the shape is going to change in your next version and here are the users that that impacts”. And then certainly being able to have a platform like Moesif to be able to look at endpoints that people call and be able to get that sort of analytics on usage and success would be another thing to help.
Streamline Your Guide
Have only a single Getting Started Guide and make it streamlined, so that it clearly shows how to make a call
Derric: Definitely. My last topic is really around startups who are just starting to think about a developer platform. Any best practices that you able to share for someone just starting out in the developer marketing space.
Adam: Well, if you’re just starting out, for sure to have those conversations that we talked about. We can maybe assume that someone is having those: talking to developers about what they actually want. Then I think kind of going and looking for the basics to get someone to “Hello World”, because as much as it’s important what happens after that, they won’t ever get to that if they don’t get to that “Hello World”. So, is there a Getting Started Guide that you can make sure really focuses on that. Some of the problems I see with Getting Started Guides is often someone wants to share all of the context you would ever need to know about using your product and that’s the wrong place for it. If you have to study up on concepts, before you can even get started, that’s a non starter for most developers. Most developers will skip that part and get to the part where I actually make a call. And so helping someone to be able to get where they want to go, so streamline that process in the Getting Started Guide. You can always layer in a little bit of concept as it’s needed and then you always can link out to the other spots where you cover that more in depth. So that’s definitely a problem I see after the bigger problem of not having that sort of content at all to get started.
Sometimes there will be multiple Getting Started Guides, so that’s also confusing. Being able to have that single place to send them through. And then certainly looking for ways to make sure that your docs will stay up to date. I mentioned OpenAPI. Can you generate from your OpenAPI at least your API reference, so that you know that that is up to date. It requires that you also keep your OpenAPI document up to date, but there must be some way to be able to do that. So that way you’re putting yourself forward with at least the basics of documentation that someone expects.
Spark Creativity Thru Use Cases
Showing devs what you can do though a use case will often spark their creativity
And then from a startup perspective you’re going to want as quickly as you can to understand how someone wants to use your product. Hopefully, you have some ideas around that already, put those out there in the form of use cases and sample apps. Validate that those are the problems that someone wants to solve. I’m seeing this less but within the last week I’ve had a conversation about this, where it’s “we’re this platform that can do anything and we wouldn’t want to hold back developer creativity by telling them what we do”, and I think that telling them what you can do is the spark that starts that creativity. So, being able to give some of those use cases is what will make someone realize “Oh well, this is sort of like this other thing that they didn’t have in mind”, then they will be creative. But just providing something and saying “do something with this”, doesn’t inspire that creativity in the way that people think it does.
Derric: That’s a really good lesson, which is it’s really good to show than tell — show the different use cases and provide that very concise and specific onboarding or getting started guide. But don’t have 10 of them, just have one. Cool, well, thank you very much for having us here Adam. It was a pleasure talking about developing marketing and developer experience.
Adam: Thanks for having me.