Smallstep focuses predominantly on automated certificate management for internal systems in order to automate certificate administration and secure infrastructures. This is an area that organizations are still grappling to get a hold of, and the move toward microservices has exacerbated this further. Using TLS to secure the communication between your microservices can require hundreds of thousands of certificates, and if organizations are not implementing best practices around certificate management, this can be challenging.

In this episode of TFiR Let’s Talk, Swapnil Bhartiya sits down with Mike Malone, Founder and CEO of Smallstep Labs, to discuss some of the challenges organizations are facing with certificate management and some of the common mistakes that are being made. Malone goes into their recent announcement of the general availability of their Certificate Manager toolkit and how it is helping organizations simplify the process for developers.

Key highlights from this video interview are:

• Securing large software systems continues to be problematic. Malone feels that there has been a maturation of processes and techniques over the past 10 years or so. He discusses how distributed systems have evolved as we move more towards cloud-native environments.
• Smallstep focuses on automated certificate management for internal systems. The company focuses on authentication. Malone discusses how cryptographic authentication can be used instead of IP addresses in order to secure cross cloud communication.
• Although X.509 certificates have been around for a while, they have not been applied on such a large scale before. However, using TLS to secure all the communication between your microservices can require hundreds of thousands of certificates. Malone discusses how automation is being used to solve these problems.
• Malone describes the main mistakes he sees developers make, such as certificate management and the need to build best practices around generating keys. He explains how Smallstep tools help simplify this process.
• Smallstep’s core technology is open source with a large community built around it. The company offers professional support services as well as lots of documentation and tutorials around best practices and implementations. Malone explains one of the key use cases they have, ACME-based certificate management.
• Malone shares his top tips for how organizations can improve their certificate management posture. He discusses the open source Certificate Management toolchain and what it offers. He also goes into detail about the hosted instance of their open source Certificate Management toolchain that adds in enterprise features, which is now GA.

Connect with Mike Malone (LinkedIn, Twitter)

Learn more about Smallstep (LinkedIn, Twitter)

The summary of the show is written by Emily Nicholls.

[expander_maker]

Here is the automated and unedited transcript of the recording. Please note that the transcript has not been edited or reviewed. 

Swapnil Bhartiya: Hi, this is your host Swapnil Bhartiya and welcome to another episode of TFiR: Let’s Talk. And today we have with us, Mike Malone, founder and CEO of Smallstep Labs. Mike, it’s great to have you on the show.

Mike Malone: Great to be here. Thanks for having me.

Swapnil Bhartiya: Yeah, it’s my pleasure, since it’s the first time we are talking to each other. I would love to know a bit about the company since you’re also a founder. So tell me a bit about what’s a specific problem area that you saw that you want to solve, which kind of led to creation of a new company.

Mike Malone: I’m a software engineer and I like to say my happy place is distributed systems architecture. I like building large software systems and building teams that build large software systems. And I’ve been lucky in that I’ve been able to do that a number of times in my career and the impetus for the business was that in those systems, security is really an unsolved problem. There are a lot of pain points in securing large software systems and that’s what we’re building to solve.

Swapnil Bhartiya: The company is a couple of years old. You’re talking around 2016. That’s when the company was created if I’m not wrong?

Mike Malone: That’s right.

Swapnil Bhartiya: Right. So the thing is that in today’s modern world, things are moving so fast. The way we talk about the Agile development model. The technologies that we are creating are also so new. So a lot of things have happened since you created the company. We are moving more and more towards a Cloud Native environment. So can you also talk about how you have seen the whole evolution of distributed systems themselves? Because that has also changed the way we look at security and a lot of other things.

Mike Malone: Well, I’d say that in the time that we’ve been around as a company… Well Kubernetes is obviously a big difference, right? Containerization. But I’d say it’s been a maturation of processes and techniques that have been in use at least in the circles that I’ve been running in for probably 10 years or so. Microservice, Agile development, DevOps, CICD, I think are really proliferating. And that above all else, I think is what we’ve been seeing.

Swapnil Bhartiya: Oh. When we talk about security, first of all, it’s no longer just one thing. Especially if you look at the Cloud Native space. Can you also talk about what are the specific areas that you focus on? Because the landscape is so large that one company cannot handle everything there.

Mike Malone: What we specialize in is automated certificate management for internal systems. For server to server, service to service, or connecting to databases, network communication, issuing the credentials, and managing identities for those interprocess communications.

Swapnil Bhartiya: So is it within the organization or because.. No once again, really within a multi-cloud-verse. So sometimes things are spread like a cloud when we do talk about [inaudible 00:03:11] there are a couple of things you talk about – identity management, access control, there’s not just one thing. From that also authorization is one authentication is one. Talk about the scope of a smaller step.

Mike Malone: Well, we focus on the authentication piece, at least for now, who knows what’s in the future, which is foundational. You can’t authorize until you’ve authenticated. You mentioned multi-cloud and hybrid, these infrastructures are getting increasingly complicated and heterogeneous. And one nice thing about cryptographic authentication, as opposed to maybe older school sort of techniques using IP addresses and MAC addresses is the network really becomes irrelevant at least from a theoretical perspective. So cross cloud communication is just as secure and works exactly the same way as communication within the cloud when you’re using cryptographic techniques.

Swapnil Bhartiya: One more thing is once again, automation, that is the key in modern and cloud work. When we do look at the certificate manager, this is not a new problem, the problem has been solved. So just talk, what value are you bringing through automation? Because, once again, when we do look at security, these are not the bugs that we have to worry about. These are the human errors that we have to worry about.

Mike Malone: Right. X.509 certificates, which is what we deal in primarily, we also do SSH certificates, but X.509 certificates are some of the oldest technologies out there, security technologies in fact, predating the web X.509 is an ITU-T standard from the 80’s that was being developed as a telco standard. So certainly the technology has been around for a while, but it hasn’t been applied at the scale that we’re seeing it being applied at now. And, and really it’s an order of magnitude or maybe multiple orders of magnitude scale change. So it’s really a different kind of problem that’s being solved. And if you look at a lot of the older technologies, it’s really about manual workflows to obtain a certificate using something like OpenSSL and manually deploy it to a piece of infrastructure.

And that works well enough if you’re talking about two or three or maybe a dozen certificates for your website or, or whatever. But if you’re trying to use TLS to secure all of the communication between all of your microservices, connecting your databases, your Kafka, for all of your Kubernetes components, your operators and admission controllers and stuff like that. You’re talking about potentially thousands in some cases, hundreds of thousands of certificates. And then if you are applying best practices of short lived certificates, they may need to be renewed daily or hourly or you can’t do that manually, obviously. Automation and good compliance governance audit observability around those processes, integration with alerting. These are all of the things that we’re bringing to the table. The certificates have been around for a while, but operationalizing certificate management at scale is a new problem.

Swapnil Bhartiya: Right. Which also kind of leads me to another question, which is like, if we just look at organizations, what are some of the pain points that are there when it does come to, you did touch upon a few there, but if I can just, broadly, hey, these are the pain point that you saw also at the same time. If you can also talk about, you see a lot of mistakes that they make, and you want to kind of take that out of the equation as well.

Mike Malone: There are a lot of sharp edges in PKI and certificate management. Some of the mistakes that we see made, a lot of complex environments end up doing some form of certificate management in an Ad hoc messy way. So when you talk about large organizations that have multiple teams, maybe many clusters multi-cloud, they end up with a lot of shoestring and bubblegum type solutions where they’re stringing stuff together. It doesn’t have a lot of good audit, a lot of good control. They don’t have alerting.

They don’t have like all of those ilities that you want in like a large scale system, just aren’t there. It’s hard to really say anything concrete about the security story. You don’t know where the keys were generated. Who’s seen them, what’s seen them, whether they’ve transited the network. Building and applying best practices here where keys are generated, where they’re being used and they’re short lived, and renewed on a regular cadence with alerting.

If something goes sideways, it’s a big challenge and really hard to build yourself. What we end up seeing are certificates that are issued for a year or multiple years, which again, presents all sorts of governance and compliance issues around what happens if there’s a compromise there. Poorly configured TLS stacks that maybe aren’t even checking revocation status for these multi-year certificates, which really provides very little real security at that point.

Just applying these practices at scale, there’s a litany of sort of detailed, nuanced things that you have to consider. One of the things that we talk about a lot with our tool chain is ease of use, first of all. We were opinionated and make a lot of these decisions and sort of apply best practices for you, make it the right thing. Also the easy thing and also misuse prevention.

So we make you sort of opt in very intentionally to do something that is maybe dangerous, just to pick a specific sort of simple example. If you wanted to generate an RSA key that’s like 1024 bits, there may be some reason for legacy systems that require that key type, but it’s generally not considered cryptographically secure anymore. So we will require that you pass in an insecure flag to our CLI indicating that, you know what you’re doing and you want to do it anyways. This is an area it’s baroque, and it’s an area that requires a lot of specialized knowledge that not a lot of people have even really smart, experienced engineers and operators. We’re very sensitive to that.

Swapnil Bhartiya: Yeah. Since you talked about knowledge, one more thing that happens is that one is tribal knowledge and the second is no technical debt. So when in today’s world, we see that teams folks are moving around. So once you know, somebody move out of your organization, they do take that tribal knowledge with them about that. So by automating you also [inaudible 00:10:08]. So can you also talk about the importance of that? Because this also has become a reality where folks are moving around a security, especially, certificate management that cannot go with the employee.

Mike Malone: Right. Going back to what I was saying about the Ad hoc, shoestring, and bubble gum, that’s a real problem in organizations where someone sort of ties something together that works well enough, but then when they leave, nobody knows how to operate it.

Swapnil Bhartiya: Forget, I’m saying though, I just want to stress that through automation, how are you making sure that whether we call it plumbing, whether you call it patching, whether you call it, call it Frankenstein’s monster that we are trying to build, you’re taking that [inaudible 00:10:49] through your automated, smallest steps, certificate manager. That’s what I want to stress on.

Mike Malone: Yeah. Right. So automation, we’re also open core. So our core technology is open source and we have a large community built around that. We obviously offer professional support services, but there’s community support as well, lots of documentation and tutorials around all of best practices and implementations for various scenarios, and then where they exist, we’re standards based. So one of the use cases that we sell into a lot is ACME based certificate management.

ACME is the protocol that it’s an IETF, standard RFC, and it was developed by Letsencrypt. So it’s a very popular protocol for automated certificate management for the web PKI, for websites. So people are often familiar with it and have tool chains that they’re used to and are already using to manage their web PKI certificates. And we can offer that same capability for internal PKI. I think all of the obvious things are done to address that. We have documented standard solutions in place that are well supported and are implementing industry best practices.

Swapnil Bhartiya: Do you have any, not necessarily sharing your playbook, but some tips for organizations, how they can kind of improve their certificate management posture. And of course your solution is there as well. Just some best practices, basically.

Mike Malone: Short-lived certificate. That’s a big one with automation. I mean, that’s 90% of the problem is getting short-lived certificates with automation and alerting in place everywhere.

Swapnil Bhartiya: Now, can you also talk a bit about the inner working, how it works, what does the product look like from the perspective of users and also it’s an open core. So talk about the balance between Open Source version and the commercial version.

Mike Malone: We are open core and we have an Open Source, complete tool chain for certificate management. We have a lot of large enterprises that have picked that up and used it. And then the product, what we’re bringing to general availability, is building around that open core. So it’s a hosted instance of our Open Source certificate management tool chain that adds a bunch of enterprise features. Governance and compliance functionality, audit, and observability alerting, we integrate with your SIM, your Splunk or your Sumo logic or whatever advanced access controls. And then of course we run it. It’s highly available, it’s secure, and it’s really easy to get started. You know, you click a few buttons in the UI and you’re up and running. So it is really designed to be sort of the easiest, safest, most secure way to just get up and running with certificate management.

Swapnil Bhartiya: Mike, thank you so much for taking time out today and talk to me about, of course, the company itself and the automated certificate manager. So thanks for sharing those insights. And I would love to have you back on the show. Thank you.

Mike Malone: Thanks. It’s been fun.

[/expander_maker]