The Importance of Code Signing in the Software Supply Chain

How do you know that your code can be trusted?

In a world where trust is hard to come by, it’s an important question to ask ourselves. How do we know that the applications we’re running, the containers we’re deploying, or code we’re delivering to our customers is genuine? How do we know it hasn’t been tampered with?

It all comes down to code signing.

In this blog, we’ll discuss the importance of code signing, the challenges of implementing it properly, and how Keyfactor Code Assure allows you to centralize and secure code signing without disrupting developer workflows (you can find the full webinar on this topic by clicking “Watch Now” below).

Today, software isn’t just the storefront. It’s the operational engine behind every business. And to keep that engine running like a well-oiled machine, we need to know that everything we build and everything we deploy is trusted.

At the heart of trust in the software supply chain is code signing. Code signing proves the identity of the source vendor of code and verifies that the code hasn’t been tampered with since it was published.

We’re quickly moving toward a reality where everything needs to be signed. Not just the software we buy from third-party vendors, but just as importantly, the software we build and deploy within our own organizations. That means everything from PowerShell scripts, Bash scripts, containers, libraries, files, and executables (you name it).

But code signing has been around for years. So, what’s changed?

Application and operations teams are moving faster than ever, thanks to the adoption of CI/CD and build and test automation tools. That means fewer human eyes with a direct line of sight into what’s happening throughout the pipeline.

Bad actors from advanced persistent threat (APT) groups to basement-dwelling hackers look for weaknesses or vulnerabilities in these fast-moving and complex environments. To keep a low profile, malicious code is often injected or modified at virtually any point in the supply chain without detection. Either that or they compromise the code-signing process itself.

An attacker could inject malicious code into open-source software, into your source code repository, or compromise build servers themselves. In the case of SolarWinds, attackers injected just a few lines of benign-looking code into a single DLL file that created a deep and widespread threat across their supply chain.

Adversaries may also acquire or steal code-signing keys found on vulnerable systems to sign their malware tools. We’ve seen this happen in a number of different scenarios, where code-signing private keys are compromised from a build server, a developer workstation, or even accidentally exposed within firmware or software itself.

So from a security perspective, it’s not enough to simply sign code anymore. Now, it’s just as important to: (1) make sure that what developers are building is what’s getting signed and delivered to your customers, and (2) that the infrastructure and the private keys used to sign code are tightly controlled.

Attackers have shifted left. Rather than target organizations directly, they’re looking for weak links in the software supply chain where they can steal tools or compromise code. State-backed APT 41 has mastered this technique for over a decade, indirectly targeting a wide span of companies by compromising their software vendors or suppliers.

Code-signing is a natural target for attackers, because it allows them to slipstream into the supply chain and breach targets from the inside. Here’s how that can happen:

• Key theft or misuse: Adversaries steal code-signing materials via unprotected systems or misconfigured account access controls.
• System compromise: Attackers can also compromise centralized signing or update servers to hijack a software update in the delivery process.
• Code compromise: Malicious code is injected into open-source libraries or source code repositories, and remains undetected before it is signed and pushed to end-users.
• Insider threats: Developers within the organization may intentionally sign malicious code or even unintentionally disclose signing keys in a publicly accessible software update or repo.

Despite these well-known threats, the reality is that most organizations we talk to today aren’t signing as much as they should, and more importantly, they typically don’t have a consistent code signing process.

In a recent study, we found that companies have an average of 25 code-signing certificates. About 51% of respondents said they store the associated private keys in a hardware security module (HSM). On the flip side, many said that private keys are also found on build servers (33%) and developer workstations (19%), and nearly two-thirds (60%) of companies have no formal access controls and approval processes for code-signing keys.

The problem is that code-signing is often loosely coupled with software development, rather than tightly integrated. The two processes are disjointed, which creates friction and delays in software delivery, or worse, it opens up the possibility for bad actors to take advantage. Here are some common barriers:

• Remote teams: Decentralized development is the new norm, which introduces several challenges. Storing keys locally on a machine isn’t an option, but transferring large files to a centralized signing system results in significant wait times.
• No process: Code-signing keys are well-protected in a HSM, but there’s still a need to control what is signed, who can sign, etc.
• Manual process: Policy guardrails are essential, but processes that are heavily dependent on hardware configuration and manual approval processes cannot scale.
• No integration: Lack of file type support or integrations with existing tools (e.g. SignTool, Authenticode, jarsigner) makes it extremely difficult to maintain centralized control while allowing developers to work transparently.

So the question becomes, how do you make it easy and transparent for developers to sign code while retaining centralized control over what and how they can sign? It’s a question we asked ourselves, but instead of searching for the answer, we built it.

As we shift toward more frequent software updates, remote development teams, and more products, we outgrew our existing signing solution that relied on highly secure, but heavily manual processes. That’s where Keyfactor Code Assure comes in.

To find the right balance between our (and our customers’) requirements for transparency, ease of use, and of course security, we built a solution that could:

• Protect private keys in a centrally controlled HSM (physical or cloud)
• Enable developers to use native tools and signing methods on their local machine, without private keys ever leaving the central HSM
• Control access and segregate duties for different teams and roles
• Enforce policy guardrails for signing based on user/group, location, signing method, and other parameters
• Maintain an audit trail and timestamp of all code signing activities and key usage
• Integrate with build servers and CI/CD tools, most notably Jenkins, to automate code-signing

Let’s take a look at how it works. Here is a quick preview of the solution and how it enables secure code signing for Java and Windows (you can find the full demo here).