Canaries are the Keys to Success in the Cloud

by | May 2, 2024

tl;dr – By scattering tripwires throughout your environment in canary tokens you can be alerted to attackers in your environment before any other tool detects something is wrong.

The adoption of Cloud and SaaS services in modern architectures makes it necessary to manage various programmatic credentials, typically in the form of API keys or tokens. The days of UI-driven infrastructure and provisioning are replaced with technologies like Terraform leveraging dozens of credentials in one workflow. For organizations with hybrid or cloud-native environments, this can mean managing thousands of keys across dozens of different teams. By design, these keys have privileged access to cloud resources, but securely managing them and detecting when they are abused is difficult.

Further complicating the situation is the usage of keys across development, engineering, IT operations, and DevOps teams–each with different use cases. Since each team has a unique way of utilizing the keys, mature organizations require appropriate privilege access management to control how these keys are deployed.

What happens when the best-laid plans of mice and access management encounter a developer on a tight deadline? Perhaps this developer hard-codes keys directly in the source code. That code is then checked into a repo and an engineer troubleshooting a service creates a curl command with a key passed in the URL. That pass key ends up in a log, just waiting for a threat actor to pounce. These real-world situations happen every day and they can create a rich environment for an attacker who has compromised your perimeter to start to gain broader access to the data in your environment.

How can you protect your access keys?

The problem of keys being checked into public code repositories is well-known and quickly exploited. AWS keys checked into a public repo are rapidly found and exploited in seconds. This has become such a well-understood attack vector that cloud providers have built scanners to detect the keys and disable them for customers typically before attackers can exploit them. This can be a major help to organizations and open-source projects working publicly, but what about private repositories?

Private repositories can present a more dangerous situation because engineers can feel less concerned about security practices when the code base is only exposed internally. They may be more willing to take shortcuts and directly expose the keys instead of using a security vault. Attackers know this and when they first get into an environment this becomes one of the first priorities for reconnaissance. They know that if they can find access keys, they can find gold in the environment.

The recent breach of Sisense and exfiltration of all their customer data came through this attack vector. Attackers breached the Identity of a Sisense employee in their Gitlab repositories and scanned the repositories for access keys. They were able to locate and utilize several AWS S3 access keys to gain access to all the sensitive data for Sisense.

How do you generate canary tokens?

First priority is to mimic the scanning approach that AWS uses to scan for keys in public repos, you should be using to scan internally against repositories, file shares, documentation, and scripts. The next step is to use a bit of Judo to leverage this very popular reconnaissance method to plant some tripwires for attackers in your environment. These trip-wires come in the form of canary tokens.

A canary token is a type of honeypot method where you deploy a resource in the environment that looks to be a real production access key or file, but instead is designed to alert you the moment it is used. This type of detection can help teams become notified of a breach well before any of their existing alarms detect an attacker or any data is exfiltrated. In the recent Change Healthcare breach, attackers were in the environment for 9 days doing reconnaissance and extracting data. By strategically deploying canary tokens, you can trap attackers in situations like Change Healthcare. 

How do I get some Canaries?

Thinkst Canaries provides an excellent free service for the community called Canary Tokens that allows security teams to generate canaries of various types. They can come in the form of AWS secrets, Azure Entra login, documents, website URLs, and more.

These resources can be scattered strategically across an environment acting as a trap for attackers to fall right into. Commit an AWS key to a test repository, drop a file called passwords.xlsx in an admin file share, and place a MySQL dump file into one of your production backups. The opportunities to get creative with these honeypots are endless and the more tempting you can make the resource look, the faster you will get alerted when a resource is accessed.

Identity is the undisputed perimeter of modern environments and the fragmentation of programmatic keys to these resources makes a hard job even harder. A layered approach of internal scanning, privileged access management, and access certifications can make it a more secure one but once that first layer of your perimeter is breached, you need to know as soon as possible. Canaries can be that detection mechanism to help alert your teams even before your SOC notices anything has gone wrong.

Author

  • Adam St. Onge

    Adam St. Onge is the Regional Director of Sales for Zilla Security, leading a team of sales professionals. He is passionate about solving complex problems and driving results that help internal and external teams reach their goals.

    Before joining Zilla Security, Adam held technical and leadership positions in organizations ranging from Fortune 500 to disruptive start-ups. These roles included responsibilities for technical architecture, engineering, IT operations, and technical sales.

    Adam holds a Bachelor's degree in Computer Science from Mount Saint Mary College and a Master's degree in Networking from Rochester Institute of Technology.

    Connect with Adam via LinkedIn.

    View all posts

Recent Posts