The authors thank John Fokker and Marcelo CaroVargas for their contributions and insights.
In our upcoming talk at the Cloud Security Alliance Summit at the RSA Conference, we will focus our attention on the insecurity of cloud deployments. We are interested in whether attackers can use compromised cloud infrastructure as viable backup resources as well as for cryptocurrency mining and other illegitimate uses. The use of containers has increased rapidly, especially when it comes to managing the deployment of applications. Our latest market survey found that 83% of organizations worldwide are actively testing or using containers in production. Applications need authentication for load balancing, managing the network between containers, auto-scaling, etc. One solution (called a cluster manager) for the automated installation and orchestration of containers is Kubernetes.
Some key components in the Kubernetes architecture appear below:
High-level Kubernetes architecture.
- Kubernetes master server: The managing machine oversees one or more nodes
- Node: A client that runs tasks as delegated by the user and Kubernetes master server
- Pod: An application (or part of an application) that runs on a node. The smallest unit that can be scheduled to be deployed. Not intended to live long.
For our article, we need to highlight the etcd storage on the master server. This database stores the configuration data of the cluster and represents the overall state of the cluster at a given time. Kubernetes saves these secrets in Base64 strings; before Version 2.1 there was no authentication in etcd.
With that knowledge, security researcher Giovanni Collazo from Puerto Rico started to query the Shodan database for etcd databases connected to the Internet. He discovered many and by executing a query, some of these databases started to reveal a lot of credentials. Beyond leaking credentials from databases and other accounts, what other scenarios are possible?
There are several ways that we can acquire credentials for cloud services without hacking into panels or services. By “creatively” searching public sites and repositories, we can find plenty of them. For example, when we searched on GitHub, we found more than 380,000 results for certain credentials. Let’s assume that half of them are useful: We would have 190,000 potentially valid credentials. As Collazo did for etcd, one can also use the Shodan search engine to query for other databases. By creating the right query for Django databases, for example, we were able to identify more cloud credentials. Amazon’s security team proactively scans GitHub for AWS credentials and informs their customers if they find credentials.
Regarding Kubernetes: Leaked credentials, complete configurations of the DNS, load balancers, and service accounts offer several possible scenarios. These include exfiltrating data, rerouting traffic, or even creating malicious containers in different nodes (if the service accounts have enough privileges to execute changes in the master server).
Creating malicious containers.
One of the biggest risks concerning leaked credentials is the abuse of your cloud resources for cryptomining. The adversaries can order multiple servers under your account to start cryptomining, enriching their bank accounts while you pay for the computing power “you” ordered.
We have heard a lot about incidents in which companies have not secured their Amazon S3 buckets. A number of tools can scan for “open” buckets and download the content. Attackers would be most interested in write-enabled rights on a bucket. For our Cloud Security Alliance keynote address at RSA, we created a list of Fortune 1000 companies and looked for readable buckets. We discovered quite a few. That is no surprise, but if you combine the read-only buckets information with the ease of harvesting credentials, the story changes. With open and writable buckets, the adversaries have plenty of opportunities: storing and injecting malware, exfiltrating and manipulating data, etc.
McAfee cloud researchers offer an audit tool that, among other things, verifies the rights of buckets. As we write this post, more than 1,200 writable buckets belonging to a multitude of companies, are accessible to the public. One of the largest ad networks in the world had a publicly writable bucket. If adversaries could access that network, they could easily inject malicious code into advertisements. (As part of our responsible disclosure process, we reported the issue, which was fixed within hours.) You can read an extensive post on McAfee cloud research and how the analysts exposed possible man-in-the-middle attacks leveraging writable buckets.
Clustering the Techniques
To combat ransomware, many organizations use the cloud to back up and protect their data. In our talk we will approach the cloud as an attack vector for spreading ransomware. With the leaked credentials we discovered from various sources, the open and writable buckets created a groundwork for storing and spreading our ransomware. With attackers having a multitude of credentials and storage places such as buckets, databases, and containers, defenders would have difficulty keeping up. We all need to pay attention to where we store our credentials and how well we monitor and secure our cloud environments.