First of all, I would not refer to myself as a security expert, but I have been in the position of having to answer this question. What I found out surprised me a bit: There is no such thing as a completely secure system. Well, I guess a completely secure system would be one where the servers are all turned off 🙂

Someone working with me at the time described designing a secure system in terms of raising the bar to intruders. So, each layer of securing decreases the opportunity for an attack.

For example, even if you could perfectly secure the private key, the system is not completely secure. But, correctly using the security algorithms and being up to date with patches raises the bar. But, yes, a super computer powerful enough and given enough time can break encryption. I’m sure all of this is understood, so I’ll get back the question.

The question is clear so I’ll first try to address each of your points:

Say the key is protected by the filesystem’s security model; but what
about (malicious) superusers, or platforms that don’t provide such
fidelity?

Yes, if you use something like Windows Key Store or a password encrypted TLS private key you are exposed to the users that have the password (or access) to the private keys. But, I think you will agree that raises the bar. The file system ACLs (if implemented properly) provide a pretty good level of protection. And you are in the position to personally vet and know your super users.

Or the key is hardcoded into software binaries, but it could always be
decompiled and what about open source software or interpreted code?

Yes, I’ve seen hardcoded keys in binaries. Again, this does raise the bar a bit. Someone attacking this system (if it is Java) has to understand that Java produces byte code (etc) and must understand how to decompile it are read it. If you are using a language that writes directly to machine code, you can see that this raises the bar a bit higher. It is not an ideal security solution, but could provide some level of protection.

If the key is generated, such an algorithm would need to be
deterministic (presumably) and then the same problem applies to the
seed.

Yes, essentially then the algorithm becomes the private key information for creating the private key. So, it would need to now be protected.

So, I think you have identified a core issue with any security policy, key management. Having a key management policy in place is central to providing a secure system. And, it is a pretty broad topic.

So, the question is, how secure does your system (and, therefore the private key) need to be? How high, in your system, does the bar need to be raised?

Now, if you willing to pay, there are some people out there that produce solutions to this. We ended up using an HSM (Hardware Security Module). It is basically a tamper-proof server that contains a key in hardware. This key can then be used to create other keys used for encryption. The idea here is that (if configured correctly), the key never leaves the HSM. HSMs cost a lot. But in some businesses (protecting credit card data lets say), the cost of a breach is much higher. So, there is a balance.

Many HSMs use key cards from maintenance and admin of the features. A quorum of key cards (5 of 9 lets say) have to be physically put into the server in order to change a key. So, this raises the bar pretty high by only allowing a breach if a quorum of super users collude.

There may be software solutions out there that provide similar features to an HSM but I’m not aware of what they are.

I know this only goes some way to answering the question, but I hope this helps.

What you want can’t be done.

Say the key is protected by the filesystem’s security model; but what
about (malicious) superusers, or platforms that don’t provide such
fidelity?

You basically want protection from people turning malicious. In your model, at some point someone will have access to the key. What if that person is malicious? What if YOU are malicious? See, the problem as you state is unsolvable except by not having a key at all.

So don’t work with Database credentials but other authentication mechanisms. But no matter what, someone at some point needs access to the data and that someone can be malicious.

One of systems I currently use works this way.

• I start with a login form of the authentication service. It uses two-factor authentication to make sure I am who I claim I am.
• I receive a temporary SSL certificate that I can use to access the protected service. The service keeps track of the certificates it accepts.
• My exchanges are encrypted by this certificate from eavesdropping. Trying to crack it is not very useful since it will expires.
• The certificate expires quickly (in several hours), but not too quickly so that I don’t need to provide a password for every interaction.
• The certificate can be instantly revoked on the other side.

Of course the protected service runs somewhere out of my reach. It could run on my machine under a different account (and possibly in a container), but I have superuser privileges and could then try to circumvent the restrictions.

Basically if you have a malicious superuser, all bets are off. And you should assume the presence of a malicious superuser in your threat model.

So you need to isolate your protected service from the client-accessible machine. Move the protected service to a machine only accessible via network. Put your clients to virtual machines that prevent them from reaching out to the rest of the physical machine where your protected service runs.

If your protected service is not so precious as to be able to command such measures, go with whatever encrypted key store the OS gives you: both Windows, Linux, and OSX have keyring implementations.

This is one of those problems that can’t really be solved at the same level that it was created. We’ll have to take a step back to a few fundamental philosophies, and follow the cascade in hope of a resolution.

The first philosophy is “never trust the client”, and the closely related “if you really want to keep a secret, don’t tell anyone!”

A simple example is database credentials, as you mentioned. Obviously you want your client to have access to it, but not any random Internet Stranger, so you need some kind of identity/verification/login system. But the core premise of hiding this is a problem: if the user itself must possess a secret, but you don’t want them to know what that secret is, all you can do is hide it or make it hard for them to open “the secret package”. But they can still find out, so you’d better have a backup plan!

The easiest solution is, “don’t do that.” Use the user’s own account credentials to permit client-level access to the database for the software, and that’s it. If the client becomes malicious, the worst-case scenario should be that the client can screw up their own data. That’s it. Your database and system should, at most, now contain junk entries from that client, solely for that client, with no one else (including you) suffering from the mayhem.

There are specific use cases where you just only want deployed software to do something but not have everyone in the world able to connect and do that same stuff, but for that you are just hiding secrets and the only good reason to do that is just to reduce headaches or system activity. If your hidden info becomes common knowledge it had better not be game-breaking, just at worst annoying.

If you are in one of those narrow use cases, it’s just about obfuscation, which is all about getting creative. It’s a lot like Code Golf, really – except you are the one creating the puzzle. That’s all it really is – a puzzle. And people like solving puzzles, so again, it better be ok when someone figures it out.

But for the majority of things in the world, it is just best to operate without worrying about having to keep a secret from the user. Instead, best practice is to just let the user in on the secret, make it their responsibility to help protect it (their username and password, for instance), and limit the downside risk of what happens if the secret gets out or gets abused.

In true “key management” cryptographic/security contexts, the answer of “where to store the private key” is “somewhere else!” Encryption is point-to-point protection, and public-private key encryption is designed to protect info in transit through time and space. The private key is inherently vulnerable, and protecting it is not really a question of overlapping encryption – it is securing it against access entirely. And if the System must access it, then the System itself must be secured – and you can’t do that on a client’s machine. They can always run a VM, dump the RAM, sniff their network traffic, install a proxy or virtual NIC, decompile the executables…don’t voluntarily get involved in that loosing battle.

Now if you need to do something like DRM, where your need to store a secret is actually based on controlling use of the software itself, that’s another situation entirely.

TLDR: Don’t keep secrets FROM the user, keep secrets WITH the user.

In my mind, the only way to keep this completely safe is to have the key/password encrypted with a passphrase to unlock. This way the passphrase must be given for each start or use of the secret.. Not too useful.

Another way might be to have the security system be the database account it self. Not very scalable…

If each user on a system has its own DB account, and those accounts was premade so that the application does not have rights to create accounts in DB on behalf of a user you could come pretty close though. Not a good solution either, so I guess you have to have very restricted read-access on the config-files and trust your super-users.