I spend a good amount of my time doing software development. I’m one of those guys that has a bad habit of starting projects, getting half or three fourths of the way through and then coming up with another project to do (leaving the original out on in the cold). Needless to say I end up playing with a lot of tools and libraries to help with projects but I’ve started to notice a pattern. The assumption that behind the firewall everyone is friends.
In a more recent project I was working on it became apparent that a queuing system of some kind was going to be needed. Instead of running out and picking the most popular flavor of the month I figured the best move would be to give a few different systems used for queuing a run and see how they worked out. In general I was impressed with their abilities but found the security lacking greatly in a number of them.
Please be aware I’m not trying to discount any of these applications.The two I tried directly I really liked from a development point of view.
One of the earlier ones I checked out was Redis. It was blazing fast but the security model is interesting.
Redis is designed to be accessed by trusted clients inside trusted environments. This means that usually it is not a good idea to expose the Redis instance directly to the internet or, in general, to an environment where untrusted clients can directly access the Redis TCP port or UNIX socket.
To make matters even more interesting it has support for a single password passed plainly over the wire. Granted, it’s possible to use an SSL proxy as the guide points out but with one user non-repudiation could be a serious problem (especially if logs go back to NAT’d addresses). In effect the security model of Redis seems to require a single tenant, well logged (at network, host and app level) and heavily ACL’d environment. With cloud hosting I’m not so sure how well one could ensure this is the case at all times. Granted, if it’s a single developer running his own infrastructure or a very small company/group/team then it could be possible that the model would work well enough. Honestly I couldn’t get over the fact I’d have to tell friends who wanted to play with the project they’d have to make a special environment before installing.
I didn’t end up trying beanstalk but did notice it had similar pitfalls. As Kurt Seifried points out in his blog:
The major downside to beanstalkd is that it doesn’t provide any encryption of network traffic or authentication capabilities, so any client with access to a running instance of a beanstalkd server will have access to all the queues in that running instance. This can be gotten around by wrapping beanstalkd in SSL (using stunnel or a similar service) to secure communications and limiting access to beanstalkd queues based on either IP address or by requiring SSL client authentication.
So again, if you want to use the service you must either setup extra hoops and/or have an incredibly locked down infrastructure.
ZeroMQ is really cool. But you end up with similar problems of network ACL’s providing all of your protection unless you write your own authentication and authorization mechanisms.
What security features does ØMQ support?
None at the moment. ØMQ does not deal with security by design but concentrates on getting your bytes over the network as fast as possible. Solutions exist for security at the transport layer which are well understood and have had many man-years of development invested in them, such as IPsec or OpenVPN.
Granted zmq is a bit lower level and used as a building block instead of a solution so it is understandable why some things are pushed back upon the developer to implement as needed.
But Who Cares?
It’s more about being aware.
- Can anyone promise that network ACL’s won’t be modified to enable a shiny new application?
- Can you be sure that the other side of the SSL connection will remain safe and trustworthy?
- Is any data making it’s way through which can have an effect on process inside the firewall guaranteed safe (example)?
- If the hosts are multi tenant or in the cloud are you sure everyone who has access to the VM’s or networks are trustworthy?
You and/or the developers of these apps wouldn’t have come up with some kind of security solution if it was OK for any random Joe to play with the service. If someone is able to interact with a service which is “soft on the inside” then it’s likely that service would be an early target.
For example, let’s imagine an attacker gets access to the service because he is able to take control of an approved host. If the service on the other side is Redis then the attacker could sit and gain information painlessly before copying work from that point forward. If it is a zmq port then an attacker could attach another process to it and get either a copy of everything (SUB, ”) or a subset of data (PULL). Beanstalk probably has similar abilities. The security on the other side of the connection, whether inside or outside the firewall, ends up being as important as the security on the inside as the level of access to the service is more or less the same. All or nothing.
Using an SSL tunnel and only allowing specific hosts may constitue as defense in depth on paper it doesn’t seem to be enough. Maybe I’m to paranoid but if there was authentication and basic authorization in or around the service an intruder would need to gain further information or perform more attacks to gain access.