can you elaborate a little on what you mean "interfaces"
In this context, interface is a set of APIs used for certain purpose (e.g. filtering registry calls).
Also you mentioned hooking userland libraries how can this be done best ?
Well, probably inline hooks will do.
More importantly if the application by-passes the hooks as you mentioned by calling the system functions directly then what is the point of implementing userland hooks ?
The key point here is that the sandboxed application itself has as less privileges as possible when talking about the Windows security model. However, there may be "permissions", you wish to grant the application, that cannot be easily expressed by the terms of the security model. As an examle, let's say you need to grant the sandboxed application access only to a file A (and just image you cannot do that via the standard security means, such as ACLs). From the security model viewpoint, the application does not have access to A, however, your userland hooks may be used to communicate with your more privileged application that can then access A on behalf of the sandboxed one.
So, the userland hooks are used to grant
the sandboxed application some extra permissions, not to restrict
it. The assumption is that a nice application will not bypass the hooks, so it will get permissions it needs. However, the malicious application that bypasses the hooks gets nothing, since the sandbox restricts it by the security model (low integrity level, restricting SIDs in the access token etc.).
heavily constrained job object
Job objects allow you to place some retrictions on applications run under them. At the time I was researching them, the most interesting was the GUI restriction – e.g. preventing the sandobxed application from sending Windows messages outside the job object. With new versions of Windows, the possible set of restrictions grew quite a bit. I just wanted to point you to job objects as an interesting contept for implementing a sandbox.