Then unencrypt them when linking, maybe? Need to iterate on the design.
Would be nice to have an extra layer of security for files that might contain sensitive information being stored in version control. No substitute for a private repository, but you never know when there's a weak link in the chain.