if xxxx
- ends with a colon @--> book name
- begins with a colon and no slashes @--> chapter name
- has just one fwd slash @===> verse (could also have colon to specify different book or different chapter)
- has 2 fwd slashes ==> it is a line => may also have colon to specify different book or different chapter
- has no colon or fwd slashes its a line name
With this notation the open chapter/verse does not change. Used in commands like safe delete | safe drag | safe drop | safe rename <> <>
Changing test to check commit
Working with remote (off-site) storage and sync-ing safe books between different computers is done using safe push and safe pull. Even with a single laptop you need a backup and restore process and this push pull is in-built and ready to go.
The process employs
- a git repository to push and pull crypt material to and from
- a usb key, mobile phone and/or email to stash a small file containing salts
Attackers would need to bring together the crypt material, the salt file and your password, in order to access the safe's credentials.
Note = you only need 10 base64 chars to hold the 40 character hex hash.
git rev-parse `git log -1 --format=%h HEAD` # acquire the full commit hash hex string
git log -1 --format=%h HEAD # pick up the 1st 7 commit hash characters
Visit this Rakefile for an example of how to build, test, version and release.
https://github.com/tslocke/hobo/blob/master/Rakefile
safe has an agile and automated delivery pipeline that assures quality, continuity and usability in the major Linux environments including Ubuntu, RHEL, CoreOS, Amazon Linux and Suse Linux.
The pipeline process is triggered when new software arrives in the safedb github repository. When this happens
- Jenkins picks up the latest software
- Rake and Minitest are used to build and unit test the software
- Docker is used to system test safedb in the key Linux environments
- versioning is applied using the date/time and Git's commit hashes
- if tests pass the safedb gem is deployed to RubyGems.org
- website documentation is built and posted to safedb.net
Note = you only need 10 base64 chars to hold the 40 character hex hash.
git rev-parse `git log -1 --format=%h HEAD` # acquire the full commit hash hex string
git log -1 --format=%h HEAD # pick up the 1st 7 commit hash characters
A safe database contains books that you login to. A book contains chapters and chapters contain verses. Each verse has a number of lines which are just key/value pairs.
Joe Bloggs wants to safely store his social media account credentials. His creates a book called joe.bloggs, a chapter called social and verses called facebook, twitter, instagram and snapchat. These verses will hold key value pairs like username, @password and signin.url (aka lines).
safe init joe.bloggs /path/to/dir # create a book called joe.bloggs
safe login joe.bloggs # login to the book
The joe.bloggs book has been created. Now create the social chapter and facebook verse.
safe open social facebook # open chapter social and verse facebook
safe put username joeybloggs9 # create a username (key/value) line
safe put @password s3cr3t # create a password (key/value) line
safe put signin.url https://xxx # create a signin url (key/value) line
Now that facebook is done - Joe creates another verse called twitter under the social chapter.
safe open social twitter # open chapter social and verse twitter
safe put username joebloggs4 # create a username (key/value) line
safe put @password secret12 # create a password (key/value) line
safe put signin.url https://yyy # create a signin url (key/value) line
safe open creates a new chapter verse or goes to one if it exists. Commands like safe put, safe show and safe delete all work on the currently opened chapter and verse.
You use safe to put and retrieve credentials into an uncrackable encrypted "safe" on your filesystem or USB key.
You interact with safe on the command line, or through DevOps scripts and pipelines. safe will soon **integrate** with storage solutions like S3, Git, SSH, Docker, Redis, the AWS key manager (KMS), Docker, Google Drive, Kubernetes Secrets, Git Secrets, OAuth2, KeePass, LastPass and the Ansible / HashiCorp vaults.
safe is simple, intuitive and highly secure. It never accesses the cloud. The crypt files it writes are precious to you but worthless to everyone else.
$ sudo apt-get install ruby-full # for OpenSSL we need full ruby
$ sudo gem install safedb # install the safe ruby gem
$ export SAFE_TTY_TOKEN=`safe token` # setup a shell variable
$ safe init joe@abc ~/safedb.creds # initialize a safe book in folder
$ safe login joe@abc # login with the created password
You initialize then login to a domain like joe@abc. In the init command we specify where the encrypted material will be stored. Best use a USB key or phone to use your secrets on any drive or computer.
You only need to run init once on a computer for each domain - after that you simply login.
More information will be provided on installing and using safe via a gem install, Ubuntu's apt-get, yum, a docker container, a development install, a unit test install and a software development kit (SDK) install.
Always typing export SAFE_TTY_TOKEN=safe token before using safe is tiresome. Ubuntu provides .bash_aliases for creating tokens like these at the genesis of each shell.
echo "export SAFE_TTY_TOKEN=\`safe token\`" >> ~/.bash_aliases
source ~/.bash_aliases # no need to create a new shell
cat ~/.bash_aliases # we expect the export command
printenv | grep SAFE_TTY_TOKEN # we expect the environment varNote the escaped back-ticks around safe token.
Now that we have created the safetty alias we can login with one line like this.
safetty; safe login joe@abcAdvanced users should avoid adding the export command to ~/.bash_profile.
When the shell closes the shell token will disappear which is good. You can clear it immediately with these commands.
$ unset SAFE_TTY_TOKEN # Delete the shell token
$ env | grep SAFE_TTY_TOKEN # Check SAFE_TTY_TOKEN is deleted
$ env -i bash # Delete every env var created by shell
Visualize your safe as a book (like the Bible or the Oxford English Dictionary).
You open the book at a chapter and verse then read, write and update a key/value dictionary.
- joe.credentials is the book we login to.
- email.accounts is the chapter we open
- [email protected] is the verse we open
Now we can put and read key/value entries at the chapter and verse we opened.
- safe open email.accounts [email protected]
- safe put username joe
- safe input password
- safe put question "Mothers Maiden Name"
- safe put answer "Rumpelstiltskin"
- safe tell
What happened? Look in the configured folder and you'll see some breadcrumbs and your first envelope. What happened was
- the "emal.accounts" envelope is created for [email protected]
- the username and a memorable question are put in
- safe input password securely collects the password
- safe tell outputs all the data at the opened path
Let's put data for the next email account into the same "email.acocunts" envelope.
- safe open email.accounts [email protected]
- safe put username joey
- safe input secret
- safe tell
Emacs tries to detect a password prompt by examining the prompt text. These will match.
- Password:
- Enter new password:
Use Alt-x send-invisible or M-x send-invisible if emacs gets it wrong.
In emas passwords entered in the special minibuffer
- are not displayed
- nor are they entered into any history list
There are ways to help Emacs recognize password prompts using regular expressions and lisp lists but this complexity is rarely warranted.
Whole files can be secured in the safe - not just a sequence of characters.
This is legacy functionality and will soon be refactored using the multi-file embedded map approach.
You can pull in (and spit out) a file into the dictionary at the opened chapter and verse using safe read and safe write
$ safetty # alias command puts token in an environment variable
$ safe login <<book>> # login to a book
$ safe open <<chapter>> <<verse>> # go to the dictionary at the opened chapter/verse
$ safe show # look at the key/value pairs in the dictionary
$ safe read ~/creds/my-key.pem # an encrypted file is added to the safe directory
$ safe write # the file is decrypted and faithfully returned
With read/write only one file can be added to a dictionary. If you safe read the second time the safe file is effectively overwritten and unretrievable. Note that safe write creates a backup if the file exists at the filepath before overwriting it.
But can we put more than one file into a dictionary?
These commands may be refactored into read and write. Suppose you have 4 openvpn (ovpn) files and you want them encrypted in just one dictionary. You can do it with safe inter and safe exhume
$ safe inter production.vpn ~/tmp-vpn-files/prod.ovpn
$ safe inter development.vpn ~/tmp-vpn-files/dev.ovpn
$ safe inter canary.vpn ~/tmp-vpn-files/canary.ovpn
$ safe inter staging.vpn ~/tmp-vpn-files/stage.ovpn
$ safe show
Against the @production.vpn key exists a sub-dictionary holding key-value pairs like in.url, out.url, permissions, is_zip, use_sudo, date_created, date_modified and most importantly content.
The actual file content is converted into a url safe base64 format (resulting in a sequence of characters) and then put into the dictionary with keys named production.vpn, canary.vpn and so on.
$ safe exhume
This powerful command excavates all files thus reconstituting them into their configured plaintext destinations.
$ safe exhume production.vpn # dig out just the one file
$ safe exhume 'production.vpn,canary.vpn' # dig out every file in the list
$ safe exhume production.vpn ~/new/live.ovpn # dig out file to specified path
In keeping with the safe tradition of zero parameter commands whenever and wherever possible the safe inter command will now reread all the files again because safe knows where they should be.
$ safe inter
@Yet to be implemented. Above inter/exhume should be read/write and the below should be the real inter/exhume File content can be presented at standard in (stdin) and ejected to (stdout) in keeping with unix command tradition.
$ cat ~/.ssh/repo-private-key.pem | safe inter repo.key
$ safe exhume repo.key > /media/usb/repository-key.pem
Internally and therefore private - inter converts the multiline text into urlsafe base 64 on the way (std)in and exhume does the opposite on the way (std)out.
Within a DevOps script, you can read from a safe and write to it without the credentials touching the ground (disk) and/or sides.
DevOps engineers often comment that this is the safe's most attractive feature. All you have to do is to tell safe that it is being called from within a script. This an example of connecting to a database maybe to create some space.
$ safetty
$ safe login [email protected]
$ safe open mysql production
$ python db-create-space.py
You've logged into a safe book and opened a chapter and verse. Then you call a script - look no parameters!
(Improve by using actual python commands).
Now within the script could be lines like this.
db_url = %x[safe print db.url --script]
db_usr = %x[safe print db.usr --script]
db_pass = %x[safe print db.pass --script]
db_conn = Connection.new( db_url, db_usr, db_pass )
Notice the credentials have not touched the disk. The decrypted form was only used in memory to connect.
The switch --script tells safe that it is being called from within a script. Safe won't give out credentials if the script in turn calls another script and that calls safe - it only obliges when you have run the command yourself.
This gives you peace of mind that sub-processes two or more levels deep will not be able to access your credentials.
You can also limit the credentials in a book. Scripts can only access credentials in books that you have logged into. Credentials in other books within your safe are out of scope.
Many DevOps scripts source credentials that then need to be stored. Scripts can use Safe's configurable random generators to produce passwords, public/private keypairs and AES keys. Or the credentials are sourced externally and the scripts then place them into the safe.
$ safe login <<book>> # login to one of the books in the safe
$ safe use <<book>> # switch to this or that book (if logged in)
$ safe open <<chapter>> <<verse>> # open email accounts chapter at this verse (specific account)
$ safe view # contents page of chapters and verses in this book
$ safe goto <<N>> # shortcut for open command (pick number from the viewed list
$ safe put <<key>> <<value>> # put in a non-sensitive key-value pair
$ safe put @<<key>> <<value>> # put in a non-sensitive key-value pair
$ safe show # show the key/value dictionary at chapter and verse
What types can safe store. Remember the
- book
- chapter
- verse
You login to a book and then "open" it up at a chapter and verse.
At that point you get a dictionary with string keys. The value types can be
- strings
- integers
- booleans
- lists
- dictionaries
- another book, chapter and verse
- files (plain, binary, zip)
We need to fix the login bug which we now workaround by init(ing) every time. On top of that we must document the behaviour for
- list management (create read add remove eject) - remove is given a value while eject is given an index
- crud operations on books, chapters, verses and key/value entries
- password changing
- hardening configuration using Hexadecimal characters
We can configure safe's behaviour
- globally for all books on a given workstation
- locally for activities pertaining to the current book
Once credentials are in safe they can be exported in different formats. Also you can start a shell, login, open a chapter and verse and then give safe the command to run.
It can then export out selected (key/value) dictionaries at the opened chapter and verse as
- environment variables
- Kubernetes Secrets formatted files
- AWS IAM user environment variables or files
- RubyGem credentials (consumable by rake)
- rclone credentials for accessing GoogleDrive, Rackspace
- openvpn (ovpn) files (with keys/certs) for VPN tunnels
- ubuntu network manager configurations fir VPN and wireless
- certificates RubyGem credentials (consumable by rake)
- git credentials for pushing (or cloning) a git repo
In effect, safe can start VPNs, wireless connections, launch Firefox with certificates installed, run Ansible and Terraform suppling vital credentials - all this without the credentials ever touching the ground (filesystem).
safe can transfer a verse (or even the whole chapter) into a Kubernetes Secrets compatible format.
Kubernetes Secrets (through the kubectl interface) require that hexadecimal (base64) encoding be applied to secrets coming in through the letterbox.
safe can output dictionary (key/value pair) configurations in a format consumable by Kubernetes secrets.
Cracking safe is infeasible for anyone other than the rightful owner. Only OpenSSL implemented tried and tested cryptographic algorithms are used. Both PBKDF2 and BCrypt are used for expensive key derivation. The content is encrypted with AES (Advanced Encryption Standard) and 48 byte random keys are employed along with initialization vectors.
Even with all this crypt technology it is important that you
- choose a robust password of between 10 and 32 characters
- align the number of salt derivation iteratios to your machine's resources
- backup the domain folders in case you lose your USB drive or phone
Your ability to access your own secrets (even after disaster scenarios) is as important as preventing the secrets being accessed.
We travel between laptops, desktops, virtual machines and even docker containers. Always run init the first time you use a domain on a different computer.
$ gem install safe
$ export SAFE_TTY_TOKEN=`safe token` # setup a shell variable
$ safe init joe@abc /home/joe/credentials # initialize a secrets domain
$ safe login joe@abc # login to the new domain
Run all four commands the first time. Then simply run the second and fourth commands whenever you open a new shell to interact with safe.
safe is designed to operate in highly secure locked down environments in which external access is not just constrained - it is non-existent.
safe does not contact nor talk to anything external. It never asks (nor needs to know) the credentials for accessing your stores - this means it compliments your storage security be it S3, Google Drive, Redis, Git and even email/pop3 solutions.
The ability to read data from drives (after the fact and) after deletion means nothing unencrypted should be put on any drive (including usb keys).
Aside from your private keys, safe keeps a small amount of configuration within the .safe folder off your home directory. A typically safe.ini file within that folder looks like
[[email protected]]
type = user
id = joe
keydir = /media/joe/usb_drive
domains = [ lecturers@harvard ]
default = true
printx = asdfasdfas65as87d76fa97ds6f57as6d5f87a
printy = asdfasdfas65as87d76fbbbasdfas0asd09080
printz = adsfasdflkajhsdfasdf87987987asd9f87987
[lecturers@harvard]
type = domain
store = git
url = https://www.eco-platform.co.uk/crypt/lecturers.git
The planned list of backend storage systems (each onlined with a plugin), is
- Git (including GitHub, GitLab, BitBucket, OpenGit and private Git installations).
- S3 Buckets from the Amazon Web Services (AWS) cloud.
- SSH, SCP, SFTP connected file-systems
- network storage including Samba, NFS, VMWare vSAN and
- GoogleDrive (only Windows has suitable synchronized support).
Access management is configured EXTERNAL to safe. SafeDb simply piggybacks the network transport if authorization is granted.
You can use safe alone or you can use it to share secrets with colleagues, friends and family, even machines.
safe is simple and holistically secure. Simple means less mistakes, less confusion and more peer reviews from internet security experts.
Every domain is tied to backend storage which is accessible by you and others in your domain. You can use Git, S3, a networked filesystem or shared drive, a SSH accessible filesystem and soon, free storage from safe.io
You can require safe (as an SDK) and interact with it directly from any other Ruby program without wrappers.
$ gem install safe
$ irb
$ > require "safe"
$ > SafeDb::CLI.version()
The above should return the installed version of SafeDb.
If you get a LoadError (cannot load such file -- safe) then try the below.
$ irb
$ > $LOAD_PATH
[ "/usr/share/rubygems-integration/all/gems/did_you_mean-1.2.0/lib", "/usr/local/lib/site_ruby/2.5.0", "/usr/local/lib/x86_64-linux-gnu/site_ruby", "/usr/local/lib/site_ruby", "/usr/lib/ruby/vendor_ruby/2.5.0", "/usr/lib/x86_64-linux-gnu/ruby/vendor_ruby/2.5.0", "/usr/lib/ruby/vendor_ruby", "/usr/lib/ruby/2.5.0", "/usr/lib/x86_64-linux-gnu/ruby/2.5.0" ]
Before we can move to siloed safe workspaces and RELEASE the software into the public domain we must refactor file handling and implement vital methodologies for evolving the software.
- delete the concepts of content.id, content.iv and content.key in the context of files.
- add one more key to file verse @file.content and store the urlsafe base64 contents of the file there
@@@@@@@@@@@@ change @@@@@@@@@@@@ change ==> maybe better to create a sub dictionary (map) for the file so will be @@@@@@@@@@@@ change ==> key value pairs. Keys could be permissions - 755 | @content - BASE64 file representation | read.url - http://shareprices/mcdonalds.yaml | write.url $HOME/shares/mcds.yaml | type - binary @@@@@@@@@@@@ change
This move means that if we wish to export and import we do not need to fiddle with chapter files vs file files.
This concept will come with more commands - like so
safe add favfoods rice safe insert favfoods |5| potato ## Note first index is 0 -> Also -2 is 2nd last | default is -1 (append at the end) safe remove favfoods chicken safe pop favfoods |3| safe place cityfacts {} safe place cityfacts { "london" => "6,200,000", "beijing" => "20,500,000", "new york" => "9,300,000" } safe get cityfacts beijing safe remove cityfacts "new york"
Also you can now print in many formats including --hex, --json, --base64, --xml, --ini, --yaml
Now build export to simply spit out everything into plain text chapter files (within safe workspace - export section). Then the json chapter files are tarred and compressed. Build import to uncompress then unzip then use the JSON to re-create the database
This move opens the door to safe's beautifully simple upgrade methodology. To upgrade safe to a major or minor version you
- use the outgoing version to export all books
- then we upgrade safe
- then we use the new safe software to import and you are done.
Now we have cleared the path for a SIMPLE Backup and Restore method.
The backup/restore MUST BE VERSION AGNOSTIC (in as far as is human and machinely possible. Employ the export first giving us first zip file. Then add a backup meta-data file with details like who when why which tag which version and most IMPORTANTLY the random IV and SALT for the key that locks the exported content file.
The backup method retars up compresse both the metadata and the locked file. The new filename is like this.
safe.backup.<<book-name-code>>.<<time-stamp-millis>>.<<version>>.tar.gz
It adds it to the local safe backup workspace. It can only be done when logged in.
safe restore /path/to/backup/file.tar.gz
A restore will override the current in-place repository (after creating a backup of it) and user given option to rollback the restore.
This method (theoretially) allows a version 3.428.24952 to restore an export of version 1.823.03497
A safe repository (book) can be changed by one branch but read concurrently by multiple branchs.
Directory Links are NOT PORTABLE to use to point to the active workspace especially if we the safe root folder is on a USB key.
A GOOD engough concurrency technique is a lock file in the BOOK's root folder that is named safe.concurrency.lockfile.<<book.id>>
The contents of the file will hold the relative directory name (branch ID based) that has the lock and the branch ID that had it before that (if not first).
The <machine.id>.<bootup.id> is used to when the first read/write login branch occurs. Subsequent logins for a read/write branch will then have 2 choices in this shell.
- safe login ali.baba --steal # take over the primary read/write branch
- safe login ali.baba --branch # leave primary branch but open one that will not change the price of sugar
- safe login ali.baba --branch=master
- safe login ali.baba --branch=experimental
- safe login ali.baba -b experimental
safe login --steal
A third choice arises if we visit the shell holding the directory pointer and logout.
Logout NEVER TOUCHES the lock file (it could have moved on multiple times so only login can act on it).
However logout DELETES the cipher.file branch ciphertext that can be unlocked by branch key to retrieve the content key. This action renders it impossible to read or write any data from logged in book.
A subsequent login can again re-instate this privilege.
At the very beginning a repository can come into being through either
- an init
- or a clone (from git,s3,ssh,local filesystem, http)
The first repo holds the live link.
Subsequent logins must perform two checks
- IS MY DIRECTORY (branch) noted as the latest in the lock file (possible if you've logged out of the same shell)
- (if other directory) - Does the branch key within that directory's cipher file have a value
The popup asking the user to STEAL or go READONLY is triggered if the answers above are NO then YES.
If branch key has no value then stealing is not necessary so the existence of the --steal flag does not change the price of sugar.
The Stealing flow of events is to
- copy the directory into a new one for this branch named
<<book.id>>.<<timestamp>>.<<branch.key>> - validate the directory for data consistency (nice to have functionality)
- collect the password and if invalid stop now
- grab the lock file and write it to point it to our directory (we are it)
- create our own branch key and write it in within our folder
Starting a BRANCH allows you to read and write to a copied branched repository but this branch does not change the price of sugar.
In the future MERGE functionality may be implemented so that the database branch can be merged back into the master line.
May a safe overthrow command can be crudely done which rudely overthrows the main (government) line and installs this dictatorish branch as the leader - possibly trashing any changes that the master line may have since the branch occured.
The prune command can delete workspaces if
- they are not the master branch AND
- they have not been changed in this bootup (or a logout has been issued againt them).
chapter files can only be written once but can be read often. This policy may make merging and diffs between branches easier in the future.
After checking out the repo, run bin/setup to install dependencies. Then, run rake test to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/safe. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
MIT License Copyright (c) 2006 - 2014
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the 'Software'), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.