Homomorphic binary circuits - hbc

This project was developed as my Master final year project at Imperial College London.

This project is still maintained (Sept 2017) by Quentin McGaw (email: quentin . mcgaw at gmail . com)

1. What is it, in one line?

It is an API of homomorphic binary operations such as binary comparison or binary Euclidian division using the library HElib.

2. What is in there?

• It is written in C++ and is cross-platform with Vagrant
• The core API is in src/he.cpp.
• The code that is ran as an example is in src/main.cpp.
• Some other classes are in src/helper_functions.cpp
• All the other src files are unit tests and timing tests for the homomorphic binary operations implemented in src/he.cpp.
• There is a Vagrantfile to setup eveything for you, cross-platform.
• There is a makefile to build hbc or setup almost everything for you (depending on your OS).
• There is this complete, detailed and updated README.md file.
• There are links in this readme to my presentation (VERY USEFUL) and my detailed report (WAY TOO LONG).

3. What does it run ?

• It runs the code in main.cpp which execute the unit tests on all the homomorphic circuits implemented.
• You can change the main.cpp with your code by inspiring from the tests.
• You can also add circuits in he.cpp and then add corresponding tests, and tell me you want to contribute !

4. What does it require ?

• Practically:

  • A Linux/Windows/OSX computer
  • At least 3GB of RAM and 2 CPU cores
  • An internet connection
  • CPU with Hardware virtualization tech ideally (you probably have it don't worry)

• In terms of software (although this is automatically installed), here are the dependencies:

  Program or Library	Requirement 1	Requirement 2	Requirement 3	Requirement 4	Requirement 5	Requirement 6
  hbc	g++	make	libboost	c++11	HElib
  HElib	g++	make	git	NTL 10.5.0	GMP 6.1.2
  NTL 10.5.0	g++	make	GMP 6.1.2
  GMP 6.1.2	g++	make	m4	perl

5. Documentation

• This readme file
• Presentation available on Dropbox, on Github and on Youtube where I present it.
• Report available on Github or on Dropbox
• Comments in the source code, especially in he.cpp

6. Abstract

This project concerns the research and development of a real-use application of homomorphic encryption for cloud computing. The application takes advantage of the various possibilities and limitations of present homomorphic encryption schemes and programming libraries to remain usable in terms of time. The foundations of the application rely on the design of binary operations using homomorphic encryption. All the binary logic gates and various binary blocks were developed and adapted to provide enough functionalities to the application. The project focuses on providing features to cloud computing such as calculating averages on large amounts of encrypted numbers in a relatively short and decent time. The result is an application program interface written in C++ allowing to perform various operations on integers. It thus shows homomorphic encryption can be used today for simple operations if the security is more important than the speed of execution.

7. How do I run it?

7.1 Using Vagrant (easiest, compatible with all, most flexible)

1. Install git on your computer
   • apt-get install -y git for Linux machines
   • or download it from git-scm.com/downloads
2. On Windows, have an ssh client or add the ssh.exe of C:\Program Files\Git\usr\bin to your environment path
3. Install Virtual Box from virtualbox.org/wiki/Downloads
4. Install Vagrant from vagrantup.com/downloads.html
5. Open a terminal and enter git clone https://github.com/qdm12/hbc.git
6. Go to the hbc directory with cd hbc
7. Enter vagrant up to launch the virtual machine which will setup and build everything for you. This takes about 30 minutes the first time, depending on your connection speed and CPU. This basically launches an Ubuntu-based virtual machine with only what is necessary for this project. WARNING: If you do not have hardware virtualization, you can still run it but you have to change trusty64 to trusty32 and vb.cpus = 2 to vb.cpus = 1.
8. Once vagrant up has completed, enter vagrant ssh to log in the virtual machine.
9. The working directory hbc on your host machine is shared with the virtual machine at /vagrant.
10. In the virtual machine, enter cd /vagrant.
11. What's nice then:
    • You can modify the files on your host machine (like Windows etc.)
    • Changes you make are automatically reflected in the Ubuntu-based virtual machine.
    • Compile hbc again with make hbc in the virtual machine.
    • Run hbc with ./hbc from the virtual machine or your host machine.
    • Note: *You can use make hbcNrun to build and automatically run the main.cpp code.
12. When you are done:
    • Enter exit in the virtual machine, bringing you back to your host machine.
    • Enter vagrant halt to shutdown the machine. Or enter vagrant destroy to delete the machine.
13. To log back in, enter vagrant up and it should take about 30 seconds ! (except if you destroy the machine)

7.2 Using more manual ways, which don't work for all OSes

1. Make sure you have installed make
2. Open a terminal as root or administrator ideally
3. Setup the necessary libraries
   • With the Makefile provided (only works for Debian and Ubuntu)
     1. Note: git, g++, m4, perl, gmp and ntl will be installed automatically*.
     2. Enter make HElib in a terminal in the hbc directory.
   • Manually (if Vagrant and Makefile are not good for you)
     • Mac OSX
       1. Install Xcode manually or with xcode-select --install
       2. Install brew with ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
       3. Install libraries with brew install wget git g++ m4 perl libboost
       4. Download GMP with curl https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2 > gmp-6.1.2.tar.bz2
       5. Extract it and go to its directory with tar -xvjf gmp-6.1.2.tar.bz2 && cd gmp-6.1.2
       6. Configure it with ./configure
       7. Build it with make
       8. Install it with make install
       9. Optionally, check it with make check
       10. Go back and remove used files with cd .. && rm -fr gmp-6.1.2*
       11. Download NTL with curl http://www.shoup.net/ntl/ntl-10.5.0.tar.gz > ntl-10.5.0.tar.gz
       12. Extract it and go to its directory with tar -xvzf ntl-10.5.0.tar.gz && cd ntl-10.5.0/src
       13. Configure it with ./configure NTL_GMP_LIP=on
       14. Build it with make
       15. Install it with make install
       16. Go back and remove used files with cd ../.. && rm -fr ntl-10.5.0*
       17. Clone HElib with with git clone https://github.com/shaih/HElib.git
       18. Go to its src directory cd HElib/src
       19. Build it with make
       20. Optionally, check it with make check and test it with make test.
       21. Go back to the working directory with cd ../..
     • Other Linux OSes
       1. Install the libaries with (add sudo maybe) apt-get install git g++ m4 perl libboost-all-dev
       2. Download GMP with wget https://gmplib.org/download/gmp/gmp-6.1.2.tar.bz2
       3. Extract it and go to its directory with tar -xvjf gmp-6.1.2.tar.bz2 && cd gmp-6.1.2
       4. Configure it with ./configure
       5. Build it with make
       6. Install it with make install
       7. Optionally, check it with make check
       8. Go back and remove used files with cd .. && rm -fr gmp-6.1.2*
       9. Download NTL with wget http://www.shoup.net/ntl/ntl-10.5.0.tar.gz
       10. Extract it and go to its directory with tar -xvzf ntl-10.5.0.tar.gz && cd ntl-10.5.0/src
       11. Configure it with ./configure NTL_GMP_LIP=on
       12. Build it with make
       13. Install it with make install
       14. Go back and remove used files with cd ../.. && rm -fr ntl-10.5.0*
       15. Clone HElib with with git clone https://github.com/shaih/HElib.git
       16. Go to its src directory cd HElib/src
       17. Build it with make
       18. Optionally, check it with make check and test it with make test.
       19. Go back to the working directory with cd ../..
     • Cygwin 32 bit and 64 bit
       • It will fail when you try to install NTL with the NTL_CMP_LIP=on because cygwin does not find the -lgmp libary for some reason.
       • So just switch to use Vagrant. I might be missing something but there is no point digging to deep here I believe.
       • Here would be the procedure:
         1. Close any Cygwin processes running
         2. Download the right Cygwin installer:
            • Cygwin 32 bit
            • Cygwin 64 bit
         3. Run the Cygwin installer previously downloaded
         4. Click Next >, Next >, Next >, Next >, Next >, Next >
         5. With the help of the search bar, select the following packages (only the Devel is necessary):
            • git: Distributed version control system
            • gcc-g++: GNU Compiler Collection (C++)
            • make: the GNU version of the 'make' utility
            • m4: GNU implementation of the traditional Unix macro processor
            • perl: Perl Programming language interpreter
            • libboost-devel: Boost C++ libraries
            • gmp: Library for arbitrary precision arithmetic
         6. Click on Next >, Next > and wait for the installation to finish and then close the window.
         7. Lauch a Cygwin terminal
         8. Download NTL with wget http://www.shoup.net/ntl/ntl-10.5.0.tar.gz
         9. Extract it and go to its directory with tar -xvzf ntl-10.5.0.tar.gz && cd ntl-10.5.0/src
         10. Configure it with ./configure NTL_GMP_LIP=on
         11. Build it with make
         12. Install it with make install
         13. Go back and remove used files with cd ../.. && rm -fr ntl-10.5.0*
         14. Clone HElib with with git clone https://github.com/shaih/HElib.git
         15. Go to its src directory cd HElib/src
         16. Build it with make
         17. Optionally, check it with make check and test it with make test.
         18. Go back to the working directory with cd ../..
4. Build hbc
   • With the Makefile provided (compatible will all platforms).
     1. Build it with make hbc
   • Manually
     1. Create the directory objects mkdir -p objects
     2. Compile the API g++ -c src/he.cpp -I HElib/src -o objects/he.o
     3. Compile the helper functions g++ -c src/helper_functions.cpp -o objects/helper_functions.o
     4. Compile the various tests
        • g++ -c src/TEST_GATES.cpp -I HElib/src -o objects/test_gates.o
        • g++ -c src/TEST_CIRC_COMB.cpp -I HElib/src -o objects/test_circ_comb.o
        • g++ -c src/TEST_CIRC_SEQ.cpp -I HElib/src -o objects/test_circ_seq.o
        • g++ -c src/TEST_CIRC_ARITHM.cpp -I HElib/src -o objects/test_circ_arithm.o
     5. Compile the main.cpp file g++ -c src/main.cpp -I HElib/src -o objects/main.o
     6. Compile the objects into hbc g++ objects/*.o HElib/src/fhe.a -o hbc -L/usr/local/lib -lntl -lgmp -lm
5. Run hbc
   • Run it with ./hbc (Careful about having enough RAM)
   • You can also build it and run the new build with make hbcNrun

8. RAM considerations IMPORTANT

• To run the default hbc program, you need at least 3GB of RAM.
• This is because the average operation (arithmetic tests) uses about 2GB of RAM for 2 bits.
• Note that you can comment it out in the main.cpp or TEST_ARITHMETIC.cpp and stick to tests of simpler circuits such as the multiplication which only require about 0.7 - 1GB of RAM.
• For Vagrant, you can modify the amount of RAM in the vb.memory field, which is set to 2600MB by default. To monitor the RAM usage, open a new host terminal, go to the working directory and use vagrant ssh -c htop.

9. CPU considerations for Vagrant

• By default, the Vagrant VM uses 2 cores of your CPU (vb.cpus = 2) so that you can run hbc and also monitor the RAM with another vagrant ssh.
• You can also run more instances of hbc if you have more than two cores available. With Vagrant, just set vb.cpus to 3 for example, log in with vagrant ssh on different host terminals and run hbc (provided you have enough RAM to run both obviously).

10. Remove and uninstall

10.1 With Vagrant

Just enter vagrant destroy from your host machine in the working directory.

10.2 Otherwise

Use the makefile and run make deepclean which uninstalls and delete:

• hbc
• HElib, NTL, GMP
• perl, m4, git, gcc-g++ and libboost-all-dev and purge them. Only the makefile will remain in the folder.

11. Acknowledgements

Credits to Shai Halevi for HElib obviously and thanks for the quick help tips in the Issues section!

Thanks to Dr. Wei Dai (Imperial College London) for introducing me to homomorphic encryption

Thanks to Christian Bodt (Coinplus) for teaching me the basics of cryptography and security, as well as useful coding skills.

Thanks to Alexandra Rouhana for her useful discussions that helped me figure out how to overcome some design restrictions.

Thanks to Wei Dai from the Vernam Group at Worcester Polytechnic Institute for keeping me updated with the status of CuHE.

Thanks to my dad, mother and step-mother for their continuous support throughout my studies.

12. Contribution

You're welcome to contribute to this repository if you find any better circuits or other circuits and implement them !

Just send me an email (see my address at the start) and I will add you as a contributor.

Don't hesitate to contact me if you have any question as well.

13. Future Work (ideas crossing my mind)

• Use *= instead of multiplyBy when the level is very low as it is faster. multiplyBy uses relinearization which serves to reduce the size of ciphertexts. This is great for complex circuits but takes a longer time than *= for simple circuits.
• Add circuits from here to the core API he.cpp.
• Other ideas are in chapter 9: Future work of my report.