Compiling Steem for Windows (on Linux host)
This is a step-by-step guide on how to cross-compile Steem for Windows on a Linux host.
Guide to build Steem for a 64-bit Windows target on an Ubuntu Linux host
I have spent a lot of time making changes to the Steem codebase and writing my own custom build scripts to make this process relatively painless for someone who is familiar with Linux.
I have not yet tried to see if I can get this (or a similar process) to work on a Windows host (using MSYS2 perhaps). So you still need access to a Linux machine to build the Windows executables, for now.
For this guide to work, you need a version of Steem that has my changes included. As of this moment on May 22, 2016, only my Steem repo has those changes. However, I have submitted pull requests to get these changes included in the official codebase.
Edit: As of v0.5.0, my changes have been included into the official codebase. So the guide below has been updated to use v0.5.0 from Steemit's steem repo.
Initial setup
This was all done on a fresh installation of Ubuntu Server 16.04. It will likely work with other Linux distros, but I cannot guarantee it and some of the instructions may need to change.
We first need to get some initial setup out of the way. Some of this will require root permissions via sudo.
First we need to install packages for the toolchains we will be using.
~$ sudo apt-get install build-essential cmake mingw-w64 autoconf libtool doxygen
Then create a directory with a short absolute path to work within. This is so that we get consistent (and privacy-preserving) strings within the compiled executables.
~$ sudo mkdir /r
~$ sudo chown 1000:1000 /r
~$ cd /r
Make sure to chown the directory with your user and group so that you have write permission in that directory.
Let us create a few more useful directories within the /r directory.
/r$ mkdir mingw
/r$ mkdir src
/r$ mkdir build
Steem has two external dependencies: Boost and OpenSSL. Both of these will need to be compiled with the same MinGW compiler as the one used to compile Steem for things to link properly.
Cross-compiling these dependencies gets a little tricky, so you will need the handy scripts I have made to help automate this process. We need to get and install these scripts now.
/r$ cd /r/src
/r/src$ wget -O steem_dependencies_build_scripts.tar.gz https://gateway.ipfs.io/ipfs/QmWuT4PoJ9snfjGL49pi32WLUeQWzrZKWw7G3DnDrLCwkT
/r/src$ tar -xzf steem_dependencies_build_scripts.tar.gz
Compiling and installing JWasm
Cross-compiling Boost for a Windows target on a Linux host is tricky because one of the libraries, Boost.Context, needs to assemble some assembly code. However, the Boost codebase is only set up to support one compiler per target, and for the Windows target that compiler is MASM. Unfortunately, MASM only runs on Windows, but we need the assembler to run on the Linux host.
The solution is to use an open-source alternative to MASM called JWasm. The command-line interface for JWasm is not exactly the same as MASM, but I have written a Python script "steem_dependencies_build_scripts/helper/ml64" which wraps jwasm and converts the arguments passed to the script into a form acceptable by jwasm. It is a fragile hack, but it gets the job done.
So let us grab the source for JWasm, compile it, and install it in a location that is within the system PATH.
/r/src$ git clone https://github.com/JWasm/JWasm.git
/r/src$ cd JWasm
/r/src/JWasm$ make -f GccUnix.mak
/r/src/JWasm$ sudo cp GccUnixR/jwasm /usr/local/bin/
/r/src/JWasm$ cd ..
From this point on we should not need sudo or root permissions anymore.
Building OpenSSL
Building OpenSSL is straightforward using my script.
/r/src$ wget https://www.openssl.org/source/openssl-1.0.2h.tar.gz
/r/src$ tar -xzf openssl-1.0.2h.tar.gz
/r/src$ cd openssl-1.0.2h
/r/src/openssl-1.0.2h$ /r/src/steem_dependencies_build_scripts/mingw_build_openssl.sh
/r/src/openssl-1.0.2h$ cd ..
Building Boost
Apparently, Steem does not currently build against Boost 1.61. It does build against Boost 1.59 and 1.60, but I found that the resulting executables for both versions were not functioning properly for some reason. The latest version of Boost that Steem (at least the cross-compiled version) does work well with is Boost 1.58.0. So we will build that.
/r/src$ wget http://downloads.sourceforge.net/project/boost/boost/1.58.0/boost_1_58_0.tar.bz2
/r/src$ tar -xjf boost_1_58_0.tar.bz2
/r/src$ cd boost_1_58_0
/r/src/boost_1_58_0$ /r/src/steem_dependencies_build_scripts/mingw_build_boost.sh -j2
/r/src/boost_1_58_0$ cd ..
Feel free to replace (or even get rid of) the -j2 flag with -jN where N is the number of cores on your machine.
Building Steem
At this point the external dependencies for Steem have been built and installed at /r/mingw.
We now need to build Steem itself.
Grabbing and setting up the appropriate sources
We will be building v0.5.0. Feel free to update these instructions to build any later versions.
/r/src$ git clone https://github.com/steemit/steem.git
/r/src$ cd steem
/r/src/steem$ git checkout v0.5.0
/r/src/steem$ git submodule update --init --recursive
/r/src/steem$ cd ../..
Cross-compiling Steem
I like to build Steem out of the source directory. This allows me to better manage all the different types of builds (release/debug, full/witness, linux/windows) for all the different versions of Steem I compile.
So let us first set up the directory for the build and cd into it.
/r$ cd build
/r/build$ mkdir -p steem/v0.5.0/windows-full-release
/r/build$ cd steem/v0.5.0/windows-full-release
We can now run cmake to configure the build. However, because the options get very complicated for a cross-compile build, I have created a helper script that simplifies the process. This script is actually included with the Steem codebase. In fact, it is not just for cross-compilation. It works well for regular compilation as well. I encourage you to play around with it. The usage and help (accessible by running /r/src/steem/programs/build_helpers/configure_build.py -h) documents how to use it pretty decently.
/r/build/steem/v0.5.0/windows-full-release$ /r/src/steem/programs/build_helpers/configure_build.py --sys-root=/r/mingw -f -r --win
After running that command, cmake should have successfully built the files into the directory.
Note that -f -r options configured the build to be a Release build of a full node (meaning -DLOW_MEMORY_NODE=OFF), which is the default if you do not pass any flags. By the way, my changes to the code (included in the pull request) means that -DENABLE_CONTENT_PATCHING=ON is no longer necessary because the code no longer depends on Qt5 to run the patching algorithm.
If you wanted to build a witness node instead, you would pass -w instead of -f. And if you wanted a Debug build, you would pass -d instead of -r.
The --win flag is to tell the build script that this is a Windows cross-compilation build. Leaving it off would make it build Steem for the native platform like usual. Because --win is specified, it is essential to also specify the --sys-root so that the build script knows where the MinGW-w64 compiled Boost and OpenSSL dependencies reside.
Now, all that is left is to actually compile Steem.
/r/build/steem/v0.5.0/windows-full-release$ make -j2
Once again feel free to change the -j2 flag.
If all went well, you should have steemd.exe located at /r/build/steem/v0.5.0/windows-full-release/programs/steemd/steemd.exe and cli_wallet.exe located at /r/build/steem/v0.5.0/windows-full-release/programs/cli_wallet/cli_wallet.exe.
These are fully static builds which means the executables work by themselves on a 64-bit Windows machine. There is no need to install any other libraries on the Windows host.
I built the steemd.exe and cli_wallet.exe executables using the process described in the OP for the code as of this commit. So that code is actually ahead of v0.4.3.
I have made those executables available in a zip file available at this link for anyone who really wants to play around with it now and doesn't have access to a Linux system (or doesn't want) to build it themselves.
Because steemit.com is being very foolish right now (it refuses to let me create a link to IPFS without automatically treating it like an image) I have to provide the link in a code block:
That download link is to an IPFS gateway. So it should download to your computer to a filename like
QmbFbd3MYj6LrRjPFKPUkzUHMEtAdnHMDqfW7NXzjMt3Ah. You need to rename it to a zip file likesteem.zipand then you can extract the two executables inside.If you have sha256sum.exe on your computer (I recommend getting it), you can verify that the zip file you downloaded is good by comparing it to the following sha256sum:
If you don't get the above sha256sum for the file you download, then it isn't the correct file and I cannot guarantee the safety of what you run.
If you have 7-zip installed, it automatically puts a "CRC SHA" context menu in Windows Explorer. Right click the .exe, CRC SHA -> SHA-256. Capitalization of the hash doesn't matter.
Great tip. Although, I should mention that the SHA256 sum I included in the post is of the downloaded zip file, not of any of the exes.
Ive used this guide to successfully make working 0.11.0 builds of Steem for Windows, thanks!
One question, unlike Bitcubes's build, the compiled versions Ive produced are massive, like 20MB executables. They are also slightly slower (5%). I even edited the your handy scripts to build shared lib versions and it made no difference.
Any idea what would cause this.. is it as its cross compiled? Id love to get better at compiling/porting stuff, so any tips appreciated! :)
I would have said they are larger because it is a fully static build, but apparently you are saying based on your attempts it isn't much better when using shared libraries. So, I am really not sure. Are the shared lib versions you made using Boost and OpenSSL as the shared libs? What is the size difference in the files (both the reduction in the cli_wallet.exe and steemd.exe files individually as well the net change, if any, between all necessary files to run cli_wallet and steemd together before and after the shared lib version)?
Perhaps it can be explained by the fact that bitcube's executables require "Visual C++ Redistributable Packages for Visual Studio 2013" as a dependency to run, but the MinGW compiled versions do not because any relevant code dependencies are already included in each of the executable? Again, just a guess and not something that I'm certain about.
I think the performance difference may be explained by the fact that this guide does not use GMP/MPIR which I believe bitcube does use in his builds (although I may be wrong about that; can I get clarification about that @bitcube?). You could follow the guides floating around on how to use GMP and make the necessary modifications to my guide and build scripts to have the MinGW Windows build use GMP as well. It is a slightly more difficult task with the cross-compile build because you first need to cross-compile GMP as an additional dependency in addition to OpenSSL and Boost.
By the way, I would love to see your modifications to get the shared libs working, if you are willing to share them. Maybe you can put up your own modified guide as a post for some upvotes. Also, if you get GMP working in the cross-compile build, I think that is worthy of a guide as well.
I basically just added "link=shared" to Boost and "shared" to the OpenSSL helper scripts in the appropriate section. This produced files about 2mb smaller (each), but still way bigger than Bitcubes (6mb vs 20'ish).
Ive made some tweaks to OpenSSL's and Boost build options and testing right now too. Looking at GMP now too, to see how possible it is!
Within configure_build.py I found this:
Maybe add Windows cross-compilation flags
--
I tried changing to DFULL_STATIC_BUILD=OFF but during compile it seemed to complain about it and put it back on. I suspect this is causing the mammoth 20mb exe's?
I want this exact static build but not for windows. I need it for linux and I'm getting an error when I leave off the --win flag:
/r/mingw/lib/libboost_unit_test_framework-mt.a(unit_test_parameters.o):unit_test_parameters.cpp:(.rdata$_ZTVN5boost6detail17basic_unlockedbufISt15basic_streambufIcSt11char_traitsIcEEcEE[_ZTVN5boost6detail17basic_unlockedbufISt15basic_streambufIcSt11char_traitsIcEEcEE]+0x70): undefined reference to `std::basic_streambuf<char, std::char_traits >::xsputn(char const*, long long)'
collect2: error: ld returned 1 exit status
libraries/fc/CMakeFiles/api.dir/build.make:115: recipe for target 'libraries/fc/api' failed
make[2]: *** [libraries/fc/api] Error 1
CMakeFiles/Makefile2:262: recipe for target 'libraries/fc/CMakeFiles/api.dir/all' failed
make[1]: *** [libraries/fc/CMakeFiles/api.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
If anyone can help me with this it would be greatly appreciated. I'm too newb to do these static builds :(
Geat job @arhag !!!
I hope you can re-build cli_wallet with ncurses & doxygen support?
helpandgethelpcommand looks ugly on windows :DI updated the above guide so that doxygen is installed on the system and so the compiled cli_wallet has proper working documentation built-in.
Nice!!!... I really appreciate it.
have you build v0.84 for windows
following above step by step but getting following error while compiling v0.85
make[2]: *** [libraries/fc/vendor/udt4/CMakeFiles/udt.dir/src/api.cpp.obj] Error 1
make[1]: *** [libraries/fc/vendor/udt4/CMakeFiles/udt.dir/all] Error 2
make: *** [all] Error 2
Yes, this guide is currently broken for v0.8.x. I haven't spent time on trying to fix the code to make it work because people had some instability issues with the MinGW build of v0.5.0, and @bitcube had a more stable build using the Microsoft Visual Studio compiler.
He has also recently made changes to get v0.8.5b to compile using Visual Studio. You can find that guide here. Or, if you don't want to compile it yourself and just want to use the executable, check out this guide instead.