Build systems

[mjpost]

Can you comment on build systems?

bjam is the default and preferred. I see you have provided compile_query_only.sh, presumably for convenience for folks who don't want to bother with boost.

What about cmake? I see that that is added to the system and in fact just build Joshua using it, but it's not clear to me that this was the right thing to do. In particular, cmake does not seem to respond to environmental settings of e.g., KENLM_MAX_ORDER. Why is cmake present, and what is its intended use, and why is it included? It also litters files all over the place.

It seems I should revert to using bjam in my own build process.

(My goal is to make it easier to depend on KenLM. Ideally I'd like to package it as a submodule. I've already separated KenLM from Joshua's wrappers and it works well, apart from the build system complication).

(Caveat: I do not understand modern build systems.)

[emjotde]

Hi,
it seems Lane did the CMakeList files. I'd love to get rid of bjam in favor of CMake.
Also the littering thing: CMake is supposed to be used as an "out-of-source" build. The idea is you create a folder anywhere you like and then you give cmake the path to the source. One typical way is to use it like this:

cd kenlm
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=release
make -j 4

No littering. Everything gets built in "build".

[mjpost]

That would be nice, but the example you provided doesn't work for me (no files are generated in build/).

I think that cmake isn't set up correctly. I also wasn't able to pass in KENLM_MAX_ORDER, unfortunately, except by directly modifying one of the files.

I'm sure this isn't that hard to setup correctly, so maybe I'll look into it, but I don't think I'll be able to in the near term.

[emjotde]

Hm, the exact commands work alright for me. The parameters are indeed missing though. Should be easy to fix.

[mjpost]

Hmm; it works for me for cmake version 3.0.2 (on Linux), but not for version 3.3.1 (OS X).

[mjpost]

How about fixing this, @dowobeha?

[kpu]

Quoth: https://twitter.com/mccv/status/350037001384361984
New interview question: "what's your favorite build system?" Correct answers: 1) Rage face 2) None of them 3) Literal table flip

bjam is annoying, but less annoying than autofools. It's like LaTeX in that regard. If somebody wants to replace the build system, I'm ok with that. Just not motivated enough to do it myself in e.g. Moses again.

[kpu]

On a tangential note related to your motivation, it might be nice to extract a general-purpose JNI wrapper.

[mjpost]

Marcin, nevermind about it not working — I just had to delete the cached files that were scattered all over, and now everything works nicely.

[mjpost]

@kpu, agreed, I'll keep it in mind.

[skystrife]

@mjpost: CMake not respecting KENLM_MAX_ORDER should be fixed as of 33c6e65; you can specify it during the call to cmake now with e.g.

mkdir build
cd build
cmake .. -DKENLM_MAX_ORDER=10 -DCMAKE_BUILD_TYPE=Release
make -j4

[mjpost]

Fantastic; it worked perfectly on both OS X (boost 1.59) and CentOS 6.7 (Boost 1.41). This is really nice, thanks a lot.

Is there anything else on this in your immediate pipeline, or should I pull this into Joshua?

[kpu]

We're working on interpolation (including @skystrife), but there's more work to be done.

[mjpost]

These instructions should be added to BUILDING and README.md, after which, I think this issue can be closed.

[kpu]

Fixed in 9662a51 by switching to cmake.