I haven't used the heightmap loader so I can't be of much help there, but I can offer a different approach.
I've been creating all my terrain with WorldMachine (http://www.world-machine.com/). It is quite frankly the diggitybomb.
I export my heightfield as RawFP32 (floats) and load these into an array with the binary streamreader and take it from there. This format suits the heightfield objects and can easily be turned into a terrain mesh for drawing. This also allows dynamic reduction for LOD.
Once you get started on doing your own terrain LOD it's a long, long road, but well worth it. Alternatively you can export it into a 3D modeling program and create a .FBX model of your terrain. No sense in going all out if all you need is a simple limited area of terrain. This gives you the benefit of including other elements in your model such as distant foliage and detail that you don't intend to collide with.