TFloat (FAST_FLOAT) work done & slightly different idea used to make code easily switchable between double & float

[GerHobbelt]

@stweil : saw your work in the TFloat branch.

This pullreq is FYI only (it's an unaldulterated copy of my tesseract fork's TFloat branch and thus has way too many commit diffs to be mergeable) -- I'll ready a decent pullreq tonight or tomorrow, but I wanted to send a heads up so you can decide whether you like this or not.

The key idea here with FAST_FLOAT is to use template<T, ST> functions for Serialize and DeSerialize, so we don't have code cluttered with lots of #ifdef/ifndef/... to make it happen.

Problem being the tesseract data files: those carry data in double format (or float for 'old' scales[]), while the run-time type is dictated by the new TFloat type you came up with: that one is either float or double, depending on a compile-time define. (FAST_FLOAT)

The idea implemented here is to have the Serialize and DeSerialize functions (in TFile and elsewhere) do any necessary conversions between the run-time type and file-storage (persisted) type.

See for example this bit of code from serialis.h:

  // Serialize data.
  bool Serialize(const std::string &data);
  bool Serialize(const std::vector<char> &data);
  template <typename T>
  bool Serialize(const T* data, size_t count = 1) {
	  return FWrite(data, sizeof(T), count) == static_cast<int>(count);
  }
  template <typename T, typename ST>
  bool Serialize(const T *data, size_t count = 1) {
	  ST* arr = new ST[count];
  	  for (size_t i = 0; i < count; i++)
	  {
		arr[i] = data[i];
	  }
	  bool rv = (FWrite(&arr[0], sizeof(ST), count) == static_cast<int>(count));
	  delete[] arr;
	  return rv;
  }
  template <typename T, typename ST>
  bool Serialize(const std::vector<T>& data)
  {
	  std::vector<ST> arr;
		size_t len = data.size();
		arr.resize(len);
		for (size_t i = 0; i < len; i++) {
			arr[i] = data[i];
		}
		bool rv = Serialize(arr);
		return rv;
  }
  template <typename T>
  bool Serialize(const std::vector<T> &data) {
    // Serialize number of elements first.
    uint32_t size = data.size();
...

which now has the extra Serialize() members:

  template <typename T, typename ST>
  bool Serialize(const T *data, size_t count = 1)

and

  template <typename T, typename ST>
  bool Serialize(const std::vector<T>& data)

where T represents the run-time type (Tfloat) and ST represents the storage type (double for new data files; float for old scales[] data).

Then the code can easily dictate what the output to disk is going to be and thus be cross-compatible with other builds, which have their FAST_FLOAT /un/defined, e.g. this snippet from weightmatrix.cpp:

bool WeightMatrix::Serialize(bool training, TFile *fp) const {
  // For backward compatibility, add kDoubleFlag to mode to indicate the doubles
  // format, without errs, so we can detect and read old format weight matrices.
  uint8_t mode = (int_mode_ ? kInt8Flag : 0) | (use_adam_ ? kAdamFlag : 0) | kDoubleFlag;
  if (!fp->Serialize(&mode)) {
    return false;
  }
  if (int_mode_) {
    if (!wi_.Serialize<int8_t>(fp)) {
      return false;
    }
    // The scales stored in memory have an extra factor applied to them
    // to allow faster operation. We have to remove that factor here
    // before writing to disc.
    auto scales = scales_;
    for (auto &scale : scales) {
      scale *= INT8_MAX;
    }
    uint32_t size = scales.size();
    if (!fp->Serialize(&size)) {
      return false;
    }
    if (!fp->Serialize<TFloat, double>(&scales[0], size)) {
      return false;
    }
  } else {
    if (!wf_.Serialize<double>(fp)) {
      return false;
    }
    if (training) {
      if (!updates_.Serialize<double>(fp)) {
        return false;
      }
      if (use_adam_ && !dw_sq_sum_.Serialize<double>(fp)) {
        return false;
      }
    }
  }
  return true;
}

Note the <float> and <double> usage there: this in the function calls: these now dictate the output format (and at a place in the code where this is relevant: now this code remains easy to read as the file format can be read off the code lines without any trouble).

I hope you like it. The commit list attached to this pullreq is a mess: disregard anything but the last couple commits (a cleaned up pullreq will follow next day), where this work was done (including work on AVX/SSE code to cope with the new float vs double TFloat approach:

• SHA-1: 31de23d = Continued work on SHA-1: d397065 --> Part 2
• SHA-1: d397065 = Part 1: redesigned the TFloat approach using templates for the Serialization and Deserialization methods. Tested Deserialization with double (i.e. standard, non-optimized) layout: run-time type == storage type.

---

There's other work in there too, which will be filed in a separate pullreq as it's only sideways related:

• SHA-1: e0a9b7c..21d5cbb

[GerHobbelt]

The cleaned-up version of this now exists as stweil#7 : using C++ function templates to DRY & still have float/double (Tfloat: #3486) selectable.