Generalize header parsing

[ybasket]

Let ParseableHeader return errors if parsing fails so that more interesting header parsers can be written and parsing can fail early if headers are not as expected. Also make CsvRow a case class to allow for pattern matching.

Use case example: CSV with translations, first column being the key, the other columns being the languages identified by the headers. As it doesn't make sense to parse the file when the languages are invalid, it's nice to be able to fail on header parsing already instead of having to revalidate the headers on every row.

[ybasket]

Thinking again about the use case and others where headers play a significant role, I decided to change header parsing from being always by column to the more general way of parsing the whole header row at once. That allows for more validation (for example, check that all expected columns are present, no duplicate column names) and for more interesting types (like pre-building a parse function based on the header). To keep the impact small and still support the previous by column approach, I added type aliases and extension methods which keep many use cases source compatible.

It's definitely a debatable change (which might even pave the way to further changes which for example, could make the Option[Header] in CsvRow easier to deal with), so discussion is very welcome :)

[ybasket]

Codacy only complains about the null literal as parent exception for HeaderError, can be safely ignored.

[satabin]

I am not sure to be a big fan of putting the operations in an implicit class. Since we control the entire class and don't extend some third party type, let's try to involve less implicit searches.

[ybasket]

How would you do it then? Note that it is an extension for CsvRow objects whose headers are Nel-based (= CsvNelRow), as these don't work on all header types.

[satabin]

Hmm indeed, I missed that part. Then I think I need to read everything through again, I must have missed something here.

[satabin]

Can you use the dot syntax please? Not a big fan of the dotless application :)

[satabin]

Also is it like really way more efficient to use NIO than just Stream.emits(s)? If the performance gain is marginal, then I would prefer not having a CharBuffer.

[satabin]

I like having my personal relative imports at the beginning and the Java ones at the end (yes, I'm that kind of guy :)).

[satabin]

Globally speaking I still miss to see the use case for a non-list header type. It adds some complexity to the types and operation resolution. And I can't seem to come up with a use case for it right now that would convince me totally. If you have such a use case, I would be really happy to hear about it :) Maybe even put it in the README, so that it's clear why we add this complexity.

[ybasket]

Globally speaking I still miss to see the use case for a non-list header type. It adds some complexity to the types and operation resolution. And I can't seem to come up with a use case for it right now that would convince me totally. If you have such a use case, I would be really happy to hear about it :) Maybe even put it in the README, so that it's clear why we add this complexity.

As I wrote above, it is debatable change as it comes with some overhead/complexity. The following is a simplified version of an example I encountered at work:

sealed trait Language
// instances...
case class I18nValue(translate: Language => String)

implicit val i18nRowDecoder: CsvRowDecoder[(String, I18nValue), String] = (row: CsvRow[String]) => {
    for {
      headers <- row.headers.toRight(new DecoderError(s"CSV file must have headers"))
      langs <- headers.tail
        .traverse { lang =>
          Language
            .withNameInsensitiveOption(lang)
            .toRight(new DecoderError(s"""Unknown supported language "$lang""""))
        }
      i18n <- I18nValue.buildFromAllLanguages(langs zip row.values.tail) // fails if not all languages are present
    } yield row.values.head -> i18n
  }

The code would be way cleaner if it was possible to fail processing when parsing the headers already (not all languages present, invalid languages present) and not while handling the rows themselves (which implies the overhead of checking language exhaustivity for each row again).

Similar early validation use cases I could see when importing CSV files into a RDBMS, by the table structure you can reject CSV files when parsing the headers, but you don't want to check again on every row you insert.

Of course, in case of an error, the stream fails on the first row, but for correct files (which should be the dominant case) it adds overhead not be able to fail header parsing. Plus the necessary workaround makes it harder to deal with bad rows as attemptDecode will alway return a Left if the headers aren't valid.

Happy to fix all the comments you left in case this convinces you.