File table engine may open too many files at once.

[alexey-milovidov]

I almost downloaded the data from http://www.gharchive.org/
64901 files, 839 GiB in size.

But I cannot simply process this data as is:

$ clickhouse-local --query "SELECT * FROM file('*.json.gz', TSV, 'data String') LIMIT 10"
Code: 76, e.displayText() = DB::ErrnoException: Cannot open file /opt/milovidov/example_datasets/gharchive/2018-12-30-10.json.gz, errno: 24, strerror: Too many open files (version 20.2.1.1)

[alexey-milovidov]

Now it works like a charm. Even with data in garbage format json.gz it is processed with more than 500 MB/sec on old E5-2650v2 and RAID-5 of 8 HDDs.