feat: implement text chunking processor with fixed token length and delimiter algorithm #431
| Job | Run time |
|---|---|
| 1m 16s | |
| 55s | |
| 1m 4s | |
| 57s | |
| 1m 10s | |
| 1m 6s | |
| 1m 3s | |
| 1m 1s | |
| 59s | |
| 50s | |
| 1m 3s | |
| 55s | |
| 55s | |
| 57s | |
| 56s | |
| 1m 0s | |
| 50s | |
| 53s | |
| 52s | |
| 53s | |
| 54s | |
| 52s | |
| 53s | |
| 56s | |
| 57s | |
| 56s | |
| 54s | |
| 51s | |
| 55s | |
| 51s | |
| 1m 7s | |
| 54s | |
| 54s | |
| 54s | |
| 54s | |
| 56s | |
| 34m 13s |