feat: implement text chunking processor with fixed token length and delimiter algorithm #429
| Job | Run time |
|---|---|
| 53s | |
| 1m 14s | |
| 59s | |
| 53s | |
| 59s | |
| 1m 8s | |
| 1m 2s | |
| 53s | |
| 1m 4s | |
| 52s | |
| 55s | |
| 54s | |
| 56s | |
| 1m 4s | |
| 55s | |
| 50s | |
| 1m 3s | |
| 58s | |
| 54s | |
| 53s | |
| 58s | |
| 56s | |
| 50s | |
| 54s | |
| 52s | |
| 52s | |
| 1m 4s | |
| 48s | |
| 53s | |
| 53s | |
| 52s | |
| 57s | |
| 54s | |
| 56s | |
| 52s | |
| 51s | |
| 33m 41s |