When running opensearch-update with date, production gives request size exceeded

[sfisher]

I tried to run update and I get request size exceeded which I haven't gotten in other environments yet. Maybe a bug in my script or maybe an Opensearch setting or maybe some different and huge data on production.

Like python -u manage.py opensearch-update --updated_since=2024-11-04T00:00:00Z 2>&1 > ../logs/opensearch_indexing.log:

  File "/apps/ezid/ezid/manage.py", line 21, in <module>
    main()
  File "/apps/ezid/ezid/manage.py", line 17, in main
    django.core.management.execute_from_command_line(sys.argv)
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/apps/ezid/ezid/ezidapp/management/commands/opensearch-update.py", line 83, in handle
    result = self._do_bulk_update(string_parts)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/apps/ezid/ezid/ezidapp/management/commands/opensearch-update.py", line 118, in _do_bulk_update
    response = OpenSearchDoc.CLIENT.bulk(body=json_string)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/opensearchpy/client/utils.py", line 176, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/opensearchpy/client/__init__.py", line 555, in bulk
    return self.transport.perform_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/opensearchpy/transport.py", line 455, in perform_request
    raise e
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/opensearchpy/transport.py", line 416, in perform_request
    status, headers_response, data = connection.perform_request(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/opensearchpy/connection/http_urllib3.py", line 308, in perform_request
    self._raise_error(
  File "/ezid/.pyenv/versions/ezid-py311/lib/python3.11/site-packages/opensearchpy/connection/base.py", line 315, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
opensearchpy.exceptions.TransportError: TransportError(413, '{"Message":"Request size exceeded 10485760 bytes"}')