[Bugfix] Fix xgrammar failing to read a vocab_size from LlavaConfig on PixtralHF.

[sjuxax]

In trying to use an updated Pixtral-HF with xgrammar, I got AttributeError: 'LlavaConfig' object has no attribute 'vocab_size'. Full traceback:

Traceback

Traceback (most recent call last):
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/starlette/middleware/base.py", line 185, in __call__
    with collapse_excgroups():
         ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/starlette/_utils.py", line 82, in collapse_excgroups
    raise exc
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/starlette/responses.py", line 255, in wrap
    await func()
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/starlette/responses.py", line 244, in stream_response
    async for chunk in self.body_iterator:
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/vllm/entrypoints/openai/serving_chat.py", line 319, in chat_completion_stream_generator
    async for res in result_generator:
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/vllm/utils.py", line 399, in iterate_with_cancellation
    item = await awaits[0]
           ^^^^^^^^^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/vllm/engine/multiprocessing/client.py", line 584, in _process_request
    params = await \
             ^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/vllm/engine/async_llm_engine.py", line 553, in build_guided_decoding_logits_processor_async
    processor = await get_guided_decoding_logits_processor(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/vllm/model_executor/guided_decoding/__init__.py", line 107, in get_guided_decoding_logits_processor
    return get_local_xgrammar_guided_decoding_logits_processor(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/vllm/model_executor/guided_decoding/xgrammar_decoding.py", line 33, in get_local_xgrammar_guided_decoding_logits_processor
    config = GrammarConfig.from_guided_params(guided_params=guided_params,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/vllm/model_executor/guided_decoding/xgrammar_decoding.py", line 151, in from_guided_params
    vocab_size=model_config.hf_config.vocab_size,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jeff/.virtualenvs/vllm312/lib/python3.12/site-packages/transformers/configuration_utils.py", line 210, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'LlavaConfig' object has no attribute 'vocab_size'

Indeed the LlavaConfig class of transformers has no vocab_size. Fixed this by requesting the text config specifically with vocab_size=model_config.hf_text_config. Sounds right to me!

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[mgoin]

I thought this wouldn't exist for text only models, but it seems this is a noop in that case - so elegant solution!

vllm/vllm/transformers_utils/config.py

Lines 551 to 562 in 6faec54

	def get_hf_text_config(config: PretrainedConfig):
	"""Get the "sub" config relevant to llm for multi modal models.
	No op for pure text models.
	"""
	if hasattr(config, "text_config"):
	# The code operates under the assumption that text_config should have
	# `num_attention_heads` (among others). Assert here to fail early
	# if transformers config doesn't align with this assumption.
	assert hasattr(config.text_config, "num_attention_heads")
	return config.text_config
	else:
	return config